Preheat the oven to 400°F (200°C).

1. Roll out the puff pastry sheet on a lightly floured surface until it’s about ¼-inch thick. Cut the pastry into three equal-sized rectangles (about 4×6 inches).
2. Place the pastry rectangles on a baking sheet lined with parchment paper and prick the dough all over with a fork to prevent it from puffing up unevenly.
3. Bake the puff pastry in the preheated oven for 15-20 minutes, or until golden brown and crisp.
4. Once baked, remove the puff pastry from the oven and allow it to cool completely on a wire rack.
5. Once cooled, dust the puff pastry rectangles with powdered sugar.
6. In a saucepan, heat the milk over medium heat until it starts to steam. If using a vanilla bean, split it lengthwise and scrape out the seeds into the milk, then add the pod as well. If using vanilla extract, add it to the milk once it’s heated.
7. In a separate bowl, whisk together the egg yolks, sugar, flour, and cornstarch until smooth and pale.
8. Slowly pour the hot milk into the egg mixture while whisking constantly to temper the eggs.
9. Return the mixture to the saucepan and cook over medium heat, whisking constantly, until the custard thickens and comes to a boil. Let it boil for 1 minute, then remove from heat.
10. Stir in the butter until smooth. If you used a vanilla bean pod, remove it from the custard now.
11. Transfer the custard to a bowl, cover with plastic wrap, and refrigerate until it’s completely chilled (about 1-2 hours).
12. Once the puff pastry is cool and the custard is chilled, start assembling the layers. Place one rectangle of puff pastry on a serving platter or plate.
13. Spread a generous layer of the vanilla custard over the puff pastry.
14. Top the custard with another rectangle of puff pastry and repeat the process until all the layers are stacked, ending with the third puff pastry rectangle on top.
15. In a small bowl, mix the powdered sugar, water, and vanilla extract until smooth and the glaze is thick but spreadable.
16. Spread the glaze over the top puff pastry layer, smoothing it out with a spatula. You can also use a fork to create a decorative pattern in the glaze if desired.
17. Chill the Mille-Feuille for 1-2 hours to allow the layers to set and the flavors to meld.
18. Slice into individual servings and serve cold or at room temperature.