Follow my directions for preparing a batch of brioche, and rest in the fridge 12 hours or overnight.
On a well-floured surface, roll the dough into an 18 x 24-inch rectangle, about 1/4-inch thick.
Spread the butter evenly across the top of the dough. In a small bowl, combine the brown and granulated sugars, the cinnamon and nutmeg, and the salt. Scatter and spread evenly over the buttered dough.
Spray a pan well with baking spray. Starting at the long edge closest to you, roll the dough into a tight log. Slice the log into about 16 pieces that are 1-inch thick. Place the buns on their side on the prepared baking tray.
Let proof at room temperature until the dough springs slowly back into place when touched lightly with your fingertip. Meanwhile, preheat the oven to 350°F. Bake for 30 minutes, until the buns are slightly browned on top.
While they are cooling, whisk together the powdered sugar and milk until smooth. Drizzle over the top of the warm buns and serve.