Pour the brownie mix into a large mixing bowl, add the milk chocolate chips to the brownie mix, and bake according to the directions on the box. Best to use a 9x13 pan and bake for about 25 minutes. Cool and then cut into 1-inch squares.
In a large bowl, blend the pudding mix and the milk for about 2 minutes. A wire whisk works well for this step. Then fold in about 3 cups of cool whip.
Begin to layer about 1/2 the brownies in the bottom of a large trifle bowl, half of the pudding mixture, about 1/2 cup toffee bits and pieces, and a layer of cool whip. Repeat with another layer. Top with remaining toffee bits and pieces.
Add additional candy toppings, if desired. Cover and store in the refrigerator at least overnight.