Place the Oreo cookies in a food processor and pulse until finely crushed, resembling dark soil. If you don't have a food processor, place the cookies in a sealed bag and crush them with a rolling pin.
Whisk the instant pudding mixes with the cold milk for about 2 minutes, until thickened. Refrigerate for 5 minutes while preparing the cream cheese mixture.
Beat the softened cream cheese and butter together with an electric mixer until smooth and creamy.
Add the powdered sugar and vanilla extract and beat until completely smooth.
Gradually add the chilled pudding mixture and beat on low speed until fully combined and creamy.
Fold in the whipped topping with a spatula until no white streaks remain.
Spread about half of the crushed Oreos evenly across the bottom of a 9×13-inch baking dish.
Spread the pudding mixture evenly over the Oreo layer, smoothing the top with a spatula.
Cover the filling with the remaining crushed Oreos, creating a thick "dirt" layer across the entire surface.
Tuck gummy worms into the Oreo crumbs, leaving some visible on top to create the classic dirt-and-worms appearance.
Cover the dish tightly and refrigerate for at least 4 hours, preferably overnight, so the cookies soften and the filling sets properly.
Serve chilled, scooping through the Oreo "dirt" and creamy pudding layers.