Frost changes the wing texture to something like sandpaper. Snow is even worse, it changes the aerodynamic shape of the wing. On a “clean” wing, the air flows smoothly. On a wing “contaminated” with frost or snow, the air has trouble sticking to the wing surface.