'On foot' is the more commonly used expression. This also stems from the fact that 'on' is usually used for actions involving body parts. 'By' is usually used to talk about a means of transport (i.e. train, car, boat, plane, etc). They both have the same meaning, albeit with different grammatical structure.