As long as they look neat, are clean and have no holes in them, it should be ok. Personally I find sweat pants to be a sloppy look. If you want that level of comfort, try some slim line jogger pants or even leggings. A much higher range of movement and comfort for both men and women.