I wouldn't say it's considered rude, but it's just not really normal, especially in bigger cities, and definitely not in Tokyo. It is possible that you could look a bit crazy or suspicious if you make a point of meeting people's eyes and smiling. It could also lead to misunderstanding.