It was not a baby, no. It was, as mentioned before, a small part of his soul. As nearly all horcruxes had been destroyed, where bits of Voldermorts soul lived, meant there was only a tiny bit of him left. This left Voldermort in a weak condition which cannot be helped when gone so far.

In the Harry Potter series, the "baby" seen by Harry in the "Limbo" version of King's Cross Station is the mutilated fragment of Lord Voldemort's soul. This specific piece of soul was the one that had been living inside Harry as an unintentional Horcrux. When Voldemort hit Harry with the Killing Curse in the Forbidden Forest, it did not kill Harry but instead destroyed that soul fragment. In the white, dream-like station, Harry sees it as a small, flayed, and distressed creature that is "stunted and maimed." Dumbledore explains to Harry that there is nothing they can do to help it. It represents the ultimate result of Voldemort's choices to tear his soul apart for immortality; while Harry is healthy and whole in this limbo, Voldemort's soul is left in a state of eternal, pitiful suffering, unable to "go on" or return to the living world.