Baku is a very walkable city. There's wide footpaths, lots of underpasses to avoid a wait at the traffic lights, and it's mostly flat. Your first port of call should be the Old City (Icheri Sheher). The Old City is a compact walled city and it's a good idea simply to lose the map and wander through the alleyways.