I think that you think that a certain borough is not all that it could be, when, in fact, it is all that it should be... And more!
Hyperlocal site Go Earlsfield has this graph, courtesy of think tank The Centre for Cities, which shows that Lewisham's population shrinks by day. It's from a report ('Inner London's Economy') that attempts to understand what happens to the populations of each borough in London's inner core on a daily basis, as commuters flood in and out.
Inner London accounts for 40 per cent of Greater London’s population, but 56 per cent of private sector jobs. The daytime population of Inner London is 68 per cent higher than its resident population; most of the boroughs import workers daily, but Lewisham, Greenwich, Newham and Wandsworth experience (surprisingly) small net outflows each day:
With thanks to Richard for the nod.