Larry Ryan is a writer and editor. He was the Founding Editor of The Long + Short, an online magazine about new ideas and innovation, and previously worked at the Independent.

Londoners have always envisaged the future, sometimes bizarrely.

The years prior to 2012 saw a stream of criticism surrounding the plans to host the London Olympics. The costs of pulling everything together; the impact on the Lea Valley; the displacement of small local businesses; the prospect of security missiles on east London roofs; the security miscalculations (troops were eventually brought in alongside G4S); the unsightliness of Anish Kapoor’s Orbit; the fast lanes to the site for grandees – all this, and more, led to predictions that Londoners would leave during the games and reduce the city to a ghost town. But once the gold medals started rolling in, the concerns dissipated. Today, we are more likely to remember the games as an expression of the UK as a multicultural, modern, open, and sportingly successful country.

To plan for future triumphs often requires us to be blinkered. Faith in an idea for the future can demand tunnel vision. The German-born economist, Albert O. Hirschman, who died in 2012, liked to tell the story of the 19th-century construction of the railway line connecting Boston and the Hudson River. Laying the line required tunnelling through a mountain, something planners assumed would be relatively easy. But the project was much more complicated and difficult than anyone had imagined, and cost ten times more than expected. If the designers and planners and construction companies had known how hard it would be, no one would have committed to it; yet in the end, the result immeasurably improved the economy of the region.

Hirschman developed the principle of the Hiding Hand, a play on Adam Smith’s Invisible Hand, which argued that humans have a natural propensity to underestimate the difficulty of preparing for the future. This leads, on the one hand, to ingenious problem-solving of a kind we wouldn’t willingly embark on if we knew what was coming, and on the other to unintended consequences and perverse outcomes.

As London tries to think itself into the world of the mid-21st century, to assess what will be required in 10, 25 or 50 years’ time, it can be instructive to look at how past Londoners have imagined the city’s future. In a previous issue of London Essays, Geoff Mulgan suggested that the city is full of membranes into the past. Arguably, the grand projects of London’s past offer a membrane of sorts into the future, even if the future for which they were built never truly materialised. They act as monuments to our ideas of how we thought we could shape things.

The modernism of the Royal Festival Hall, for example, tells of a brief exciting moment amid post-war austerity when the Festival of Britain celebrated the country’s modernity and energy – although its attendant monument, the Skylon, was famously dismantled and sold for scrap. Further down river, Canary Wharf suggests a less communitarian, more Thatcherite-free-market vision of the future. The Barbican offers up a utopian democratic vision, although its brutalism can also sometimes be uncomfortable: it turned out that Londoners were less keen to learn a new way of using urban space than its architects, Chamberlin, Powell and Bon, anticipated. The elevated thoroughfares and areas originally intended for shops remain deserted. But the Barbican continues to thrive, as a visit any weekend will attest. And an apartment, if one ever becomes available, is well out of the price range of most Londoners.

At least as instructive are the plans that never materialised. In 1855, Joseph Paxton proposed the Great Victorian Way, a ten-mile glass loop circling a portion of central and west London: a spectacular arcade of glass-covered streets, roadways, shops, railway stations, and three river crossings. New technology would enable the use of glass strong and cheap enough for the project, which received the backing of parliament. In the event, the cholera epidemic of 1858’s Great Stink meant funds had to be diverted and the plan shelved; ultimately, London’s sewer system was created instead. The unbuilt loop is said to have provided the basis for the route of the Central line.

In 1909, the writer Ford Madox Ford published an essay titled The Future in London, offering a provocative vision of a planned city circumscribed by a 60-mile sweep of a compass point set in Threadneedle Street. As Iain Sinclair has noted, this anticipated the vision of Britain’s most famed town planner, Patrick Abercrombie (the moving spirit behind the M25 and the green belt) in reading “London as a series of orbital hoops, ring roads and parkland”.

Bizarrely, in the 1930s, Charles Glover proposed turning King’s Cross into an airport with eight runways arranged in an octagon on stilts. Later, he suggested planting a heliport on the roofs of Covent Garden. These plans didn’t take off, so to speak, though in some respects London City Airport is a descendant.

There were other attempts to elevate the city: during the postwar reconstruction of the 1950s and 1960s, the City of London Corporation proposed a network of elevated walkways between the buildings of the Square Mile. These “pedways” would take pedestrians off the street and give them their own walkways on higher ground. Some were built, albeit in a scattershot manner: the plan only really found expression in the Barbican (with limited success), and elsewhere, the pedways ran into dead ends, or failed to join up with each other. The Corporation eventually abandoned the policy. Fragments stand today as small glimpses into a planned post-war future that didn’t quite come to pass.

In 1954, Geoffrey Jellico, Ove Arup and Edward Mills devised a scheme to demolish Soho bit by bit and replace it with several large towers sitting on top of a platform, below which gardens and canals would have traced the shape of former streets.

In the late 1960s, a scheme was hatched to build the London Ringways – miles of elevated motorways encircling and crossing the city. Thousands of people would have had to be moved and entire districts of the capital disrupted for this to come to pass. Public resistance, the first stirrings of the road protest movement and high costs brought the scheme back down to earth.

In 1982, there was an idea to pedestrianise Oxford Street by raising the cars onto a flyover running the length of the street, about two stories above the existing road. Escalators would allow pedestrians to access buses above. The whole thing would have been encased in glass (the Great Victorian Way lives on), effectively creating a shopping mall out of Oxford Street. After initial interest in the plan by architect Brian Avery (who would later design the BFI Imax and the London Transport Museum), the project ran aground on questions of cost, logistics and fears over pollution. Giant malls to rival Oxford Street would follow later. Attempts to “fix” Oxford Street would continue, as would the pollution.

Muddling through

These wacky schemes do little to moderate the cynicism that is often expressed when our own sense of the future is articulated. But too much scepticism can be unhelpful, inducing helplessness: we do need to plan in some way for the future. And the future can be bright, and much-loved: the Barbican and the Southbank Centre now feel as indelible to London as Hyde Park and Regent Street.

In his 1959 essay, ‘The Science of “Muddling Through”’, the American political scientist Charles Lindblom made the case against too much theory when it comes to future planning, and for “building out from the current situation, step-by-step and by small degrees”.

The problem with grand visions of the future, Lindblom argued, is that “on many critical values or objectives, citizens disagree, congressmen disagree, and public administrators disagree”.  Schemes that start as if the present were a blank slate, proposing their own fundamental values, are almost invariably doomed to failure: “A wise policy-maker consequently expects that his policies will achieve only part of what he hopes and at the same time will produce unanticipated consequences he would have preferred to avoid. If he proceeds through a succession of incremental changes, he avoids serious lasting mistakes in several ways.”

Adapting to fast-moving times with gradual incremental changes can feel like playing catch-up. But it has often been London’s way. When John Nash designed Regent Street in the early 19th century, he imagined a long straight boulevard like those of French cities, running from Portland Place to Carlton House Terrace. Private ownership of land put paid to this design, as did St James’s Square. Instead, his street had to curve to avoid some places along the route, although various streets and buildings were still demolished, whether people liked it or not. The street also had to be moved a little further west. To get down to Pall Mall, Regent Street takes an awkward hard right at Piccadilly. Towards the end of the street’s development, a separate plan to construct Piccadilly Circus was added into the mix. And the buildings were rather soon redeveloped, some of them as little as 70 years later: little of the original remains beyond the shape of things.

A dose of scepticism is a useful asset when it comes to envisaging the city of the future, as is an acceptance that divergent interests and demands will force upon us awkward turns here and there. But ambition and vision are important too. Big ideas have shaped the city and will continue to do so. From Regent Street to the Olympic Park, Londoners’ visions of the future are all around us, muddling through and showing off their optimistic futurism.