Will Hutton / The Guardian

Power is leaching from the centre, even as the complexities of national and international challenges multiply. It is the hallmark of our times. Whether political or religious leaders, CEOs or five-star generals – all are more constrained in what they can do.

This is a pattern across all societies. The digital age, globalisation and higher levels of education have equipped more people to become insurgents or to form single-issue pressure groups. It is a world where the opportunities to be a Julian Assange, Beppe Grillo, Osama bin Laden, George Soros or Nigel Farage grow by the day. Power is draining away from those in whom it is formally placed, but with no obvious substitute in sight.

As Moisés Naím writes in The End of Power, there are three interrelated dynamics gnawing away at formal power structures – what he calls "the more revolution, the mobility revolution and the mentality revolution". There are more literate, educated people worldwide than ever before who refuse to be regimented and controlled as they once were. They are mobile, migrating and exchanging information to an unparalleled degree. Moreover, fewer will take anything for granted: they expect their voice to be heard, whether on the streets of Cairo, on social media in China or in anti-fracking protests in Sussex.

Naím is the first to concede that the dispersion of power is frequently a force for good. One obvious benefit is that autocracy is on the wane. In 1989, Freedom House reckoned that only 69 countries could be counted as democracies: today, the number has reached 117.

It is also good that the chances of mass war on a 20th-century scale are shrinking: in a world of declining power,, as US generals are relearning in Afghanistan, war is won differently today. It is the fast-moving insurgent who can capture hearts and minds or the terrorist cyberhacker who ends up ahead. But as Naím wryly remarks, the decay of power even undermines the insurgent terrorist groups themselves. He reports that 26 of 45 terrorist organisations dissolved, not from being beaten militarily, but from internal strife and challenge. Al-Qaida's greatest weakness is its own factionalisation, as will be the Taliban's.

It is this tendency to fragmentation and the chorus of often irrational voices insisting that their demands be met that most concerns Naím. It may be easier to establish a single-issue NGO, a religious movement or a political faction, but that does not mean that the consequence is necessarily always beneficial.

There is now certainly hyper-competition from new religious groups, or from blogs, tweets and websites trying to sway your opinion. But the consequences can be perverse. The rise of charismatic religion may challenge centralised, formerly powerful religious groupings that have lost their way, such as Catholicism, under increasing siege from the rise of Pentecostal churches in Africa and South America. But religious fundamentalism's grip on logic and rationality is even more tenuous.

Similarly, millions of blogs and tweets have forced news consumers to fall back on trusted, established sources, aiding media concentration rather than diminishing it.

But the "more, mobility and mentality revolutions" have their most obvious malign impact on politics in general and the political party in particular. Naím observes that parties are the engine room of democracies: they gather a constellation of interest groups around a common set of principles that offer a compass for government. Everywhere, political parties are succumbing to the rise of uncompromising single-issue pressure groups, lobbyists and funders, and the corresponding decline of supporters who want common values expressed. It is now not just parties but whole countries that are held to ransom by a faction or interest group holding a simplified but impossible view of the world – Naim's "terrible simplifiers".

One obvious example is the US Republican party, now in thrall to the Tea Party movement, which sees no value in compromise, but instead worships at the altar of an imagined US constitution that allegedly guarantees a nightwatchman state. Another example is the emergence of the Pirate party in Sweden and Germany. However, interestingly, Naím sees Britain as the laboratory that conclusively proves his point. The rise of Scottish and Welsh nationalism and Ukip, along with the hollowing out of both the Conservative and Labour parties, make the country increasingly hard to govern .

This must, in part, explain the collapse in Labour and Conservative party membership over the past 50 years and the consequent weakening of their capacity to create formal and informal coalitions of a broad set of interest groups around common values.

Ed Miliband's and Labour's failures predictably get the most attention from the press – identikit pieces about his lack of forcefulness, clarity and too much equivocation – as if a new leader could magically solve the problem, but with no understanding of the much wider context in which any political leader now operates.

The Tory party's problems, driven by similar forces, are arguably even more acute. At any other time, David Cameron would be seen as a classic mainstream Tory. Today, he is marginalised by as many as 200 backbenchers owing their position to constituency association selectorates, some of no more than 100 activists, in thrall to "terrible simplicities" on tax, Europe, immigration and welfare.

Any genuinely tough call – to put property taxation on 2013 rather than 1991 values, accept the need for immigration, cigarette packaging or even build the HS2 train line – is made incomparably harder or is simply off-limits because of the veto of a single-issue pressure group that a party is no longer strong enough to take on.

It is the decay of power. The centre fragments and power devolves to myriad new forces that often exercise their power with narrow obsessions in mind. Who now speaks for the whole? Who keeps a macro view, mediating competing interests and conflicts and has the courage to make decisions based on a strategic view of all our interests, not just sectional ones?

Parties have to fight back –arguing better, crystallising policies better, running primaries to select their candidates to widen their appeal – as does our democracy. Representative government was a great invention. It now has to be saved from the single-issue, monomaniac, simplifying, self-interested vandals – a much more interesting position for Mr Miliband to take than a belated "me too" conversion to a referendum on the EU.