An article by Ali Soufan that poses this question, although it can be applied to other Islamist groups. A passage that summs up his views (lightly edited):
Today’s al-Qaeda can boast tens of thousands of fighters under its command, and that is not even counting the thousands more who still swear allegiance to al-Qaeda’s wayward progeny, the Islamic State.Why have jihadi groups survived and grown? In short, because their ideology remains strong. That evolution, Fazul predicted, would make al-Qaeda much harder to defeat. Unfortunately, he was right. Al-Qaeda, the Islamic State, and other jihadi groups have become adept at luring disaffected young men with false claims of an epochal war between Islam and the West and fraudulent promises of history-shaping adventure.
He has some thoughts, none startling, on as he concludes:
we must dedicate ourselves to undermining the resource that underpins each of its branches: its store of ideas.

Very little seems to be done on the 'store of ideas', even after a succession of events that have undermined the optimism in the 'West' that terrorism is a rare, painful event. Then we look around parts of the world and as the author writes there is little room for optimism where most deaths form Islamist activity happens.

There is a thread (with 301,490 views and 274 posts) that provides the background and may be one day this thread will be merged into it: