Bob:

You are rapidly becoming my hero.
US AID does development, and as an institution they (not surprisingly) tend to see effectiveness and development as paths to stability.

US State does governance, and not surprisingly, they tend to see democracy and rule of law as paths to stability.

US Defense does security, and not surprisingly, they tend to see reduction of threats and clearing of secure space as the path to stability.

"3D" proponents typically say "no, you must do all of these in concert, and they one achieves stability."

All of this scratches at the obvious symptoms of the problem, and may well create some temporary window of stability, and may even facilitate over long hard effort actual progress to be made in a very indirect, "even a blind squirrel finds a nut" kind of way.

But at some point the West will need to set the lessons of "how to sustain a colony" aside as the basis for COIN doctrine and look instead to "why colonized people revolt and how to avoid creating, or if created, best repair such conditions."
The core problem is the failed "Failed State" construct that has been undermining our efforts in these places for years.

If only the plumber can fix the toilet, the divorce will not happen.

These are real people in these places who, like Ken suggested, have their own way of doing things.

We are a foreign power, and always viewed as such, whether for good or bad.

We overwhelm domestic government and its legitimate connections, whether intentionally or inadvertently, by our presence, force and wealth.

The entity and theory that is lacking is one that can understand and influence relevant changes in the areas that legitimately concern us---whether by force (at times) or more subtly.

Our current approaches have been less than effective, and a new theory of effective non-US engagement has yet to emerge---probably drowned out by the bureaucratic dominance of the debate by the parties that have been mis-assigned to the tasks, and lack any effective tools and theories.

The legitimate debates are place and people specific, and there are no one size fits all external models.

It is just hard.