Quote Originally Posted by William F. Owen View Post
My personal experience of a systematic, well defined, repeatable, and well understood assessment process, is that it is of limited use, when carried out by human beings dealing with incomplete, changing and ambiguous data.

Some old guy called Carl Von something called it "friction" and I think specifically warned against trying to quantify it.

However, as someone passionate about military science and thought, I'd love to see you post of one these cards or some deeper explanation so we may better assess it.
I mentioned this briefly before in a similar topic some weeks ago, but I think the methodology used by the strategic warning community, often called "indications analysis" might be useful for some of the difficulties in judging effectiveness and or progress toward various goals in coin. Indications analysis is designed to be carried out by "human beings dealing with incomplete, changing and ambiguous data" - it comes with the territory of the strategic warning problem.

Probably the best primer on the subject is Cynthia Grabo's now-declassified and updated text from the 1970's. In a simplistic nutshell, indications analysis works backward from a particular end-state. A series of indicators, or signposts, on the way to that end-state are developed and then monitored. Although developed for warning, any end-state or scenario can be broken down and analyzed using this methodology. The advantages are two-fold: It can be used to make predictions (which is what it was originally designed to do) but it also can provide a means to analyze and compare various courses of action one might take.

To borrow from Mark's post below, indications analysis is both an "art" and a "science" and tries to combine the strengths of both philosophies. Although the framework might be science-based, the nature and ambiguity of information requires human judgment, particularly since indicators and indications are not limited to hard, quantifiable data. From Grabo:

An indication can be a development of almost any kind. Specifically, it may be a confirmed fact, a possible fact, an absence of something, a fragment of information, an observation, a photograph, a propaganda broadcast, a diplomatic note, a call-up of reservists, a deployment of forces, a military alert, an agent report, or anything else. The sole provision is that it provide some insight, or seem to provide some insight, into the enemy’s likely course of action. An indication can be positive, negative or ambiguous (uncertain).
and

An indicator is a known or theoretical step which the adversary should or may take in preparation for hostilities. It is something which we anticipate may occur, and which we therefore usually incorporate into a list of things to be watched which is known as an “indicator list.’’ Information that any step is actually being implemented constitutes an indication. The distinction between expectation and actuality, or between theory and a current development, is a useful one, and those in the warning trade have tried to insure that this distinction between indicators and indications is maintained. Many non-specialists fail to make this careful distinction.
Instead of "preparation for hostilities" you can substitute any theoretical end-state you desire. In fact, Indications Analysis within the intelligence community has expanded beyond the traditional role of warning of hostile actions by adversaries to monitoring a variety of issues of interest to the US. "Warning problems" have been established on a variety of topics that have little to do with the potential of an adversary's attack. I see no reason why this tested framework cannot be used in analysis of COIN and LIC, particularly since in my experience so much effort is put towards current intelligence, which has little value for this kind of estimation and analysis.

I do see two potential problems however. First, indications analysis requires a lot of resources, time and effort. Often, the strategic warning community is under-resourced in lieu of other requirements - it seems likely that a COIN-focused effort would suffer to an even greater extent.

Secondly, indications analysis requires a focused collection effort that may not be available in COIN scenarios. IOW, indicator development and monitoring are not possible if there isn't a significant body of baseline knowledge and the ability to get information in the necessary areas.

Of course, both these limitations apply equally to any other methodology or analytical framework that one might use.