Do Metrics Matter?

[Bill Moore]

While my argument is too absolute above, the underlying point in my opinion remains valid which is that we have allowed metrics (the tail) to wag the dog (strategy). Our search for the perfect metric will always be illusionary, but that doesn't mean some limited metrics and assessments don't have a role in conflict, but that role should be much more limited than it is now. Metrics and assessments will never tell us if we're winning if winning means defeating an adversary, or getting the adversary to stop fighting (political agreement, etc.). As I stated above what we measure has little impact on the adversary's will or means to continue fighting for whatever his objectives may be. When metrics distract us from this reality we lose sight of what is really important. I'm sure our metrics would have told us Germany and Japan should have surrendered long before they did based on the futility of continued struggle, but people are proud and stubborn beyond reason. The same holds true for insurgents and the counterinsurgent who both wage war/conflict until both sides tire enough to resolve their differences politically. All our measures/assessments prior to that point are of questionable value. Providing a couple of starting papers to inform the debate that I hope will follow since this is an important topic for all conflict, not just small wars.

https://csis.org/publication/afghani...-ten-years-war

Afghanistan: The Failed Metrics of Ten Years of War

This analysis looks at the reporting available on the state of the war at the end of 2011, in terms of the data, trends, and maps available from the US Department of Defense (DoD), the US National Counterterrorism Center (NCTC), the NATO/ISAF command, and the UN. It attempts to explore the meaning of these data, the reasons for the sharp differences between them, and what they say about the fighting to date and its progress.

http://wih.sagepub.com/content/19/1/...ication_detail

The Problem of Metrics:
Assessing Progress and Effectiveness in the Vietnam War

A lot in this article, but I found this particular paragraph of particular interest.

South East Asia expert Bernard B. Fall similarly highlighted the complexities of measuring success in an unconventional environment. Fall had spent nearly 15 years in Indochina becoming one of its most respected scholars. His 1961 publication Street without Joy, a classic of the eight-year French-Indochina War, influenced a wide range of American officers preparing for deployment to Vietnam. Fall defined victory in revolutionary war as ‘the people and the army … emerg[ing] on the same side of the fight’.12 He realized, though, that assessing progress toward such victory required suitable indicators.
The French criteria for ‘control’ often had ‘no real meaning when it came to giving a factual picture of who owned what (or whom) throughout the Vietnamese countryside inside the French battleline, much less outside’.13 Fall argued that trends in levels of security and population control could be plotted objectively on a map, given accurate reporting of assassinations, insurgent raids, and Vietcong taxation. Measuring ‘administrative control’, however difficult, if done properly, provided military commanders with the most accurate assessment of their progress.14

O.K. I admit my bias I a big Bernard Fall fan, and yes it appears he is making an argument for metrics, but hey a big fan doesn't mean a die hard fan, we can have our disagreements. On a serious note, I recall reading comments Mr. Fall made in an article about how the U.S. village assessment process was highly inaccurate in Vietnam, and while that article didn't use the term administrative control his example did. He pointed to the schools we built (sound familiar, Kodak moments for our Civil Affairs Teams) being used as a metric of success, but the reality was the Viet Cong administered the schools and appointed the teachers. I had a senior Afghan security official tell me the same thing in 2010. I wasn't able to confirm it, but he said Mullahs were teaching in two of the schools we built radicalizing the children and parents didn't want to send their kids there, but were threatened if they didn't. I suspect our assessments and metrics never captured that uncomfortable truth.

By late 1967 the sheer weight of numbers had become crushing. At MACV commander conferences, staff officers deluged Westmoreland with 65 charts during the Measurement of Progress briefing alone. According to the briefing officer, no senior general expressed any interest in one indicator over another.52 All the while, the US mission in South Vietnam tempered any signs of failure with a blizzard of statistics suggesting progress that led only to confusion on the American home front. Both optimists and pessimists easily justified their positions from the mounds of conflicting data. Without
linkages to coherent strategic aims and sound threat assessments, it seemed any balance sheet or prognosis was as good as the next. Even the establishment of an operations research and systems analysis office in MACV (MACEVAL) and an increasing reliance on automation did little to facilitate analysis of the war’s trends. Given the obsession with statistics and measurement, the war’s complexity had simply overwhelmed MACV’s
capacity for understanding.53

and part of the last paragraph captures it all rather nicely

In the end, the United States Army failed in Vietnam in part because its metrics for success masked important operational and organizational deficiencies. Flawed measurements validated imperfect counter-insurgency methods and provided MACV with a false sense of progress and effectiveness. These measurements were symptomatic of a larger failure in thinking about the war’s deeper issues.

[Bob's World]

IMO, the biggest problem is that we insist on forcing every conflict, regardless of nature or form, into the Clausewitzian war box.

I think we are better served if we recognize two broad categories of conflict, each of which requires a completely different understanding and logic to understand, address or assess.

The first of these is your classic interstate conflict, or "state-based conflict". This is war and apply Clausewitz liberally (or literally) as you choose. While having a smart strategy is preferable, one can win through tactical victories regardless of their strategic shortfalls. All one must do is sieze and hold ground, defeat or destroy warmaking capacity, and coerce the Army, Government (and people if one intends to stay) of the opponent and one wins. This lends itself to neat metrics of tanks destoyed, miles advanced, rivers crossed, etc, etc, etc. In short, the sum of tactics will ultimately add up to strategy. Objective measures are relevant. Each commander can measure what he did in his sector during his tenure to assess his personal and unit success. These separate reports can then be added to tell a comprehensive story of if one is "winning" or "losing."

The other category of conflict is intrastate conflict, or "populace-based conflict." This may be war, but often is not war at all and is better thought of as an extreme form of civil emergency. In these types of conflicts success does not come from defeating the capacity of the challenger's ability to fight or controlling his terrain, or capturing his flag. Rather, success comes from understanding and repairing the perceptions of governance resident among some population (or populations) that have been driven (or neglected) to the point where they feel compelled to illegally, and often violently, challenge these existing, offending, systems of governance.

In these types of conflicts all tactical action should be designed and executed with primary strategic purpose of shaping these perceptions in positive directions, even when the immediate, tactical purpose my be to defeat some force, clear some village, or extend electricity to a region previously dark. One can certainly measure in objective ways all of those tactical actions - but all one learns from such measures is what one has done. Much more important to appreciate is HOW the actions were done, and what effect they had on the critical perceptions of the populations the illegal challengers are coming from - but also non-problematic domestic populations and populations abroad with only indirect or perhaps no stake at all in the conflict.

This is incredibly subjective. It cannot be measured during a single commander's tenure, nor within the confines of his battle space. One cannot tell which commander is good or bad, as what they did that can be measured is largely meaningless in of itself - rather it is how they did what they did and how those actions were perceive that matters. That, however only manifests over time and is impossible to attribute to any particular man, unit or action.

We must evolve in how we think about conflict in general before we can evolve in how we assess the effectiveness of our actions. I personally see little inclination to learn or evolve in that direction. Meanwhile we have our "Made in Iraq" generals (I see HR McMaster is nominated for his 3rd star) and our "Made in Afghanistan" generals. All possible because we could objectively measure the superiority of these commanders over their peers by what they did during their tenures and within their battle space. We celebrate tactical excellence, and turn a blind eye to strategic failure.

Strategic failure in the face of such tactical excellence cannot be our fault, so it is blamed on convenient foils, such as: "complexity" or "ideology" or host governments/security forces/populations (or our own at home) who "lack the will" to win. We widely recognize that the global strategic environment has fundamentlaly changed and that power is shifting to be more balanced between those within government and those outside of government. Yet we cling to our old thinking and practices.


Bottom line is that we suck stategically at populace-based conflicts primarily because it is we who lack the skill or will to recognize that they are fundamentally different than state-based conflicts. Afterall, doing so would require a much fuller accounting of the causal role of governments (the local one as well as those who intervene) in these conflicts - and besides, how would we know who to promote??

[slapout9]

You are dealing with Systems and not countries or in many cases not even governments as we think of them. Until we accept this and begin to learn about reality as it is as opposed to viewing it through the eyes of dead people...we are just doomed. Link to video on System of Systems.

http://www.youtube.com/watch?v=h2br2_twHfw

[Steve Blair]

Metrics can be useful, but the issue I keep seeing in many areas is that they often become an end in themselves. And that's when they become dangerous. In law enforcement you see crime reporting twisted to keep certain "hot" categories low, in academia you see actual student learning subordinated to metrics based on five year completion rates and situations where resources are focused on the top 5% of students and the bottom 5%, leaving the average student in the morass of being "average" and thus not requiring resources.

A metric is a tool. Nothing more. Elevating it to a goal is dangerous, especially when you're using the wrong metric or trying to quantify something that can't easily be quantified (or shouldn't be quantified).

[davidbfpo]

I have long argued - in the UK policing context - that 'Key Performance Indicators' were twisted into 'Measures' or metrics. A very different function.

Rudy Giuliani in his book on NYC writes well on the need for having metrics to assess performance, IIRC he called them performance measures.

[carl]

Good leaders will make good use of metrics, good, bad or indifferent. If the right ones aren't there, they will figure the situation out. The quality of the leaders is the key.

I think our fixation with the right metrics, the wrong metrics, being interpreted this way or that way is in keeping with what appears to me to be a cultural proclivity to eliminate the human element in war and especially in combat leadership. It seems that we are always looking for some machine or process that will make good leadership moot. If we could only find the right thing, then the hard and uncertain task of finding and promoting good leaders, there would be no need for.

Bill M: Gregory Daddis also wrote a book about that subject.

http://www.amazon.com/No-Sure-Victor.../dp/0199746877

I thought it a good book.

[TheCurmudgeon]

Metrics are just numbers. It is a question of what you believe the number represent. If you believe that the complexities of human motivations and feelings can be easily and simply reduced to a metric then you will find your answer in metrics, even if you have to create the data points and the conclusions to draw from them. If you are willing to assume that humans are more complicated than that, then you can still find indicators, but they will not yield absolute proof of success or failure, only the potential for movement in a specific direction.

Even then, you must have a complete understanding of what it is you are trying to achieve. I don't think most military types have any idea the political and cultural impetus behind our ultimate objectives. We happily delude ourselves into believing we understand how other people think. I wish I had a better answer. All I can do is advise against the blind belief that everything can be reduced to simple metrics.