Body Counts and Metrics

[Bob's World]

This is true, however, of almost all data in intelligence assessment--the data rarely has a single unambiguous interpretation. Assessing the validity and reliability of indicators, contextualizing them, and making collective sense of them (and their broader significance) is what analysts ought to be doing for a living.

As a general rule, I would rather have more information than less, even if the information might be inflated or indicative of multiple possible trends.

and are typically oblivious to how collecting and counting it might be affecting the very operations they are working to support.

Or just as often, will fight tooth and nail to keep a data source on line when the ops guys want to go turn it off for the negative effects its day to day operational presence has on the mission as a whole.

BL, I would discount the intel community's perspective on this by the "factor of 2 described by Fuchs," I would proably do the same for commanders at any level higher than that which the casualites were actually produced at, say company level.

[Ken White]

However, knowing that you killed 'X' AND got his weapon AND will or even may be able to exploit that is not pointless.

The key is to know your enemy and really understand who you're fighting. Bill Moore makes the valid point that our overly competitive system encourages misuse of such data (and I repeat that public release exacerbates that factor) but that same system with it's forced short rotations (and repetitive deployments deliberately tasked to different AOs) does not allow Commanders or units to get to know their enemy. A body count is then sort of pointless.

However, as Wilf said when he started this thread:

"Historically best practice body counts were based on recorded kills, verified by physical control and recovery of the body AND Weapons - usually for some form of exploitation."(emphasis added /kw)

However, Wilf cited that usage in some Commonwealth campaigns where a three year tour (or longer, particularly in Rhodesia) was the norm. We don't do that (unfortunately) so many do not see the value for us. It was not and is not pointless.

If you know your AO and your enemy. If you do not, then it probably is pointless...

As for Bob's World's comment on the Intel community and their shortfalls, I've seen that. He's right. I've also seen commanders that would not tolerate that attitude as they wanted to do what was right as opposed to doing what they thought their Boss might want...

He also says:

I would discount the intel community's perspective on this by the "factor of 2 described by Fuchs," I would proably do the same for commanders at any level higher than that which the casualites were actually produced at, say company level.

Hmm. That sounds like an indictment of a lot of Field Grade and Flag Officers...

[Bob's World]

If you brief senior leaders on certain stats every day, those stats take on an importance for the sake of the stat itself.

So effects are more important. Also way more difficult. I've worked with guys who have developed and sold effects measureing and reporting systems that would crush the biggest computers IBM every built, I know they certainly crushed the staffs they were foisted upon; and in my opinion the results were virtually worthless. This actually drove me to the quote below, as I told my fellow officer that he was 'A master of complexity - able to devine insanely complex solutions to complex problems.' I told him "don't complify - simplicate!"

Body counts are easy. We killed 5 of them, and suffered one WIA in the process. Easy. Sounds like we're winning. I recall as kid in kindergarden watching the news about Vietnam, and they would post the daily box score. US 30, NVA 240. etc. It was like following sports. Except, of course, that it isn't sports, and the score in of itself doesn't mean anything. More "how do you feel" about the score over time...measure that, if you can.

[Ken White]

...I told him "don't complify - simplicate!"

Body counts are easy. We killed 5 of them, and suffered one WIA in the process. Easy. Sounds like we're winning.

You want to simplicate to the point of irrelevance...

You left out "...physical control and recovery of the body AND Weapons - usually for some form of exploitation." I would totally agree that what you posted was foolish -- I do not agree that if you add the important bit about their weapons (give that some thought...) and the potential for exploitation.

I recall as kid in kindergarden watching the news about Vietnam, and they would post the daily box score. US 30, NVA 240. etc. It was like following sports. Except, of course, that it isn't sports, and the score in of itself doesn't mean anything.

While you were in kindergarten I was adding to that box score in a unit that would not allow a KIA to be counted unless there was a weapon to go with it and that diligently tried to exploit each case and successfully did so on a number of occasions. So I know there is value if done properly as opposed to basing my opinion on dimly recalled TV Follies (That's what the MACV Press Briefings were called, with good reason).

The fact that a bunch of staff Colonels prostituted a process originated by the SecDef over the objections of many in-country makes your opening comment above even more poignant (in the painful and pertinent sense of the word):

If you brief senior leaders on certain stats every day, those stats take on an importance for the sake of the stat itself.

As I said, those are your contemporaries and their Bosses you're impugning, not mine.

That said, I do agree that can occur but that is misuse of the information and it amounts to doing what I specifically said should not be done -- and which you apparently ignored as you often do while searching for a riposte. I'll just repeat some of that for you:

""I would add that those figures should not be publicized in any way or released to the media because they will either misunderstand or misuse them -- more likely both -- and that will skew the military application (as it did in Viet Nam after mid 1966).(ADDED:A briefing is publicizing)
. . .
P.P.S

John and Bob's World are correct also in that it can be a useful measure of performance but not of effectiveness/outcome.[/B]""

More "how do you feel" about the score over time...measure that, if you can.

Apparently you changed your mind about the use as a measure of performance:

I see this as a measure of performance though, not effectiveness. Many factors go into what it takes to break the will of an opponent, so merely counting bodies only tells you that you are killing people.

Totally agree now as I did when you first wrote it.

I'm not a Sports fan so don't do scores and didn't get to watch Viet Nam on TV though I'm glad you did, you must've been a precocious little Kindergartener. However, I can measure an inability to determine the difference between appropriate and inappropriate uses of information and the appreciation of that difference...

[Bob's World]

This problem is a big part of my current job. Not so much to devise effective metrics, but to try to ensure that we are emphasizing the right bits of information, drawing the right conclusions, (and questioning the rightness of those), etc.

One unit will go out on a CRP, engage in a daylong TIC, reduce several IEDs, and get confirmed BDA on half a dozen Insurgents (with their weapons) from attack helo's brought in to support them.

Another will swoop into a compound in the middle of the night a discover and destroy a cache of dope or amonium nitrate; or snatch a person or two of interest and swoop back out.

Another will conduct a Medical event at a district center, treat hundreds of locals and engage in meaningful conversations with important local leaders and return to base without firing a shot.

Who was most productive today? Who was most effective today? What do we do more of? What do we do less of? How do I convince conventional commanders to allocate critcal enablers to one form of engagement over another?

I have a counterpart who reads the "scorecard" to the commander every morning. No one ever asks what it means. I try to identify and highlight important nuances that come from all of these various types of engagement; and sometimes I get a "WTF?" look in return. Sometimes you pimp the actions that are easily understood in order to get the enablers to go out and do what might not be quite so obvious.

How does that go, "you sell the sizzle, not the steak?" That's fine. But never forget you will quickly starve eating nothing but sizzle. But boy, to people love sizzle.

[Ken White]

Who was most productive today? Who was most effective today? What do we do more of? What do we do less of? How do I convince conventional commanders to allocate critcal enablers to one form of engagement over another?

I'm pretty sure that varies a great deal depending on whether the location and overall relevance of those IEDS was critical, nice or unimportant and if the weapons of those' insurgents' were obtained and how well their deaths can be exploited; whether the cache was significant and/or the snatch was worthwhile (and the snatchees can be exploited); and whether the meeting and MedCap were in a critical or humdrum area.

IOW, the factors of METT-TC apply as they pretty much do in all wars. Importance and productivity vary from day to day and each type operation can range from unnecessary to deadly to so-so to good work to super important. That's war for you...

Frustrating, isn't it?

I have a counterpart who reads the "scorecard" to the commander every morning. No one ever asks what it means.

That could mean they're all self explanatory or self evident -- or that those items are irrelevant. Either way, sounds awfully bureaucratic to me. What do you report on those days when there is nothing to report?

How does that go, "you sell the sizzle, not the steak?" That's fine. But never forget you will quickly starve eating nothing but sizzle. But boy, to people love sizzle.

I'm not sure if that's a metaphor that insinuates that overrated DA is more glamorous than is the FID/SFA business which is really far more important but it seems like it might be. If it is, I agree.

Illegitimi non carborundum. Keep on pushing.

[Bill Moore]

You win when you win, not when you have the highest score (or right metrics)

We're attempting to incorporate industrial age methodologies (business management) by measuring performance and effects using some sort of pseudo-scientific methodology, which as Bob correctly points out simply crushes the staff and prevents them from doing effective work. This whole MOP/MOE nonsense that one of the worst SECDEF's in history forced down the military's throat just won't die.

What stats do we need to measure to see if we're in black vice being in the red when we're at war? Can we really measure a man's will to resist? I think they enemy is at a 6 today, yesterday they were at a 7. Tomorrow they may be at an 8 so we'll need to do something different. We also seem to forget that most effects are accumulate over time and are not readily measurable. We need more analysis, more ground truth about what makes people tick, more leadership instead of management, and less focus on diverting staffs to chase useless metrics.

[Entropy]

Who was most productive today? Who was most effective today? What do we do more of? What do we do less of? How do I convince conventional commanders to allocate critcal enablers to one form of engagement over another?

Well, that is the crux, and it seems to me one can't determine anything from simply counting actions or metrics. The effect of those actions is what matters and too often we don't really understand those effects until much later, if at all. One also has to consider synergies and discordance between discrete events or actions.

That is something intel should be much more focused on instead of current intel, which receives far too many resources. There's also a big collection problem in that we often don't have access to sources that can answer the important questions which usually have a "why" in them.

Personally, I don't see any easy solutions. We are too risk-averse to do proper humint, we are not properly training our intel people to the tasks required to support this war, and Commanders are too often making the problem worse by settling for substandard intel.

[Seahorse]

Thanks to Wilf for initiating this thread and to the many contributions which are richly exploring the issues related to metrics. I wish to contribute to this exchange with my own observations and experiences.

My experience began with the development of EBO concepts and theory and the ability to apply measures to assess the desired effects with JFCOM in 2002. I have since had the dubious benefit of actually applying the theory of MOP/MOE development, collection, assessment, analysis and reporting at the strategic, operational and tactical levels of the ISAF mission with separate deployments to both Kabul and Kandahar as an operational analyst.

Notwithstanding the vigorous debates surrounding EBO itself, the continued and insatiable demand for measures has frustrated me considerably. I echo many of the identifed problems several posts have described. Unlike many of my contemporaries (scientists) I feel there are serious limitations to the collection, analysis and reporting of measures. It is my position that reliance on the measurable has led to a situtaion I describe as the real EBO effects blurred operations. This blurring is based on several factors such as:

i) Reliance on MOPs versus MOEs - MOPs are necessary to evaluate if we are doing things right. Numbers killed in an AO can be used to evaluate insurgent activity. However numbers of KIA, development projects or numbers of Shura meetings are meaningless when divorced from the requirement for patrols, development or local interaction. MOEs are intended to evaluate if we are doing the right things - such as the earlier post regarding the value and priority associated with combat patrols, intelligence driven operations, or medical and humanitarian assistance. True MOEs could be used to support transitioning ink spot strategies from clear, hold and build dominant operations in an AO for example. Unlike MOPs (which are straightforward and easier and faster to measure), the selection and development of MOEs is not easy and their evaluation lags the operational tempo.

ii) measuring the measurable versus the important - too often there is far too much reliance on what we can measure and then associating that measure with the progress of an operation. Number of schools built may make one feel good, but what has it to do with education requirements/need? Are the schools located where they are needed, are they built to specified standards, are they protected from Taliban vandalism, are they staffed with qualified teachers, do they have secured funding for operations, do they have teaching materials, are the local children attending the school etc? I have seen Unicef tents that were more effective schools than were dedicated buildings. Measuring 'things' also results in what I call an effects explosion. This refers to the ever increasing numbers of metrics that are foisted on deployed personnel to collect and report irrelevent data (ISAF progressed from 25 to 100 to over 700 monthly measures over a one year period for example). Couple this with the problems in collection and reporting this information across the provinces results in inconsistencies, ommissions, real and unintentional errors, inflationary or creative reporting, etc.

iii) reporting measures and producing slides or excel tables instead of analysing their significance and providing advice or recommendations. Often analysts divorced from the actual operations are given the responsibility to assess and report on the data. The results are mostly divorced from the operations and are virtually useless in terms of influencing the operations themselves, they are predominantly used to report to higher headquarters and later repackaged and distilled to form IO materials that are used to show progress of the mission to national audiences and media.

iv) metrics obsession - certain metrics (KIAs, TICs and other SIGACTs for example) become THE metric for reporting and this detracts from the purpose of collecting the metric in the first place which should be determining effectiveness and supporting HQ decision analysis requirements. Too often I witnessed senior staff that were more concerned with the numbers than they were with the associated trend and analysis. We need to emphasize the relationship between metrics and operations. The counter narcotics reliance on # of acres of poppy eradicated is a useless and overly relied on metric when one considers that despite the ever increasing (presumably good) statistic, poppy cultivation has exploded in terms of its proliferation both within problem provinces and to a large number of previopusly poppy free provinces. Where is the analysis of the eradication program in relation to the growth of the problem. Secondly, is there a secondary relationship between eradication and insecurity which may be counter to the dominant mission in Afghanistan?

v) distillation of metrics - Too often I witness the development of metrics that are so distilled (down to the tactical level) that they are relatively meaningless to the original purpose. This distillation is the result of combinations of the above problems. Any one measure can be rationalized to be important to a particular purpose in and of itself, however in the roll up of the comprehensive assessment the relative weighting of the metric becomes too prominent and can dominate the reported results. I believe certain metrics are key and quite dominant (numbers of IEDs could be one example) on their own, however the overall assessments need to account for more than the constituent elements. I assert that qualitative measures are more important to MOEs, which should form the basis of higher level assessments.
Quantitative assessments which dominate metric measures can inform qualititative assessments however the subjectivity of these qualitative measures detracts from their adoption and use. I would assert that an informed and effective force understands qualitatively their AO. This understanding fails to be conveyed within our dominant quantitative metric measurement systems.

These are a few of my observations. I continue to work in this field and am attempting to develop proposals for how to more effectively implement measures in support of decision making and influenceing operations.

V/R

David

[Seabee]

This may be oversimplified, so I apologize in advance...

Is a "body count" not effective in some areas, and bad in others?

If I was fighting a German panzer division, the more kills the better.

if I was fighting a population that hold grudges over generations and generations... I would be weary of killing more than I would need to. If i was to kill an uncle, and have 5 Nephews stand up to avenge him, I would need to start killing at a rate that exceeds the possible pop ups...

Anyone remember "space invaders"? You shoot the little widgets as fast as you can... but at some stage there are just too many, they are too fast... and getting more and more....

Like I said, over simplified, but I am a simple mind... :-)

[Bob's World]

A basic rule of thumb is that when you are fighting your own populace (COIN); or assisting another country fight their own populace (FID); you really need to keep the killing to a minimum and focus on addressing the issues of poor governance that set that segment of the populace onto a separate course.

If you are fighting another state, even if the state used to be part of your state (as in the American Civil war), then it is game on and you are competing for state survival. In states with empowered populaces you have to be pretty hard on them to ensure that the entire populace, not just the military, understands that it has been defeated.

In COIN you are not fighting for state survivial, it is often really just an "illegal violent election" in a country that denies effective, legal processes to at least the insurgent segment, if not the entire populace. In COIN you are typically fighting to preserve a particular team or type of governance; but the state continues on as a sovereign under new management if the insurgent prevails. You will all still need to get along, win or lose, to be an effective state.

This is why I think it is more helpful to think of insurgency as a "civil emergency" rather than as "war," and military intervention as "Military Support to Civil Authorities" rather than as "warfare" to help set the proper mindset for the military. Call it a war and they will fight it like a war.

[Sylvan]

This may be oversimplified, so I apologize in advance...

Is a "body count" not effective in some areas, and bad in others?

If I was fighting a German panzer division, the more kills the better.

if I was fighting a population that hold grudges over generations and generations... I would be weary of killing more than I would need to. If i was to kill an uncle, and have 5 Nephews stand up to avenge him, I would need to start killing at a rate that exceeds the possible pop ups...

Anyone remember "space invaders"? You shoot the little widgets as fast as you can... but at some stage there are just too many, they are too fast... and getting more and more....

Like I said, over simplified, but I am a simple mind... :-)

Metrics are hard to come by in COIN.
A better metric in fighting a german panzer division is how many guidons you captured as that indicates units taken out of action. An insurgancy doesn't have units in the traditional sense. You can have fuzzy metrics such as atmospherics, but hyper Type-As don't do fuzzy. Dead bad guys is about the only true quantifiable that you can use that is of any (albiet very limited) value. Its hard to get away from. Even the messiah (peace be upon him) has found his administration crowing about how he has killed more bad guys then Bush has.
You really want to go down that road?
BTW, killing bad guys is an integral, important part of COIN, but basing success or failure on the numbers is stupid.