Crowdsourcing on AQ and Analysis (new title)

[CWOT]

I think there were probably considerable Red Teams and all sorts of analysis done. I am certain that some segments of the Intel Community never lost focus.

In general though, I think the common political narratives thrown about in the media hampered how analysis was done; whether it be WMD in Iraq, Bin Laden in a Cave, or if you are in the U.S. the famous "DC Sniper might be in a White Box Truck". In these cases, narratives designed to give the populace comfort result in analysis being skewed to find that which has been declared rather than sticking to structured analytical approaches and hard data.

I was just thinking about it last night, the perceptions placed in my mind and others and how that can really hinder long run objectives.

[Bill Moore]

CWOT,

Excellent post and I highly recommend the video link to a much wider audience than the intelligence community. I think it reinforces my view (my pre-conceived pattern) of why psychology should be a mandatory part of professional military education. I can see the pre-conceived patterns (an old concept) in numerous discussions throughout SWJ. We have some dogmatic individuals who embrace the anti-COIN arguments, the pro-COIN argument, and the government is always at fault. It is clear when they look at a picture with no pattern they'll find one that fits their assumption.

I think there are two take aways from this. First is need to be much more self critical of ourselves (recognize we all suffer from the shortfall of being human), and encourage ideas that rub against the accepted grain. Second, in my view this justifies the pursuit for ever greater analytical/knowledge based technology that can analyze MBs of data to identify potential patterns without the bias that an analyst would bring. The human will always make the ultimate decision (at least for the forseeable future), but this type of technology will help the analyst avoid getting stuck in ruts. An analyst when sort through tons of data has no choice but to form a hypothesis and then search for data that confirms or supports it. He or she can't randomly sort through volumes and volumes of data and randomly create links.

[CWOT]

Thanks Bill, I appreciate the compliment.

I watched the TED video on Patternicity as a complete coincidence while riding on the train. The entire time he was talking about looking for things that are not there, I was thinking," I've done that before."

I agree that we should have a better process for accepting alternative views within our analysis. I have seen the red team crews before and the idea is good. But, I don't know, it just never seems to come up with something plausible and different. That's part of the reason I started the Bin Laden poll this week. I wanted to see how perspectives differed between groups of people and the results thus far are really striking. I have the poll running through the end of this week and will start cross posting the results here for those that are interested.

[slapout9]

Maybe we should teach to stop looking for patterns and teach what they teach Cops......follow the evidence. B.F. Skinner was a big believer in Systems Thinking and I have real issues of how his pigeon experiments were portrayed. Skinner developed the idea of "Hunting Behavior" from those experiments not patternicity(spelling).

[jmm99]

based on what works and has worked for me and others I've observed over the years (lawyers, cops and investigators) - follow the evidence, recognize systems and use fuzzy (not cast in concrete) patterns.

A quick graphic down and dirty. The Deep and Narrow, and the Broad and Shallow, are recipes for disasters. So, use the more difficult "Evidence Tree" which is a compromise (time is not infinite - "better" leads have to be chosen):

Evidence Trees.jpg

Have to do some of my Real World stuff.

Regards

Mike

PS: A couple of years ago, Slap and I had a conversation re: systems. I said I was dumb on systems - I was thinking about all the academic stuff (Bravo Sierra in Stanian terms) on systems. Slap said: No, you use systems every day. So, I looked about and concluded he was right. I use them, have used them before I was a lawyer; but just haven't called them that. E.g., you can start with the justice system and find many systems in and linking to that more generalized system. Thus, HT to Slap.

[Backwards Observer]

Here are some examples of a system commonly used among a number of peoples in Asia.

Dependent Origination:

Dependent origination or dependent arising (from Sanskrit: प्रतीत्यसमुत्पाद, pratītyasamutpāda; Pali: paticcasamuppāda; Tibetan: rten.cing.'brel.bar.'byung.ba; Chinese: 緣起) is a cardinal doctrine in Buddhism. It describes the causal relations between the psychophysical phenomena that sustain dukkha (dissatisfaction) in worldly experience. It is variously rendered into English as "dependent origination", "dependent arising", "conditioned genesis", "dependent co-arising", and "interdependent arising" and is an elaboration of the second Noble Truth.

Pratītyasamutpāda - Wikipedia

[Backwards Observer]

I agree that we should have a better process for accepting alternative views within our analysis.

Are there adaptive constraints involved with legacy systems that prioritise doctrinal conformity over independent situational and contextual observation? Is the system feedback loop tuned to legacy survivability over adaptive potential? If so, is legacy survivability viewed as the key to foundational strength?

To be honest, I have no idea what I'm talking about.

[Entropy]

I agree that we should have a better process for accepting alternative views within our analysis. I have seen the red team crews before and the idea is good. But, I don't know, it just never seems to come up with something plausible and different. That's part of the reason I started the Bin Laden poll this week. I wanted to see how perspectives differed between groups of people and the results thus far are really striking. I have the poll running through the end of this week and will start cross posting the results here for those that are interested.

As an intel guy, this kind of thing is near and dear to my heart.

Red-teaming can be useful, but it's best when composed of outsiders who are less likely to share organizational mindsets. Structured analytical techniques are another option (ACH being the most famous) that can help people break out of their mindset and consider alternatives. The intel community uses a variety of methods, but of course it can't do that for everything given time and resource constraints. Training and experience count for a lot IMO as does a decent level of introspection.

It's an enduring problem to be sure and there are no easy or certain solutions.

[CWOT]

Red-teaming can be useful, but it's best when composed of outsiders who are less likely to share organizational mindsets.

I worked for many years trying to bring in outside academics for different perspectives and the results were really mixed. Sometimes it was the government folks that just couldn't get in the mindset to collaborate. Other times, it was the reverse, the academic folks couldn't provide something useful for policy/actions. When it went well, it was great. Achieving the right mix was very challenging though.

[davidbfpo]

I struggled with a title; the author IMO is a 'disruptive thinker', a theme that appears on SWJ in particular and using his own, long title would be wrong.

CWOT's interesting short article on this difficult topic, which often appears here and more widely in intelligence studies. It starts with a German (Prussian IIRC) quote pre-WW1:

In war you will generally find that the enemy has at any time three courses of action open to him. Of those three, he will invariably choose the fourth.

—Helmuth Von Moltke

With that quip, Von Moltke may have launched a spirited debate within his intelligence staff. The modern version of the debate can be said to exist in
the cottage industry that has been built on the examination and explanation of intelligence failures, surprises, omissions, and shortcomings.

The contributions of notable scholars to the discussion span multiple analytic generations, and each expresses points with equal measures of regret, fervor, and hope. Their diagnoses and their prescriptions are sadly similar, however, suggesting that the lessons of the past are lost on each succeeding generation of analysts and managers or that the processes and culture of intelligence analysis are incapable of evolution. It is with the same regret, fervor, and hope that we offer our own observations on avoiding intelligence omissions and surprise. Our intent is to explore the ingrained bias against outliers, the potential utility of outliers, and strategies for deliberately
considering them.

Link:https://www.cia.gov/library/center-f...iers-13Jan.pdf

CWOT cites several crowd sourcing surveys, which have appeared here and draws on those.

[davidbfpo]

I have merged into this thread eight of CWOT's crowd sourcing threads. They were: Does Bin Laden Matter?, Crowdsourcing AQ's Strategy 2011-2012, Vote Now - 1 Year After Bin Laden: What has happened?, Results of the Post Bin Laden & AQ Strategy Poll, Results "How big is al Qaeda?", Vote: What will terrorism be post Bin Laden? and Financial Implications of Bin Laden's death.

I have also renamed the thread's title, it was How Bin Laden Narratives Hindered Analysis.