intelligence analysis, overcoming bias and learning

[RedTEamGuru]

All Source Analysts Training

After a recent DGS DART working group my office was having a discussion on all source analysts training. There seems to be some misconception at least in the DGS community as to what training should be required for those Analysts who are working in an All Source capacity. I think part of this stems from the fact that the AF does not have an All Source Analyst AFSC. Each AFSC traditionally has been INT focused with the exception on the 1N0s, who are Ops focused assigned generally to flying squadrons and the 14N Intelligence Officer.

The line of thinking at the working group tended to steer towards mission specific training and application training rather than core skills that could be used at any unit.

I wanted to get everyone’s thoughts on some of the basic skills training that any all source analysts should have in the toolkit.

The idea we have come up with is a three track approach for all source training, regional training and mission specific training.

The All Source track would be a pyramid with courses built on a foundation of basic skills.

Foundation Skills
• Analytical Methods
– Problem definition
– Process (scientific method)
– Statistics
• Research
– Strategies
– Source evaluation
• Communications
– Technical and Editorial Writing
– Briefing
101 Training
• AFSC/MOS Technical School
– 1NX
• By-fire hose training
– 1N1: Receives SIGINT, HUMINT, AGI, OSINT
– Intel: Receives Ops familiarization;
– AF: Rcvs Army-centric, Navy-centric, Joint
• Military Capabilities
• General Intelligence (CIA, NSA, NGA, etc)

Advanced Skills
• Scenario-based training/exercises
• Specialized analytical techniques
– HUMINT-specific support
– MASINT-specific support
• Region-centric integration
– Able to correlate language, geo-political, and cultural aspects to specific intelligence problems

[milnews.ca]

This from MERX, Canada's public tender posting page (bolding mine):
"....Defence Research & Development Canada (DRDC) have a requirement to retain the services of a contractor to provide support for DRDC Toronto's data collection and collection involving a series of behavioural science experiments with human subjects. The experiments described in this Statement of Work (SOW) are motivated by DRDC Toronto's Applied Research Program (ARP) project, entitled "Understanding and Augmenting Human Capabilities for Intelligence Production," which is under the project management of Dr. David R. Mandel, Group Leader of the Thinking, Risk, and Intelligence Group within DRDC Toronto's Adversarial Intent Section. The overarching objectives of the ARP project and the experiments described herein for which contractor support is sought are: (a) to identify systematic biases in human performance that may effect the quality of intelligence analysis; (b) to identify factors that may attenuate or amplify such biases or otherwise influence judgmental performance; and (c) to examine the viability of counter-measures aimed at reducing or eliminating them...."

More details in Statement of Work here (.pdf download).

[Ken White]

With some slight experience as one of them and with Analysts, I noted the problem, acknowledge that many can park their bias and do an effective job (subject and issue dependent) but am firmly convinced it is a problem. glad to see someone delving into it. Look forward to the result.

[IntelTrooper]

What about the bias of the people conducting the research? Is there another panel to investigate their possible bias?

While there's been some discussion of this in American intelligence circles, I haven't seen any definitive counter-measures. For example, there was no method for us to challenge the bizarre analysis of a battalion S-2 (so-called battalion senior intelligence analyst) who proclaimed that the main problem in our province was "criminal activity" and not the Taliban. We figured that since this conclusion wasn't based on any actual reporting, she must have some kind of cognitive bias against, say, reality.

[marct]

For example, there was no method for us to challenge the bizarre analysis of a battalion S-2 (so-called battalion senior intelligence analyst) who proclaimed that the main problem in our province was "criminal activity" and not the Taliban. We figured that since this conclusion wasn't based on any actual reporting, she must have some kind of cognitive bias against, say, reality.

No, no, no! It's not a bias against reality, it's a definition of the Taliban as criminals !

Now, it would have been a bias against reality if she had defined the Taliban as Smurfs... then again, maybe not !

[IntelTrooper]

No, no, no! It's not a bias against reality, it's a definition of the Taliban as criminals !

Curses, now I'm the one with the bias...

Now, it would have been a bias against reality if she had defined the Taliban as Smurfs... then again, maybe not !

I thought I recognized the guy in those briefings! Mullah Papa Smurf AKA Abu Smurfette...

[Ken White]

It also is a significant aid to keeping everyone honest...

The Canadians know that and are smart enough to have someone else also take a look and pull best ideas from both. We, on the other hand...

The efficiencies of consolidation and centralization are known, what is often ignored is the adverse impact of those moves on effectiveness. Two minds / approaches are always better than one; three even mo' betta...

There's another aspect aside from effectiveness. Put another way, you can always have your best gunners do the shooting or the best guy on point -- but no one else will learn much or become good at what they do and your bench will not be very deep.

All that has been known for centuries; "Quis custodiet ipsos custodes?" is not new and is practiced in most of the world; only the insane American predilections for one size fits all, one size does all, "whatever the boss wants" and "always show your good side" ignore that logic.

You're not likely to see any US countermeasures. We prefer those who totally support what the Boss wants, no matter how inane or even criminal -- as your example proves...

[IntelTrooper]

You're not likely to see any US countermeasures. We prefer those who totally support what the Boss wants, no matter how inane or even criminal -- as your example proves...

Yes we do... as I had the distinct impression that her "analysis" was thinly-disguised propaganda to make her battalion commander look good...

[Rex Brynen]

The Canadians know that and are smart enough to have someone else also take a look and pull best ideas from both. We, on the other hand...

We have some absolutely top-notch analysts—and, obviously, some less so. HOWEVER, one disadvantage of having a very small community is... that we have a very small community. Consequently, there is less opportunity for exchanging ideas/debating/challenging assumptions/etc. Virtually all of the folks that work on the Middle East, for example, could comfortably fit into the Sparks Street Tim Hortons for a double-double.

There are ways of offsetting that, of course--for example, by bringing folks into discussions who aren't in the IC, but rather in the aid, diplomatic, or even (heaven forbid) the academic and NGO communities. Some agencies and managers do it. Some don't. It is obviously harder in MI (especially on the deployed, pointy end) than it is in political assessment however, and the folks in the LE and security intelligence communities aren't always used to working that way either.

There's also interchange with allied communities, and especially the US, not just at the level of data but also in terms of conferences/meetings/discussions/etc.

From what I can see, the US has got much better at this post-9/11 than before.

On the original public tender that milnews posted, you'll find some of the very interesting work that the lead researcher (Dr. David Mandel's) is doing on these and other issues listed here.

[davidbfpo]

Last month I attended this international conference in London, hosted by the intelligence and security Studies of Brunel University and the University of Mississippi, a rather unconventional mix was on offer:

...cross-disciplinary discussion about the value of learning from other fields to improve both the understanding and the practice of intelligence analysis.

Professionals in other fields—including medicine, the social and behavioural sciences, history and historiography, anthropology and other disciplines engaged in ethnographic research, econometric forecasting, and legal reasoning—also face many similar challenges to those that exist in intelligence analysis, including:

Difficulties acquiring information from a wide variety of sources
Vetting and evaluating the information that is acquired
Deriving understanding and meaning from that information
Impact of deadlines, editing, and other production processes on accuracy of analysis and assessment
Problems in dissemination and distribution to consumers or customers
Managing relationship between producer and consumer (role, responsibility, independence & objectivity)
Developing professional infrastructure (recruit, select, train, & develop personnel; code of ethics)
Overcoming impact of changing technology and alternative information distribution systems
How do practitioners in various non-intelligence fields overcome these kinds of challenges? How are their challenges similar to or different from those that exist in the intelligence arena? What can be learned from the comparison?

Most of the presentations are on:http://sas.olemiss.edu/bciss/materials.php

[SWJ Blog]

Fixing Intelligence Analysis: From Specialists to Experts

[SWJ Blog]

Bias and Perception: How if Affects Our Judgment in Decision Making and Analysis

[davidbfpo]

There are just over seventy threads that have 'analysis' in the title; most a specific to a campaign or theme, whereas this one is concerned with intelligence analysis, overcoming bias and learning. Three threads have been merged here and the former phrase has become the title. All prompted by the next post.

[davidbfpo]

The CIA has released this book 'Psychology of Intelligence Analysis' by Richard Heuer, as a free PDF, spotted via Twitter:

Did you know that the CIA released a free PDF on how to analyze OSINT Well, it wasn't JUST for OSINT. It discusses bias and how to keep an open mind when performing analyses. Worth a download and read.

Amidst all the explanations this is the best:

Dick Heuer's writings make three fundamental points about the cognitive challenges intelligence analysts face:

• The mind is poorly "wired" to deal effectively with both inherent uncertainty (the natural fog surrounding complex, indeterminate intelligence issues) and induced uncertainty (the man-made fog fabricated by denial and deception operations).
• Even increased awareness of cognitive and other "unmotivated" biases, such as the tendency to see information confirming an already-held judgment more vividly than one sees "disconfirming" information, does little by itself to help analysts deal effectively with uncertainty.
• Tools and techniques that gear the analyst's mind to apply higher levels of critical thinking can substantially improve analysis on complex issues on which information is incomplete, ambiguous, and often deliberately distorted. Key examples of such intellectual devices include techniques for structuring information, challenging assumptions, and exploring alternative interpretations.

Link:https://www.cia.gov/library/center-f...gence-analysis

[AdamG]

In the spring of 1997, a 14-year-old’s school science fair project made a convincing argument to ban a dangerous chemical compound: dihydrogen monoxide, known as DHMO. Nathan Zohner, a junior high school student in Idaho, gave 50 of his fellow students a report called "Dihydrogen Monoxide: The Unrecognized Killer,” which accurately laid out the dangers of DHMO, convincing the majority of students to call for its ban. The experiment caused enough of a splash that it was picked up by The Washington Post

http://mentalfloss.com/article/50190...rogen-monoxide

[SWJ Blog]

Dangers of Ethnicity in Analysis

Entry Excerpt:



--------
Read the full post and make any comments at the SWJ Blog.
This forum is a feed only and is closed to user comments.

[SWJ Blog]

Community Ecosystem Analysis

Entry Excerpt:



--------
Read the full post and make any comments at the SWJ Blog.
This forum is a feed only and is closed to user comments.

[SWJ Blog]

Qualitative Analysis Concept in Support of Force 2025 and Beyond (F2025B) Maneuvers

Entry Excerpt:



--------
Read the full post and make any comments at the SWJ Blog.
This forum is a feed only and is closed to user comments.

[SWJ Blog]

Using Target Audience Analysis to Aid Strategic Level Decisionmaking

Entry Excerpt:



--------
Read the full post and make any comments at the SWJ Blog.
This forum is a feed only and is closed to user comments.

[SWJ Blog]

Strategic Studies Institute: Research and Analysis

Entry Excerpt:



--------
Read the full post and make any comments at the SWJ Blog.
This forum is a feed only and is closed to user comments.