Chaotic Dynamics: A Novel Approach to Intelligence Analysis in Asymmetric Warfare

[JeffC]

This is an essay that I originally wrote for the AFCEA Intelligence Essay competition but didn't get it finished in time to meet the 10/31/07 deadline. I'm posting it here for feedback and review. It's still a work in progress.

-------------

Chaotic Dynamics: A Novel Approach to Intelligence Analysis in Asymmetric Warfare

By Jeffrey Carr

"Consider yourself lightly. Consider the world deeply." - Miyamoto Musashi

In 1999, Psychology researchers at Harvard University organized what has now become a very famous study . The participants were asked to watch a video of a basketball game, and count the number of times that a “white shirt” team member passed the basketball. At the end of the video, participants were asked to record their count…, and whether or not they saw the person in the black gorilla suit in the middle of the action. About 50% completely missed the “guerilla”. Researchers call this phenomenon “inattentional blindness”, i.e., the failure to see something that’s in plain sight.

May 11, 1998 (New Delhi):“Today, at 1545 hours, India conducted three underground nuclear tests in the Pokhran range.”

This announcement by the Prime Minister of India sent reverberations throughout the entire U.S. Intelligence Community, but particularly at CIA where analysts who had received satellite evidence 6 hours in advance of the test did nothing with it because no one believed that such an event was likely to occur. In the words of one official, “They would have been more vigilant if the policy community believed this was likely.”

Within 24 hours, DCI George Tenet named retired Admiral David Jeremiah to head an investigation into the IC’s embarrassing failure. According to Tenet’s letter to the Senate Select Committee on Intelligence:
“the site has been under periodic surveillance by photoreconnaissance and electronic eavesdropping satellites, which recorded increasing activity. But the images and activities they recorded in recent days were not interpreted clearly or quickly by the CIA.”

The evidence was there, but it wasn’t seen by the people who are responsible for applying objective clarity in the analysis of evidence. This is classic mirror-imaging, and Adm. Jeremiah addressed it in his recommendations :

Analytic Assumptions and Tradecraft

1. Add rigor to analysts’ thinking when major events unfold. Two mechanisms would help:

a. Bring in outside substantive experts in a more systematic fashion.
b. Bring in experts (who)… would serve, together with substantive specialists, as “Red Teams” on major analytic problems and work with analysts to study assumptions, mirror-imaging, and complex analytic processes.

In 2001, Dr. Rob Johnston accepted a fellowship with the CIA's Center for the Study of Intelligence. His mandate was to “identify and describe conditions and variables that negatively affect intelligence analysis … using an applied anthropological methodology that would include interviews, direct and participant observation, and focus groups.” Dr. Johnston’s research was published in 2005 under the title “Analytic Culture in the U.S. Intelligence Community: An Ethnographic Study”. In Chapter Six, he writes about the problem of ethnocentric bias (a more precise term than “mirror-imaging”):

"Ethnocentrism is a phenomenon that operates on a conscious level, but it is difficult to recognize in oneself and equally difficult to counteract. In part, this is because, in cases of ethnocentric thinking, an individual does not recognize that important information is missing or, more important, that his worldview and problem-solving heuristics interfere with the process of recognizing information that conflicts or refutes his assumptions."

In other words, the mandate to “try to think like them” is doomed to fail because analysts who make the effort still have to address the biological facts about how they perceive information to begin with.

The Biology of Perception
The question about how information travels from “out there” to “in here” has been a subject of inquiry by philosophers and scientists for several thousand years. The Platonic view is that light carries shadows of the external world into the brain, and the brain then matches those shadows with its own ideal forms. For Plato, perception was passive.

Not surprisingly, Aristotle disagreed. The brain, argued Aristotle, received information through the body’s interaction with its environment. As the body moved, touched, tasted, and listened to its surroundings, it sent information back to the brain where it was shaped by expectation and experience. One of the obvious problems with this model is that, practically speaking, we see what we expect to see; the cold slap of reality notwithstanding.

A leading researcher in this area is Dr. Walter J. Freeman, Professor Emeritus of Neurobiology, UC Berkeley. Freeman’s research demonstrates how our brains interact with the world through a cycle of Intention-Action-Perception-Assimilation, with Socialization as the platform. Imagine an early version of the Etch-O-Sketch. By turning a couple of knobs, you could draw shapes and figures. When you wanted to erase your work, you just flipped it upside down, shook it a couple of times, and your old work disappeared, leaving a blank slate in its place.

In Dr. Freeman’s model, chaos attractors replace the aluminum dust inside the Etch-O-Sketch. The two knobs that a user draws with become Action and Perception. The urge to flip the Etch-O-Sketch over, shake it up, and begin anew occurs in the brain when a person wants to be accepted into a family or group environment (gangs, the military, fraternities, terrorist organizations); a desire that is prompted by the pressures of socialization. Dr. Freeman explains the science behind the theory:

“Multiple chaos attractors, each representing one component of the overall landscape that is our environment; and each a receptor of an external stimulus, assemble themselves into a replica of what our action/perception chain tells us is out there. We learn through action and perception, but knowledge is gained through socialization. If the artificial construct that a person’s brain has organized does not equate with a different person’s construct, how can different people share experiences? The answer lies in socialization as a feedback mechanism…. (T)he brain induces chaos that dissolves its intentional structure and enables the emergence of new habits, beliefs and values through cooperative actions with others.

“These techniques do not change individuals through forgetting or loss of memory. They restructure the intentionality of individuals. They induce deep, often dramatic, rarely catastrophic, changes in values and points of view that typically are life-long. They provide additional evidence, if any is needed, that brains are dynamical systems and not logical devices.“

Applying Chaotic Dynamics Theory to Intelligence Analysis
Yorim Wind, Colin Crook, and Robert Gunther of The Wharton School have written a business book based on Dr. Freeman’s work in chaotic dynamics and the brain entitled “The Power of Impossible Thinking (Wharton School Publishing, 2004). In it they discuss how ground-breaking ideas have launched innovations long thought to be impossible such as the 4 minute mile and IBM’s adoption of Open Source, among many others. The seemingly impossible problems plaguing the Intelligence Community, such as Mirror-imaging, may be solved with the same application of insight and awareness.

The first step is realizing that what we think we know about any situation is several steps removed from that situation’s authentic components, based on how the brain processes information coming from outside of itself. The solution is to consciously break down and rebuild the brain’s internal model of the environment in question (i.e., the cultural and societal influences of religious terrorists in the Middle East or Central Asia). Based on Dr. Freeman’s research, we know that the brain can be influenced to break down its “intentional structure” through the pressures of socialization; i.e., the desire to belong, to be accepted, by a particular social group. In the Long War that the U.S. is presently engaged in, religious fanaticism is the primary building block upon which all of the enemy’s actions derive from. Analysts who seek to discern the intentions of that enemy should strive to adopt a parallel mindset.

For example, a Red Team of intelligence analysts are deployed to a facility where they are immersed in an environment as realistically constructed as possible to simulate the daily experience of a member of Al-Qae’da in Iraq, or the Taliban in Afghanistan. This may include religious education in the Koran, daily prayers to Allah, all written and spoken communication done only in Arabic, and other like-minded activities. Any scenario that displaces the previous mindset of the analyst and replaces it with one more closely aligned with the terrorist would accomplish the goal.

Once the internal structures of the brain have been dissolved and rebuilt, these analysts will be able to discern and predict enemy movements and plans from a much deeper and richer foundation of knowledge than ever before. While the axiom that “we don’t know what we don’t know” will still apply, these analysts will be much better prepared to spot the guerilla in the center of the basketball court.

[Rex Brynen]

That is indeed interesting stuff, Jeff. It would be interesting how it would work out in a prolonged Red Team exercise--and whether the "immersion" could really display ingrained assumptions and world views enough to make a difference.

Personally, I'm not sure you need to go that far to develop the sort of empathy and understanding required to get inside the opponent's head, nor am I convinced that it can it can adequately accommodate the variety of non-cultural and ideological factors that shape insurgent decision-making. This can often be highly idiosyncratic, shaped by leadership styles, risk tolerance, adaptability, immediate (and highly dynamic) political context, group dynamics, etc. In other words, if I think of all the Islamists or members of armed non-state groups that I've known, they've varied dramatically in terms of attitudes and behaviours (just like, say, Republicans, US military officers, or environmentalists).

The broader issue you're addressing is absolutely vital. I've always thought that the best analysts combine subject knowledge, empathy, common sense, and a degree of imagination that is also firmly based in available evidence. It is not that hard to recognize, in my experience--but I haven't the faintest idea how you would test for it.

[Brian Hanley]

I would say this sounds like some guys reaching into the stratosphere to get support for implementing expensive ideas that are all self-circumscribed. And I'm the sort who likes math and has studied chaos theory. In other words, you don't fix an inability to see outside the box by staying inside the box.

There is no substitute for field experience and wide reading. I'd say forget about trying to duplicate Al Qaeda. Send a group of guys out into the field alone, without any support but $10-$20K to spend 1 year without any help at all. Have them set up a business and survive in Liberia, Sierra Leone, Uganda, Uzbekistan, Azerbaijan, etcetera. Have them required to spend at least 50% of their time in country, and don't allow them to employ anyone that isn't a local. (Yes, some may not make it back. That's the price for learning what you need to.) Now, the really good info will start coming in the 2nd and 3rd years, but even 6 months will do a lot.

There are so many places this can happen. It's cheap as dirt by comparison. In the process, some of them will turn out to be pretty good at getting inside Al Qaeda, and others like them. Hell, I'm 4 handshakes from bin Laden myself. That sort of thing will pay off for years.

Look, there's just no way to do any kind of classroom exercise that teaches a man what he'll learn in 1 week in a bad neighborhood where he has no backup and just has to make it happen.

Here's another simple method of getting in touch with the world. There are women all over the world looking for dates with Americans. (And men too.) Just conversing with them will teach you a fair amount about their world and how they think if you talk about politics. They can also give you a guaranteed "in" to families. It's a basic fact that the women's network all over the world knows pretty much everything. Get the women to talk to you and you'll find out a hell of a lot. (That's true in the USA too...)

[JeffC]

Thanks Rex and Brian for your feedback. I'm sure that there are many alternatives to the one that I recommended, and they should probably all be tried within certain cost and security parameters. In my opinion, the most important takeaway is the way that the brain processes external data, and what needs to be done to break apart and rebuild our internal models.

The other issue, which I didn't specifically call out in the essay, but perhaps I should, is that of translation. Accurate translation is frequently dependent on cultural context. That's why native speakers are so valuable versus American graduates from the Defense Language Institute. If you can't recruit a native speaker, then perhaps some "cultural context" can be instilled into Americans learning Arabic, Chinese, etc. through various innovative ways, such as we've discussed in this thread.

[Beelzebubalicious]

Jeff, I agree with what you're saying in principle, but to do so, you'd essentially have to "erase" someone's mental models, memories, etc. and then replace them with these artificial ones. Makes me think of the Jason Bourne book and movie series...

I think what Brian writes echoes what's written in the "Power of Impossible Thinking". That is (paraphrasing), you need to continuously challenge your mental models, test them, reconstruct them and remain flexible and open to intuition (vs. the voice that says it should be this or that). The best way to do this is to immerse yourself in other cultures, preferrably other countries.

I've lived in 3 countries (more than 2 years each - Japan, Eritrea and now Ukraine) and while my degree of immersion varied, in the first two, I was 90% immersed and while I learned a whole lot about the culture(s) I was living in, I learned as much about myself and my own cultural and personal models. I still make a hell of a lot of mistakes, but one thing I have learned is never, never assume that I know what's going on or that my assumptions are right or real. I always try and triangulate my information sources if I can.

The problem with analysts is that if they're analyzing data in an office in Langely, then they're at a double disadvantage. They have their own American models and then they have their CIA training and culture to overcome. Kind of hopeless if you ask me. Hence, the efforts like these, I suppose.

By the way, Brian, the model you propose is pretty damn similar to the Peace Corps (I am a returned volunteer). I have a lot of interesting examples from my time in Eritrea. One construct or model I could never overcome was "time" and my sense of it. I tried, but could never fully embrace the Eritrean concept of time. Another simplistic, but interesting example, was how many Peace Corps volunteers used to "dress down" in an attempt to be closer to their colleagues, however, this was their own model and assumption. Eritreans are a very proud people and even the poorest would take meticulous care of their clothes and alwasy try to be as presentable as possible. Dressing down was indulgent and even offensive to some (even your poorest American is much wealthier than most Eritreans).

Well, I'm now too long-winded, but it's a subject I find very interesting. By the way, I found the link to Dr. Johnston's work. His whole study is published here.

[Rex Brynen]

The problem with analysts is that if they're analyzing data in an office in Langely, then they're at a double disadvantage. They have their own American models and then they have their CIA training and culture to overcome. Kind of hopeless if you ask me. Hence, the efforts like these, I suppose.

I do have to say that there is some real quality product coming out of the IC--much, much better, I think, than many people presume.

That having been said, there is the problem of the analyst being cocooned in an organizational box. It is more of a potential problem in the US IC, where its possible to spend 95%+ of your time at (say) interacting with colleagues from your own agency, and very little reaching across DC to share (differing) perspectives with people in other branches of government, let alone the NGO, academic, (etc) communities. In the past couple of years ODNI and the NIC have been trying to address this, with some success.

In smaller ICs, there simply aren't enough folks in the analytical community to make this sort of organizational cocooning possible (although there are obviously other serious problems with a smaller analytical community).

A related complication for the US IC is that US diplomats are either restricted, self-restricted in their functioning in some countries of high interest, or not even present at all. Given the very considerable value of quality diplomatic reporting, this can be quite the disadvantage.

[Jedburgh]

The title of the piece implies that it will discuss "intelligence analysis and asymetric warfare" - yet the operational example (India's nuclear test) is of analysis failing in a very conventional Cold-War type collection and analytic task.

.....The seemingly impossible problems plaguing the Intelligence Community, such as Mirror-imaging, may be solved with the same application of insight and awareness....

Mirror imaging is a common analytic failing of new and poorly trained analysts. However, to categorize it as a "seemingly impossible problem" is a gross exaggeration. Overall, the piece demonstrates a fundamental lack of knowledge about the community whose problems it presumes to address.

Solution to such problems are not "seemingly impossible"; they are simple. Hire good people, train and mentor them well, ensure that they have regular opportunities to travel within their areas of focus (for those who aren't already regionally-based) and have in place a functional professional-development program that ensures they keep up with emerging technology and methodology within the field.

[wm]

I am going to go out on a limb here and suggest that the reason that the folks in the Harvard study did not see the gorilla is less due to the fact of “inattentional blindness” and more due to the fact of intentional focussing, AKA tunnel vision. If I am directed to look for a certain thing, I will tend to devote myself to that hunt. I will not attend to other things that might merit my attention because I have chosen to exclude them. Consider a “Where’s Waldo” picture as a case in point. In the back of Martin Handford’s books, one will usually find a list of other things to find in the pictures besides Waldo. When hunting for Waldo though, how often does one notice his dog Woof, his arch-enemy Odlaw, Professor Whitebeard, Wanda, or Wilma, not to mention all the other “punnily” described items in those lists?

Over and above the issue of selective attention, I suspect that we ought to consider some things like the nature of certainty, proof, and truth with more than just a passing handwave. The article’s single paragraph summations of Platonic and Aristotelian epistemology are massive oversimplifications, and therefore, distortions of some very detailed analyses. They set up the poles of a debate that still rages today. In that light, I’d suggest, as a minimum, that the work of a few 20th Century philosophers and mathematicians be consulted. Ludwig Wittgenstein’s work, On Certainty is extremely insightful. In it he suggests that what often passes for verification of the truth is like buying several copies of the same newspaper and re-reading the same story (speaks to Rex's point about staying in the same agency). I heartily endorse R.G. Collingwood’s work on re-enactment in the philosophy of history as well as his consideration of absolute presuppositions and philosophical method. Godel’s incompleteness theorem is noteworthy for the fact that it shows the limits to proofs. I recommend reading the debate on the problems with translation and conceptual schemes between WVO Quine and Donald Davidson. The Confucian notion regarding rectification of names might be worth looking at too. And, while we’re at it, how about a look at a few Zen koans and the parables of Jesus to try to reframe the boxes in which we tend to do our conceptualizing?

One last point about using native speakers as translators: ever have anyone ask you if you want gravy on your spaghetti?*

Usually, one is a native speaker of only one language. Native speakers of Iraqi Arabic are English As A Second Language (ESL) speakers. They will have the same kinds of cultural disconnects when trying to get the words right for their English native speaking employers. And, that is without consideration for such things as idiolects, regionalisms, colloquialisms, and slang. My Nebraska-born father described a heavy rain storm as “a trash mover and a gully washer,” my New Hampshire-born grandmother talked about “ it raining pitchforks and hammer handles,” my Paris-born French teacher said it was “raining halberds” and most other folks I know say ”it’s raining cats and dogs.” While I might describe a certain car as “cool” or “hot,” my 10-year old describes it as a “sick” car. My brother-in-law might describe that same car as “the balls” or “the nuts.” We live within a few miles of each other and are, supposedly, speaking the same English. Now, that’s chaos!





*If you live in and around Chelsea, Massachusetts, "gravy" is what you probably call the tomato sauce put on pasta.

[Brian Hanley]

Beelzebubalicious - The difference betweent that and the Peace Corps is that you are going in to develop a network of people that work for you in a business. Seriously, I think there is no better way to create a network that will give you "ins" into the real world. You aren't going to hobnob with the poor, you are going to get into the world of the rich and powerful. But aside from that, yes, it's got its similarities.


JeffC - The real concern I have with your proposal above is that it presumes that you even could create a simulated society of terrorists. The problem with that is that anything you come up with will be from your own map of the world. In this, Osama and his boys have an advantage over us. They know how we think and work because we broadcast it everywhere. We don 't even try to listen to what they are saying. If you are serious about it, and have no other resource, I would recommend following Al Jazeera and the major newspapers in Tehran, Lahore and Riyadh. It will be one hell of an eye-opener, that's for sure. I'm sure there must be a translated version of those available somewhere in the vast federal government bureacracy.

The first time you read an article on the front page of a major Arab paper that says, in all seriousness, that Israelis are harvesting Arab children to put their blood in passover matzoh, and you see that the author is a PhD professor at the university - well, you'll begin to realize how very different things are.

To really get inside the head of a fanatic though, one either has to have been one (of some religious persuasion) or else have been very close to one or more. Personally, I think one of the best resources we have in the USA on how those guys think is people like Falwell. They, like Osama, are "people of the book" and take its dictates as literally as possible. They both motivate large numbers of people. (Hope I didn't offend anyone here by saying that. They have different ideological maps, very different.)

[Rex Brynen]

We don 't even try to listen to what they are saying.

That's simply not the case, Brian--the IC is full of people who know their files well, spend enormous time listening to public (and not-so-public) discourse, and write some excellent analytical product.

There will always be some things that we don't know because of the challenges of collection, the "other side's" OPSEC and CCD efforts, the complexities of human decision-making, the array of relevant variables, the challenges of prediction, etc.

Ted is absolutely right in highlighting the central importance of good recruitment, training, professional development and mentoring, and field (and, i would add, cross-disciplinary or interagency) familiarity.

[Brian Hanley]

That's simply not the case, Brian--the IC is full of people who know their files well, spend enormous time listening to public (and not-so-public) discourse, and write some excellent analytical product.

OK. I sit corrected. However, I have not seen much evidence of it getting to people that need it or want it. And I'd say that includes the public. This is an ideological war first and foremost. I would still recommend going direct to translations before analysis for most. It makes the case better.

[nichols]

Hell, I'm 4 handshakes from bin Laden myself.

Brian,

You should seriously think about joining the CIA

You post some interesting thoughts yet weave comments like the above into your posts.....your back ground in biology...please help me connect the dots.

What exactly is your intend / end state in this forum?

[Brian Hanley]

My intentions are to have conversations with intelligent people and perhaps help some to be more effective. In my current incarnation I am in graduate school in microbiology. You may or may not be aware that college campuses are not places where such discussions can occur in a useful fashion very easily in most cases.

My reasons for selecting my current graduate study area is biological warfare, which is something that Al Qaeda has said they want to acquire.

I suspect that my thoughts and comments will make people think. I tell it like I see it and call things as I see them.

[nichols]

My reasons for selecting my current graduate study area is biological warfare, which is something that Al Qaeda has said they want to acquire.

You post that you're four handshakes away from bin Laden.

Why?

[JeffC]

The title of the piece implies that it will discuss "intelligence analysis and asymetric warfare" - yet the operational example (India's nuclear test) is of analysis failing in a very conventional Cold-War type collection and analytic task..

--- I chose that example because it was a clear cut demonstration of missing the obvious, and because Red-teaming was one of Tenet's recommendations.

Mirror imaging is a common analytic failing of new and poorly trained analysts. However, to categorize it as a "seemingly impossible problem" is a gross exaggeration.

--- It is a pervasive problem according to Johnston's study and many more critiques of Intelligence analysis over the years. I could have filled a page with just references to that one issue.

Solution to such problems are not "seemingly impossible"; they are simple.

--- If the problem were simple, it would no longer be a problem. Clearly, for anyone who reads the literature, that's not the case.

[JeffC]

JeffC - The real concern I have with your proposal above is that it presumes that you even could create a simulated society of terrorists. The problem with that is that anything you come up with will be from your own map of the world.

--- I'm certain that such a facility can be built and staffed inside Iraq or Egypt, possibly even Israel. And while it certainly wouldn't be perfect, it definitely would break apart the former mindset or mental model of the analysts that attended it, and replace it with one more in tune with that of an Islamic extremist.

--- More importantly, my suggested scenario is just a starting off point. I'm sure it would require a great deal of testing with alternate scenarios before the best solution would be arrived at.

[Rex Brynen]

--- If the problem were simple, it would no longer be a problem. Clearly, for anyone who reads the literature, that's not the case.

I think Jedburgh has done more than just read the literature, Jeff.

In a robust assessment environment, people are on a look-out for this. Assumptions about social/cultural/political context, motive, intentions, and behaviour are questioned by analytical team members, team leaders, and higher-ups, and tested against evidence. (They had best also be on the look-out for the reverse of mirror-imaging too: that is, an almost caricatured image of the opponent.)

I'm not saying that its not a problem, but that it is perhaps less common of a problem than you suggest, and more readily dealt with by existing recruitment and training mechanisms than you presume.

[Brian Hanley]

You post that you're four handshakes away from bin Laden. Why?

Sorry to be opaque there, or appear a bit dense. My intent was to signal how one can network by just getting out in the field and doing it. The context was to support what I was saying to JeffC about how nothing can replace that experience. I gave it as an example of the potential value of getting out there and mixing it up.

If someone wants to try to make some practical use of that info, they can write me and I can dig up the contact info that was 2 handshakes away from OBL. For the record, those parties are in Almaty. They are friends of a family there that I had a connection with through a relative. The friends of the family were reliably reported to have worked directly with bin Laden for a time in Afghanistan. So it's a cold lead now, but so is everything. At the time it was reported as fairly warm.

[JeffC]

I'm not saying that its not a problem, but that it is perhaps less common of a problem than you suggest, and more readily dealt with by existing recruitment and training mechanisms than you presume.

Actually, my essay is based largely on reports issued by the CIA's Center for the Study of Intelligence. I relied on experts who have studied the problem for years. That's not to say that they're infallible, however I don't think that their findings should be dismissed without proof which counters their conclusions.

The same applies to Freeman's research findings. If the brain does create its own internal models, then certain things need to be done to break those models down. The fact remains that each person cannot know what he or she doesn't know, regardless of good intentions or re-doubled effort.

[Jedburgh]

....--- It is a pervasive problem according to Johnston's study and many more critiques of Intelligence analysis over the years. I could have filled a page with just references to that one issue....

....If the problem were simple, it would no longer be a problem. Clearly, for anyone who reads the literature, that's not the case....

As stated, mirror imaging, along with falling into logical fallacies and permitting various biases to determine judgments, are a common failing among new and poorly trained analysts. However, "common" does not equate to "pervasive".

There are also significant differences in quality and type of analysis produced by the different agencies and services. There is perhaps an agency or two out there that hires far too many people who don't really deserve the title of "intelligence analyst"; but there are others that are lucky enough to have several consummate professionals on the payroll. Throughout the community we have many outstanding analysts who are very much in touch with reality as it exists on the ground and in the heads of the threat.

Reads the literature? Well, I've spent a couple of years as a collector and worked a bit of analysis here and there, and, as limited as my academic qualifications may be, I stand solidly by my opinions.

The disparagement of seemingly simple solutions is in itself an analytic failing. Despite the apparent simplicity, they are still dependent upon the human vagaries of the people implementing them. The existing problems with selection, training and professional development of career analysts may have a simple solution - but the primary obstacle is not so simple, as it tends to lie with bureaucrats outside the field of intelligence analysis. If you were able to conduct such a study in-depth, you would find that the degree of severity of these quality problems ebb and flow as much with changes in senior management as it does with hiring and training of analysts.

In any case, even if a brilliant analyst produces a truly insightful and timely product for national-level consumption, there may still exist yet another obstacle at the national strategic decision making level. The personal beliefs, biases and perceptions of a given policy maker make cause him to simply reject outright the conclusions, judgments and recommendations of the analyst. Then when everything goes tango uniform, "intelligence" is blamed. But the greater public rarely learns that the "intelligence" at fault was cherry-picked to conform to the policy-maker's predilictions, while the good intel that did exist was ignored or dismissed.

Along the lines of my last statement, I recommend giving a read of Knowing One’s Enemies – Intelligence Assessment Before the Two World Wars. The book is a collection of essays that amply illustrate the point that, even when a nation is in possession of sufficient intelligence of a quality to make effective policy decisions, it can all come to disaster due to the inherent biases, proclivities and abilities of key policy makers. The harmful effects of internal disputes within intelligence agencies, and turf battles between competing agencies, are also laid out in careful detail.

This is where you complexity in reaching an effective solution. Hell, its been over six years since the wake-up call and we're still not ready for work yet. Hiring, training and developing good analyst is the easy part.