Intelligence failure: get the right IT system thinking
I've looked through the Intelligence thread and cannot immediately find an appropriate thread for this.
Bear with me, it could fit in the Detroit bombing thread: http://council.smallwarsjournal.com/...ead.php?t=9331 and FBI investigations: http://council.smallwarsjournal.com/...ead.php?t=8828 - both are useful cross references, hence the links.
Robert Haddick today has written 'Computers must take over counter-terrorism analysis', which at first I thought was another "IT can fix it"; pg. 2 of this article
:http://www.foreignpolicy.com/article...nment?page=0,1
Then I recalled Jeff Jonas is an IT expert (at IBM) and was well worth reading, having thought hard on the issues around data. His blogsite is: http://jeffjonas.typepad.com/ and just to illustrate try his post-9/11 ppt on the hijackers associations:http://jeffjonas.typepad.com/SRD-911-connections.pdf
After a long absence he has now commented on what he calls 'The Christmas Day Intelligence Failure', note this is Part One:http://jeffjonas.typepad.com/jeff_jo...elligence.html
He advocates that "data finds data":
Quote:
The December 25th event is a classic case of enterprise amnesia. Enterprise Amnesia is the condition of knowing something on one hand and knowing something on another hand and never the two data points meet....
Abdulmutallab applies for a multi-entry visa. The terrorist database (TIDE) is checked and found to contain no such record. The State Department issues a visa. Later, a TIDE record for Abdulmutallab is added to TIDE. The split-second this record is added to TIDE, the State Department is notified the visa may need reconsidered.
Devil in the details. For all this to work, the system needs to realize that despite name variations and inconsistent data, the identity in the terrorist database is the identity in the visa system...
Jeff raises difficult issues for non-IT outsiders to think about - as we should be the ones setting the requirements for IT help - and I will add subsequent parts as they appear.
He is a very entertaining speaker on these issues.
Change needed - another view
Another respected commentator on security issues, Bruce Schneier, adds this comment:http://www.schneier.com/blog/archive..._intellig.html
Quote:
We don't need new technologies, new laws, new bureaucratic overlords, or -- for heaven's sake -- new agencies. What prevents information sharing among intelligence organizations is the culture of the generation that built those organizations....sharing is far more important than secrecy. Our intelligence organizations need to trade techniques and expertise with industry, and they need to share information among the different parts of themselves....We need the bottom-up organization that has made the Internet the greatest collection of human knowledge and ideas ever assembled. The problem is far more social than technological.
Does Bruce Schneier contradict himself?
Quote:
Critics have pointed to laws that prohibited inter-agency sharing but, as the 9/11 Commission found, the law allows for far more sharing than goes on. It doesn't happen because of inter-agency rivalries, a reliance on outdated information systems, and a culture of secrecy. What we need is an intelligence community that shares ideas and hunches and facts on their versions of Facebook, Twitter and wikis. We need the bottom-up organization that has made the Internet the greatest collection of human knowledge and ideas ever assembled.
I agree with his points about culture change, but to deny the benefits of "value-added" technology is being overly dismissive of a "needed" capability. He even states in the paragraph quoted above (emphasis is mine) that we're working with outdated information systems. Many government agencies have their own Facebook now and can share information with other contacts in other agencies, but that hardly allows one fuse all the data available, then to connect the dots in a way that tells a story. The real challenge isn't sharing the information (we're much better than he gives the community credit for, but there is still much room for improvement), but the bigger challenge is making sense of the volume of information. We desparately need better information technology that helps analysts sort through volumes of data and then connect the dots (analytical support) and display it in a meaningful way. The culture that needs to change quickest is for each government agency/department and military service to stop storing their data in databases that are not accessible to the community of interest at large. Too much data resides in data banks that is not sharable outside their individual system, etc. The result is intelligence failures because the data was not available to the analyst who had a hunch, and if he/she had the all the data available and had the right analytical tools to quickly pull and sort through the relevant data, and then display it in a way that tells a story (visualize the data through link analysis and using temporal analysis), then we will have made a change that will actually result in our intelligence and law enforcement communities being more effective. Facebook and Twitter are only baby steps, they are far from being revolutionary enough to truly move us into the information age.
Another technology he may be bashing is technology to detect explosives and other potential weapons in airports. IMO it would be foolish not to invest in these technologies. Technology in many cases can do a better job at this and other tasks than humans, so why not use it? If it effectively reduces risk to a critical economic system (our air transportation system), why not invest in it? I'm sure if we did a cost comparison of what one attack costs when you consider all the ripple effects we would it find it a worthwhile investment.
On the periphery - not overlooked?
Earlier today I read this NYT article on the radicalization route for the Detroit bomber:http://www.nytimes.com/2010/01/17/wo...l?pagewanted=2 . I have seen similar before and worth a read, although much I fear is news reporting and not careful, verified investigation.
Then Leah Farrell, an Australian CT analyst, adds her viewpoint - having cited the NYT article:
Quote:
Still, while he was seen to be “reaching out” to known extremists and appearing on “the periphery of other investigations” into radical suspects there, he was not considered a terrorist threat himself, according to a British counter-intelligence official.
Leah adds:
Quote:
Edge of network connections–again. Of course the problem is always resourcing. There is never enough time to track down everything. But still, it seems to me that we see this over and over and over again.
No answers provided, some pointers to her earlier thinking on the issue and to an IT "guru" who has tried an answer.
I also wonder how many for example have attended a meeting on a controversy, listened, even spoken to a speaker, who might be a 'known extremist'. Does that merit a CT record? In the Detroit incident, an exchange of information between the UK and USA which apparently did not happen.
I just finally threw the clot. . .
The reality is that we do not have a technical problem. While I won't pretend to understand the advanced arithmatic above, I suspect that the mathmatical solution to predicting behavior is not so far from a reality. Looking at a person, observing their behavior and applying those behaviors against a model of a "terrorist" (No, Liles, this is not a boolean - more of a sliding scale, that someone is more likely than someone else to be a baddie). In turn we can then focus our efforts on those people. Naturally, this type of system will seldom capture the angry guy who just goes off and drives his SUV through a university or a nut case who is otherwise "ok" and shoots up Ft. Hood, but it _should_ provide us a list of people to observe more closely, and so we can stop searching cub scouts.
However, there is a policy issue and a people issue at hand here. Regardless of what we want to believe, our government doesn't like to share. This is often promoted by contractors who are protecting their own turf (Data makes you king, and sharing data is seen as weakening your realm). In turn, we find that various agencies can "collect" on someone, and failure to share is not met with a firing squad.
Conversely, I posit that data entry is annoying at best, and hard at worst. Given human nature and *our* desire to find the most leisure whenever possible, people don't bother to collect on the details. I flew through RDU this morning at 0600. The woman in front of me was meddling with her personal toiletries. The TSA rep told me I could jump into another line to bypass her. However, our practice should have been to report her by name on this behavior. Was it criminal? No. Suspicious? Not really. But when taken in conjunction with other behavior, it could show trends or patterns that might indicate negative or dangerous behaviors in the future. Sadly, my crack TSA agent instead made a smart assed comment and I was on my way.
The reality is that during a survey of any data store in the intelligence or C2 arena, we might be surprised at how many fields of data we ask for and how few are actually completed. It is really hard to do trending when 80% of your database is blank.
So, at the end of the day, the reality is that the ideas you all are promoting are sound mathematically and technologically, and if used to highlight individuals, organizations or even regions, these can be effective to help us plan. But until we are really serious (I mean firing some senior people in both the Government and Industry), I will just continue fighting the good fight and hoping that we get lucky again.
And the second clot - for Social Networking
Until we solve for all of the rest, the Social Media - Faceyspaces, and TwitteryTweets are all just a time suck. Trust me - I use all of them, and they all waste time. To try and use these to paint a picture, when we can't mine structured, normalized data is simply a bridge too far.
During last year, I read about a use case from the intel community for twitter - Imagine two patrols twitter about the same event (like an IED blast) from two vantage points. Or that all of the patrol members twitter about the event. Now we have 24 reports (or whatever) about the event, and in turn our intel studs can form a complete picture based on the 24 strories of 140 characters each.
Seriously? The market has been blown sky high, and I am jumping on my iPhone? I don't know how to text and return accurate fire. Moreover, how do we know it was one event or 24? Location, separated by time could create multiple events? Is that 1 or 24?
Again, until we are mature enough to use the systems in play, let's keep reporting out of MilBook and Twitter. (OK - We can use Wikis - Intellipedia is supposed to be pretty hot - though it is just another island of information not accessible to the enterprise half the time)