Intelligence failure: get the right IT system thinking

[davidbfpo]

I've looked through the Intelligence thread and cannot immediately find an appropriate thread for this.

Bear with me, it could fit in the Detroit bombing thread: http://council.smallwarsjournal.com/...ead.php?t=9331 and FBI investigations: http://council.smallwarsjournal.com/...ead.php?t=8828 - both are useful cross references, hence the links.

Robert Haddick today has written 'Computers must take over counter-terrorism analysis', which at first I thought was another "IT can fix it"; pg. 2 of this article
:http://www.foreignpolicy.com/article...nment?page=0,1

Then I recalled Jeff Jonas is an IT expert (at IBM) and was well worth reading, having thought hard on the issues around data. His blogsite is: http://jeffjonas.typepad.com/ and just to illustrate try his post-9/11 ppt on the hijackers associations:http://jeffjonas.typepad.com/SRD-911-connections.pdf

After a long absence he has now commented on what he calls 'The Christmas Day Intelligence Failure', note this is Part One:http://jeffjonas.typepad.com/jeff_jo...elligence.html

He advocates that "data finds data":

The December 25th event is a classic case of enterprise amnesia. Enterprise Amnesia is the condition of knowing something on one hand and knowing something on another hand and never the two data points meet....

Abdulmutallab applies for a multi-entry visa. The terrorist database (TIDE) is checked and found to contain no such record. The State Department issues a visa. Later, a TIDE record for Abdulmutallab is added to TIDE. The split-second this record is added to TIDE, the State Department is notified the visa may need reconsidered.

Devil in the details. For all this to work, the system needs to realize that despite name variations and inconsistent data, the identity in the terrorist database is the identity in the visa system...

Jeff raises difficult issues for non-IT outsiders to think about - as we should be the ones setting the requirements for IT help - and I will add subsequent parts as they appear.

He is a very entertaining speaker on these issues.

[davidbfpo]

Another respected commentator on security issues, Bruce Schneier, adds this comment:http://www.schneier.com/blog/archive..._intellig.html

We don't need new technologies, new laws, new bureaucratic overlords, or -- for heaven's sake -- new agencies. What prevents information sharing among intelligence organizations is the culture of the generation that built those organizations....sharing is far more important than secrecy. Our intelligence organizations need to trade techniques and expertise with industry, and they need to share information among the different parts of themselves....We need the bottom-up organization that has made the Internet the greatest collection of human knowledge and ideas ever assembled. The problem is far more social than technological.

[Bill Moore]

Critics have pointed to laws that prohibited inter-agency sharing but, as the 9/11 Commission found, the law allows for far more sharing than goes on. It doesn't happen because of inter-agency rivalries, a reliance on outdated information systems, and a culture of secrecy. What we need is an intelligence community that shares ideas and hunches and facts on their versions of Facebook, Twitter and wikis. We need the bottom-up organization that has made the Internet the greatest collection of human knowledge and ideas ever assembled.

I agree with his points about culture change, but to deny the benefits of "value-added" technology is being overly dismissive of a "needed" capability. He even states in the paragraph quoted above (emphasis is mine) that we're working with outdated information systems. Many government agencies have their own Facebook now and can share information with other contacts in other agencies, but that hardly allows one fuse all the data available, then to connect the dots in a way that tells a story. The real challenge isn't sharing the information (we're much better than he gives the community credit for, but there is still much room for improvement), but the bigger challenge is making sense of the volume of information. We desparately need better information technology that helps analysts sort through volumes of data and then connect the dots (analytical support) and display it in a meaningful way. The culture that needs to change quickest is for each government agency/department and military service to stop storing their data in databases that are not accessible to the community of interest at large. Too much data resides in data banks that is not sharable outside their individual system, etc. The result is intelligence failures because the data was not available to the analyst who had a hunch, and if he/she had the all the data available and had the right analytical tools to quickly pull and sort through the relevant data, and then display it in a way that tells a story (visualize the data through link analysis and using temporal analysis), then we will have made a change that will actually result in our intelligence and law enforcement communities being more effective. Facebook and Twitter are only baby steps, they are far from being revolutionary enough to truly move us into the information age.

Another technology he may be bashing is technology to detect explosives and other potential weapons in airports. IMO it would be foolish not to invest in these technologies. Technology in many cases can do a better job at this and other tasks than humans, so why not use it? If it effectively reduces risk to a critical economic system (our air transportation system), why not invest in it? I'm sure if we did a cost comparison of what one attack costs when you consider all the ripple effects we would it find it a worthwhile investment.

[davidbfpo]

Earlier today I read this NYT article on the radicalization route for the Detroit bomber:http://www.nytimes.com/2010/01/17/wo...l?pagewanted=2 . I have seen similar before and worth a read, although much I fear is news reporting and not careful, verified investigation.

Then Leah Farrell, an Australian CT analyst, adds her viewpoint - having cited the NYT article:

Still, while he was seen to be “reaching out” to known extremists and appearing on “the periphery of other investigations” into radical suspects there, he was not considered a terrorist threat himself, according to a British counter-intelligence official.

Leah adds:

Edge of network connections–again. Of course the problem is always resourcing. There is never enough time to track down everything. But still, it seems to me that we see this over and over and over again.

No answers provided, some pointers to her earlier thinking on the issue and to an IT "guru" who has tried an answer.

I also wonder how many for example have attended a meeting on a controversy, listened, even spoken to a speaker, who might be a 'known extremist'. Does that merit a CT record? In the Detroit incident, an exchange of information between the UK and USA which apparently did not happen.

[Stan]

I also wonder how many for example have attended a meeting on a controversy, listened, even spoken to a speaker, who might be a 'known extremist'. Does that merit a CT record? In the Detroit incident, an exchange of information between the UK and USA which apparently did not happen.

David,
It was just a few years ago while attending MET training, the instructors feared their POI was copied and sent to addresses in the Middle East.

Of the countless so-called seminars and conferences held in Europe, I don't recall one instance where my credentials were checked or my name vetted. In fact, most of these folks are so hungry for participants they dump their advertisements onto the internet. I'm a little tired of the spam every morning, but wonder who their audience actually ends up being.

General so and so is your guest speaker with decades of experience fighting (insert key country or conflict here)

[Presley Cannady]

Oh and for Chrissakes, why is it every social networking tool in the last decade's been promoted as the next must-have thing for collaboration? I find it particularly disturbing whenever I hear someone say that the IC can benefit from something like Twitter, Facebook or wikis. It's rate to see such a claim accompanied with an explanation of what these tools bring to the table, and you'll never see any analysis of the pitfalls. Take the Twitter and Facebook models for example. Are we going to be ranking the relevance of take based on the popularity of the source? That's what a friend or follower model entails. A wiki is a bit more defensible, but no more so than any other content repository with versioning and open access to anyone--a wiki is no more innovative than say git or svn or Alfresco.

[Presley Cannady]

Robert Haddick today has written 'Computers must take over counter-terrorism analysis', which at first I thought was another "IT can fix it"; pg. 2 of this article
:http://www.foreignpolicy.com/article...nment?page=0,1

Automation is already spreading into the counterterrorism field, but simply saying we need more of it isn't going to produce technology that simply doesn't exist yet. There is no single thing you can point to and call it data-mining. It is a family of hundreds of loosely connected problem spaces and solutions orbiting storage and recall. If we have to resort to a very crude simile, you might liken the field today to that of the vast, also loosely connected realms of neuroscience, cognition, and linguistics. An even less compelling, but still useful parallel might be to the study of episodic memory.

Then I recalled Jeff Jonas is an IT expert (at IBM) and was well worth reading, having thought hard on the issues around data. His blogsite is: http://jeffjonas.typepad.com/ and just to illustrate try his post-9/11 ppt on the hijackers associations:http://jeffjonas.typepad.com/SRD-911-connections.pdf

Ten bucks Jeff Jonas hasn't done serious work in twenty years with any database model other than relational. That is to say that the trick to his analysis here is devising a system of relations--tables in a database--that admits the properties of events to be correlated in hindsight. That is to say even if future terrorists were careful enough not to input identical or even similar contact information, the system would break down if the method of input (say, in this day an age, Travelocity v. Expedia) changed. There is an easy enough way to fix this (and I hope they've done it), which brings us to this:

After a long absence he has now commented on what he calls 'The Christmas Day Intelligence Failure', note this is Part One:http://jeffjonas.typepad.com/jeff_jo...elligence.html

First off, the visa office wouldn't consult TIDE, they'd consult the TSDB--which is sourced from TIDE. This is a non-classified subset of the information contained in the IC's database. The key problem, from news reports thus far, is that information in TIDE was not transferred to TSDB. This is a classic failure in information sharing.

The point is that there are still filters between source repository of collected data (which could be an airline's manifest, a booking agent's order list, or an aggregator like TIDE) and the databases operated on by analysts. Law, I imagine, plays a role in keeping those walls up; I leave that to someone with the appropriate background.

Another wall might simply be competing data structure. The vBulletin software driving this forum has a database schema that is for all intents and purposes fixed during operation. I can input no more data than a relation specifies and in most cases no less than the constraints allow. The only way to change that is to change the underlying structure, which anyone whose ever even played with SQL should understand is a dicey, manual process that should never be taken lightly or without adequate testing before hand.

He advocates that "data finds data":

That's nice, but he has a lot of technical obstacles to overcome first. The walls I listed above are not insurmountable, but they are difficult to overcome. The legal issues have to be resolved by legislation or jurisprudence. The variety in data schema out there is tremendous. And finally, the real world's databases are not self-evolving (yet), and wishing for mature enough technology is not going to change that fact. Research may change that fact in the future, but for the time being human beings are going to be the principal glue that moves information from one large database to another.

Jeff raises difficult issues for non-IT outsiders to think about - as we should be the ones setting the requirements for IT help - and I will add subsequent parts as they appear.

The Non-IT folk--the stakeholders--have laid out clear requirements, in public and on multiple occasions. I hate to see it when folks in the computer sciences hide behind so-called ambiguities in the requirements to hide the fact that a problem may be intractable at this time. This is probably because in advance of the release planning, a ton of promises were made about what a technology could do without any sort of thought into what it couldn't.