Intelligence Collection and Sharing

SWJ Blog - Intelligence Collection and Sharing by CPT Tim Hsia.

Quote:

---

Years from now after the Iraq and Afghanistan wars have ended; historians will pore over the operations and tactics of the U.S. Army during both campaigns. They will likely applaud the all-volunteer force and the courage of the individual soldier; just as likely, however, they will criticize the lack of information sharing and management between the militarily and civilian departments of the U.S. government. Specifically, they will note the military’s poor record in information management, accessibility of intelligence gathered, and the inability to apply years of accumulated intelligence to current battlefield operations. A way to patch the current intelligence gap within the U.S. government would be to adopt an information collection program that accumulates data similar to major internet stock market trackers. Market trackers absorb information continuously, rigorously track trends, and enable traders to formulate decisions based off the latest news combined with historical data. The ability of market trackers to store and quickly recall historical data should be mimicked by the U.S. government so that commanders and diplomats possess relevant records that enable them to make decisions which take into account the economic, historical, cultural, political, anthropological, and environmental aspects of the region they are operating within...

---

I think we'll see this sooner rather than later but how soon the business communities are willing to share their secrets has yet to be determined.:wry:

Quote:

---

...A solution to the current intelligence blackhole would be to collect, store, and sift this data into a “geointel” site organized in a manner replicating stock market data....

---

I dislike the comparison and feel it is yet another false analogy based on misconceptions about intelligence. The piece shows little understanding of multidisciplinary all-source collection and how it affects the analytic effort at the tactical and operational levels.

However, tossing aside the market comparisons, the concept of organizing finished intel by geographic location in an all-encompassing user-friendly database that can support analysis at all levels is a good one. At its most fundamental, this resembles an old analysis tool that is very handy in the COIN environment - the incident map/coordinates register.

Unfortunately, the coordinates register gets short shrift in current doctrine. The most current version of FM 2-33.4 Intelligence Analysis provides one short paragraph on this tool, with no real instructions on its use. It is treated the same way in ST 2-91.1 Intelligence Support to Stability Operations and Support Operations. FM 2-91.4 Intelligence Support to Urban Operations doesn't mention it all - and I personally have found it to be particularly effective as an analytic tool in the unconventional urban environment. There is also no discussion of it at all in the new FM 2-01.301 Specific TTPs for IPB, although the manual does briefly mention an "incident overlay" in an almost dismissive manner.

(A lot of what LE is doing today with GIS support to LE analysis is similar to what I learned in the old days with a stubby pencil, notebook and paper maps. However, LE cross-jurisdictional issues with information sharing in support of analysis makes the Army's problems as discussed in the CPT Hsia's piece look positively minor by comparison.)

My understanding of the intent of CPT Hsia's concept is that it takes the old coordinates register tool and upgrades it a couple of generations - and takes it from tactical to strategic applicability - with current technology. Looking at it from an analyst's perspective, in many operational environments it would be nice to be able to use current high-res overhead imagery of your target location (from country level down through province, city, district, block and even to single building) as a dashboard to pull up resources based on what you are looking at. i.e. for the single building it could provide every product that even tangentially mentions that building or an occupant as well as everything reported within a given distance of the building. (As an almost-aside, it would pretty much require dual monitors to use the capability effectively.)

The technology exists. Hell, we are able to do bits and pieces of that right now. But the overarching all-encompassing user-friendly database does not exist. To build it to the level that he states - with complete (and timely) joint service, interagency and private sector colloboration and input - is a tough row to hoe. If we could get something that works that well with just the military services, it would be a near miracle.

Of course, there's always the human factor. We will always have leaders who, despite having all the substantive information they need to support a specific course of action, instead permit their personal biases and perceptions to dismiss valid intelligence. On the other hand, we continue to have a few outstanding leaders (never enough though) who, despite the lack of substantive information, manage to make the right decisions, or to find innovative ways of turning tragedy into success. Technology won't replace character or instinct (or be a substitute for the lack of), although it can certainly be leveraged as a powerful tool in support of good leaders.

Jedburgh it sounds like you are describing the google suite.

Quote:

---

Originally Posted by selil

Jedburgh it sounds like you are describing the google suite.

---

Then I guess I did a really lousy job of describing my thoughts. That's not what's in my mind's eye at all. Although there are aspects of Google's Advanced Search that are useful, Google Earth is but a pale reflection of the quality of overhead imagery available to the IC and the interface I was pondering was like nothing Google offers.

Google Pro has a lot better imagery. For Google Earth there is an API that you can do some pretty swift GIS data stuff with. I'm not familiar with what you're using at all but Space Imaging in Thornton, CO has some pretty cool tools (sort of civillian company).

One other thing about Google is that what you see with search tools are only part of the capability. With the Google appliance suite you can put your data up for searching and it (and the software) can start doing analysis and drawing relationships. That gets your easy to use database.

When you're dealing with content rather than images or geographic locations context aware searching is important. There is another similar technology to Google for context aware searching being modeled by a company called Hakia. It uses heurestic semantics and the more questions you ask the better it gets at searching.

Sorry I didn't understand.

If I might ask a few questions....

01 First off, from your posts, I've developed this vision of being able to look at visual imagery (almost like driving down a road, or a network of roads, only from above) and eventually focusing in on a specific location (whatever it may be).

02 Then based upon the location selected (with a number of different locator options), not just coordinates, because you may not have that specific piece of information), tell me everything I might want to know about that specific location, with additional search options (like a diameter search, or a "side of road" search, or "Persons associated with" search).

03 My main question is, who would be the primary consumer (user) of the system? I know that sounds like a stupid question, but from reading the original article vrs. your comments, I don't ever see one unified software system being able to reliably perform all the required functions for all the different parties involved (folks in the trenches vrs. folks in command, being outside the primary activity area).

For example, I noted all the "high security" requirements in the original article. Well, if it takes an extra 5+ seconds to get through all the security access crap out in the field, that can be several eternities if people are shooting at you. Result will be that people won't use the crap (software) because it's just too slow. They won't tell you that, they just won't use it.

Reason I'm so interested in this is because we do software work, but for local government units across a number of states, and GIS/attribute database integration is a giant issue, so am always on the lookout for new ideas and concepts.

04 You night want to check out Google Neighborhood. It's extremely new, I just very recently got clued into it, and just started looking at it. Could be the type of software (first steps) that you are looking for.

Can tell you that the perfect training for this type of work would be "Siamese Cat Herder".:wry:

Quote:

---

Can tell you that the perfect training for this type of work would be "Siamese Cat Herder".

---

Naw, calicos are way tougher.

Seriously, I see the problem with the whole article in that it confuses/conflates information/knowledge management with intelligence. The first can support or cripple the latter but it cannot replace reall intelligence--especially in the realm of intelligence analysis.

Tom

Quote:

---

Originally Posted by Tom Odom

Seriously, I see the problem with the whole article in that it confuses/conflates information/knowledge management with intelligence. The first can support or cripple the latter but it cannot replace reall intelligence--especially in the realm of intelligence analysis.

Tom

---

Concur wholeheartedly. What seems to be described is little more than an advanced form of near-real time targeting folders. That is a subspecies of the entire realm of intelligence operations.

Just because one has spiffy systems for filing and retrieving a bunch of data in a lot of different ways, one cannot therefore conclude that one has good intelligence. Among other things, one needs to be able to structure queries in appropriate ways to extract data from the storage and retrieval systems. One also needs to be able to look at the results of those queries, decide if they contain gaps, and decide what additional queries may be needed to fill in the gaps in the initial returns. And, IMHO, most important, one needs to be able to do the "so what" piece. In other words, to have intelligence, one needs to be able to answer the question, "What does all of the data returned by the queries allow me to infer about what is going to happen next and with how much confidence?"--

By analogy--When you try to get directions from Google Maps, you need to tell it were you want to go, when, and which type of route you want to take. The system doesn't know that first.

Quote:

---

Originally Posted by Tom Odom

...Seriously, I see the problem with the whole article in that it confuses/conflates information/knowledge management with intelligence. The first can support or cripple the latter but it cannot replace reall intelligence--especially in the realm of intelligence analysis.

---

Tom, I read the article a bit differently. My perception was the he was focused on an information management concept as a tool for analysis, not the analysis itself. What I saw as severely lacking was any understanding of the complexities of collection by the various disciplines, as well as reporting by tactical units, and how all of that raw intel feeds into analysis. That is where the false analogy of the market comes into play - because he doesn't truly understand how the spectrum of collection assets are tasked and raw intel is fed to analysts his concept of information management falls a bit flat.

Quote:

---

Originally Posted by wm

Just because one has spiffy systems for filing and retrieving a bunch of data in a lot of different ways, one cannot therefore conclude that one has good intelligence. Among other things, one needs to be able to structure queries in appropriate ways to extract data from the storage and retrieval systems. One also needs to be able to look at the results of those queries, decide if they contain gaps, and decide what additional queries may be needed to fill in the gaps in the initial returns. And, IMHO, most important, one needs to be able to do the "so what" piece. In other words, to have intelligence, one needs to be able to answer the question, "What does all of the data returned by the queries allow me to infer about what is going to happen next and with how much confidence?"

---

wm, I don't believe CPT Hsia claims in his piece that more efficient filing translates to good intelligence. My take on his concept was that, if you trim out all the extraneous narrative, he is basically stating that by having all the data geo-coded it would facilitate more efficient pattern/trend analysis.

As I stated in my rambling bit above, I concur with that at a fundamental level. The incident map / coordinates register is without question an extremely useful tool for analysis in the unconventional warfare environment. But it is just that - a tool. One of many.

Commentary from Kent's Imperative:

Quote:

---

Via the incomparable Small Wars Journal, we note a currently serving officer’s contribution to the literature in the form of articulated longing for the solutions that can help end his ongoing frustrations with the intelligence system as it is now structured.....

....We empathize with his frustrations, although we do not know that another “central” portal system or new repository is necessarily the answer to the ever expanding complexity of classified holdings and networks. The captain’s pain, we think, would better be served by a situated software application – a portal or other tool that could create, for his unit and those like his serving throughout the far reaches of the Long War, the functional appearance of centralization, at least insofar as this means the kind of “one stop shopping” for products in support of ongoing operations that seems to be needed. Frankly, we believe that the creation and maintenance of such a portal is a clear role for a reachback function – if such an element can identify and stay current with the constantly shifting needs of those on the ground.....

---

They can tweak all the databases or systems they want. Unfortunately, until an intel analyst is able to turn the information and data into something insightful, it will still be difficult to operationalize the content to a usable product. As we jam data into the meat grinding funnel that is pushed down for the intel user, context of the information still remains lost (in many cases). The more complex the churn is-- the more experienced the intel analyst must be. I'm still seeing weakness on the human side. I'm also still seeing push intel that does not related to the initial PIRs. Sad, but it is cooler to buy and play with more toys than fix an HR problem.

Quote:

---

Originally Posted by S2MSSI

They can tweak all the databases or systems they want. Unfortunately, until an intel analyst is able to turn the information and data into something insightful, it will still be difficult to operationalize the content to a usable product. As we jam data into the meat grinding funnel that is pushed down for the intel user, context of the information still remains lost (in many cases). The more complex the churn is-- the more experienced the intel analyst must be. I'm still seeing weakness on the human side. I'm also still seeing push intel that does not related to the initial PIRs. Sad, but it is cooler to buy and play with more toys than fix an HR problem.

---

Agree and that is what I was targeting in my earlier comments. The blender only mixes what get's put into it; it does not determine quality.

Tom

Quote:

---

Originally Posted by S2MSSI

...The more complex the churn is-- the more experienced the intel analyst must be. I'm still seeing weakness on the human side. I'm also still seeing push intel that does not related to the initial PIRs. Sad, but it is cooler to buy and play with more toys than fix an HR problem.

---

True to a certain degree. I've spoken on the board several times before regarding the quantity vs quality issue for the HUMINT side. But this applies to analysts a bit differently.

In the Army at least, the Intelligence Analyst MOS has always been the largest one in the MI field. The problem isn't numbers, its training. Like everyone else in the Big Army, training was always focused on conventional maneuver warfare. This has shifted significantly in the past couple of years, but the main weakness of training remains an over-reliance on systems and a failure to adequately focus on critical thinking skills. The new FM 2-33.4 Intelligence Analysis actually goes into quite a bit of detail on critical thinking, the reasoning process and analytic pitfalls. But from what I've seen in the field, little of that is impressed into the heads of the cherry analysts at the schoolhouse and the focus is on using the spectrum of available tech tools instead. The smart ones with natural talent do rise to the occasion, but that is not an acceptable method for developing enough capable analysts to support current ops.

I'll never forget the time in Afghanistan, when I asked the NCOIC of the terrain team at Bagram for analytic products for certain key locations in-country I was in the process of assessing. He offered me some great overhead imagery, with key points clearly demarcated and a nice glossy unit logo on it. But when I pressed and asked again for an analytic terrain product, he just looked at me like I had something growing out of my forehead.

Jedburgh, isn't one of the standing principles that intelligence analysis is a lossful process? You take a large volume of data and distil it into a subset of high value data and that becomes a work product as a deliverable to a specific audience. So, is it that the deliverable needs to have specific rules or methods for the audience to manipulate the rules?

As I reflect on just how many reports I read, corrected, opined on and contributed to, one thing still sticks in my mind today; the analysts and desk officers at home replying with either ‘of significant intelligence value’ or ‘of no intelligence value’.

The often-overlooked common denominator was the source’s spin on the report. Typical OB for example is in and of itself boring, especially to an individual who has never been to country X, or perhaps barely understands why all those numbers are relevant today, but weren’t a week ago.

I would be very concerned about openly sharing ‘raw or analyzed intel’ when the user is Joe the ordinary soldier, with no skills in comprehending or applying said in a country and/or culture he/she doesn’t fully understand in the first place.

Quote:

---

Originally Posted by Stan

The often-overlooked common denominator was the source’s spin on the report.

---

Fully agree. The source's spin is huge but so is the collectors reference of atmospherics in the transmission. How the information was derived clarifies the response. If the individual was in a group setting, who was in the group, who could observe and influence the group, was the person under duress, alone, scared, having cha, trying to gain favor, ...

The disposition of the setting will twist what was said in multiple directions.

Quote:

---

Originally Posted by S2MSSI

The disposition of the setting will twist what was said in multiple directions.

---

I'm glad you brought that up. The variables are endless, but must be addressed in the final report if one is to gain anything from the raw data. I was recently asked by a 'brand spankin' new' DATT "just how many reports are accomplished on average in a typical shop"?

While avoiding the obvious response (quality vs quantity), I explained that some two-man shops in the middle of a civil war cranked out 10 to 12 a day (situation and priority driven), and others with 13 folks grinding away, less than two substantive reports.

His puzzled look was all I needed :rolleyes:

Quote:

---

Originally Posted by Jedburgh

I'll never forget the time in Afghanistan, when I asked the NCOIC of the terrain team at Bagram for analytic products for certain key locations in-country I was in the process of assessing. He offered me some great overhead imagery, with key points clearly demarcated and a nice glossy unit logo on it. But when I pressed and asked again for an analytic terrain product, he just looked at me like I had something growing out of my forehead.

---

This anecdote captures almost perfectly what was in essence the point in my earlier post. Annotated data is just that--annotated data--not intelligence. High quality logic and critical thinking skills are not a given in people--they have to be groomed, and as with race horses, all the grooming in the world may not produce a Derby Winner.

Quote:

---

Originally Posted by Stan

I'm glad you brought that up. The variables are endless, but must be addressed in the final report if one is to gain anything from the raw data. I was recently asked by a 'brand spankin' new' DATT "just how many reports are accomplished on average in a typical shop"?

While avoiding the obvious response (quality vs quantity), I explained that some two-man shops in the middle of a civil war cranked out 10 to 12 a day (situation and priority driven), and others with 13 folks grinding away, less than two substantive reports.

His puzzled look was all I needed :rolleyes:

---

Hmmm you can always point to Brazza 93-95 and just say, none.:D

Quote:

---

Originally Posted by Tom Odom

Hmmm you can always point to Brazza 93-95 and just say, none.:D

---

Damn ! That was my trump card :D
I don't suppose it would then be blatantly obvious where that 'none' got him :o

:wry:

Quote:

---

Originally Posted by Stan

Damn ! That was my trump card :D
I don't suppose it would then be blatantly obvious where that 'none' got him :o

---

Hmmm promoted to full Colonel as Al Gore's advisor on Africa, n'est pas? After of course, I relieved him in Goma at the request of the JTF-A Commander and demand of the French...:wry:

Thanks for the feedback on the article. To clarify, the intent of the article was not to imply that raw information is the end all and be all. Additionally, thorough scrutiny and analysis of intelligence will always be necessary.

The problem I sought to highlight is the fact that there is simply no overarching information network collection and sharing for units in Iraq and Afghanistan. The Army is oftentimes accused of fighting the last war, but in essence we are fighting the last battle because we aren't keeping information and properly passing it on to units that will operate in a region 2-3 years from now. Think about how many times we have fought the "battle of Baghdad, Mosul, Falluja, Baqubah, etc..." What is happening, is units redeploy back home and all their information is lost and irretrievable to units who operate in that very same region 6-12 months down the road. This is compounded when units are being shifted continuously. All of that intelligence from the mundane (terrain, trafficability, key terrain, census info) to key data (important sheikhs, imams, cell network) has to all be researched and mapped out again.

Quote:

---

Originally Posted by tim

Thanks for the feedback on the article. To clarify, the intent of the article was not to imply that raw information is the end all and be all. Additionally, thorough scrutiny and analysis of intelligence will always be necessary.

The problem I sought to highlight is the fact that there is simply no overarching information network collection and sharing for units in Iraq and Afghanistan. The Army is oftentimes accused of fighting the last war, but in essence we are fighting the last battle because we aren't keeping information and properly passing it on to units that will operate in a region 2-3 years from now. Think about how many times we have fought the "battle of Baghdad, Mosul, Falluja, Baqubah, etc..." What is happening, is units redeploy back home and all their information is lost and irretrievable to units who operate in that very same region 6-12 months down the road. This is compounded when units are being shifted continuously. All of that intelligence from the mundane (terrain, trafficability, key terrain, census info) to key data (important sheikhs, imams, cell network) has to all be researched and mapped out again.

---

Tim,

I have no doubt about that issue but it is tied to larger IM issues, exacerbated by the need for RIP/TOA. We work it here (JRTC) but I understand where you are coming from. That said, I would frame the discussion in terms that were more operations-centric (even though they support both intel and ops). In any case, your article is well worth the read.

In a larger sense, we always do this. I am in the business of lessons learned and as a history guy I will tell you that is often an oxymoron when it comes to the US Army.

Best

Tom

PS

Keep writing!

Quote:

---

Originally Posted by Tom Odom

:wry:

Hmmm promoted to full Colonel as Al Gore's advisor on Africa, n'est pas? After of course, I relieved him in Goma at the request of the JTF-A Commander and demand of the French...:wry:

---

Vraiment...pas une petite merde ;)
I was probably more relieved with your decision than all the French in Bukavu and Goma combined :D

Hi Tim !

Quote:

---

Originally Posted by tim

Thanks for the feedback on the article.

...All of that intelligence from the mundane (terrain, trafficability, key terrain, census info) to key data (important sheikhs, imams, cell network) has to all be researched and mapped out again.

---

I'm surprised that both the Army and DIA have not in 3 or 4 years fixed this. Then again, Tom and I were looking at each other after hearing of 'intel folks' in the camp (thinking quietly that someone heard our requests for assistance finally)... "the Cav has arrived" ...only to find one of the most disinterested people on creation in the midst of 800,000 refugees.

Welcome and please do continue to write !

Regards, Stan

Quote:

---

Originally Posted by tim

....What is happening, is units redeploy back home and all their information is lost and irretrievable to units who operate in that very same region 6-12 months down the road. This is compounded when units are being shifted continuously. All of that intelligence from the mundane (terrain, trafficability, key terrain, census info) to key data (important sheikhs, imams, cell network) has to all be researched and mapped out again.

Quote:

---

Originally Posted by Tom Odom

....tied to larger IM issues, exacerbated by the need for RIP/TOA.

---

---

It is a huge problem when units don't do a proper intel handover. Blame can go both ways: sometimes the departing unit hands over everything and the gaining unit files and forgets. Sometimes the departing unit is in such haste to unass the AO that they don't bother covering important details or transferring important data. Often it is a mix of both to varying degrees, with many other RIP/TOA issues impacting the intel handover. Then there are the instances in which there is no handover at all; one unit moves in after the other has departed with zero transfer of anything other than responsibility.

Quote:

---

Originally Posted by Tom Odom

....I would frame the discussion in terms that were more operations-centric.

---

Tom, I agree 100%. When it comes to the handover affecting intel transfer it is really an operational issue, not an intel issue. Like damn near everything else, it comes down to a leadership issue. At what level that leadership issue comes to rest is dependent upon the situation at hand.

Certainly operational necessity can severely impact the ability of units to execute an effective hand-off, but in reality far less often than for which the excuse is made.

One thing that has always bothered me when entering any new environment is that it is exactly that, an environment. Made up of all the social interactions, politics, etc which serve to make it unique.

There really should not be so much difficulty in at least maintaining the outline or shadow of that particular social enclave considering that travel companies have been putting together travel guides for such a long time.

A travel guide is not supposed to be able to tell you everything you need to know about an area but it sure enough gets you into the vicinity of understanding what and who to look for / to.

I know several individuals who have made a good effort in providing exactly this kind of thing and we can only hope that would continue.

Gathering and holding information isn't the problem, getting it where it is available to those who need it is where the answers can be found.

Quote:

---

Originally Posted by Ron Humphrey

One thing that has always bothered me when entering any new environment is that it is exactly that, an environment. Made up of all the social interactions, politics, etc which serve to make it unique.

There really should not be so much difficulty in at least maintaining the outline or shadow of that particular social enclave considering that travel companies have been putting together travel guides for such a long time.

A travel guide is not supposed to be able to tell you everything you need to know about an area but it sure enough gets you into the vicinity of understanding what and who to look for / to.

I know several individuals who have made a good effort in providing exactly this kind of thing and we can only hope that would continue.

Gathering and holding information isn't the problem, getting it where it is available to those who need it is where the answers can be found.

---

Ron, the living, breathing entity characterization is spot on.

Although the transfer of data is extremely important, in current ops relationships are truly critical. When an area is handed off between units, a poorly handled transfer (or no transfer at all) of HUMINT sources can be catastrophic to the new unit's initial collection efforts. By the time they rebuild the network, it is time to rotate home - and the cycle can start all over again. Even in the best of circumstances a transfer of sources has to be handled carefully.

Aside from HUMINT sources, the local relationships of sheikhs, mukhtars, imams, key civic leaders, local security/police commanders etc. with the US forces is absolutely critical. The handoff should include face-to-face meets with the most important of these living breathing represenations of the unit's area of operations. A "travel guide" type listing of local key leaders will not effectively replace a couple of days spent conducting a joint meet-and-greet with the locals by the necessary outgoing and incoming personnel. But on the other hand, its better than nothing.

Quote:

---

Originally Posted by Jedburgh

It is a huge problem when units don't do a proper intel handover. Blame can go both ways: sometimes the departing unit hands over everything and the gaining unit files and forgets. Sometimes the departing unit is in such haste to unass the AO that they don't bother covering important details or transferring important data. Often it is a mix of both to varying degrees, with many other RIP/TOA issues impacting the intel handover. Then there are the instances in which there is no handover at all; one unit moves in after the other has departed with zero transfer of anything other than responsibility.

Tom, I agree 100%. When it comes to the handover affecting intel transfer it is really an operational issue, not an intel issue. Like damn near everything else, it comes down to a leadership issue. At what level that leadership issue comes to rest is dependent upon the situation at hand.

Certainly operational necessity can severely impact the ability of units to execute an effective hand-off, but in reality far less often than for which the excuse is made.

---

One would think that after doing these handoffs for almost 5 years now that CENTCOM, CFLCC/3rd Army, MNF-I or some senior HQ would have developed an SOP with a complete checklist for RIP/TOA activities. It ought to be much like one's installatiuon clearance papers--you ain't leaving until you get all the boxes signed off--and an adequate data handoff ought to be one off the biggest boxes on the checklist.

But, wm, that would make sense. And we know that that cannot possibly be permitted.