Open Source Intelligence (OSINT): merged thread

[SWJED]

From Secrecy News (FAS Blog) - Army Seeks to Catalyze Open Source Intelligence.

A new U.S. Army Field Manual is intended to advance the development and use of open source intelligence (OSINT), which is intelligence that is derived from publicly available data legally obtained.

"The value of publicly available information as a source of intelligence has... often been overlooked in Army intelligence operations. This manual (pdf) provides a catalyst for renewing the Army's awareness of the value of open sources; establishing a common understanding of OSINT; and developing systematic approaches to collection, processing, and analysis of publicly available information."

The growing military appreciation of open source intelligence arises from the ever-increasing quality of public sources and the evident limitations of traditional classified approaches...

[bismark17]

This is a good manual. While there is a lot of jargon that is DOD or DA specific the content is very interesting and applicable to Domestic L.E. Intel. It seems like the Army FMs just keep getting better. They have come a long way from the old FM 7-8 or 7-70 manuals that I used to read.

The manual is well laid out and organized. It's very relevant and useful for end users and not just "doctrine" specialists who are just skimming for the latest buzz words to throw out to impress the boss. It includes some very useful appendixes that add to the overall content. Interested readers who are not up on the PIR/SIR/IR methodology may find it useful to develop some knowledge of the Army Intel doctrine or process before digging in.

[marct]

Interested readers who are not up on the PIR/SIR/IR methodology may find it useful to develop some knowledge of the Army Intel doctrine or process before digging in.

Any recommended readings on getting up to speed in he PIR/SIR/IR area?

Marc

[Stan]

Humint, we used to call it globally. That included OSINT, but only a tinny part of HUMINT....can't hardly wait for Tom to log-on with his two cents.

OSINT, more or less what the more than 250 ARMY NCOs and Officers did for years (that would be 12 for me). Some were freebies at the local cocktail curcuits, others you had to work for.

Without a suitable background and language, you're up against a difficult task. Written press has its own slang, and context can get way outta hand if you have no idea what they are saying.

45 weeks in California does not get you there. Reading open source materials is one thing, but did you understand the overall context from say, an African's point of view ?

Regards, Stan

[bismark17]

I learned a lot from a great friend over coffee just outside of the gates of Ft. Lewis. Until my eyes started drifting over to more interesting sights in the coffee shop......The new Army Intel Analysis manual is pretty good. FM34-3??? There is also a great one on Intel Analysis in Urban environments. I will send you the links.

The great thing is all of this stuff is on the net. At the risk of making the military guys mad at me, I don't think the methodology is that critical. It's good and works well for them in their realms. But they have to do deal with a very complex variety of potential situations and areas of operation that civilians don't have to worry about.

I don't have to worry about having a pre built process to set up an intel shop in Baghdad or North Korea, I just have to know what's going on in that barber shop where my players are hanging in as an example.

[marct]

Hi John,

I learned a lot from a great friend over coffee just outside of the gates of Ft. Lewis. Until my eyes started drifting over to more interesting sights in the coffee shop......The new Army Intel Analysis manual is pretty good. FM34-3??? There is also a great one on Intel Analysis in Urban environments. I will send you the links.

Thanks, I'd appreciate it.

The great thing is all of this stuff is on the net. At the risk of making the military guys mad at me, I don't think the methodology is that critical. It's good and works well for them in their realms. But they have to do deal with a very complex variety of potential situations and areas of operation that civilians don't have to worry about.

I don't have to worry about having a pre built process to set up an intel shop in Baghdad or North Korea, I just have to know what's going on in that barber shop where my players are hanging in as an example.

Good point. Actually, I'm also wondering how much of it can be adapted for Anthropological fieldwork. I can just see the reactons ofsome of my colleagues when I assign a US Army manual as a textbook .

Marc

[Steve Blair]

I'd appreciate those links as well, if you don't mind. Some of this bleeds over into my own theories of insurgencies and may have some historical links as well.

[Jedburgh]

I learned a lot from a great friend over coffee just outside of the gates of Ft. Lewis. Until my eyes started drifting over to more interesting sights in the coffee shop......The new Army Intel Analysis manual is pretty good. FM34-3??? There is also a great one on Intel Analysis in Urban environments. I will send you the links.

The great thing is all of this stuff is on the net...

As far as I know, the current FM 2-91.4 Intelligence in Support to Operations in the Urban Environment (AKO log-in required), dated Jun 05, is not available on the general 'net - if that is the one you are referring to.

FM 34-3 Intelligence Analysis, dated Mar 90, is readily available - but that version is dated. There was a much-improved draft dated May 98, but that isn't generally available. A new version has been in the works for a long time....

[Jedburgh]

...The new Army Intel Analysis manual is pretty good. FM34-3???...

...FM 34-3 Intelligence Analysis, dated Mar 90, is readily available - but that version is dated. There was a much-improved draft dated May 98, but that isn't generally available. A new version has been in the works for a long time....

A Draft - Not For Implementation copy of the new FOUO FM 2-33.4 Intelligence Analysis in now available for review to those with an AKO log-in. It has significantly changed from the previous versions that I referred to in my earlier post.

[BILL]

thank you santa: You are the most rare "OSINTithus" very rare breed of suspect authenticity. We really have OSINT guys? Ok that is both scary and hopeful at the same time. Hopeful because we maynot be as far behind as I thought. Scary concept! You read OSINT and did what? Our OSINT CAPABILITIES are awesome, impresses me continually. We have been active for 1 1/2 yrs. But we treat the "Trooper" like an infant baby. We outline his areas of Interest, names, concepts, an open feed he can change/adjust at will. We feed him a steady diet 24/7 of things of INTEREST to him. A big steady vacuume of the Internet, sites, RSS, news, blogs, forums, docs, pdfs, etc. We make him the expert on real time OSINT intel, in his area, on a very pragmatic level. And they chose and run the ops etc. The end user directs the feed, amount, quality, targets, definitions, etc. We have developed a new concept, "Paradigm Intelligence", a very fancy inductive reasoning system. We have found we can pull data from a closed cell with it, it is anthropology based, as it looks more at actions and less at words. I hope to get some sense of what you DID. Thanks Bill

Humint, we used to call it globally. That included OSINT, but only a tinny part of HUMINT....can't hardly wait for Tom to log-on with his two cents.

OSINT, more or less what the more than 250 ARMY NCOs and Officers did for years (that would be 12 for me). Some were freebies at the local cocktail curcuits, others you had to work for.

Without a suitable background and language, you're up against a difficult task. Written press has its own slang, and context can get way outta hand if you have no idea what they are saying.

45 weeks in California does not get you there. Reading open source materials is one thing, but did you understand the overall context from say, an African's point of view ?

Regards, Stan

[Stan]

thank you santa: You are the most rare "OSINTithus" very rare breed of suspect authenticity. We really have OSINT guys? Ok that is both scary and hopeful at the same time. Hopeful because we maynot be as far behind as I thought. Scary concept! You read OSINT and did what?

Bill,
First and foremost, welcome to the SWC!
One normally does not commence with personal attacks and skepticism for his/her first post. As time permits you, please go here and introduce yourself.

Yes, Bill, we really did, and do have OSINT ‘guys’ out there. Forgive me if I don’t feel good about sharing my past and training with someone who has ‘N/A’ listed as his/her background in the user profile.

Our OSINT CAPABILITIES are awesome, impresses me continually. We have been active for 1 1/2 yrs. But we treat the "Trooper" like an infant baby. We outline his areas of Interest, names, concepts, an open feed he can change/adjust at will. We feed him a steady diet 24/7 of things of INTEREST to him. A big steady vacuume of the Internet, sites, RSS, news, blogs, forums, docs, pdfs, etc. We make him the expert on real time OSINT intel, in his area, on a very pragmatic level. And they chose and run the ops etc. The end user directs the feed, amount, quality, targets, definitions, etc. Thanks Bill

I was not referring to our typical soldiers in non-intelligence roles and capacities; rather intel-related MOSs and being poorly prepared to do same. My initial intelligence training is more than two decades old, so I’ll assume there have been significant advances in OSINT (such as the internet, blogs and forums).

Regards, Stan

[Jedburgh]

CSS, 9 Apr 08: Open Source Intelligence: A Strategic Enabler of National Security

Open Source Intelligence (OSINT) has gained considerable prominence in recent years. Traditionally, intelligence has been the business of discovering secrets using a closed system of collection and analysis. Key sources included human intelligence (HUMINT), signals intelligence (SIGINT), and imagery intelligence (IMINT). Although open sources were frequently used in the intelligence process, their value was seen as secondary. Classified information was deemed more valuable and often more credible. The systematic acquisition of non-classified information was rarely seen as an intelligence priority.

Today, OSINT’s importance is widely acknowledged. It is estimated that OSINT provides between 80 and 95 per cent of the information used by the intelligence community. There is a growing debate within and between the various branches of government and the national security apparatus on how best to use open source information. However, the role and potential of OSINT remain a matter of some dispute. OSINT’s advocates believe it to be the answer to many of today’s intelligence challenges. They call for a new intelligence paradigm marked by a preponderance of open source information and a trans-sector intelligence collaboration that includes a broad network of public and private actors. But there are others who warn against treating OSINT as more than a component of a continuing, all-source approach to intelligence-gathering and analysis.....

[Jedburgh]

...The growing military appreciation of open source intelligence arises from the ever-increasing quality of public sources and the evident limitations of traditional classified approaches...

The new OSINT FM is only the latest in the evolution of the military approach to OSINT. Within the US Army we've had the INSCOM OSINT Handbook, published in May 03, and the Joint Military Intelligence Training Center Open Source Exploitation: A Guide for Intelligence Analysts, published in Apr 04 (this is the second edition, I never saw the first). Both had relatively limited distribution.

As far as I know, NATO was ahead of the game with a formal manual published in Nov 01, the NATO OSINT Handbook and the NATO OSINT Reader, published in Feb 02.

For those with AKO access, the OSINT Knowledge Center contains copies of the presentations from a number of intelligence assets and their approaches to OSINT as put out during a series of OSINT conferences held in 05 and 06.

[Stan]

I learned a lot from a great friend over coffee just outside of the gates of Ft. Lewis. Until my eyes started drifting over to more interesting sights in the coffee shop.

Hi Bismark !
That's not too far fetched. Most of Humint today is little more than common sense and eyes open. DIAs course goes farther with lessons and experiences over the last 25 years. That "book" at AKO is much the same. The folks that put that together are former Army NCOs and Officers who once worked for DIA.

Regards, Stan

[Beowulf]

We used to joke that OSINT was just an excuse to surf the niprnet, check email, buy crap from amazon, etc.

We started to get reports that some bad guys in the area were going to meet with a reporter, and we were tracking that, trying to find out where and when.

We missed the meet, but got lots of reports about it after it happened. Sure enough a few days later an analyst from higher emails us some pics from Reuters showing a group of BG's meeting in a typical sit down. Weapons, unifroms ak's, commo etc. The main BG's had their faces covered but some did not.

OSINT, actually proved useful, and will continue to do so.

This is especially true considering the BG's effective use of IO, along with the "good" of getting their message, videos, etc out there they get increased exposure and may end up showing us something we need to know.

[bismark17]

It's amazing how much information can be lifted from social networking sites like myspace, facebook or just googling the target's email address. Obviously, these things are more relevant on the CONUS L.E. side.

[Stan]

Deception and bias are of particular concern. These sources may also convey one message in English for US or international consumption and a different non-English message for local or regional consumption. It is important to know the background of open sources and the purpose of the public information in order to distinguish objective, factual information from information that lacks merit, contains bias, or is part of an effort to deceive the reader.

Bill Mera has some good strong points on this. My experiences in Sub-Sahara and Estonia often led to confrontation at the embassy. Reporting variations were at times near opposite and worse, irreconcilable. Those with greater rank, simply jammed their version down the tube.

The FM is certainly better than some of the "100 dash Ones" from my days in the early 80's.