SMALL WARS COUNCIL
Go Back   Small Wars Council > Military Art & Science Applied > Intelligence

Intelligence What do we know, need to know, and how do we get there?

Reply
 
Thread Tools Display Modes
Old 11-09-2005   #1
SWJED
Small Wars Journal
 
SWJED's Avatar
 
Join Date: Sep 2005
Location: Largo, Florida
Posts: 3,988
Default Open Source Intelligence (OSINT): merged thread

9 Nov. New York Times - Intelligence Center Is Created for Unclassified Information. Excerpt follows:

"Top intelligence officials announced on Tuesday the creation of a new agency, the Open Source Center, to gather and analyze information from the Web, broadcasts, newspapers and other unclassified sources around the world."

"The premise of the center, announced as part of the restructuring of the nation's intelligence agencies by the director of national intelligence, John D. Negroponte, is that some critical information to understand threats to national security requires neither spies nor satellites to collect."

"This 'open source' information can include anything from sermons broadcast from radical mosques in the Middle East to reports in the provincial Chinese press of possible avian flu outbreaks. Such material has often been undervalued by government policymakers, in part because it lacks the cachet of information gathered by more sensitive methods, intelligence officials said..."

Moderator's Note

Six threads have been merged here. Note there is separate one on Google Earth, which remains untouched.

Last edited by davidbfpo; 01-13-2013 at 12:29 AM. Reason: Add note
SWJED is offline   Reply With Quote
Old 11-10-2005   #2
KenDawe
Registered User
 
Join Date: Nov 2005
Location: Kent, WA
Posts: 9
Default

Re-inventing the Early Bird? Or institutionalizing it?
(And is it true that the Early Bird has been politicized in the last 5 years?)
KenDawe is offline   Reply With Quote
Old 11-10-2005   #3
Jedburgh
Council Member
 
Join Date: Oct 2005
Posts: 3,097
Default

Quote:
Originally Posted by KenDawe
Re-inventing the Early Bird? Or institutionalizing it?
(And is it true that the Early Bird has been politicized in the last 5 years?)
A bit more than the Early Bird.
Quote:
Originally Posted by DNI
In executing the National Strategy for Open Source, the Center will be a comprehensive Intelligence Community Center focused on all open sources, to include the Internet, databases, press, radio, television, video, geospatial data, photos, and commercial imagery. The Center's functions will include collection/acquisition; open source analysis and research; centralized training; and facilitation of government-wide access and exploitation.
Extensive details are available in a couple of documents that have been uploaded to AKO KC Intelligence Reference Files, for those with an account.
Jedburgh is offline   Reply With Quote
Old 11-11-2005   #4
DDilegge
Guest
 
Posts: n/a
Default Neither...

Quote:
Originally Posted by KenDawe
Re-inventing the Early Bird? Or institutionalizing it?
(And is it true that the Early Bird has been politicized in the last 5 years?)
I read the Early Bird (Current News) every morning. It is simply coverage of major news items and Op-Ed pieces that address issues associated with the DoD. No opinion is offered, simply copies of news articles. While one might argue that the Early Bird only posts articles deemed "favorable", I would argue the opposite (for the most part) as I have seen news and opinions that slam the DoD on certain issues. The purpose here is to inform DoD leadership and others on what is being reported in the mainstream news media.

One criticism I have of the Early Bird is that they did away with the Supplement several months ago. The supplement offered a whole slew of additional reporting that might not be deemed "headline news" but otherwise offered deeper insight and background on important issues. The reason given for dropping the supplement was required manpower resources so the main Early Bird could go from a five-day to a seven-day format.

All that said, the SWJ tries to fill in and add depth to the Early Bird by posting Daily News Links. If you look at the address line (url) you can see our take-off (inside joke) on the Early Bird....

Referencing the new Open Source Intelligence Center - whole different ballgame - it will (or should) mine all open source data - not just headline news. Moreover, it will add analysis, not simply a regurgitation...
  Reply With Quote
Old 04-19-2006   #5
SWJED
Small Wars Journal
 
SWJED's Avatar
 
Join Date: Sep 2005
Location: Largo, Florida
Posts: 3,988
Default CIA Mines 'Rich' Content From Blogs

19 April Washington Times - CIA Mines 'Rich' Content From Blogs.

Quote:
... The new Open Source Center (OSC) at CIA headquarters recently stepped up data collection and analysis based on bloggers worldwide and is developing new methods to gauge the reliability of the content, said OSC Director Douglas J. Naquin.

"A lot of blogs now have become very big on the Internet, and we're getting a lot of rich information on blogs that are telling us a lot about social perspectives and everything from what the general feeling is to ... people putting information on there that doesn't exist anywhere else," Mr. Naquin told The Washington Times.

Eliot A. Jardines, assistant deputy director of national intelligence for open source, said the amount of unclassified intelligence reaching Mr. Bush and senior policy-makers has increased as a result of the center's creation in November.

"We're certainly scoring a number of wins with our ultimate customer," said Mr. Jardines, who became the first high-level official in charge of the government's nonsecret intelligence in December.

"I can't get into detail of what, but I'll just say the amount of open source reporting that goes into the president's daily brief has gone up rather significantly," Mr. Jardines said. "There has been a real interest at the highest levels of our government, and we've been able to consistently deliver products that are on par with the rest of the intelligence community."...

The OSC uses powerful computers and software technology to "sift" the Internet for valuable intelligence. It also buys information from commercial databases.

In the past, open-source reports were used mainly by intelligence analysts...
SWJED is offline   Reply With Quote
Old 12-11-2006   #6
SWJED
Small Wars Journal
 
SWJED's Avatar
 
Join Date: Sep 2005
Location: Largo, Florida
Posts: 3,988
Default Seeking Iran Intelligence, U.S. Tries Google

11 December Washington Post - Seeking Iran Intelligence, U.S. Tries Google by Dafna Linzer.

Quote:
When the State Department recently asked the CIA for names of Iranians who could be sanctioned for their involvement in a clandestine nuclear weapons program, the agency refused, citing a large workload and a desire to protect its sources and tradecraft.

Frustrated, the State Department assigned a junior Foreign Service officer to find the names another way -- by using Google. Those with the most hits under search terms such as "Iran and nuclear," three officials said, became targets for international rebuke Friday when a sanctions resolution circulated at the United Nations.

Policymakers and intelligence officials have always struggled when it comes to deciding how and when to disclose secret information, such as names of Iranians with suspected ties to nuclear weapons. In some internal debates, policymakers win out and intelligence is made public to further political or diplomatic goals. In other cases, such as this one, the intelligence community successfully argues that protecting information outweighs the desires of some to share it with the world.

But that argument can also put the U.S. government in the awkward position of relying, in part, on an Internet search to select targets for international sanctions...
SWJED is offline   Reply With Quote
Old 12-11-2006   #7
Jedburgh
Council Member
 
Join Date: Oct 2005
Posts: 3,097
Default

Sounds almost like another urban myth.

Hell, even after the Agency refuses an RFI from State, State has its own Bureau of Intelligence and Research staffed with professional analysts. They are just as good as any analysts at the Agency, although there is a significant difference in type, capability, and availability of collection resources.

I just don't see an RFI of consequence being filled by a cherry FSO in a cubicle when they have professional analysts on-hand. I see this story being leaked in this manner by someone at State as providing a public view of ongoing bureaucratic infighting.
Jedburgh is offline   Reply With Quote
Old 12-11-2006   #8
bismark17
Council Member
 
bismark17's Avatar
 
Join Date: Mar 2006
Location: Seattle, Wa
Posts: 206
Default

My understanding is that State's Intel Unit used to be the Open Source Unit for the OSS. Once the OSS was disbanded they were placed with the State Department.
bismark17 is offline   Reply With Quote
Old 12-11-2006   #9
Tom Odom
Council Member
 
Tom Odom's Avatar
 
Join Date: Oct 2005
Location: DeRidder LA
Posts: 3,949
Default State INR

I can't speak to the OSS connection but my experience with State INR as a historian wading through the 1964 Congo Crisis, as an intelligence analyst in Gulf War 1, and as an intel operator in Zaire and Rwanda was that INR did first rate work.

In many ways that work was possible because INR operated semi-independently of the regional bureaus and was not subject to policy pressures as are the bureaus and the embassies.

Finally my associates at INR stayed in their roles longer and developed greater depth than the analysts at DIA or CIA.

Tom
Tom Odom is offline   Reply With Quote
Old 12-12-2006   #10
CSC2005
Council Member
 
CSC2005's Avatar
 
Join Date: Oct 2005
Location: VA
Posts: 30
Default Inr

It was not by accident that the new head of analysis under the DNI was the former head of State INR. Despite having fewer analysts then the CIA has lunchroom workers, INR has always prided themselves on long term deep analysis by analysts who cover an area for decades. Instead of playing 5-year old soccer "all the crowd the ball" intel or try to be a classified CNN, INR focuses on quality of analysis. A trait the 9/11 report said was "lacking" at CIA and DIA.
CSC2005 is offline   Reply With Quote
Old 12-11-2006   #11
Steve Blair
Moderator
 
Steve Blair's Avatar
 
Join Date: Oct 2005
Location: Montana
Posts: 3,195
Default

Quote:
Originally Posted by Jedburgh View Post
Sounds almost like another urban myth.

Hell, even after the Agency refuses an RFI from State, State has its own Bureau of Intelligence and Research staffed with professional analysts. They are just as good as any analysts at the Agency, although there is a significant difference in type, capability, and availability of collection resources.

I just don't see an RFI of consequence being filled by a cherry FSO in a cubicle when they have professional analysts on-hand. I see this story being leaked in this manner by someone at State as providing a public view of ongoing bureaucratic infighting.
I would tend to agree. This just sounds too "pat" to me. It could also be someone with the proverbial political ax to grind, since the Google spin would be a good way to discredit the accuracy of anyone's name who turned up in said memo.
Steve Blair is offline   Reply With Quote
Old 01-25-2007   #12
SWJED
Small Wars Journal
 
SWJED's Avatar
 
Join Date: Sep 2005
Location: Largo, Florida
Posts: 3,988
Default Army Seeks to Catalyze Open Source Intelligence

From Secrecy News (FAS Blog) - Army Seeks to Catalyze Open Source Intelligence.

Quote:
A new U.S. Army Field Manual is intended to advance the development and use of open source intelligence (OSINT), which is intelligence that is derived from publicly available data legally obtained.

"The value of publicly available information as a source of intelligence has... often been overlooked in Army intelligence operations. This manual (pdf) provides a catalyst for renewing the Army's awareness of the value of open sources; establishing a common understanding of OSINT; and developing systematic approaches to collection, processing, and analysis of publicly available information."

The growing military appreciation of open source intelligence arises from the ever-increasing quality of public sources and the evident limitations of traditional classified approaches...
SWJED is offline   Reply With Quote
Old 01-26-2007   #13
bismark17
Council Member
 
bismark17's Avatar
 
Join Date: Mar 2006
Location: Seattle, Wa
Posts: 206
Default Manual Review

This is a good manual. While there is a lot of jargon that is DOD or DA specific the content is very interesting and applicable to Domestic L.E. Intel. It seems like the Army FMs just keep getting better. They have come a long way from the old FM 7-8 or 7-70 manuals that I used to read.

The manual is well laid out and organized. It's very relevant and useful for end users and not just "doctrine" specialists who are just skimming for the latest buzz words to throw out to impress the boss. It includes some very useful appendixes that add to the overall content. Interested readers who are not up on the PIR/SIR/IR methodology may find it useful to develop some knowledge of the Army Intel doctrine or process before digging in.
bismark17 is offline   Reply With Quote
Old 01-26-2007   #14
marct
Council Member
 
marct's Avatar
 
Join Date: Aug 2006
Location: Ottawa, Canada
Posts: 3,682
Default

Quote:
Originally Posted by bismark17 View Post
Interested readers who are not up on the PIR/SIR/IR methodology may find it useful to develop some knowledge of the Army Intel doctrine or process before digging in.
Any recommended readings on getting up to speed in he PIR/SIR/IR area?

Marc
__________________
Sic Bisquitus Disintegrat...
Marc W.D. Tyrrell, Ph.D.
Institute of Interdisciplinary Studies,
Senior Research Fellow,
The Canadian Centre for Intelligence and Security Studies, NPSIA
Carleton University
http://marctyrrell.com/
marct is offline   Reply With Quote
Old 01-26-2007   #15
Stan
Council Member
 
Stan's Avatar
 
Join Date: Dec 2006
Location: Estonia
Posts: 3,817
Default DIA would be a good start

Humint, we used to call it globally. That included OSINT, but only a tinny part of HUMINT....can't hardly wait for Tom to log-on with his two cents.

OSINT, more or less what the more than 250 ARMY NCOs and Officers did for years (that would be 12 for me). Some were freebies at the local cocktail curcuits, others you had to work for.

Without a suitable background and language, you're up against a difficult task. Written press has its own slang, and context can get way outta hand if you have no idea what they are saying.

45 weeks in California does not get you there. Reading open source materials is one thing, but did you understand the overall context from say, an African's point of view ?

Regards, Stan
Stan is offline   Reply With Quote
Old 01-26-2007   #16
bismark17
Council Member
 
bismark17's Avatar
 
Join Date: Mar 2006
Location: Seattle, Wa
Posts: 206
Default Just an idea

I learned a lot from a great friend over coffee just outside of the gates of Ft. Lewis. Until my eyes started drifting over to more interesting sights in the coffee shop......The new Army Intel Analysis manual is pretty good. FM34-3??? There is also a great one on Intel Analysis in Urban environments. I will send you the links.

The great thing is all of this stuff is on the net. At the risk of making the military guys mad at me, I don't think the methodology is that critical. It's good and works well for them in their realms. But they have to do deal with a very complex variety of potential situations and areas of operation that civilians don't have to worry about.

I don't have to worry about having a pre built process to set up an intel shop in Baghdad or North Korea, I just have to know what's going on in that barber shop where my players are hanging in as an example.
bismark17 is offline   Reply With Quote
Old 12-24-2007   #17
BILL
Council Member
 
BILL's Avatar
 
Join Date: Dec 2007
Posts: 41
Default YOU did OSINT for 12 years

thank you santa: You are the most rare "OSINTithus" very rare breed of suspect authenticity. We really have OSINT guys? Ok that is both scary and hopeful at the same time. Hopeful because we maynot be as far behind as I thought. Scary concept! You read OSINT and did what? Our OSINT CAPABILITIES are awesome, impresses me continually. We have been active for 1 1/2 yrs. But we treat the "Trooper" like an infant baby. We outline his areas of Interest, names, concepts, an open feed he can change/adjust at will. We feed him a steady diet 24/7 of things of INTEREST to him. A big steady vacuume of the Internet, sites, RSS, news, blogs, forums, docs, pdfs, etc. We make him the expert on real time OSINT intel, in his area, on a very pragmatic level. And they chose and run the ops etc. The end user directs the feed, amount, quality, targets, definitions, etc. We have developed a new concept, "Paradigm Intelligence", a very fancy inductive reasoning system. We have found we can pull data from a closed cell with it, it is anthropology based, as it looks more at actions and less at words. I hope to get some sense of what you DID. Thanks Bill
Quote:
Originally Posted by Stan View Post
Humint, we used to call it globally. That included OSINT, but only a tinny part of HUMINT....can't hardly wait for Tom to log-on with his two cents.

OSINT, more or less what the more than 250 ARMY NCOs and Officers did for years (that would be 12 for me). Some were freebies at the local cocktail curcuits, others you had to work for.

Without a suitable background and language, you're up against a difficult task. Written press has its own slang, and context can get way outta hand if you have no idea what they are saying.

45 weeks in California does not get you there. Reading open source materials is one thing, but did you understand the overall context from say, an African's point of view ?

Regards, Stan
BILL is offline   Reply With Quote Received Infraction
Old 01-26-2007   #18
Jedburgh
Council Member
 
Join Date: Oct 2005
Posts: 3,097
Default

Quote:
...The growing military appreciation of open source intelligence arises from the ever-increasing quality of public sources and the evident limitations of traditional classified approaches...
The new OSINT FM is only the latest in the evolution of the military approach to OSINT. Within the US Army we've had the INSCOM OSINT Handbook, published in May 03, and the Joint Military Intelligence Training Center Open Source Exploitation: A Guide for Intelligence Analysts, published in Apr 04 (this is the second edition, I never saw the first). Both had relatively limited distribution.

As far as I know, NATO was ahead of the game with a formal manual published in Nov 01, the NATO OSINT Handbook and the NATO OSINT Reader, published in Feb 02.

For those with AKO access, the OSINT Knowledge Center contains copies of the presentations from a number of intelligence assets and their approaches to OSINT as put out during a series of OSINT conferences held in 05 and 06.

Last edited by Jedburgh; 01-26-2007 at 08:08 PM.
Jedburgh is offline   Reply With Quote
Old 01-26-2007   #19
Stan
Council Member
 
Stan's Avatar
 
Join Date: Dec 2006
Location: Estonia
Posts: 3,817
Default The Coffee Shop

Quote:
I learned a lot from a great friend over coffee just outside of the gates of Ft. Lewis. Until my eyes started drifting over to more interesting sights in the coffee shop.
Hi Bismark !
That's not too far fetched. Most of Humint today is little more than common sense and eyes open. DIAs course goes farther with lessons and experiences over the last 25 years. That "book" at AKO is much the same. The folks that put that together are former Army NCOs and Officers who once worked for DIA.

Regards, Stan
Stan is offline   Reply With Quote
Old 02-08-2007   #20
Beowulf
Council Member
 
Join Date: Nov 2005
Posts: 13
Default

We used to joke that OSINT was just an excuse to surf the niprnet, check email, buy crap from amazon, etc.

We started to get reports that some bad guys in the area were going to meet with a reporter, and we were tracking that, trying to find out where and when.

We missed the meet, but got lots of reports about it after it happened. Sure enough a few days later an analyst from higher emails us some pics from Reuters showing a group of BG's meeting in a typical sit down. Weapons, unifroms ak's, commo etc. The main BG's had their faces covered but some did not.

OSINT, actually proved useful, and will continue to do so.

This is especially true considering the BG's effective use of IO, along with the "good" of getting their message, videos, etc out there they get increased exposure and may end up showing us something we need to know.
Beowulf is offline   Reply With Quote
Reply

Bookmarks

Tags
ctc, infantry weapons, osint, pakistan, terrorism, training camps, uzbekistan, west point

Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump

Similar Threads
Thread Thread Starter Forum Replies Last Post
The David Kilcullen Collection (merged thread) Fabius Maximus Doctrine & TTPs 451 03-31-2016 04:23 PM
Open Source Analysis of WikiLeaks? BobKing Intelligence 15 09-11-2010 05:08 PM
Human Intelligence and Counterintelligence Operations in Iraq- New Book Out Danilo Intelligence 0 11-25-2008 10:28 PM
The Frontiers of Global Security Intelligence SWJED Intelligence 0 08-23-2008 01:49 AM
Terrorist Groups Thesaurus / Open Source Guide SWJED Adversary / Threat 1 11-17-2006 03:19 PM


All times are GMT. The time now is 12:13 PM.


Powered by vBulletin® Version 3.8.9. ©2000 - 2017, Jelsoft Enterprises Ltd.
Registered Users are solely responsible for their messages.
Operated by, and site design 2005-2009, Small Wars Foundation