Open Source Intelligence (OSINT): merged thread

[bismark17]

It's amazing how much information can be lifted from social networking sites like myspace, facebook or just googling the target's email address. Obviously, these things are more relevant on the CONUS L.E. side.

[sgmgrumpy]

Remove if already posted.

On July 16-17, Chris Pallaris, Sean Costigan and I attended the DNI Open Source Intelligence (OSINT) Conference in Washington, DC. The most important message that emerged from the two day session was that OSINT is no longer "nice to have", rather it is an absolute imperative in our contemporary world of complex, asymmetric and non-linear security threats.

Read here descriptions of the panel sessions and download selected power point presentations.

[Dominique R. Poirier]

In my own opinion, a good specialist armed, once and for all, with solid real world knowledge in politics in general and about a given country or region can make surprisingly good analysis, and even accurate forecasts in politics and strategy on the sole basis of open sources.
However, it is understood that, in that case, analysis and forecast are likely to loose in accuracy with time since real world knowledge are not exempt from changes and evolutions. The pace of this observable decrease in accuracy depends of the observed country or region and of the significant political events or upheavals (publicly known or not) happening in them.

I see that correct deductions and forecasts exclusively based upon open sources owe to two ways of analyzing open sources which are: the formal analysis of the open sources per se; and perception management analysis. Other factors linked to the specifics of the observed country (when the object of our attention happens to be a country in particular) will provide the analyst a basis which will help detecting deception attempts. I’ll probably not tell anything new if I say that if the formal content of a given media known for its inclination to practice deception is therefore of relatively poor value, then we may consider it otherwise when analyzing it under the angle of perception management.

What happens when it comes to Army intelligence operations (a field about which I bloody don't know anything)? I admit that things might be ruled otherwise, then; especially if the enemy is not a nation or a country with a government and its own national media and is, or is not, acting as proxy.

So I’ll read this manual with great interest.

[sgmgrumpy]

Open Source Intelligence Familiarization Documents

The intent of this single-page document is to share basic knowledge about Open Source Intelligence (OSINT). From OSS net

Version 2, May 2006

[Jedburgh]

CRS, 5 Dec 07: Open Source Intelligence (OSINT): Issues for Congress

Open source information (OSINT) is derived from newspapers, journals, radio and television, and the Internet. Intelligence analysts have long used such information to supplement classified data, but systematically collecting open source information has not been a priority of the U.S. Intelligence Community (IC). In recent years, given changes in the international environment, there have been calls, from Congress and the 9/11 Commission among others, for a more intense and focused investment in open source collection and analysis. However, some still emphasize that the primary business of intelligence continues to be obtaining and analyzing secrets.

A consensus now exists that OSINT must be systematically collected and should constitute an essential component of analytical products. This has been recognized by various commissions and in statutes. Responding to legislative direction, the Intelligence Community has established the position of Assistant Director of National Intelligence for Open Source and created the National Open Source Center. The goal is to perform specialized OSINT acquisition and analysis functions and create a center of excellence that will support and encourage all intelligence agencies.

The effort has been only underway since late 2005 but the Center is up and running, and providing support, including training, to OSINT professionals throughout the Intelligence Community. Administrative mechanisms are in place to ensure that there is a comprehensive community-wide open source effort. It appears, however, to some observers that not all agencies have as yet made comprehensive commitments to acquiring and using open source information, nor that the ODNI has taken sufficient steps to ensure that open sources are appropriately exploited. Observers suggest that congressional oversight of the OSINT process might provide insight into current progress as well as identify areas that need modification. A particular focus of congressional interest might be potential tradeoffs between classified and open source collection to ensure that needed information is obtained in the best and most cost-effective manner. Proponents maintain that this approach helps to ensure that agents and expensive surveillance systems are focused on obtaining information that is being actively hidden.

The collection and analysis of OSINT information will be ultimately judged by its contribution to the overall intelligence effort. Collecting information from open sources is generally less expensive and less risky than collection from other intelligence sources. The use of OSINT may result not only in monetary savings but also in less risk than utilizing sensitive technical and human sources. OSINT can also provide insights into the types of developments that may not be on the priority list for other systems or may not be susceptible to collection through other intelligence approaches — innovative applications of new technologies, shifts in popular attitudes, emergence of new political and religious movements, growing popular discontent, disillusionment with leadership, etc. Supporters of OSINT maintain that the future contribution of the Intelligence Community will be enhanced by its ability to provide detailed information and incisive analyses of such developments. This report will be updated as new information becomes available.

Complete 27 page report at the link.

[JeffC]

CRS, 5 Dec 07: Open Source Intelligence (OSINT): Issues for Congress

Complete 27 page report at the link.

I just finished reading it during my ferry commute home. I need some time to think about the implications, but I'm concerned about the lack of commitment to OSINT on the part of so many in leadership positions.

I see that DHS is farming out it's OSINT duties to the SITE Institute. I'm inclined to support that model for the entire IC - i.e., farm out OSINT collection and analysis to small niche firms that have the skills, the people, and the agility to do it properly.

[JeffC]

Remove if already posted.





Read here descriptions of the panel sessions and download selected power point presentations.

Wow. This is a treasure trove. Thanks, grumpy.

[milnews.ca]

Mod Squad: If this belongs elsewhere, feel free to move w/my thanks for your patience.

DHS has issued a Privacy Impact Assessment (PDF at DHS site - PDF at Scribd.com) highlighting some of the web sites they'll be monitoring to assess threats to the 2010 Olympics.

In spite of my bitterness at not making it into the Appendix , I've developed a page with their selected list o' links here.

[AdamG]

An excellent example of how astute observers can take the innocuous and start piecing together puzzles.

It is, according to a new book, part of the hidden reality behind the Pentagon’s classified, or “black,” budget that delivers billions of dollars to stealthy armies of high-tech warriors. The book offers a glimpse of this dark world through a revealing lens — patches — the kind worn on military uniforms.

http://www.nytimes.com/2008/04/01/sc...prod=permalink

Then again, when the media gets it wrong, they really get it wrong -

http://trekmovie.com/2011/05/06/gema...laden-mission/

[GeorgePBurdell]

I argue that when it comes to open source collection, the easiest target is the United States by a very wide margin.

[davidbfpo]

Leah Farrell's blog consistently provides insight, aided by being Australian too and from a police background:http://allthingscounterterrorism.com/

A few weeks ago she posted a series of photos, obtained from a jihadi website, on an IMU training camp in Pakistan:http://allthingscounterterrorism.com...an-apparently/

After some input and open source research she's posted an update:http://allthingscounterterrorism.com...ing-camp-pics/

The IMU crop up irregularly, especially due to their German links and maybe there is nothing of note here. It is IMHO an example of what can be done from open sources.

[AdamG]

For the Peanut Gallery

If you have anyone in your chain who can take the Open Source course at Leavenworth, send them - it's well worth the time & effort.

[bourbon]

To Defeat Terrorists, Start Using the Library, by Scott Helfstein. Bloomberg, Aug 30, 2011.

The information glut that marks the 21st century is evidenced in some unexpected places. Last month, my organization, the Combating Terrorism Center at West Point, released a report that sharply disputed conventional wisdom about terrorism along the Afghanistan-Pakistani frontier.

The report argued that the Haqqani Network, a border- spanning tribal group with deep ties to Pakistan’s government, had been more influential than the Taliban in aiding al-Qaeda’s rise.

How did we support this thesis, which has vast implications for reconciliation efforts in the region as well as for the distribution of U.S. military aid? Not with data culled from clandestine operations in Pakistan’s tribal areas or from Osama bin Laden’s computer hard drive. The report was based on the public statements and writings of individual extremists over the past 30 years. Rather than ferreting out secret information, researchers merely took extremists at their voluminous word.

It seems terrorists, too, are susceptible to the syndrome known as Too Much Information.

[Bill Moore]

bourbon, thanks for sharing this article. While in no way meaning to degrade or insult the excellent work our intelligent analysts do, intelligence analysis based on classified information (which is frequently just as questionable as unclassified information) often lacks critical context, especially historical context. And as in any other organization ideas tend to go viral and become accepted wisdom, even when they're incorrect. It is harder to correct the record when the idea is based on classified that not all the analysts (or subject matter experts) have access to.

OSINT is an underused discipline in my view. I think part of the blame for our shortfalls is in our process, where commanders want dumbed down summaries on targets without context, so the intell community (not all of it, but definitely on the pointy end of the tactical spear) becomes target fixated, and they miss the bigger picture, a picture that may indicate their targeting process needs to be adjusted to have the desired effect.

[carl]

Bill and Bourbon:

It occurs to me that most all civilian PHD experts on various areas and peoples of the world get to be PHDs by studying open sources. Those same PHDs are then sometimes turned to by the military in order to pass on the knowledge they learned from open sources. I can see local commanders wanting dumbed down reports but why don't the higher ups put more emphasis on the open sources the civilian experts used to get to be experts?

[davidbfpo]

Hat tip to a "lurker" for this Swiss think tank's fifteen minute podcast:http://www.isn.ethz.ch/isn/Digital-L...g=en&id=154122 and the speakers bio:http://www.i-intelligence.eu/about/team/

The summary says:

The growing availability of open source information has profoundly impacted intelligence agencies and how they have operated over the last 10 years. In today’s podcast, Chris Pallaris discusses the risks and opportunities that open source intelligence poses. He also describes what makes a good intelligence analyst in the open source era.

[AdamG]

The National Security Agency just declassified a hefty 643-page research manual called Untangling the Web: A Guide to Internet Research (PDF) that, at least at first, doesn't appear all that interesting. That is, except for one section on page 73: "Google Hacking."

http://news.yahoo.com/search-spy-goo...140400536.html

[OUTLAW 09]

An excellent image forensics tool.....for open source analysis work.

Reveal Project @RevealEU
Did you use our #image #forensics tool?
Do you have feedback for us?
Let us know!
Or try it now:
http://reveal-mklab.iti.gr/reveal/

Stop Fake @StopFakingNews
Here is your weekly update on Kremlin disinformation efforts
http://www.stopfake.org/en/kremlin-w...ruary-9-2017/#

[OUTLAW 09]

Stop Fake @StopFakingNews

Tracking a Mysterious Missile Launcher Inside an Information War

http://www.stopfake.org/en/tracking-...ormation-war/#

[OUTLAW 09]

2015 "The Rise of iWar" report from @SSInow mentions @EliotHiggins' work and @bellingcat's vehicle tracking project.
http://www.strategicstudiesinstitute...fm?pubID=1292#