What we really need is a better crystal ball

[davidbfpo]

Thanks to Rick M for drawing attention to the forecast energy shortage soon (from the Energy Security thread:http://council.smallwarsjournal.com/...?t=7629&page=2) and a shorter edition placed above (Post 12): "Why haven't more people in government, and the oil industry itself, seen this particular crisis coming? Why aren't they acting proactively to soften the blow?"

The Leggett article is far wider in analysis: http://www.independent.co.uk/opinion...t-1766551.html

I quote (shortened):

The same question can be asked, with hindsight, of the bonus cultists who gave us the credit crunch... Gillian Tett of the Financial Times, a trained anthropologist, describes the effort made by the banking elite at "ideological domination" ahead of the financial crash. Elites do this to maintain power, she explains. They decide what is talked about and what is not. There was a major "social silence" around the epidemic growth of derivatives....This is exactly what I see going on among my old friends in the oil industry when it comes to weighing their assets....One of the few financiers who saw the credit crunch coming said derivatives were financial hydrogen bombs built by 26-year-olds with MBAs. Here is another set of similarities and differences. The oil crunch is an economic hydrogen bomb. But it is being built by men close to retirement.

So the search for a crystal ball involves generations and reducing 'silence'.

[Rick M]

Thanks for your support, David.

The following was posted on the Energy Security thread this morning, but since it evolved out of a study which has everything to do with intelligence (in more than one sense of the word) and little to do with energy, I will offer it here as well.
I welcome your observations, particularly if someone spots something which is felt to be incorrect.

Review and application of SSI study, “Known Unknowns: Unconventional Strategic Shocks in Defense Strategy Development” (Nathan Freier, Nov. 2008).

The Strategic Studies Institute (SSI) at the US Army War College has issued many stimulating research papers in recent years, several of which deal with energy security issues.
Nathan Freier’s recent SSI paper does not focus on energy issues.
Rather, the central purpose of his study is to present a paradigm for the examination of potential strategic shocks.

I recently summarized the key points of Nathan Freier’s “Known Unknowns” and then applied his paradigm to emerging energy security issues, primarily the phenomenon known as “peak oil” and its corollary, export decline.

The central point of my application of Freier’s paradigm to the issue of peak oil may be summed up thus: there are well-established trend-lines which point to impending energy security concerns (witness yesterday's warning from the IEA, #12 above).
Meanwhile, industry and government officials largely deny these concerns.
This ongoing situation provides a classic and real-time illustration of an evolving potential shock.
It is precisely the sort of situation which Freier says needs to be noted and scrutinized by military analysts.

This analysis was posted at Energy Bulletin this morning:
http://www.energybulletin.net/node/49779

[davidbfpo]

An interesting, (UK) short review entitled 'How vital were Cold War spies?', which has a variety of views from insiders: http://news.bbc.co.uk/1/hi/world/europe/8184338.stm

davidbfpo

[Rick M]

The IEA released its most recent WEO last November.
Its Executive Summary is available for free downloading, but it does not contain any of the graphs.

Two of these graphs are particularly telling, and after much fiddling I was able to post them on the Energy Security thread:
http://council.smallwarsjournal.com/...?t=7629&page=3

[Bill Moore]

http://www.bbc.co.uk/news/technology-14841018

A study, based on millions of articles, charted deteriorating national sentiment ahead of the recent revolutions in Libya and Egypt.

While the analysis was carried out retrospectively, scientists say the same processes could be used to anticipate upcoming conflict.

The system also picked up early clues about Osama Bin Laden's location

Maybe this has some potential, a lot more at:

http://www.uic.edu/htbin/cgiwrap/bin...view/3663/3040

News is increasingly being produced and consumed online, supplanting print and broadcast to represent nearly half of the news monitored across the world today by Western intelligence agencies. Recent literature has suggested that computational analysis of large text archives can yield novel insights to the functioning of society, including predicting future economic events. Applying tone and geographic analysis to a 30–year worldwide news archive, global news tone is found to have forecasted the revolutions in Tunisia, Egypt, and Libya, including the removal of Egyptian President Mubarak, predicted the stability of Saudi Arabia (at least through May 2011), estimated Osama Bin Laden’s likely hiding place as a 200–kilometer radius in Northern Pakistan that includes Abbotabad, and offered a new look at the world’s cultural affiliations. Along the way, common assertions about the news, such as “news is becoming more negative” and “American news portrays a U.S.–centric view of the world” are found to have merit.

[davidbfpo]

I'd forgotten this thread.

Having immersed myself in data-mining, data collection and similar subjects I remain a critic of this approach - both in the domestic and overseas expeditionary contexts.

Sticking to the 'Small Wars' context and the strong possibility they will happen in the developing world - where will the data come from? Even allowing for the spread of mobile phones and computers there are large areas of the world which are electronic / data deserts, albeit maybe ones with a sparse population.

Jobu asked this question a few months ago in a RFI:http://council.smallwarsjournal.com/...ad.php?t=13252.

Data collection is fraught with problems, so is access to such places and how much expertise exists outside?

A 'crystal ball' with very little data and insight is what we have now and for sometime to come.

If our enemies reduce their electronic exposure and do not travel widely the IT solutions so beloved of late will simply not work.

[Entropy]

Bill,

I'm skeptical of computational approaches, though further research is warranted.

[Bill Moore]

Posted by davidbfpo,

Having immersed myself in data-mining, data collection and similar subjects I remain a critic of this approach - both in the domestic and overseas expeditionary contexts.

As you should, to date most efforts have produced little of value, but on the other hand (and to my knowledge they are not published publicly) there have been examples of successes that show what is possible now, and of course more will be possible in the future.

Of course a lot data isn't digitized, and even if it is it isn't readily available so it be fused with other sets of data, or it isn't structured, and the list of challenges goes on and on, but over time many of those challenges can be addressed.

Artificial intelligence will never replace the power of the human brain (I might be an exception) in our life times, but if you look at a cyborg capability where it simply augments our ability to see patterns and potentially identify links between events and actors not previously visible then I think there is considerable value added.

Additionally, and I haven't seen this discussed yet in any of my readings, to maximize these capabilities we will have to re-engineer our information/intelligence collection processes to stream line the process from collection to transition to structured data that is available to be fused by those with access with these programs. Note that this study used open source data from the media, imagine if they were able to also fuse open source with classified data.

We're definitely not there yet, but I think we're on the road to a "better" crystal ball that will only serve to augment the most important element in the process, which is the human mind. The danger is we'll get some people in the system (seen it before) that wants a system that will replace humans, so hopefully we can keep those idiots marginalized.