Distributed Networked Systems Theory and Practice

I'm currently working on a project trying to get its arms around what makes something a "Distributed networked system". For the most part its like pornography - people recognize it when they see it, and tend not to agree to what someone else thinks is "it" when THEY see it.

Anybody out there doing work in this area in the other services? FCS? Air force info to the cockpit stuff? Marine "Dragon of the month"?

At what point do you depart from "network enabling an platform-centric, but fundamentally heirarchical operating environment" and get beyond the lightning bolts and pixie dust? Since we are not going to just lead with both feet from heirarchical platform based (though web-enabled) is there a tipping point where you suddenly become "network-centric"and get to enjoy all the cool supposed benefits? (and just add a bunch of network vulnerabilites to a platform-centric operating environment for marginal gains? Is "web-enabled platform - centric" all we really need?

Then there is the "enterprise -wide system of system engineering" stuff... Talk about culture changes!

Well, anybody with thoughts on the topic, cynical and jaded, or wide - eyed and bushy tailed, I'd be interested!

Thanks,

Paul

Hi Paul,

Can you clarify what context you are looking at? I've seen the term "Distributed networked system" used in sociology, anthropology, computer science, and in a whole slew of other areas.

Marc

The context is operational command and control of naval "distributed networked systems to perform tactical missions (Anti-submarine warfare, Anti-Air warfare, Anti-Surface Warfare, Electronic attack/defense, etc.)

All the "way we want to do things" slides show a grunch of unmanned autonomous systems with a plethora of lighning bolts interconnecting them (I see the same sort of diagrams associated with FCS, Sea Dragon, various constellation sof aircraft linked back to the AOC, etc.

When you look at what is being bought, and how we engineer things, and how the culture we have tends to think about things, what we seem unable to get past is a "legacy archetecture" of platforms and C2 nodes in a heirarchical architecture "net-enabled" to a limited extent.

This provides a limited benfit that is difficult to quantify "increased situational awareness" but with all the potential vulnerabilities (to network attack, but also in time expended achiving the assumed beneficial "increased situational awareness" - aside: in nearly a dozen networked experiments and excercises I've yet to see people people spend LESS time dealing with RFIs and SA issues, the more network resources you add - always more, the "just one more look and I might find the vulnerability to destroy the Death Star" syndrome).

Attached is a slide depicting the two extremes - Business practice indicates that its typically not possible to transform an organization from one into another - you stand up new organizations (spin off subsidieries or create organizational structures) to get from one to another. Even in sales force transition in business thee seems to be an "out with the old in with the new" recapitalization, rather than incremental retraining. The desire in teh NAvy appears to be incremental, evolutionary transition from one to another - but is that possible? Desireable?

Are there business examples or examples in the other domains you mention of incremental vice "revolutionary" transitions from heirarchical to "distributed" "operating environmemts"

Edit - the unreadable captions on the two sides are "Hierarchical System architecture - structured information flows and feedback" and the other "Network-centric System architechture - adaptive information flows and ad hoc feedback"

If you're looking at systems I can give you some ideas (and heck being the greedy academic I am throw money at me and I'll let you into my lab where we have built the Internet...)

I'm not military but I stayed at a Holiday Day Inn Express...

You're mixing some terms and we can get into specifics if you wish. Distributed in the computer systems world is a buzz word. It takes on a couple connotations. Distributed can mean processing, data, communications, accessibility, etc.. Network enabling can mean "webbifying" an application, putting telemetry on the network, linking entities, or creating applications that share the medium like instant messenger. Distributed and hierarchical when used together create "systems paralysis" where one goal is nearing the polar and contradiction of another stated goal. See some examples that follow. .

If we're talking about network centric warfare using computers for command and control and for situational awareness there are some web2.0 technologies that can accomplish some of this...

Consider tools like Twitter. It allows for instantaneous sentences to pop up asking questions or making statements. Talk about situational awareness. That does mean somebody has to be watching it though.

Consider tools like blogging. If commanders were using a blogging tools rather than a flat file hierarchical tool for writing their reports standard context and heuristic software could search for key phrases and items nearing on instantly. The difference between key search phrases (Google) and heuristic searching (Hakia) is awesome.

To get an idea where in systems design, and how we can get it wrong so many times draw an org chart on a piece of paper. Now draw a network chart of information flows. I'm willing to bet you did both in a 2D space. Information networks are not even 3D they are 4D spaces just like the Internet. Distributed does not taking one silo (vertical data information set) of information and breaking it up into many silo's. You want NO silo's.

Change the terms around a little and start thinking about entities. I'm not into euphemisms an entity is processing, data, communications, rules, etc.. and becomes a "node" that is interlinked to other "nodes" in a "web" rather than flat structure. No entity has the entire information set but any few entities can create a missing entity. This is a key concept for distributed data and processing but you might notice communications is missing.

Similarly high speed, high volume, high availability (the triad) are three key concepts in distributed computing. (We're way up there in concepts so I know there are contrarian examples for experts reading this)... The rule is you can have any two of the triad.

These examples are examples of application space changes rather than actual distributed networks. If you want to get to the systems engineering aspects we have to add the actual communications medium and that is a sticky wicket. You already mentioned security but what kind of security? The MaConahay, Schou Ragsdale model provides us with five key security services (confidentiality, integrity, availability, non-repudiation, and authentication). There are two other dimensions to the model but let's think about this first.

If you have a fully distributed network the architecture makes availability nearing on a zero issue. Common security practices like encryption take care of non-repudiation, confidentiality, and integrity. Fill the distributed network with throttled white noise and ensure a strong authentication mechanism at access points and security becomes less of an issue. The problem is that in the Navy the granularity of an entity is an aircraft carrier or submarine when it should be a sailor. If I take out an entity that has substantial functional components then you've got issues.

Closing items: This knowledge space/area is filled with vendors who haven't got a clue, academicians who don't understand the requirements, and users who are only looking for a solution. When in doubt go more towards the root of the system. For me this area of endeavor has been a no go for research. No money, no interest, owned by the corporations nobody will look at it other than as fundamental theoretical research. If you have any questions let me know, if I'm off base let me know too.

Thanks Selil - a lot of food for thought there.

On the "distributed, networked" as buzzword - thee is plenty of that, and cutting through it to get at the "Meat" part of what I am after.

Terms like this tend to be "code word labels" on complex, sets of ideas that individual groups have spent a lot of times thinking through. And unfortunatley disparate groups (deliberatly or out of ignorance) use the same code-word labels. Software development has it easy with nice sequential numbers (or at lesat used to before things like "Vista Extreme Gamers Special Edition").

In this case "Distributed netorked system" is a code-word label on a family of ideas to disaggregate sensors, weapons, and command from platforms. That is the "far-term" ppt answer. The near term effect has been to allow platforms to share info at the individual watchstander level without having to go up the chain of command and back down again - and allow platforms to be "net enabled" to communicate in more robust ways than radio telphone and teletype.

This disaggregation desire stems from two sources, one that increasing aggregation of every possible capability into every platform have nearly made ships unaffordable (and leaving a capability off opens up the decision - maker for being pilloried if a ship is ever lost becasue of the missing system not being there...) The other being a more formal argument based on the work of Capt Wayne Hughes (ret) pointing out a "tactical instability" problem when you have a few "super capable" ships - putting too many eggs in one basket. A notion Adm Cebrowski tried to address with the "Streetfighter" concept that got corrupted and mismanaged into the debacle that is LCS.

So looking ahead (partly in response to what may well be a short run of LCS, what is the "right way" (not necessarily exculding an LCS-like thing, but recognizing it is woefully insufficient) How do you disaggregate (distribute) sensors, weapons, information fusion and other C2 functions?

We are already spending billions on the network part, so that is a fait accompli, at the CVN level and "higher up" but the question is open as to how much of that needs to trickle down to the escorts. And then there are the "disadvantaged users" like subs, who suffer from Mother Natures presumption of laws that do not allow radio waves to go very far through water...(How dare she! Though I heard it put this way - The greatest desire of the silent service is... comms at speed and depth, and the greatest fear of the sub skipper is... comms at speed and depth).

So a network is coming, is it the right one to allow 'distribution' - and distribution of "what". Do you need "fighting ships" if you can achieve effective disaggregation leaving you a fleet of trucks for "stuff"? What sort of "stuff"? SOme systems are being directed top down (PBD 753) but are they the "right ones" and as this has a big effect on "Undersea Superiority systems" how do get joint leverage and buy in for the 99 and 44/100 % Navy mission of ASW ("undersea superiority system" is code word label for "ASW 2.0").

Anyway that is a pretty stream of concsiousness response, but hopefully further explains the context of what the issue is...

Like I said I will think some more about your response and hopefully post some more organizaed thoughts after a few more (I hope) repsonses...

Just a quick question as I've got a class starting in a minute. Is there a public document showing data flow? I know there is a NAVSEA document listed on Amazon but something that show's the totality (or unclassified) of data channels and directions? The Navy has always been a proponent of "systems of systems" design.

The "totality of data channels and directions" in theoryis contained in the "Architecture framework" a set of things that look like INTEL CPU design documents on rolls or 3 foot by N foot plotter paper. Those trying to engineer it from teh top down are doing some interesting things, but the scope is just incomprehensible.

Here is a public website looking at the problem:

http://homepages.nildram.co.uk/~esuk...daf/dodaf.html

This discusses the full range of "views" or ways to try to diagram all the interactions between entities, but tends to be in the context of trying to "top-down engineer" in the traditional sense.

If you believe the math (or at least how the implications of the math are spun) regarding comples adaptive systems this is a dead end. Other alternatives have been suggested. PArt of my question is wondering if anybody has any experience with the alternatives.

A good systems engineer can talk about the different metaphors and models. In large scale design you can use esoteric methods like object oriented design from software to summarize ideas and entities. I finished reading the website and it looks like they're using the Booch Rumbaugh (gang of four) UML (object oriented design method). It also looks like they're mixing a few things like taxonomies for the XML.

You said something interesting in "Top Down Engineer"... A common enterprise design failure is "Top Down Design" without out bottom up requirements gathering. When you collect from the bottom you can summarize and get consistency.

I wouldn't consider the scope of what you're trying to describe as a Complex Adaptive System (CAS) because all of the entities are known, they all have form, function, input, output, and more importantly they are linked to a logistics route, communications route, and leadership route (an open framework but still a framework). So they are known unlike truly complex systems that the entities can only be percieved after they've acted and their results seen (not actually required but often seen with CAS).

Most of my experience has been designing and implementing and upgrading metropolitan area networks (MCIWorldcom CPE/DTE), world wide area networks (Sun Microsystems SWAN & SUNMC), consulting on follow the sun help desks, and such. And, no I'm not trying to get a consulting gig you can't afford me. I might be able to help you though. I'm just trying to figure out if you're trying to get a handle on the current architecture or solve the architecture issues that are already there.

Note that that websie is an example of the sort of research people are doing into the Department of Defense architecture framework. There are all sorts of folks oput there tryin to "touch the elephant".

Interesting remark re Complex adaptive systems...The "Architechture" - particularly as defined in the DoD literature is not a CAS. The current architecture, given the lack of such engineering based design is. That is a big part of the problem - the current 'architecture" (or lack thereof) is based not on system engineering, but social networking. When you look at what the architecture framework is trying to do, you get into Joint Mission Essential Task LIsts and the information flows to perform the tasks, metrics and requirements for decision tools, etc etc. Currently "getting the job done" is a function of "good officers figuring out what they need to do and doing it" - which comes not by studying the METLS (though they are what any training he might have gotten prior to arriving in his job are based on) - but by interacting with the other people he works with and for.

Why is this not "good enough"? WHy can't people learn jobs by "OJT" as they have in the Navy for centruries? Becasue if you look at what the Navy is doing in its various future visions and what its leaders say they are for, and how the training comunity is leveraging computer aided training, is that human decision-making to accomplish the tasks in the METLS is being reduced to "flow charts" of decision-making. In effect turning human operators into performs of pseudo-code instruction sets to optimize efficiency and reduce mistakes. IF you then read the visions for Unmanned Autonomous vehicels, the vision is for swarms and constellations of vehicles, in the land, sea and air, exhibiting "creativeness" through exploitation of emergent responses and adaptive behaviors.

In essence the goal is to make humans operate like machines, and machines to think like humans. Maybe there is a "meet in the middle" in there, but sure seems bassackward to me... What I am involved in is trying figure out how all that "architechture stuff" meets the Task LIst stuff and the current mindset of task improvement through social networking, vs task improvement through systems engineering.

The system engineering stuff is being "Done in a vacuum"and I see a complete disconnect with any way to "evolve" the current ways of doing business toward this new way of thinking.

We have a potential opportunity (if the frickin shipyards can build it affordably build them) to "spin off a subsudiary" in the LCS community to look at ways of finding the right balance of "human as machine" and "human as natures best pattern recognition engine and CAS" and working with distributing machines (a long way from being "creative" if ever...Ray Kurzweil notwithstanding ;) )

So the problem boils down to, given a small mission set (Say ASW), with a Mission essential task list, this framework for architechting "Really really complicated systems" (with some potentially complex subelements) and a focus on top down engineering, flow chart and metric-based decision-making, and "human out of harm's way" desires; how do you reconcile that with the current state of "the Fleet" as a social network based, run to the sound of the guns, command by negation, culturally independant, adaptive and ad hoc (CMM level 1 "Heroic -effort based) organization?

I just finished a short paper on the subject of Network-Centric Warfare. As far as it being revolutionary, I doubt it. As far as I can see, it is mearly a way of further integrating C2ISR much like signal flags, mounted messengers, the telegraph, wireless communications, and the advent of the computer did in the past. So far, much of what I've read indicates that much of the superior performance attributed to networked organizations can be filtered away when you notice that they fought with superior weapons and after long periods of uninterrupted training.

So, I feel I have a good understanding of the networked stuff, but what I don't get is the comparison to "system-centric" organizations. As far as I can tell, we're still just acquiring new systems.

Quote:

---

Originally Posted by frmoss

I just finished a short paper on the subject of Network-Centric Warfare. As far as it being revolutionary, I doubt it. As far as I can see, it is mearly a way of further integrating C2ISR much like signal flags, mounted messengers, the telegraph, wireless communications, and the advent of the computer did in the past. So far, much of what I've read indicates that much of the superior performance attributed to networked organizations can be filtered away when you notice that they fought with superior weapons and after long periods of uninterrupted training.

So, I feel I have a good understanding of the networked stuff, but what I don't get is the comparison to "system-centric" organizations. As far as I can tell, we're still just acquiring new systems.

---

Good points. One thing about the technology factors of the network is that with increased reliability, speed, and scalability instead of communication conduits being parasitic on decision cycles the network becomes a force multiplier.

As an example. The World Trade Organization riots/demonstrations would not have been possible without highly trained technologists (and adaptive self interest in the users) integrating the social mechanisms (social networks) of the demonstrators into the technology networks (communication). The demonstrators uses C4IT/C4ISR so effectively it prolonged the demonstration (in Seattle) and nullified police attempts at ending the demonstration. That would be an example of a real world network centric insurgency using hybridized communications.