Adam Gartenberg's Blog

Business Analytics and Optimization, IBM and Social Marketing

InfoSphere Streams drives ground-breaking covert (shhhh!) sensor intelligence


This looks like it might be shaping up to be national security week on the blog this week.  A colleague just pointed me to an article (Adelos Offers Ground-Breaking Sensor Intelligence) detailing how the unique, highly parallel processing capabilities of IBM InfoSphere Streams are providing the analytical power behind a new method of covert intelligence and surveillance gathering.

I'm not going to pretend to understand half of the science behind this system, but from what I can gather, by "detecting, classifying and localizing signals from disturbances along a single strand of commercial-grade fiber-optic cable" (leveraging a novel use of "the electro-optical properties of Rayleigh's Backscatter for a continuous phased array," naturally) it is possible to detect and find meaning for geospatial detection and measurement.  (Note:  Any of you more well versed in geospatial detection/measurement and/or covert intelligence and surveillance gathering should feel free to jump in with a intel-to-english translation.)

Needless to say, to find software to interpret the data and "manage sensor alert messages from the Adelos digital signal processor and... convert User Datagram Protocol data into standardized XML-based messages for distributed situational awareness in a network environment," you can't just head over to your local Best Buy.  

The article describes what they needed, and what they found in InfoSphere Streams:

Adelos would need an architecture capable of embarrassingly* parallel calculations and visualization, a variety of analytical and pattern-recognition utilities, and machine learning for advanced classification of remotely sensed events. The process needed to consume additional sensor streams from other sensor platforms for near real-time classification with higher probability rates based on sensor fusion techniques.

After an extensive review, TerraEchos chose Infosphere Streams, a relatively new product developed by IBM for the U.S. government. Infosphere Streams is a paradigm-shifting computational architecture developed to manage and co-mingle massive data streams of varying spatial, temporal and ontological characteristics. The product has an embarrassingly* parallel flexibility designed to scale to the real-time data challenges inherent to Adelos S4™ and can process specific energy phenomena in difficult pattern-recognition problems that concern commercial and government customers. Using the IBM SPADE programming language, Infosphere Streams users can associate a variety of analytical functions with the product's core and achieve unparalleled performance results....

With all the strategic force protection challenges the Navy faces, S4 provides a world-class application of electro-optical MASINT with advanced computational scalability for managing real-time, mission-critical sensor arrays. Combined with IBM network-aware, high-performance computing classified clouds, Adelos delivers covert surveillance and intelligence necessary to face land, air, water, "man-inside" and cyber-physical threats that define the asymmetric terrorism landscape. As the program continues, TerraEchos and IBM seek to move from a posture of harvesting sensor data to achieving insight through sensor intelligence.


(*Note: The article keeps using the phrase "embarrassingly parallel," which I am taking to mean "incredibly, ridiculously, highly parallel."  I am also not going to comment on the publishing of an article on "covert intelligence and surveillance" in a public journal.)

  Link:  Adelos Offers Ground-Breaking Sensor Intelligence