Objectives

The canonical use case for STREAMS is the cyber-infrastructure to be deployed for the SKA, an international project to develop the world’s largest radio observatory of the next decades. The computational requirements of the SKAO are estimated to be 3-4 orders of magnitude larger than the current state of the art, making this one of the most challenging parts of the project. While a number of processing steps will be investigated, these can be roughly separated into two main categories: buffer-based batch processing and streaming near real-time processing.

Thales LAS France will also conduct a pilot study on STREAMS, aiming at prototyping a large number of possible architectures, to experiment with different ways of scaling (up and out), and to validate new COTS technologies for Terabit networking and real-time high performance computing as well as new algorithms for future radar systems.

Our research agenda is organized around 4 technical / technological objectives:

• massive parallelization of processing algorithms while complying with real-time constraints and exploring the specific capabilities of processors (specialized computing cores, variable arithmetic precision, etc.);
• integration of these algorithms with very high bandwidth data ingest and non-disruptive synchronization of data acquisition, processing, distribution and storage processes in a distributed environment;
• integration of new data analytics methodologies, based on deep learning and targeting different parts of the workflow: filtering / compression of raw data, signal processing and analysis of reduced data. In this area, two sub-topics will be particularly targeted: coupling of training resources with the time critical pipeline and integration of the inference process into it, ensuring compliance with real-time constraints;
• study and validation of strategies for incremental upgrades and long-term maintenance of computing platforms.