The Product Generation architecture for the GOES-R Ground System

Gerald Dittberner, Satya Kalluri, Allan Weiner, Michael Blanton, Anderson Tarpley

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Abstract

    The GOES-R Ground System (GS) will produce a much larger set of products with higher data density than previous GOES systems. This requires considerably greater compute and memory resources to achieve the necessary latency and availability for these products. Over time, new algorithms could be added and existing ones removed or updated, but the GOES-R GS cannot go down during this time. To meet these GOES-R GS processing needs, the Harris Corporation will implement a Product Generation (PG) infrastructure that is scalable, extensible, extendable, modular and reliable. The primary parts of the PG infrastructure are the Service Based Architecture (SBA) and the Distributed Data Fabric (DDF). The SBA is the middleware that encapsulates and manages science algorithms that generate products. The SBA is divided into three parts, the Executive, which manages and configures the algorithm as a service, the Dispatcher, which provides data to the algorithm, and the Strategy, which determines when the algorithm can execute with the available data. The SBA is a distributed architecture, with services connected to each other over a compute grid and is highly scalable. This plug-and-play architecture allows algorithms to be added, removed, or updated without affecting any other services or software currently running and producing data. Algorithms require product data from other algorithms, so a scalable and reliable messaging is necessary. The SBA uses the DDF to provide this data communication layer between algorithms. The DDF provides an abstract interface over a distributed and persistent multi-layered storage system (memory based caching above disk-based storage) and an event system that allows algorithm services to know when data is available and to get the data that they need to begin processing when they need it. Together, the SBA and the DDF provide a flexible, high performance architecture that can meet the needs of product processing now and as they grow in the future.

    Original languageEnglish (US)
    Title of host publicationRemote Sensing System Engineering III
    Volume7813
    DOIs
    StatePublished - 2010
    EventRemote Sensing System Engineering III - San Diego, CA, United States
    Duration: Aug 2 2010Aug 2 2010

    Other

    OtherRemote Sensing System Engineering III
    CountryUnited States
    CitySan Diego, CA
    Period8/2/108/2/10

    Fingerprint

    products
    Processing
    Architecture
    Infrastructure
    Data storage equipment
    data products
    Distributed Architecture
    Necessary
    Data Communication
    plugs
    Middleware
    Caching
    Storage System
    availability
    Large Set
    resources
    Latency
    communication
    grids
    Availability

    Keywords

    • GOES-R
    • Information technology
    • IT Architecture
    • Remote sensing
    • Service based architecture

    ASJC Scopus subject areas

    • Applied Mathematics
    • Computer Science Applications
    • Electrical and Electronic Engineering
    • Electronic, Optical and Magnetic Materials
    • Condensed Matter Physics

    Cite this

    Dittberner, G., Kalluri, S., Weiner, A., Blanton, M., & Tarpley, A. (2010). The Product Generation architecture for the GOES-R Ground System. In Remote Sensing System Engineering III (Vol. 7813). [781305] https://doi.org/10.1117/12.859902

    The Product Generation architecture for the GOES-R Ground System. / Dittberner, Gerald; Kalluri, Satya; Weiner, Allan; Blanton, Michael; Tarpley, Anderson.

    Remote Sensing System Engineering III. Vol. 7813 2010. 781305.

    Research output: Chapter in Book/Report/Conference proceedingConference contribution

    Dittberner, G, Kalluri, S, Weiner, A, Blanton, M & Tarpley, A 2010, The Product Generation architecture for the GOES-R Ground System. in Remote Sensing System Engineering III. vol. 7813, 781305, Remote Sensing System Engineering III, San Diego, CA, United States, 8/2/10. https://doi.org/10.1117/12.859902
    Dittberner G, Kalluri S, Weiner A, Blanton M, Tarpley A. The Product Generation architecture for the GOES-R Ground System. In Remote Sensing System Engineering III. Vol. 7813. 2010. 781305 https://doi.org/10.1117/12.859902
    Dittberner, Gerald ; Kalluri, Satya ; Weiner, Allan ; Blanton, Michael ; Tarpley, Anderson. / The Product Generation architecture for the GOES-R Ground System. Remote Sensing System Engineering III. Vol. 7813 2010.
    @inproceedings{691746d772d24f9c80bc8c93ebec1284,
    title = "The Product Generation architecture for the GOES-R Ground System",
    abstract = "The GOES-R Ground System (GS) will produce a much larger set of products with higher data density than previous GOES systems. This requires considerably greater compute and memory resources to achieve the necessary latency and availability for these products. Over time, new algorithms could be added and existing ones removed or updated, but the GOES-R GS cannot go down during this time. To meet these GOES-R GS processing needs, the Harris Corporation will implement a Product Generation (PG) infrastructure that is scalable, extensible, extendable, modular and reliable. The primary parts of the PG infrastructure are the Service Based Architecture (SBA) and the Distributed Data Fabric (DDF). The SBA is the middleware that encapsulates and manages science algorithms that generate products. The SBA is divided into three parts, the Executive, which manages and configures the algorithm as a service, the Dispatcher, which provides data to the algorithm, and the Strategy, which determines when the algorithm can execute with the available data. The SBA is a distributed architecture, with services connected to each other over a compute grid and is highly scalable. This plug-and-play architecture allows algorithms to be added, removed, or updated without affecting any other services or software currently running and producing data. Algorithms require product data from other algorithms, so a scalable and reliable messaging is necessary. The SBA uses the DDF to provide this data communication layer between algorithms. The DDF provides an abstract interface over a distributed and persistent multi-layered storage system (memory based caching above disk-based storage) and an event system that allows algorithm services to know when data is available and to get the data that they need to begin processing when they need it. Together, the SBA and the DDF provide a flexible, high performance architecture that can meet the needs of product processing now and as they grow in the future.",
    keywords = "GOES-R, Information technology, IT Architecture, Remote sensing, Service based architecture",
    author = "Gerald Dittberner and Satya Kalluri and Allan Weiner and Michael Blanton and Anderson Tarpley",
    year = "2010",
    doi = "10.1117/12.859902",
    language = "English (US)",
    isbn = "9780819483096",
    volume = "7813",
    booktitle = "Remote Sensing System Engineering III",

    }

    TY - GEN

    T1 - The Product Generation architecture for the GOES-R Ground System

    AU - Dittberner, Gerald

    AU - Kalluri, Satya

    AU - Weiner, Allan

    AU - Blanton, Michael

    AU - Tarpley, Anderson

    PY - 2010

    Y1 - 2010

    N2 - The GOES-R Ground System (GS) will produce a much larger set of products with higher data density than previous GOES systems. This requires considerably greater compute and memory resources to achieve the necessary latency and availability for these products. Over time, new algorithms could be added and existing ones removed or updated, but the GOES-R GS cannot go down during this time. To meet these GOES-R GS processing needs, the Harris Corporation will implement a Product Generation (PG) infrastructure that is scalable, extensible, extendable, modular and reliable. The primary parts of the PG infrastructure are the Service Based Architecture (SBA) and the Distributed Data Fabric (DDF). The SBA is the middleware that encapsulates and manages science algorithms that generate products. The SBA is divided into three parts, the Executive, which manages and configures the algorithm as a service, the Dispatcher, which provides data to the algorithm, and the Strategy, which determines when the algorithm can execute with the available data. The SBA is a distributed architecture, with services connected to each other over a compute grid and is highly scalable. This plug-and-play architecture allows algorithms to be added, removed, or updated without affecting any other services or software currently running and producing data. Algorithms require product data from other algorithms, so a scalable and reliable messaging is necessary. The SBA uses the DDF to provide this data communication layer between algorithms. The DDF provides an abstract interface over a distributed and persistent multi-layered storage system (memory based caching above disk-based storage) and an event system that allows algorithm services to know when data is available and to get the data that they need to begin processing when they need it. Together, the SBA and the DDF provide a flexible, high performance architecture that can meet the needs of product processing now and as they grow in the future.

    AB - The GOES-R Ground System (GS) will produce a much larger set of products with higher data density than previous GOES systems. This requires considerably greater compute and memory resources to achieve the necessary latency and availability for these products. Over time, new algorithms could be added and existing ones removed or updated, but the GOES-R GS cannot go down during this time. To meet these GOES-R GS processing needs, the Harris Corporation will implement a Product Generation (PG) infrastructure that is scalable, extensible, extendable, modular and reliable. The primary parts of the PG infrastructure are the Service Based Architecture (SBA) and the Distributed Data Fabric (DDF). The SBA is the middleware that encapsulates and manages science algorithms that generate products. The SBA is divided into three parts, the Executive, which manages and configures the algorithm as a service, the Dispatcher, which provides data to the algorithm, and the Strategy, which determines when the algorithm can execute with the available data. The SBA is a distributed architecture, with services connected to each other over a compute grid and is highly scalable. This plug-and-play architecture allows algorithms to be added, removed, or updated without affecting any other services or software currently running and producing data. Algorithms require product data from other algorithms, so a scalable and reliable messaging is necessary. The SBA uses the DDF to provide this data communication layer between algorithms. The DDF provides an abstract interface over a distributed and persistent multi-layered storage system (memory based caching above disk-based storage) and an event system that allows algorithm services to know when data is available and to get the data that they need to begin processing when they need it. Together, the SBA and the DDF provide a flexible, high performance architecture that can meet the needs of product processing now and as they grow in the future.

    KW - GOES-R

    KW - Information technology

    KW - IT Architecture

    KW - Remote sensing

    KW - Service based architecture

    UR - http://www.scopus.com/inward/record.url?scp=77958121699&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=77958121699&partnerID=8YFLogxK

    U2 - 10.1117/12.859902

    DO - 10.1117/12.859902

    M3 - Conference contribution

    SN - 9780819483096

    VL - 7813

    BT - Remote Sensing System Engineering III

    ER -