Explicit state representation and the ATLAS event data model

Theory and practice

Marcin Nowak, David Malon, Peter Gemmeren, Arthur Schaffer, Scott Snyder, Sebastien Binet, Kyle Cranmer

    Research output: Contribution to journalArticle

    Abstract

    In anticipation of data taking, ATLAS has undertaken a program of work to develop an explicit state representation of the experiment's complex transient event data model. This effort has provided both an opportunity to consider explicitly the structure, organization, and content of the ATLAS persistent event store before writing tens of petabytes of data (replacing simple streaming, which uses the persistent store as a core dump of transient memory), and a locus for support of event data model evolution, including significant refactoring, beyond the automatic schema evolution capabilities of underlying persistence technologies. ATLAS has encountered the need for such non-trivial schema evolution on several occasions already. This paper describes the state representation strategy (transient/persistent separation) and its implementation, including both the payoffs that ATLAS has seen (significant and sometimes surprising space and performance improvements, the extra layer notwithstanding, and extremely general schema evolution support) and the costs (additional and relatively pervasive additional infrastructure development and maintenance). The paper further discusses how these costs are mitigated, and how ATLAS is able to implement this strategy without losing the ability to take advantage of the (improving!) automatic schema evolution capabilities of underlying technology layers when appropriate. Implications of state representations for direct ROOT browsability, and current strategies for associating physics analysis views with such state representations, are also described.

    Original languageEnglish (US)
    Article number042024
    JournalJournal of Physics: Conference Series
    Volume119
    Issue number4
    DOIs
    StatePublished - Jul 1 2008

    Fingerprint

    costs
    loci
    maintenance
    physics

    ASJC Scopus subject areas

    • Physics and Astronomy(all)

    Cite this

    Nowak, M., Malon, D., Gemmeren, P., Schaffer, A., Snyder, S., Binet, S., & Cranmer, K. (2008). Explicit state representation and the ATLAS event data model: Theory and practice. Journal of Physics: Conference Series, 119(4), [042024]. https://doi.org/10.1088/1742-6596/119/4/042024

    Explicit state representation and the ATLAS event data model : Theory and practice. / Nowak, Marcin; Malon, David; Gemmeren, Peter; Schaffer, Arthur; Snyder, Scott; Binet, Sebastien; Cranmer, Kyle.

    In: Journal of Physics: Conference Series, Vol. 119, No. 4, 042024, 01.07.2008.

    Research output: Contribution to journalArticle

    Nowak, M, Malon, D, Gemmeren, P, Schaffer, A, Snyder, S, Binet, S & Cranmer, K 2008, 'Explicit state representation and the ATLAS event data model: Theory and practice', Journal of Physics: Conference Series, vol. 119, no. 4, 042024. https://doi.org/10.1088/1742-6596/119/4/042024
    Nowak, Marcin ; Malon, David ; Gemmeren, Peter ; Schaffer, Arthur ; Snyder, Scott ; Binet, Sebastien ; Cranmer, Kyle. / Explicit state representation and the ATLAS event data model : Theory and practice. In: Journal of Physics: Conference Series. 2008 ; Vol. 119, No. 4.
    @article{9d25013eae614210bf128d91c05103c6,
    title = "Explicit state representation and the ATLAS event data model: Theory and practice",
    abstract = "In anticipation of data taking, ATLAS has undertaken a program of work to develop an explicit state representation of the experiment's complex transient event data model. This effort has provided both an opportunity to consider explicitly the structure, organization, and content of the ATLAS persistent event store before writing tens of petabytes of data (replacing simple streaming, which uses the persistent store as a core dump of transient memory), and a locus for support of event data model evolution, including significant refactoring, beyond the automatic schema evolution capabilities of underlying persistence technologies. ATLAS has encountered the need for such non-trivial schema evolution on several occasions already. This paper describes the state representation strategy (transient/persistent separation) and its implementation, including both the payoffs that ATLAS has seen (significant and sometimes surprising space and performance improvements, the extra layer notwithstanding, and extremely general schema evolution support) and the costs (additional and relatively pervasive additional infrastructure development and maintenance). The paper further discusses how these costs are mitigated, and how ATLAS is able to implement this strategy without losing the ability to take advantage of the (improving!) automatic schema evolution capabilities of underlying technology layers when appropriate. Implications of state representations for direct ROOT browsability, and current strategies for associating physics analysis views with such state representations, are also described.",
    author = "Marcin Nowak and David Malon and Peter Gemmeren and Arthur Schaffer and Scott Snyder and Sebastien Binet and Kyle Cranmer",
    year = "2008",
    month = "7",
    day = "1",
    doi = "10.1088/1742-6596/119/4/042024",
    language = "English (US)",
    volume = "119",
    journal = "Journal of Physics: Conference Series",
    issn = "1742-6588",
    publisher = "IOP Publishing Ltd.",
    number = "4",

    }

    TY - JOUR

    T1 - Explicit state representation and the ATLAS event data model

    T2 - Theory and practice

    AU - Nowak, Marcin

    AU - Malon, David

    AU - Gemmeren, Peter

    AU - Schaffer, Arthur

    AU - Snyder, Scott

    AU - Binet, Sebastien

    AU - Cranmer, Kyle

    PY - 2008/7/1

    Y1 - 2008/7/1

    N2 - In anticipation of data taking, ATLAS has undertaken a program of work to develop an explicit state representation of the experiment's complex transient event data model. This effort has provided both an opportunity to consider explicitly the structure, organization, and content of the ATLAS persistent event store before writing tens of petabytes of data (replacing simple streaming, which uses the persistent store as a core dump of transient memory), and a locus for support of event data model evolution, including significant refactoring, beyond the automatic schema evolution capabilities of underlying persistence technologies. ATLAS has encountered the need for such non-trivial schema evolution on several occasions already. This paper describes the state representation strategy (transient/persistent separation) and its implementation, including both the payoffs that ATLAS has seen (significant and sometimes surprising space and performance improvements, the extra layer notwithstanding, and extremely general schema evolution support) and the costs (additional and relatively pervasive additional infrastructure development and maintenance). The paper further discusses how these costs are mitigated, and how ATLAS is able to implement this strategy without losing the ability to take advantage of the (improving!) automatic schema evolution capabilities of underlying technology layers when appropriate. Implications of state representations for direct ROOT browsability, and current strategies for associating physics analysis views with such state representations, are also described.

    AB - In anticipation of data taking, ATLAS has undertaken a program of work to develop an explicit state representation of the experiment's complex transient event data model. This effort has provided both an opportunity to consider explicitly the structure, organization, and content of the ATLAS persistent event store before writing tens of petabytes of data (replacing simple streaming, which uses the persistent store as a core dump of transient memory), and a locus for support of event data model evolution, including significant refactoring, beyond the automatic schema evolution capabilities of underlying persistence technologies. ATLAS has encountered the need for such non-trivial schema evolution on several occasions already. This paper describes the state representation strategy (transient/persistent separation) and its implementation, including both the payoffs that ATLAS has seen (significant and sometimes surprising space and performance improvements, the extra layer notwithstanding, and extremely general schema evolution support) and the costs (additional and relatively pervasive additional infrastructure development and maintenance). The paper further discusses how these costs are mitigated, and how ATLAS is able to implement this strategy without losing the ability to take advantage of the (improving!) automatic schema evolution capabilities of underlying technology layers when appropriate. Implications of state representations for direct ROOT browsability, and current strategies for associating physics analysis views with such state representations, are also described.

    UR - http://www.scopus.com/inward/record.url?scp=50849113759&partnerID=8YFLogxK

    UR - http://www.scopus.com/inward/citedby.url?scp=50849113759&partnerID=8YFLogxK

    U2 - 10.1088/1742-6596/119/4/042024

    DO - 10.1088/1742-6596/119/4/042024

    M3 - Article

    VL - 119

    JO - Journal of Physics: Conference Series

    JF - Journal of Physics: Conference Series

    SN - 1742-6588

    IS - 4

    M1 - 042024

    ER -