Better extensibility through modular syntax

Robert Grimm

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

We explore how to make the benefits of modularity available for syntactic specifications and present Rats!, a parser generator for Java that supports easily extensible syntax. Our parser generator builds on recent research on parsing expression grammars (PEGs), which, by being closed under composition, prioritizing choices, supporting unlimited lookahead, and integrating lexing and parsing, offer an attractive alternative to context-free grammars. PEGs are implemented by so-called packrat parsers, which are recursive descent parsers that memoize all intermediate results (hence their name). Memoization ensures linear-time performance in the presence of unlimited lookahead, but also results in an essentially lazy, functional parsing technique. In this paper, we explore how to leverage PEGs and packrat parsers as the foundation for extensible syntax. In particular, we show how make packrat parsing more widely applicable by implementing this lazy, functional technique in a strict, imperative language, while also generating better performing parsers through aggressive optimizations. Next, we develop a module system for organizing, modifying, and composing large-scale syntactic specifications. Finally, we describe a new technique for managing (global) parsing state in functional parsers. Our experimental evaluation demonstrates that the resulting parser generator succeeds at providing extensible syntax. In particular, Rats! enables other grammar writers to realize real-world language extensions in little time and code, and it generates parsers that consistently outperform parsers created by two GLR parser generators.

Original languageEnglish (US)
Title of host publicationACM Sigplan Notices. Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI)
Pages38-51
Number of pages14
Volume2006
DOIs
StatePublished - 2006
EventPLDI 2006 - 2006 ACM SIGPLAN Conference on Programming Language Design and Implementation - Ottawa, ON, Canada
Duration: Jun 10 2006Jun 16 2006

Publication series

NameProceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI)
Volume2006

Other

OtherPLDI 2006 - 2006 ACM SIGPLAN Conference on Programming Language Design and Implementation
CountryCanada
CityOttawa, ON
Period6/10/066/16/06

    Fingerprint

Keywords

  • Extensible syntax
  • Module system
  • Packrat parsing
  • Parser generator
  • Parsing expression grammar
  • Rats!

ASJC Scopus subject areas

  • Computer Graphics and Computer-Aided Design
  • Software

Cite this

Grimm, R. (2006). Better extensibility through modular syntax. In ACM Sigplan Notices. Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI) (Vol. 2006, pp. 38-51). (Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI); Vol. 2006). https://doi.org/10.1145/1133255.1133987