Overview ======== Dex is a domain specific language for generating "glue" code between a parser generated by Daedalus, and an application that needs to use the parser. It's purpose is to translate from the data representation used by Daedalus into an application specific data representation. Using Dex ========= The ``dex`` tool translates a Dex specification file (e.g., ``my-app.dex``) to some code that implements the data translation (e.g., ``my-app.h`` and ``my-app.cpp``). While Dex specifications are agnostic to the target language, at present we've only implemented concrete support for exporting to C++, as this is one of the main backends supported by Daedalus. We plan to support more backends in the future. A simple invocation of ``dex`` looks like this: .. code-block:: bash > dex my-spec.dex Command Line Flags ------------------ By default, the name of the generated files will be derived from the name of the Dex specification. ``dex`` supports the following flags: .. data:: --ddl-path=DIR Adds a directory to be searched when looking for Daedalus specification. .. data:: --dex-path=DIR Adds a directory to be searched when looking for Dex specifications. .. data:: --output=FILE Generate code in this file, instead of deriving it from the name of the spec. In the case of C++ we generate two files---a header and an implementation, whose names are derived from the given name by adjusting the file extension. C++ Namespaces -------------- In the generated C++ code, the declarations from each Dex module are wrapped in a namespace matching the module name. For example, all definitions in the standard ``CPP`` module, would be declared in namespace called ``CPP``. Writing Dex Specifications ========================== A Dex specification consists of a sequence of top-level declarations. There are five types of declarations, described in more detail below. .. code-block:: Dex import DAEADLUS_MODULE(PARSERS) ``import`` declarations are used to specify what Daedalus types we are going to be working with. In parens, the declaration specifies which Daedalus parsers we are working with. Only the main parsers need to be specified, our tooling will automatically infer all types that are needed to support exporting the results of the given parser. .. code-block:: Dex using DEX_MODULE ``using`` declarations make it possible to define modular Dex specifications. Such a declaration brings into scope all the things defined in ``DEX_MODULE``. The definitions from the module may be referred to either by ``NAME`` or qualified, as ``DEX_MODULE::NAME`` to avoid name clashes. We provide a special module named ``CPP``, which has definitions for exporting various Daedalus types to standard C++ types. .. code-block:: Dex extern CODE_BLOCK extern def CODE_BLOCK ``extern`` declarations allow for arbitrary code to be added to the generated translation file. Such code will always be added at the top of the file. For the C++ backend, ``extern`` declarations are added to the ``.h`` file, and ``extern def`` declarations are added to the ``.cpp`` file. Typically, ``extern`` declarations should be used to ``#include`` dependencies, and ``extern def`` should be used to add local helper functions. The CODE_BLOCKS in ``extern`` may not contain any escapes. See :ref:`Writing External Code` for more details on ``CODE_BLOCK``. .. code-block:: Dex type NAME CODE_BLOCK ``type`` defines a Dex way to refer to types in the target language (e.g., C++). In the rest of the Dex specification, we use ``NAME`` to refer to the given external type. The ``CPP`` module also defines Dex aliases for common C++ types. The escapes in the code blocks may be used to refer to the type parameters of the declaration. .. code-block:: Dex def NAME(NAME: DAEDALUS_TYPE): EXTERN_TYPE DEFINITION Definitions are at the heart of Dex specifications, as they are used to specify the mapping between Daedalus values of type ``DAEDALUS_TYPE`` and values in the target language for type ``EXTERN_TYPE`` (which should be declared with ``type``). Optionally, definitions may be preceeded by the ``default`` keyword, which specifies that unless otherwise specified this exporter will be used for Daedalus values of the given type. At present, only definitions without ``FUN_PAPRAMS`` may be declared as ``default``. Before we go into more details on how to write definitions, let's look at how to write external code. Writing External Code ===================== A number of constructs in Dex require the user to write some code in the language of their application (e.g., C++). In Dex, the symbol ``->`` is used to signify the beginning of an external code block. The code block begins at the first non-white space character following ``->`` and contains all text that is indented equally or more than the first entry in the block. Here is an example of an external block: .. code-block:: DexBlock -> The block starts here. This is also part of the block. As is this. This is not a part of the block. Dex code blacks may contain *escapes* which signify that what follows is not part of the external language but is Dex code instead. Escapes start with ``$`` an extend for either a single identifier, or need are enclosed between parens (between ``(`` and ``)``). To write a literal ``$`` in a code block (i.e., one that *does not* start an escape) you need to write 2 dollars ``$$``. Here is an example of a code block with some escapes: .. code-block:: Dex -> External language code, here comes single identifier escape $x. More external code, more complex escape $(f(x)). Finally, this is just a single $$. The escape on the first line contains the Dex expression ``x`` while the one one the second line contains ``f(x)``. Exporter Definitions ==================== We provide a few different ways to define an exporting function, depending on the Daedalus type in question. In this section, we are describing the various ways to provide ``DEFINITION``. .. code-block:: Dex def NAME(NAME: DAEDALUS_TYPE): EXTERN_TYPE DEFINITION Structs ------- For Daedalus types with fields (i.e., structs/records), the definitions can be just a ``CODE_BLOCK``. The escapes of the code block will Typically specify how to export the fields of the structure. For example, if we are working with a Daedalus parser that produces values of type ``Point``, which have two fields ``x`` and ``y``, we might write an exporter like this: .. code-block:: Dex def as_point(pt: Point): CustomAppPoint -> CustomAppPoint($(pt.x), $(pt.y)) Note that in this example the escapes ``pt.x`` and ``pt.y`` do not specify *how* to export the fields, so Dex will use the types of the fields to try and find a ``default`` exporter. Discriminated Unions --------------------- For discriminated union types (i.e., types with multiple constructors), the definition should be a ``case`` expression, that specifies how to handle each possible shape of a value. Here's an example that exports a Daedalus value of type ``Maybe (uint 8)`` to a C++ ``uint8_t``: .. code-block:: Dex def mb_to_u8(x: maybe (uint 8)): uint8_t = case x of nothing -> 0 just v -> $v Each alternative in the ``case`` should match one of the possible alternative of the union type, followed by a ``CODE_BLOCK`` that specifies how to export it. In this example, we map Daedalus ``nothing`` values to 0, and defined values using the default exporter for ``uint 8``, which is defined in module ``CPP`` and maps the value to ``uint8_t`` in C++. The escapes in each case alternative may refer to the field of the constructor, if any. Iteration ---------- For Deadalus values that support iterations (i.e., arrays and maps), we provide a custom iteration form that makes it easy to iterate over all the elements. For example, here is how we might export a Daedalus array of bytes to a C++ string (this is already defined in the ``CPP`` module, but it serves as a nice example) .. code-block:: Dex def as_string (xs: [uint 8]): string = init -> std::string result; for x in xs -> result.push_back($(as_char(x))); return -> result As the example illustrates, iteration definitions always have 3 parts: initialization, traversal, and final result. The ``init`` code block contains statements that setup some initial state. The ``for`` block contains statements that are executed for each element in the collection. Note that in this example we are using an explicit exporter, ``as_char`` to indicate that we want to export the Daedalus ``uint8_t`` to ``char``, and not to the default ``uint8_t``. Finally the ``return`` code block should contain an expression that extracts the result from the state. Type and Function Parameters ---------------------------- Dex also supports the definition of polymorphic exporters. A polymorphic exporter may be used for Daedalus values of different types. For example, the standard ``CPP`` module provides the following exporter for processing ``maybe`` values: .. code-block:: Dex def as_optional T>(x: maybe A): optional = case x of nothing -> std::optional<$T>() just a -> std::optional($(f(a))) This exporter maps Daedalus ``maybe`` types to the standard C++ type ``optional``. This is a standard ``case`` based exporter. The new thing here are the type and function parameters, declared between ``<`` and ``>``. Here we have a Daedalus type parameter ``A`` and a C++ template argument ``T``. In addition, we have an *exporter parameter* ``f``, that we use to export the data in the ``just`` case. Also note that the escapes in the code block may refer to type parameters, as illustrated by the ``nothing`` case.