Overview

Dex is a domain specific language for generating “glue” code between a parser generated by Daedalus, and an application that needs to use the parser. It’s purpose is to translate from the data representation used by Daedalus into an application specific data representation.

Using Dex

The dex tool translates a Dex specification file (e.g., my-app.dex) to some code that implements the data translation (e.g., my-app.h and my-app.cpp). While Dex specifications are agnostic to the target language, at present we’ve only implemented concrete support for exporting to C++, as this is one of the main backends supported by Daedalus. We plan to support more backends in the future.

A simple invocation of dex looks like this:

> dex my-spec.dex

Command Line Flags

By default, the name of the generated files will be derived from the name of the Dex specification. dex supports the following flags:

--ddl-path=DIR

Adds a directory to be searched when looking for Daedalus specification.

--dex-path=DIR

Adds a directory to be searched when looking for Dex specifications.

--output=FILE

Generate code in this file, instead of deriving it from the name of the spec. In the case of C++ we generate two files—a header and an implementation, whose names are derived from the given name by adjusting the file extension.

C++ Namespaces

In the generated C++ code, the declarations from each Dex module are wrapped in a namespace matching the module name. For example, all definitions in the standard CPP module, would be declared in namespace called CPP.

Writing Dex Specifications

A Dex specification consists of a sequence of top-level declarations. There are five types of declarations, described in more detail below.

import DAEADLUS_MODULE(PARSERS)

import declarations are used to specify what Daedalus types we are going to be working with. In parens, the declaration specifies which Daedalus parsers we are working with. Only the main parsers need to be specified, our tooling will automatically infer all types that are needed to support exporting the results of the given parser.

using DEX_MODULE

using declarations make it possible to define modular Dex specifications. Such a declaration brings into scope all the things defined in DEX_MODULE. The definitions from the module may be referred to either by NAME or qualified, as DEX_MODULE::NAME to avoid name clashes. We provide a special module named CPP, which has definitions for exporting various Daedalus types to standard C++ types.

extern CODE_BLOCK
extern def CODE_BLOCK

extern declarations allow for arbitrary code to be added to the generated translation file. Such code will always be added at the top of the file. For the C++ backend, extern declarations are added to the .h file, and extern def declarations are added to the .cpp file. Typically, extern declarations should be used to #include dependencies, and extern def should be used to add local helper functions. The CODE_BLOCKS in extern may not contain any escapes. See Writing External Code for more details on CODE_BLOCK.

type NAME<TYPE_PARAMS> CODE_BLOCK

type defines a Dex way to refer to types in the target language (e.g., C++). In the rest of the Dex specification, we use NAME<TYPE_PARAMS> to refer to the given external type. The CPP module also defines Dex aliases for common C++ types. The escapes in the code blocks may be used to refer to the type parameters of the declaration.

def NAME<TYPE_PARAMS,FUN_PARAMS>(NAME: DAEDALUS_TYPE): EXTERN_TYPE
  DEFINITION

Definitions are at the heart of Dex specifications, as they are used to specify the mapping between Daedalus values of type DAEDALUS_TYPE and values in the target language for type EXTERN_TYPE (which should be declared with type).

Optionally, definitions may be preceeded by the default keyword, which specifies that unless otherwise specified this exporter will be used for Daedalus values of the given type. At present, only definitions without FUN_PAPRAMS may be declared as default.

Before we go into more details on how to write definitions, let’s look at how to write external code.

Writing External Code

A number of constructs in Dex require the user to write some code in the language of their application (e.g., C++). In Dex, the symbol -> is used to signify the beginning of an external code block. The code block begins at the first non-white space character following -> and contains all text that is indented equally or more than the first entry in the block.

Here is an example of an external block:

->
  The block starts here.
  This is also part of the block.
    As is this.
This is not a part of the block.

Dex code blacks may contain escapes which signify that what follows is not part of the external language but is Dex code instead. Escapes start with $ an extend for either a single identifier, or need are enclosed between parens (between ( and )). To write a literal $ in a code block (i.e., one that does not start an escape) you need to write 2 dollars $$. Here is an example of a code block with some escapes:

->
  External language code, here comes single identifier escape $x.
  More external code, more complex escape $(f(x)).
  Finally, this is just a single $$.

The escape on the first line contains the Dex expression x while the one one the second line contains f(x).

Exporter Definitions

We provide a few different ways to define an exporting function, depending on the Daedalus type in question. In this section, we are describing the various ways to provide DEFINITION.

def NAME<TYPE_PARAMS,FUN_PARAMS>(NAME: DAEDALUS_TYPE): EXTERN_TYPE
  DEFINITION

Structs

For Daedalus types with fields (i.e., structs/records), the definitions can be just a CODE_BLOCK. The escapes of the code block will Typically specify how to export the fields of the structure. For example, if we are working with a Daedalus parser that produces values of type Point, which have two fields x and y, we might write an exporter like this:

def as_point(pt: Point): CustomAppPoint ->
  CustomAppPoint($(pt.x), $(pt.y))

Note that in this example the escapes pt.x and pt.y do not specify how to export the fields, so Dex will use the types of the fields to try and find a default exporter.

Discriminated Unions

For discriminated union types (i.e., types with multiple constructors), the definition should be a case expression, that specifies how to handle each possible shape of a value. Here’s an example that exports a Daedalus value of type Maybe (uint 8) to a C++ uint8_t:

def mb_to_u8(x: maybe (uint 8)): uint8_t =
  case x of
    nothing -> 0
    just v  -> $v

Each alternative in the case should match one of the possible alternative of the union type, followed by a CODE_BLOCK that specifies how to export it. In this example, we map Daedalus nothing values to 0, and defined values using the default exporter for uint 8, which is defined in module CPP and maps the value to uint8_t in C++.

The escapes in each case alternative may refer to the field of the constructor, if any.

Iteration

For Deadalus values that support iterations (i.e., arrays and maps), we provide a custom iteration form that makes it easy to iterate over all the elements. For example, here is how we might export a Daedalus array of bytes to a C++ string (this is already defined in the CPP module, but it serves as a nice example)

def as_string (xs: [uint 8]): string =
  init        -> std::string result;
  for x in xs -> result.push_back($(as_char(x)));
  return      -> result

As the example illustrates, iteration definitions always have 3 parts: initialization, traversal, and final result. The init code block contains statements that setup some initial state. The for block contains statements that are executed for each element in the collection. Note that in this example we are using an explicit exporter, as_char to indicate that we want to export the Daedalus uint8_t to char, and not to the default uint8_t. Finally the return code block should contain an expression that extracts the result from the state.

Type and Function Parameters

Dex also supports the definition of polymorphic exporters. A polymorphic exporter may be used for Daedalus values of different types. For example, the standard CPP module provides the following exporter for processing maybe values:

def as_optional<A,T,f: A => T>(x: maybe A): optional<T> =
  case x of
    nothing -> std::optional<$T>()
    just a  -> std::optional($(f(a)))

This exporter maps Daedalus maybe types to the standard C++ type optional. This is a standard case based exporter. The new thing here are the type and function parameters, declared between < and >. Here we have a Daedalus type parameter A and a C++ template argument T. In addition, we have an exporter parameter f, that we use to export the data in the just case. Also note that the escapes in the code block may refer to type parameters, as illustrated by the nothing case.