Getting Help
To see a list of available commands and options that apply to all commands:
daedalus --help
To see a list of options for a specific command:
daedalus COMMAND --help
Command show-types
To type-check a DaeDaLus specification and see the types of the declarations:
daedalus show-types MyParserSpec.ddl
The command prints information about the types and parsers defined by the given specification. If type checking fails, the command still prints declarations that were successfully type checked before the error. The type error is reported and the command exits unsuccessfully as usual.
The output may be filtered with the following options:
--externalsShow only declarations without a definition.
--name NAMEShow only the declaration named
NAME.--module MODULEShow declarations from the named module instead of the module containing the input file. The module must be imported, directly or indirectly, by the input specification.
The options may be combined. For example, this shows the type of the external
declarations in module Primitives:
.. daedalus show-types --module Primitives --externals
show-typesTypes
======
def List a =
union
nil: {}
cons: NonEmptyList a
def NonEmptyList a =
struct
head: a
tail: List a
Parsers and Semantic Values
===========================
def List:
for any type a:
parameter: parser of a
defines: parser of List a
def NonEmptyList:
for any type a:
parameter: parser of a
defines: parser of NonEmptyList a
def Main:
defines: parser of List (uint 8)
The resulting types have the following form:
def ParserName:
parameter: <type A>
parameter: <type B>
...
defines: <type C>
This resembles a C type declaration as follows:
<type C> ParserName(<type A>, <type B>, ...);
The types themselves may be simple types such as integers or arrays, but they
often have the form parser of <type A>. This indicates that the parameter or
result is a parser, that itself generates semantic values of type A.
Command run
This command is used to run a DaeDaLus specification using the interpreter:
daedalus run MyParserSpec.ddl --input=input.txt
This command will run the parser defined in MyParserSpec.ddl on the
input from file input.txt.
If successful, the resulting semantic value will be shown on stdout.
Flags
- --input=FILE
Specifies that the input for the parser should be read from the given file. If no
--inputis specified, then the interpreter will run with an empty input.
- --entry=[MODULE.]NAME
Specifies which parser defined by the specification should be executed. If no
--entryis specified, then the interpreter will run the parser calledMain.The entry parser should have a fixed type (i.e., it should not be polymorphic) and it should not have any parameters, implicit or explicit.
- --json
Show the parsed semantic valus using a JSON based format.
- --html
Show the parsed semantic value rendered as HTML.
- --core
Use the Core IR interpreter.
- --vm
Use the VM IR interpreter.
Command: compile-hs
To compile a DaeDaLus parser specification to Haskell:
daedalus compile-hs MyParserSpec.ddl --out-dir=some_dir_name
Generates a small Cabal application consisting of a Main.hs and a
Cabal package file. The generated Main.hs uses Template Haskell to load
the DaeDaLus specification and compile it through the VM pipeline when the
application is built.
The generated application expects a parser named Main with a fixed result
type and no parameters. It accepts an optional input filename and writes the
result, or a parse error, as JSON:
cd some_dir_name
cabal build
cabal run some_dir_name -- input.bin
If no input filename is provided, the parser is run on an empty input.
Important
The Haskell project is not self-contained at build time. Its Template
Haskell splice requires the daedalus, rts-vm-hs, and
rts-hs-data Haskell packages to be available to Cabal. Having only a
standalone daedalus executable is not sufficient.
The generated directory does not contain a cabal.project. When local
source checkouts of the Daedalus packages are needed, create a project file
that lists the generated package and those source packages.
The original specification and any imported DaeDaLus modules must also remain available while the generated application is being built. Once built, the resulting executable does not need the compiler packages or the specification at runtime.
Flags
- --out-dir=DIRECTORY
Save the generated files in the given directory. This option is required when generating the VM Haskell application.
- --vm
Explicitly select the VM Haskell backend. This is the default.
- --old
Use the legacy Haskell backend. This generates Haskell parser modules, a sample driver, and a Cabal package description using
rts-hs.
- --no-error-stack
Disable grammar call stacks and source locations in parse errors generated by the VM Haskell backend. Error tracking remains enabled, but produces less detailed output with lower runtime overhead.
The VM-generated driver currently supports only the default Main entry
point; it may not be combined with --entry.
Command: compile-c++
To compile a DaeDaLus package specification C++:
daedalus compile-c++ MyParserSpec.ddl --out-dir=some_dir_name
Generated Files
- The compiled parser is in two files:
main_parser.hcontains the interface to the generated parser, andmain_parser.cppcontains the implementation of the parser.
In addition, the DaeDaLus compiler will generate a sample executable driver
and Makefile illustrating how to build the parser and generate Doxygen
documentation. The sample executable expects that the compiled specification
contains a parser named Main that has a fixed type and no parameters
of any kind—if this is not the case, the sample driver will fail to compile.
The sample executable driver will be generated when:
there is no explicit
--entryspecified, andthere is no custom user state
--user-state, andthe flag
--lazy-streamsis not enabled
Flags
- --out-dir=DIRECTORY
Save generated files in the given directory. The directory will be created if it does not exist.
- --out-dir-header=DIRECTORY
Save generated header files in the given directory. The directory will be created if it does not exist. If this is not specified, then the headers will be saved in
--out-dir
- --file-root=STRING
Use the given string for the names of the generated parser implementation and header. If not specified this will be
main_parser.
- --user-namespace=NAMESPACE
Place generated type declarations in the given name space. Note that at present this applies only to generate types. Generated parser entry point always reside in the default namespace. If this is not specified, this will be
User
- --no-error-stack
Disable call stack tracking in errors. Without this option, we generate code that leads to more detailed parse error messaged, at some runtime cost. Adding this flag leads to less detailed parse errors, but some potential performance gain.
- --user-state=SATE_TYPE
Generate a parser where the parser’s state will be extended with some user state of the given type. This is useful in certain situations (e.g., caching of data), but care needs to be taken that updates to this state make sense even when the parsers backtracks.
- --add-include=INCLUDE
Add an additional #include to the generated parser header file. Typically, this is used to support custom user state, or integration with an pre-compiled DaeDaLus parser. The parameter should just contain the thing to be incuded (e.g.,
"file.h"or<file.h>).
- --entry=[MODULE.]NAME
Declare that the given parser is an entry point for the generated parser. Specifying an entry point will disable the generation of a sample driver executable. The name of the C++ function corresponding to DaeDaLus declaration
FisparseF. The entry point should have a fixed type, but MAY have argument.Sample entry point for parserMainwith no extra arguments, returning a semantic value of typeT.void parseMain ( DDL::ParseError &error , std::vector<T>> &results , DDL::Input input );
If an entry point has arguments, they’ll come after the
inputargument. The entry point returns any parsed semantic values in theresultsvector, there could be multiple results if the grammar is ambiguous. If there are no results (i.e.,resultsis empty), thenerrorwill contain information about the parser error.
- --inline
Perform aggressive inlining—all parsers that can be inlined will be inlined at each use site. Sometimes this may improve performance, but sometimes it may reduce it, due to a significant increase in the size of the generate code.
- --inline-this=[MODULE.]NAME
Inline uses of a specific parser.
- --extern=MODULE[:NAMESPACE]
Do not generate declarations for the types declared in the given module. This is useful when interfacing with a pre-compiled DaeDaLus parser, which already contains declarations for the given types. Often, this option will be combined with
add-includeto include the declarations of the already compiled code. If a namespace is provided, then uses of types in the given module will be qualified with the given namespace. Otherwise, the default namespaceUserwill be used.
- --lazy-streams
Generate code that uses streams where not all data is available when the parser starts. The generated code uses coroutines, as defined by the
fiberclass fromboost-contextlibrary. If the parser reaches the end of an incomplete data stream, it yields to another coroutine which should either provide more data or terminate the stream before resuming the parser. For more details seeddl/stream.h. The data streams used by the parser are as defined in classDDL::Stream. The interaction between the data provider and the parser should be done using classDDL::ParserThread.
Command: compile-rust
To compile a DaeDaLus parser specification to Rust:
daedalus compile-rust MyParserSpec.ddl
By default, the output will be saved in a directory whose name is derived
from the name of the input file (e.g., MyParserSpec.ddl becomes
MyParserSpec). You can specify a different output directory using
the --out-dir flag.
The result is a directory containing a complete Rust crate with the generated parser and type definitions. The generated code uses the Daedalus Rust runtime system (RTS), which provides core parsing functionality.
When integrating a parser into an existing Rust crate, use --output-file
to generate just a Rust source module instead of a complete crate.
Generated Files
- The compiled parser is placed in a standard Rust crate structure:
src/lib.rscontains the generated parser implementation and type definitionsCargo.tomlprovides the Rust package configurationA sample executable driver
src/main.rs(if no explicit--entryis specified)
The sample driver is not generated when --user-state is specified, because
constructing the custom state is the responsibility of the embedding program.
To build the generated parser:
cd MyParserSpec
cargo build --release
The --save-rts flag can be used to include the RTS source code directly
in the output directory, which is useful for standalone builds that do not
depend on an external RTS installation.
When --c-bindings is specified, the generated crate builds static and
shared libraries suitable for use by C programs. Currently, C wrappers are
supported only for entry parsers without explicit parameters and using the
default parser state.
Flags
- --out-dir=DIRECTORY
Save generated files in the given directory. The directory will be created if it does not exist. If not specified, a directory name is derived from the input file name.
- --output-file=FILE
Save only the generated parser and type definitions as a Rust source file. Parent directories are created as needed. No
Cargo.toml, driver, or runtime source is generated.This is useful when the parser should be a module in an existing application crate. For example:
daedalus compile-rust Format.ddl \ --output-file=src/format.rs \ --entry=Main \ --user-state=crate::native::State \ --user-fun=crate::native
The application may then declare
mod format;andmod native;. Because both are in the same crate, native functions can refer directly to types generated informat.This option may not be combined with
--out-dir,--rts-path,--save-rts, or--c-bindings. The application crate is responsible for declaring its dependencies ondaedalus-rts-rustandserde.
- --rts-path=PATH
Specify the path to the Daedalus Rust runtime system. This is used when the RTS is located in a non-standard location. If not specified, the default path is
../rts-rustrelative to the output directory.
- --save-rts
Include the source code for the Daedalus RTS in the output directory. This creates a standalone build that does not depend on an external RTS installation. The RTS will be copied to a
rts-rustsubdirectory within the output directory.
- --c-bindings
Generate a Rust
staticlibandcdylibwith a C-compatible interface instead of a native Rust library and sample executable. One C parser wrapper is generated for each parser selected with--entry; when no entry is specified, the default entry isMain.The generated crate produces two headers when built:
parser_types.hdeclares opaque parser result and semantic value types.parser_bindings.hdeclares parser entry points, accessors, union variant queries, and result destructors.
Entry points used with C bindings currently may not have explicit parameters or use a custom user state. Bounded integer results and fields are returned by value; structs, unions, arrays, and other compound values are exposed through borrowed pointers. A pointer returned by an accessor remains valid only while its owning parsed result is alive. Release a successful parse result with the
free_ParsedNAMEfunction.This option may not be combined with
--output-file.
- --glue-path=PATH
Specify the path to the
gluefactoryprocedural macro crate used to generate the C interface. If not specified, the default path is../gluefactoryrelative to the output directory. This option applies only when--c-bindingsis used.
- --save-gluefactory
Copy the
gluefactorysource into agluefactorysubdirectory of the generated crate and use that local copy when building C bindings. This is useful for a standalone output directory that should not depend on the source repository.
- --no-error-stack
Disable call stack tracking in parse errors. Without this option, the generated code includes grammar stack traces in error messages, which provide more detailed diagnostic information at some runtime cost. Adding this flag produces less detailed parse errors but may improve performance.
- --user-state=STATE_TYPE
Generate parser functions with custom user state of the given Rust type. For example,
--user-state=MyStatecauses generated parser functions to accept&mut ddl::ParserStateWith<MyState>instead of the default&mut ddl::ParserState.The embedding program should construct the parser state with
ddl::new_parser_state_with(user_state). Generated or external parser functions may access the state through the publicuser_statefield. Care should be taken that state updates make sense when a parser backtracks.
- --add-import=IMPORT
Add
use IMPORT;to the generated parser module. This option may be specified multiple times. It can be used, for example, to bring a custom user-state type into scope:daedalus compile-rust MyParser.ddl \ --add-import=my_crate::MyState \ --user-state=MyState
IMPORTshould contain the Rust import specification without the leadinguseor trailing semicolon. Rust import forms such asmodule::Type,module::*, andmodule::{TypeA, TypeB}are supported.
- --user-fun=QUAL
Provide implementations for external Daedalus functions using native Rust functions under the qualifier
QUAL. For each external functionf, the generated parser defines an#[inline(always)]wrapper that callsQUAL::f, forwarding the parser state and all function arguments.For example,
--user-fun=nativemaps an external Daedalus function whose generated Rust name isget_valuetonative::get_value. Use--add-importas needed to bringQUALinto scope.
- --entry=[MODULE.]NAME
Declare that the given parser is an entry point for the generated parser. The entry point should have a fixed type, but MAY have arguments. Multiple entry points can be specified by using this flag multiple times. Specifying at least one entry point will disable the generation of the sample executable driver. When
--c-bindingsis used, each selected entry gets a C wrapper and entries with arguments are not currently supported.
- --determinize
Apply determinization transformations to the Core IR. This optimization can improve parser performance by eliminating non-deterministic choice where possible.
- --keep-match
Do not remove matching constructs during compilation. By default, the compiler removes certain matching patterns as an optimization.
- --no-shrink-biased
Do not attempt to shrink the scope of biased choice operations. By default, the compiler optimizes biased choice by reducing its scope where possible.
- --no-case-of-case
Do not attempt to eliminate case-of-case patterns. By default, the compiler applies this optimization to simplify nested pattern matching.
- --inline
Perform aggressive inlining—all parsers that can be inlined will be inlined at each use site. This may improve or reduce performance depending on the specific parser structure and the size of the generated code.
- --inline-this=[MODULE.]NAME
Inline uses of a specific parser. This flag can be used multiple times to inline different parsers selectively.
- --strip-fail
Remove failure nodes from the Core IR during optimization. This can reduce code size but may make debugging more difficult.
- --no-loops
Remove loop constructs from the Core IR, transforming them into recursive functions.
- --no-bitdata
Remove bitdata constructs from the Core IR, transforming them into equivalent operations on standard integer types.
- --spec-types
Specialize polymorphic types during compilation. This can improve performance but may increase code size.
- --no-core-check
Skip validation of the Core IR after transformations. This can speed up compilation but should be used with caution as it may allow invalid IR to reach code generation.
- --extern=MODULE[:NAMESPACE]
Do not generate definitions for the types declared in the given module. This is useful when interfacing with another pre-compiled DaeDaLus parser that already contains definitions for the given types. If a namespace is provided, uses of types in the given module will be qualified with that namespace.
Unsupported Features
The Rust backend currently does not support the following features:
Stack capture: Parsers that capture the execution stack (closures that capture parser state) are not supported.
External functions: Externally defined functions (primitives defined outside of DaeDaLus) cannot be compiled to Rust.
User types iwth numeric parameters: Types that are parameterized by numeric values are not supported.
Builder emission: The
emitBuilderoperation for emitting builder values into other builders is not yet implemented.Integer ranges: These only work for
sintanduint, but not forint(i.e., arbitrary-precision integers).
If your parser uses any of these features, the compiler will report an error indicating which feature is unsupported.
Command: dump
It shows the various IR structures of the DaeDaLus compiler. This command is largely for debugging.
- --parsed
Show the results of just parsing a specification. This ensures that a specification is syntactically correct.
- --resolved
Show the results of name resultion. This associates uses of names with their definitions.
- --tc
Show the results of type-checking. This validates the specification and esnures that parsers are used correctly (e.g. have the correct number of arguments with the correct types). The
dumpcommand will show this phase, unless another is specified.
- --spec
Show the IR after specialization. Specialization eliminates polymorphism and parsers with other parsers as arguments. This works by generating custom instances of a parser (similar to C++ template expansion). After this pahse, parsers are monomorphic and have only semantic values as parameters.
- --core
Show the Core IR representation. A number of DaeDaLus constructs are desugared into simpler constructs.
- --vm
This is the lowest level IR, which is used to generate code. It is essentially a control flow graph in SSA form, except that instead of using “phi” blocks, basic blocks have parameters.