Home · Docs · Primer · Examples · Prefixing · Syntax · Rules · Parser · Tree · Actions · Debug


This page documents Hapy runtime debugging facilities.

Table of Contents


Debugging is necessary when the parser results do not match user expectations. The parser may fail to accept valid input, fail to reject invalid input, or fail to produce an expected parsing tree. The mismatch between user expectations and parser behavior could be due to user misunderstanding of Hapy functionality or a Hapy implementation bug.

The following three kinds of information are provided for debugging parsing problems:


By default, Hapy debugging is not enabled. Debugging mode can be turned on from within the program or by using an environmental variable. When debugging is on, a rule compilation trace and a parsing trace are produced if applicable.

To enable Hapy debugging from the program, call Hapy::Debug(true), which is a static method of the Rule class. To disable debugging, call Hapy::Debug(false). Using this approach, debugging can be turned on or off many times, at any time.

It is also possible to enable debugging using the HAPY_DEBUG environment variable: Set HAPY_DEBUG to USER to enable debugging and to NONE to disable it. Other modes may be added in the future.

Explicit debugging instructions in the program (if any) overwrite environmental settings. If you do not want your program to react on the HAPY_DEBUG environment variable setting, call Hapy::Debug(false) before using Hapy objects.

All debugging info is dumped to std::clog.

To dump a parsing tree, call the Pree::print(ostream &) method of the tree. Parsing tree nodes can be dumped many times, at any time.

Annotating Rules

Internally, Hapy parser assigns rules unique identifiers. While those identifiers can be used to grok rule compilation and application traces, it is often much easier for humans to refer to rules using user-provided names. To name a parsing rule, use the Rule(const string &name, RuleId *) constructor. For example,

	using namespace Hapy;

	Rule rGrammar("grammar", 0);
	Rule rExpression("expression", 0);
	rGrammar = *(rExpression >> ';');

As the above example illustrates, you can set the second parameter to null if you are not interested in knowing the rule identifier (which is dynamically assigned to the rule at construction time). You can always retrieve that identifier later, using the Hapy::id() method.

Interpreting Output

This section explains how to grok debugging output produced by Hapy rules and parsing tree nodes.

Rule Compilation Trace

When the Parser::parse() method or equivalent is called the first time, each rule in the supplied grammar is "compiled" to accommodate various user settings (e.g., trimming) and to optimize future parsing. If debugging is enabled, then each compiled rule is printed on the standard error stream (std::clog).

For example, here is the rule compilation trace produced by the calculator grammar (see docs/calc.cc):

compiled rule:  _white#r-105 = white
compiled rule:  r-123 = *_white#r-105
compiled rule:  _digit#r-103 = digit
compiled rule:  _digit#r-103 = digit
compiled rule:  r-121 = *_digit#r-103
compiled rule:  r102 = _digit#r-103 >> r-121
compiled rule:  _;_trimmer#r108 = r-123 >> _;#r-109 >> r-123
compiled rule:  r-110 = r101 >> _;_trimmer#r108
compiled rule:  r103 = +r-110

The rDDD and r-DDD labels after the '#' sign are rule identifiers that Hapy assigned to the rule (e.g., r-105. Negative IDs correspond to intermediate rules. Positive IDs correspond to rules explicitly declared by the user.

The labels in front of the '#' sign are rule names automatically assigned by Hapy to some simple built-in rules. Most rules will not have these by default because Hapy cannot guess an appropriate name. To improve the readability of the trace, you should name rules. For example, if calculator rules are named to match their variable names, the compilation trace reveals these user-defined rules (among the same of intermediate rules with negative IDs):

compiled rule:  number#r102 = _digit#r-103 >> r-121
compiled rule:  number_trimmer#r104 = r-123 >> number#r102 >> r-123
compiled rule:  expression#r101 = number_trimmer#r104 | r-118 | r-114
compiled rule:  expression#r101 = number_trimmer#r104 | r-118 | r-114
compiled rule:  expression#r101 = number_trimmer#r104 | r-118 | r-114
compiled rule:  expression#r101 = number_trimmer#r104 | r-118 | r-114
compiled rule:  grammar#r103 = +r-110

This is still not perfect and even not pretty, of course. Improvement suggestions are welcome.

Some rules (e.g., expression#r101 above) are repeated because they are cloned during the compilation process and each clone is compiled. Cloning is necessary when the same rule can be reached by different paths from the start rule of the grammar and those paths have different user-preferences such as trimming. At the time of writing, Hapy does not try to minimize the number of clones.

The compilation trace is often essential for understanding the parsing trace because it maps rule IDs to rule definitions and because it shows auto-generated trimming rules.

Parsing Trace

The parsing trace shows parsing progress. Each time a rule is tried it is printed with a "try" label and the indentation level is increased. Each time the rule returns, the indentation level is decreased and the rule execution result is printed with a "match", "miss", "error", or "more" label.

Buffer contents or its prefix is printed after the "buf:" label, with the total buffer content length printed after the "clen:" label. The buffer is rendered so that it always appears on one line, with non-printable characters escaped using C++ character escaping conventions.

1/1-  try: grammar#r103::firstMatch buf: 1+1;\n clen:5
2/2-    try: r-110::firstMatch buf: 1+1;\n clen:5
3/3-      try: expression#r101::firstMatch buf: 1+1;\n clen:5
4/4-        try: number_trimmer#r104::firstMatch buf: 1+1;\n clen:5
5/5-          try: r-123::firstMatch buf: 1+1;\n clen:5
6/6-            try: _white#r-105::firstMatch buf: 1+1;\n clen:5
6/6-            miss: _white#r-105 buf: 1+1;\n clen:5
5/5-          match: r-123 buf: 1+1;\n clen:5
7/5-          try: number#r102::firstMatch buf: 1+1;\n clen:5
7/5-          match: number#r102 buf: +1;\n clen:4
11/5-          try: r-123::firstMatch buf: +1;\n clen:4
12/6-            try: _white#r-105::firstMatch buf: +1;\n clen:4
12/6-            miss: _white#r-105 buf: +1;\n clen:4
11/5-          match: r-123 buf: +1;\n clen:4
4/4-        match: number_trimmer#r104 buf: +1;\n clen:4
3/3-      match: expression#r101 buf: +1;\n clen:4
1/1-  match: grammar#r103 buf:  clen:0

The first trace column contains two numbers separated by a slash: the rule invocation identifier and the nesting level. The rule invocation identifier increases with every "try" statement. Searching for the same invocation identifier links the "try" statement with the corresponding rule execution result. For example, to find how the first expression rule matched, one could search for a "^3/" pattern.

Searching for a given nesting level allows to skip details at lower levels.

Parsing Tree Dump

For each parsing tree node, Hapy dumps the corresponding rule identifier (see the "Parsing trace" Section for details). For non-leaf nodes, the number of kids or subtrees is printed in parathesis. The line ends with the node image prefix (escaped as discussed in the "Parsing trace" Section). Kids, if any, are dumped on the next line(s) with increased indentation.

Here is a complete parsing tree produced by the docs/calc.cc grammar and "1+1;" input.

grammar#r103(1): 1+1;\n
  r-110(2): 1+1;\n
    expression#r101(1): 1+1
      r-114(3): 1+1
        expression#r101(1): 1
          number#r102: 1
        _+#r-112: +
        expression#r101(1): 1
          number#r102: 1
    _;#r-109: ;

Parsing tree nodes produced by implicit trimming rules are not shown and are not visible when traversing the parsing tree with user-level interfaces.

Context-sensitive Debugging

Using Actions, it is simple to enable and/or disable debugging depending on what rules are being tried. For example, the following code enables debugging when a particular rule has matched:

	using namespace Hapy;

	void TurnDebuggingOn(Action::Params *) {

	Rule particular;
	particular = ...;

Please see the Actions page for a discussion and more examples.

SourceForge Home