HAPY DEBUGGING

When the Parser::parse() method or equivalent is called the first time, each rule in the supplied grammar is "compiled" to accommodate various user settings (e.g., trimming) and to optimize future parsing. If debugging is enabled, then each compiled rule is printed on the standard error stream (std::clog).

For example, here is the rule compilation trace produced by the calculator grammar (see docs/calc.cc):

compiled rule:  _white#r-105 = white
compiled rule:  r-123 = *_white#r-105
compiled rule:  _digit#r-103 = digit
compiled rule:  _digit#r-103 = digit
compiled rule:  r-121 = *_digit#r-103
compiled rule:  r102 = _digit#r-103 >> r-121
...
compiled rule:  _;_trimmer#r108 = r-123 >> _;#r-109 >> r-123
compiled rule:  r-110 = r101 >> _;_trimmer#r108
compiled rule:  r103 = +r-110
	

The rDDD and r-DDD labels after the '#' sign are rule identifiers that Hapy assigned to the rule (e.g., r-105. Negative IDs correspond to intermediate rules. Positive IDs correspond to rules explicitly declared by the user.

The labels in front of the '#' sign are rule names automatically assigned by Hapy to some simple built-in rules. Most rules will not have these by default because Hapy cannot guess an appropriate name. To improve the readability of the trace, you should name rules. For example, if calculator rules are named to match their variable names, the compilation trace reveals these user-defined rules (among the same of intermediate rules with negative IDs):

compiled rule:  number#r102 = _digit#r-103 >> r-121
compiled rule:  number_trimmer#r104 = r-123 >> number#r102 >> r-123
compiled rule:  expression#r101 = number_trimmer#r104 | r-118 | r-114
compiled rule:  expression#r101 = number_trimmer#r104 | r-118 | r-114
compiled rule:  expression#r101 = number_trimmer#r104 | r-118 | r-114
compiled rule:  expression#r101 = number_trimmer#r104 | r-118 | r-114
compiled rule:  grammar#r103 = +r-110
	

This is still not perfect and even not pretty, of course. Improvement suggestions are welcome.

Some rules (e.g., expression#r101 above) are repeated because they are cloned during the compilation process and each clone is compiled. Cloning is necessary when the same rule can be reached by different paths from the start rule of the grammar and those paths have different user-preferences such as trimming. At the time of writing, Hapy does not try to minimize the number of clones.

The compilation trace is often essential for understanding the parsing trace because it maps rule IDs to rule definitions and because it shows auto-generated trimming rules.

The parsing trace shows parsing progress. Each time a rule is tried it is printed with a "try" label and the indentation level is increased. Each time the rule returns, the indentation level is decreased and the rule execution result is printed with a "match", "miss", "error", or "more" label.

Buffer contents or its prefix is printed after the "buf:" label, with the total buffer content length printed after the "clen:" label. The buffer is rendered so that it always appears on one line, with non-printable characters escaped using C++ character escaping conventions.

1/1-  try: grammar#r103::firstMatch buf: 1+1;\n clen:5
2/2-    try: r-110::firstMatch buf: 1+1;\n clen:5
3/3-      try: expression#r101::firstMatch buf: 1+1;\n clen:5
4/4-        try: number_trimmer#r104::firstMatch buf: 1+1;\n clen:5
5/5-          try: r-123::firstMatch buf: 1+1;\n clen:5
6/6-            try: _white#r-105::firstMatch buf: 1+1;\n clen:5
6/6-            miss: _white#r-105 buf: 1+1;\n clen:5
5/5-          match: r-123 buf: 1+1;\n clen:5
7/5-          try: number#r102::firstMatch buf: 1+1;\n clen:5
...
7/5-          match: number#r102 buf: +1;\n clen:4
11/5-          try: r-123::firstMatch buf: +1;\n clen:4
12/6-            try: _white#r-105::firstMatch buf: +1;\n clen:4
12/6-            miss: _white#r-105 buf: +1;\n clen:4
11/5-          match: r-123 buf: +1;\n clen:4
4/4-        match: number_trimmer#r104 buf: +1;\n clen:4
3/3-      match: expression#r101 buf: +1;\n clen:4
...
1/1-  match: grammar#r103 buf:  clen:0
	

The first trace column contains two numbers separated by a slash: the rule invocation identifier and the nesting level. The rule invocation identifier increases with every "try" statement. Searching for the same invocation identifier links the "try" statement with the corresponding rule execution result. For example, to find how the first expression rule matched, one could search for a "^3/" pattern.

Searching for a given nesting level allows to skip details at lower levels.

For each parsing tree node, Hapy dumps the corresponding rule identifier (see the "Parsing trace" Section for details). For non-leaf nodes, the number of kids or subtrees is printed in parathesis. The line ends with the node image prefix (escaped as discussed in the "Parsing trace" Section). Kids, if any, are dumped on the next line(s) with increased indentation.

Here is a complete parsing tree produced by the docs/calc.cc grammar and "1+1;" input.

grammar#r103(1): 1+1;\n
  r-110(2): 1+1;\n
    expression#r101(1): 1+1
      r-114(3): 1+1
        expression#r101(1): 1
          number#r102: 1
        _+#r-112: +
        expression#r101(1): 1
          number#r102: 1
    _;#r-109: ;
	

Parsing tree nodes produced by implicit trimming rules are not shown and are not visible when traversing the parsing tree with user-level interfaces.