Cpp: make DFAState edge lookups lock-free to speed up lexing#4953
Open
analog-cbarber wants to merge 1 commit into
Open
Cpp: make DFAState edge lookups lock-free to speed up lexing#4953analog-cbarber wants to merge 1 commit into
analog-cbarber wants to merge 1 commit into
Conversation
The C++ runtime stored each DFA state's outgoing edges in a
FlatHashMap<size_t, DFAState*> and guarded every access with a single
shared ATN-wide mutex (ATN::_edgeMutex). Because the lexer consults the
edge table once per input character, the hottest loop in the runtime paid
a shared-lock acquire plus a hash-map probe on every codepoint. The Java
reference runtime has neither cost: DFAState.edges is a plain array indexed
by symbol, and reads are lock-free (a benign miss simply recomputes the
edge, which is idempotent), while writes are serialized per state.
This change brings the C++ representation in line with Java:
- DFAState::edges becomes a lazily allocated, fixed-size array of
std::atomic<DFAState*>. getEdge() reads a slot with acquire ordering
and needs no lock; setEdge() publishes slots with release ordering and
is still serialized by the caller via ATN::_edgeMutex.
- Lexer and parser getExistingTargetState() now read edges lock-free,
removing the per-symbol shared-lock acquire from the hot path.
- The lexer and parser edge tables are allocated once at their natural
full size (the char range and maxTokenType+1 respectively) and never
reallocated, so a concurrent lock-free reader can never observe a moved
table. The only table that grows is the precedence DFA's start-state
table, which is read exclusively under ATN::_edgeMutex and never via
the lock-free path, so resizing it cannot race a reader.
- DFASerializer::getEdgeLabel shifts its index back by one when printing
edge labels (edges are indexed by t + 1 so EOF occupies slot 0),
matching the Java serializer; slot 0 renders as "EOF".
The concurrency contract is therefore identical to Java's: lock-free reads
tolerating a benign recompute-on-miss, with writes serialized. There is no
public API change.
Measured on an 84 MiB JSON input (Apple M5 Max, single-threaded interpreted
parse), the lexer stage goes from 26.7 to 47.9 MiB/s (~1.8x) and total
native parse from 15.0 to 20.9 MiB/s (~1.4x); a 54 MiB netlist input shows
1.7x / 1.3x. Output is byte-for-byte identical to the previous runtime
across all test inputs.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Signed-off-by: Christopher Barber <analog.cbarber@gmail.com>
analog-cbarber
force-pushed
the
cpp-lockfree-dfa-edges
branch
from
July 18, 2026 16:56
9066854 to
baa5db5
Compare
analog-cbarber
added a commit
to zuzukin/antlrope
that referenced
this pull request
Jul 18, 2026
The lock-free DFA-edge patch indexes edges by t + 1 (EOF in slot 0), but DFASerializer::getEdgeLabel still printed the raw slot index, shifting every DFA-dump edge label by one token type. Print getDisplayName(i - 1) as the Java serializer does. Caught by the upstream runtime testsuite on antlr/antlr4#4953/#4954; diagnostic output only — parse behavior and the event stream are unaffected. Updates the pinned fork SHAs (amended: baa5db5ba, 96ea7bcc5). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Replaces
DFAState::edges— aFlatHashMap<size_t, DFAState*>guarded by theATN-wide
_edgeMutex— with a lazily-allocated array ofstd::atomic<DFAState*>, giving a lock-freegetEdgeon the per-character hotpath. Writers still take the existing mutex; only the read path changes.
Why
The lexer consults the DFA edge table once per input character, so the hottest
loop in the C++ runtime pays a shared-lock acquire plus a hash-map probe per
codepoint. The Java reference runtime has neither cost:
DFAState.edgesthereis a plain array indexed by symbol, read without locking. This change brings
the C++ runtime to parity with the Java design.
Edges are indexed by
t + 1(as in the Java runtime) so EOF (t == -1) mapsto slot 0; the table is sized
maxTokenType + 2.Performance
Measured on a 4.7 MB JSON parse (Apple M5 Max, best of 7): ~12% off the
single-thread wall time (551 ms → 486 ms). JSON is lexer-heavy, which flatters
this change; parser-heavy grammars see less.
Safety / verification
t + 1offsetand produced an out-of-bounds write when a parser decision cached an edge on
EOF lookahead. That is fixed in this submission and the scenario is covered:
the patch has been running under AddressSanitizer in CI (single-token /
empty / truncated / malformed inputs across two grammars) in a downstream
project, clean.
array change does not alter which edges are cached.
pure-Python-runtime oracles) runs against this patch continuously.
Context
This patch (with #4954) is shipped today in antlrope, a
C++-accelerated Python runtime that drives
ParserInterpreter/LexerInterpreterfrom serialized ATNs. antlropecurrently vendors a patched copy of this runtime; once this change is merged
and released, it can drop the patch and vendor the unmodified upstream runtime
again. Benchmarks and methodology:
https://zuzukin.github.io/antlrope/concepts/ (the "vendored runtime and its
patches" section) and https://github.com/zuzukin/srdl-bench.
Authorship note: this patch was developed with AI assistance (Claude Opus
4.7, reflected in the commit trailer); I reviewed the design and code, and the
verification above (ASan CI, downstream test suite, benchmarks) is what I'd
stake it on.