Compare commits

...

77 Commits

Author SHA1 Message Date
David Given
07ebed83bf Fix some documentation from the global options change. 2025-08-21 19:14:26 +02:00
David Given
1def87fdc3 Remove the rawReadDiskCommand() function, as it's no longer used. 2025-08-21 19:00:29 +02:00
David Given
d91fed7dd4 Update documentation. 2025-08-21 18:59:02 +02:00
David Given
5f2f7e70ef Remove rawread and merge. 2025-08-21 18:55:46 +02:00
David Given
83432beff6 Merge pull request #822 from davidgiven/config
Overhaul the config file CLI.
2025-08-21 18:42:21 +02:00
David Given
979b550178 Looks like our string_view fix hasn't worked --- tweak. 2025-08-21 01:19:19 +02:00
David Given
9062a531f3 Migrate the 40track etc extension configs to actual options. Add the
ability to have --group=value options to make this cleaner.
2025-08-21 00:53:50 +02:00
David Given
e2a6fbcf3c Update a few places which used -c for other purposes. 2025-08-20 21:30:06 +02:00
David Given
ec16931f3a Update documentation for -c. 2025-08-20 21:23:02 +02:00
David Given
0ec0ca7495 Config files are now specified with -c, rather than via filename
arguments, because otherwise you get really unhelpful error messages
when you get things wrong.
2025-08-20 21:19:34 +02:00
David Given
51fa7c9371 Fix broken link.
Closes: #799
2025-08-20 00:23:21 +02:00
David Given
6c69f10fe7 Merge pull request #821 from davidgiven/protobuf
Expose the .app on OSX (in a zipfile).
2025-08-20 00:17:43 +02:00
David Given
206e85a356 Expose the .app on OSX (in a zipfile).
Closes: #800
2025-08-20 00:02:43 +02:00
David Given
8d7dd4867b Merge pull request #820 from davidgiven/protobuf
Apply the fix from #811 to make everything build against Protobuf 31.
2025-08-19 23:43:26 +02:00
David Given
d1524f78fb Apply the fix from #811 to make everything build against Protobuf 31. 2025-08-19 23:28:19 +02:00
David Given
b26735d520 Merge pull request #819 from davidgiven/fl2
Add some flux file manipulation tools.
2025-08-19 22:58:05 +02:00
David Given
603baee777 Fix a subtle bug that was causing misparsing of indexed fields on OSX. I hate
C++.
2025-08-19 22:43:45 +02:00
David Given
e105b7f498 Merge from master. 2025-08-19 20:13:41 +02:00
David Given
bb3fbccb50 Merge pull request #818 from davidgiven/convert
Add a fe-convert (plus all the necessary backend work).
2025-08-19 19:59:12 +02:00
David Given
c8edcd963d Merge pull request #817 from davidgiven/ab
Update ab.
2025-08-19 01:32:40 +02:00
David Given
3b60cdc707 Remove the -j from the build scripts for OSX. 2025-08-19 01:15:47 +02:00
David Given
ea061d65c9 Update ab. 2025-08-19 01:14:33 +02:00
David Given
da64c0237f Update documentation. 2025-08-19 01:11:40 +02:00
David Given
d2b1602881 Add a working fluxfile cp. 2025-08-19 00:55:53 +02:00
David Given
1afd45068c Merge. 2025-08-19 00:18:59 +02:00
David Given
f01b30e112 Make fluxfile rm work. 2025-08-19 00:18:47 +02:00
David Given
b5f7fbe14e Finally come up with a fluxfile ls I can live with. 2025-08-18 23:59:57 +02:00
David Given
8b6073ccbb Try making the error collector non-constexpr? 2025-08-18 22:14:48 +02:00
David Given
f902c759df Try the suggested workaround in lexy for older compilers. 2025-08-18 22:10:06 +02:00
David Given
996fdbc0f5 More overhauling of the proto layer; fluxfile ls now works. 2025-08-18 00:37:42 +02:00
David Given
9ff3e3b42a Finally make the getters and setters work with repeated fields. 2025-08-17 23:04:14 +02:00
David Given
0a5604521e Merge in fluxfile stuff. 2025-08-17 21:12:27 +02:00
David Given
786636ef5d Don't allow writing Apple 2 flux images to SCP files, because there
isn't space for the quarter-step tracks.
2025-08-17 11:42:34 +02:00
David Given
18bdb27225 fluxengine convert now uses the same syntax as the other tools. 2025-08-17 11:26:16 +02:00
David Given
faca35dec0 Update documentation. 2025-08-17 10:51:50 +02:00
David Given
f8813daae3 Attempt to make work on Windows. 2025-08-17 10:47:54 +02:00
David Given
da5a20390f Fix unhelpful message. 2025-08-17 10:40:34 +02:00
David Given
3ab3db92f5 Add basic support for TI-99 disks. 2025-08-17 10:40:07 +02:00
David Given
a3cd3dd9dc Adjust dependencies. 2025-08-17 09:45:54 +02:00
David Given
918868e9e8 Try updating the Ubuntu version. 2025-08-17 09:43:10 +02:00
David Given
cf05a25445 Does _error_collector need a constexpr constructor and destructor? 2025-08-17 01:01:32 +02:00
David Given
5d5399a267 Add another weirdly missing file. 2025-08-17 00:55:10 +02:00
David Given
2de7af0ba5 Add weirdly missing file. 2025-08-17 00:52:01 +02:00
David Given
0382c304ad Warning fix. 2025-08-17 00:46:50 +02:00
David Given
182d9946fe Add missing file. 2025-08-17 00:40:55 +02:00
David Given
f24e4029b4 Flux sources now add the locations of their data to _extraConfig ---
which is now honoured. Fix a bunch of bugs in some of the flux sources
and sinks. The converter now actually works, maybe.
2025-08-17 00:38:25 +02:00
David Given
4ebda29171 Rename track -> cylinder in lots of places. 2025-08-16 17:39:55 +02:00
David Given
53026f3d02 Rework the way locations are handled to use the new locations
microformat rather than the old RangeProto.
2025-08-16 16:59:44 +02:00
David Given
99c0e95a2f Added a routine for parsing location lists using Lexy. 2025-08-15 23:39:21 +02:00
David Given
dfa56c6b08 Raw import of Lexy. 2025-08-14 23:36:31 +02:00
David Given
0419df4b2d Another archival checkin... 2025-08-13 23:00:08 +02:00
David Given
70bdcd0978 Non-functioning archival checkin. 2025-08-12 20:31:54 +01:00
David Given
022df995aa Update for newer C++. 2025-08-11 16:21:03 +01:00
David Given
dcbe7ec41d Raw import of alphanum. 2025-08-11 16:14:27 +01:00
David Given
df4d27eefe Better support for repeated fields in the config language. Add a helper
for showing all config fields in a proto.
2025-08-10 22:22:58 +01:00
David Given
8f233f55e9 Add fluxfile ls. 2025-07-28 23:20:41 +01:00
David Given
7db49aec21 Merge pull request #814 from davidgiven/build
Update ab.
2025-07-28 13:36:21 +02:00
David Given
b5eaec0778 Try more Windows fix? 2025-07-28 12:23:41 +01:00
David Given
06b126a2e7 Typo fix. 2025-07-27 23:20:32 +01:00
David Given
ed96ebac79 Another Windows fix. 2025-07-27 23:08:37 +01:00
David Given
c6e34d2d88 Alternative Windows fix. 2025-07-27 22:50:43 +01:00
David Given
53ac8bad79 Hopefully fix Windows. 2025-07-27 21:43:26 +01:00
David Given
d2e163bc3b More Windows build debugging. 2025-07-27 21:33:52 +01:00
David Given
1404123281 Windows debugging. 2025-07-27 21:20:28 +01:00
David Given
01a7afd28a Merge from master. 2025-07-27 20:48:27 +01:00
David Given
3a42911e6f Update ab. 2025-07-27 20:48:10 +01:00
David Given
8e5d52f2c7 Update ab. 2025-07-24 23:25:41 +02:00
David Given
dfff5d7230 Merge pull request #796 from davidgiven/layout
Fix and expand the layout support to allow libdsk `altback` layouts.
2025-07-21 00:29:10 +02:00
David Given
19b63786c8 Merge from master. 2025-07-21 00:01:41 +02:00
David Given
5293e1c18b Merge pull request #792 from davidgiven/builds
Make an attempt to switch to WSL 1 for better builds.
2025-04-24 21:07:12 +02:00
Märt Põder
36b120bdbe Add Juku 5104 floppies with a new filesystem_track_order option 2024-11-30 15:31:24 +02:00
David Given
cc169d414f Add experimental support for libdsk 'altback' mode layouts. 2024-11-29 22:39:34 +01:00
David Given
0fcb2075e0 Move filesystem_track_ordering from ImageReaderProto/ImageWriterProto to
ImgInputOutputProto; it now only applies to img files. Make it honour the
appropriate track layout setting too.
2024-11-29 22:30:33 +01:00
David Given
2bda78fb40 Distinguish between filesystem track ordering and image track ordering
(although currently only the filesystem ordering is used).
2024-11-29 22:07:58 +01:00
David Given
e878c6eef6 Remove the unused sector_order field from FilesystemProto. 2024-11-29 21:24:32 +01:00
David Given
9ce405cec5 Remove the broken install rule. 2024-11-24 23:10:48 +01:00
David Given
f064d413b3 Add a docker test for Manjaro Linux. 2024-11-24 22:50:44 +01:00
277 changed files with 29960 additions and 1321 deletions

View File

@@ -20,7 +20,7 @@ jobs:
path: 'fluxengine-testdata'
- name: apt
run: |
sudo apt install libudev-dev libsqlite3-dev protobuf-compiler libwxgtk3.0-gtk3-dev libfmt-dev libprotobuf-dev wx-common
sudo apt install libudev-dev libsqlite3-dev protobuf-compiler libwxgtk3.0-gtk3-dev libfmt-dev libprotobuf-dev
- name: make
run: CXXFLAGS="-Wp,-D_GLIBCXX_ASSERTIONS" make -j`nproc` -C fluxengine
@@ -65,12 +65,14 @@ jobs:
run: |
brew install sqlite pkg-config libusb protobuf wxwidgets fmt make coreutils dylibbundler libjpeg
- name: make
run: gmake -C fluxengine -j2
run: gmake -C fluxengine
- name: Upload build artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ github.event.repository.name }}.${{ github.sha }}.fluxengine.${{ runner.arch }}.pkg
path: fluxengine/FluxEngine.pkg
path: |
fluxengine/FluxEngine.pkg
fluxengine/FluxEngine.app.zip
build-windows:
runs-on: windows-latest

View File

@@ -97,8 +97,9 @@ jobs:
- name: make
run: |
gmake -j2
gmake
mv FluxEngine.pkg FluxEngine-${{ runner.arch }}.pkg
mv FluxEngine.app.zip FluxEngine-${{ runner.arch }}.app.zip
- name: tag
uses: EndBug/latest-tag@latest
@@ -115,6 +116,7 @@ jobs:
tag: dev
assets: |
FluxEngine-${{ runner.arch }}.pkg
FluxEngine-${{ runner.arch }}.app.zip
fail-if-no-assets: false
- name: release
@@ -123,6 +125,7 @@ jobs:
name: Development build ${{ env.RELEASE_DATE }}
files: |
FluxEngine-${{ runner.arch }}.pkg
FluxEngine-${{ runner.arch }}.app.zip
tag_name: dev
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

View File

@@ -25,8 +25,10 @@ ifeq ($(BUILDTYPE),windows)
EXT = .exe
else
CC = gcc
CXX = g++ -std=c++17
CFLAGS = -g -O3
CXX = g++ -std=c++20
CFLAGS = -g -O3 \
-Wno-deprecated-enum-float-conversion \
-Wno-deprecated-enum-enum-conversion
LDFLAGS =
AR = ar
PKG_CONFIG = pkg-config
@@ -34,11 +36,10 @@ else
else
LDFLAGS += -pthread -Wl,--no-as-needed
endif
endif
HOSTCC = gcc
HOSTCXX = g++ -std=c++17
HOSTCXX = g++ -std=c++20
HOSTCFLAGS = -g -O3
HOSTLDFLAGS =
@@ -85,33 +86,23 @@ binaries: all
tests: all
README.md: $(OBJ)/scripts/+mkdocindex/mkdocindex$(EXT)
@echo $(PROGRESSINFO) MKDOC $@
@echo $(PROGRESSINFO)MKDOC $@
@csplit -s -f$(OBJ)/README. README.md '/<!-- FORMATSSTART -->/' '%<!-- FORMATSEND -->%'
@(cat $(OBJ)/README.00 && $< && cat $(OBJ)/README.01) > README.md
.PHONY: tests
.PHONY: install install-bin
install:: all install-bin
clean::
$(hide) rm -rf $(REALOBJ)
install-bin:
@echo "INSTALL"
$(hide) install -D -v "$(OBJ)/src+fluxengine/src+fluxengine" "$(DESTDIR)$(BINDIR)/fluxengine"
$(hide) install -D -v "$(OBJ)/src/gui+gui/gui+gui" "$(DESTDIR)$(BINDIR)/fluxengine-gui"
$(hide) install -D -v "$(OBJ)/tools+brother120tool/tools+brother120tool" "$(DESTDIR)$(BINDIR)/brother120tool"
$(hide) install -D -v "$(OBJ)/tools+brother240tool/tools+brother240tool" "$(DESTDIR)$(BINDIR)/brother240tool"
$(hide) install -D -v "$(OBJ)/tools+upgrade-flux-file/tools+upgrade-flux-file" "$(DESTDIR)$(BINDIR)/upgrade-flux-file"
include build/ab.mk
DOCKERFILES = \
debian11 \
debian12 \
fedora40 \
fedora41
fedora41 \
manjaro
docker-%: tests/docker/Dockerfile.%
docker build -t $* -f $< .

View File

@@ -125,6 +125,7 @@ choices because they can store multiple types of file system.
| [`hplif`](doc/disk-hplif.md) | Hewlett-Packard LIF: a variety of disk formats used by HP | 🦄 | 🦄 | LIF |
| [`ibm`](doc/disk-ibm.md) | IBM PC: Generic PC 3.5"/5.25" disks | 🦄 | 🦄 | FATFS |
| [`icl30`](doc/disk-icl30.md) | ICL Model 30: CP/M; 263kB 35-track DSSD | 🦖 | | CPMFS |
| [`juku`](doc/disk-juku.md) | Juku E5104: CP/M | | | CPMFS |
| [`mac`](doc/disk-mac.md) | Macintosh: 400kB/800kB 3.5" GCR | 🦄 | 🦄 | MACHFS |
| [`micropolis`](doc/disk-micropolis.md) | Micropolis: 100tpi MetaFloppy disks | 🦄 | 🦄 | |
| [`ms2000`](doc/disk-ms2000.md) | : MS2000 Microdisk Development System | | | MICRODOS |
@@ -136,6 +137,7 @@ choices because they can store multiple types of file system.
| [`rx50`](doc/disk-rx50.md) | Digital RX50: 400kB 5.25" 80-track 10-sector SSDD | 🦖 | 🦖 | |
| [`smaky6`](doc/disk-smaky6.md) | Smaky 6: 308kB 5.25" 77-track 16-sector SSDD, hard sectored | 🦖 | | SMAKY6 |
| [`tartu`](doc/disk-tartu.md) | Tartu: The Palivere and variations | 🦄 | 🦖 | CPMFS |
| [`ti99`](doc/disk-ti99.md) | TI-99: 90kB 35-track SSSD | 🦖 | | |
| [`tids990`](doc/disk-tids990.md) | Texas Instruments DS990: 1126kB 8" DSSD | 🦖 | 🦖 | |
| [`tiki`](doc/disk-tiki.md) | Tiki 100: CP/M | | | CPMFS |
| [`victor9k`](doc/disk-victor9k.md) | Victor 9000 / Sirius One: 1224kB 5.25" DSDD GCR | 🦖 | 🦖 | |
@@ -257,6 +259,15 @@ package, written by Robert Leslie et al, taken from
https://www.mars.org/home/rob/proj/hfs. It is GPL 2.0 licensed. Please see the
contents of the directory for the full text.
As an exception, `dep/lexy` contains a partial copy of the lexy package, written
by foonathen@github, taken from https://github.com/foonathan/lexy. It is BSL 1.0
licensed. Please see the contents of the directory for the full text.
As an exception, `dep/alphanum` contains a copy of the alphanum package,
written by Dave Koelle, taken from
https://web.archive.org/web/20210207124255/davekoelle.com/alphanum.html. It is
MIT licensed. Please see the source for the full text.
__Important:__ Because of all these exceptions, if you distribute the
FluxEngine package as a whole, you must comply with the terms of _all_ of the
licensing terms. This means that __effectively the FluxEngine package is

View File

@@ -7,6 +7,12 @@ from glob import glob
import config
import re
# Hack for building on Fedora/WSL; executables get the .exe extension,
# build the build system detects it as Linux.
import build.toolchain
toolchain.Toolchain.EXE = "$(EXT)"
package(name="protobuf_lib", package="protobuf")
package(name="z_lib", package="zlib")
package(name="fmt_lib", package="fmt", fallback="dep/fmt")
@@ -22,7 +28,7 @@ else:
("acorndfs", "", "--200"),
("agat", "", ""),
("amiga", "", ""),
("apple2", "", "--140 40track_drive"),
("apple2", "", "--140 --drivetype=40"),
("atarist", "", "--360"),
("atarist", "", "--370"),
("atarist", "", "--400"),
@@ -32,17 +38,17 @@ else:
("atarist", "", "--800"),
("atarist", "", "--820"),
("bk", "", ""),
("brother", "", "--120 40track_drive"),
("brother", "", "--120 --drivetype=40"),
("brother", "", "--240"),
(
"commodore",
"scripts/commodore1541_test.textpb",
"--171 40track_drive",
"--171 --drivetype=40",
),
(
"commodore",
"scripts/commodore1541_test.textpb",
"--192 40track_drive",
"--192 --drivetype=40",
),
("commodore", "", "--800"),
("commodore", "", "--1620"),
@@ -54,17 +60,17 @@ else:
("ibm", "", "--1232"),
("ibm", "", "--1440"),
("ibm", "", "--1680"),
("ibm", "", "--180 40track_drive"),
("ibm", "", "--160 40track_drive"),
("ibm", "", "--320 40track_drive"),
("ibm", "", "--360 40track_drive"),
("ibm", "", "--180 --drivetype=40"),
("ibm", "", "--160 --drivetype=40"),
("ibm", "", "--320 --drivetype=40"),
("ibm", "", "--360 --drivetype=40"),
("ibm", "", "--720_96"),
("ibm", "", "--720_135"),
("mac", "scripts/mac400_test.textpb", "--400"),
("mac", "scripts/mac800_test.textpb", "--800"),
("n88basic", "", ""),
("rx50", "", ""),
("tartu", "", "--390 40track_drive"),
("tartu", "", "--390 --drivetype=40"),
("tartu", "", "--780"),
("tids990", "", ""),
("victor9k", "", "--612"),
@@ -103,6 +109,13 @@ export(
"brother240tool$(EXT)": "tools+brother240tool",
"upgrade-flux-file$(EXT)": "tools+upgrade-flux-file",
}
| ({"FluxEngine.pkg": "src/gui+fluxengine_pkg"} if config.osx else {}),
| (
{
"FluxEngine.pkg": "src/gui+fluxengine_pkg",
"FluxEngine.app.zip": "src/gui+fluxengine_app_zip",
}
if config.osx
else {}
),
deps=["tests", "src/formats+docs", "scripts+mkdocindex"] + corpustests,
)

View File

@@ -26,7 +26,7 @@ def main():
print("link", sf)
os.makedirs(dirname(sf), exist_ok=True)
try:
os.link(abspath(f), sf)
os.symlink(abspath(f), sf)
except PermissionError:
shutil.copy(f, sf)
@@ -38,6 +38,11 @@ def main():
df = dirname(f)
if df:
os.makedirs(df, exist_ok=True)
try:
os.remove(f)
except FileNotFoundError:
pass
os.rename(sf, f)

View File

@@ -87,7 +87,12 @@ $(PKG_CONFIG_HASHES) $(HOST_PKG_CONFIG_HASHES) &:
include $(OBJ)/build.mk
MAKEFLAGS += -r -j$(shell nproc)
ifeq ($(OSX),yes)
MAKEFLAGS += -r -j$(shell sysctl -n hw.logicalcpu)
else
MAKEFLAGS += -r -j$(shell nproc)
endif
.DELETE_ON_ERROR:
.PHONY: update-ab

View File

@@ -501,15 +501,17 @@ def emit_rule(self, ins, outs, cmds=[], label=None):
emit(f"OUTS_{outsn}", "=", *fouts, into=lines)
emit(f"INS_{insn}", "=", *fins, into=lines)
emit(name, ":", f"$(OUTS_{outsn})", into=lines)
emit(hashfile, ":", into=lines)
emit(f"\t@mkdir -p {self.dir}", into=lines)
emit(f"\t@touch {hashfile}", into=lines)
emit(
name,
":",
hashfile,
f"$(OUTS_{outsn})",
"&:" if len(fouts) > 1 else ":",
f"$(INS_{insn})",
hashfile,
into=lines,
)
emit(f"$(OUTS_{outsn})", ":", hashfile, into=lines)
emit(hashfile, ":", f"$(INS_{insn})", into=lines)
if label:
emit("\t$(hide)", "$(ECHO) $(PROGRESSINFO)" + label, into=lines)
@@ -537,9 +539,6 @@ def emit_rule(self, ins, outs, cmds=[], label=None):
emit(name, ":", *fins, into=lines)
outputFp.write("".join(lines))
if outs:
emit(f"\t$(hide) touch {hashfile}")
emit("")

View File

@@ -444,7 +444,7 @@ def programimpl(
simplerule(
replaces=self,
ins=cfiles + libs,
outs=[f"={self.localname}$(EXT)"],
outs=[f"={self.localname}{toolchain.EXE}"],
deps=deps,
label=label,
commands=commands,

View File

@@ -1,5 +1,10 @@
import platform
_is_windows = (platform.system() == "Windows")
class Toolchain:
PREFIX = ""
EXE = ".exe" if _is_windows else ""
class HostToolchain(Toolchain):

2
dep/alphanum/UPSTREAM.md Normal file
View File

@@ -0,0 +1,2 @@
Downloaded from:
https://web.archive.org/web/20210918044134/http://davekoelle.com/files/alphanum.hpp

450
dep/alphanum/alphanum.h Normal file
View File

@@ -0,0 +1,450 @@
#ifndef ALPHANUM__HPP
#define ALPHANUM__HPP
/*
Released under the MIT License - https://opensource.org/licenses/MIT
Permission is hereby granted, free of charge, to any person obtaining
a copy of this software and associated documentation files (the "Software"),
to deal in the Software without restriction, including without limitation
the rights to use, copy, modify, merge, publish, distribute, sublicense,
and/or sell copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included
in all copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM,
DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE
USE OR OTHER DEALINGS IN THE SOFTWARE.
*/
/* $Header: /code/doj/alphanum.hpp,v 1.3 2008/01/28 23:06:47 doj Exp $ */
#include <cassert>
#include <functional>
#include <string>
#include <sstream>
#ifdef ALPHANUM_LOCALE
#include <cctype>
#endif
#ifdef DOJDEBUG
#include <iostream>
#include <typeinfo>
#endif
// TODO: make comparison with hexadecimal numbers. Extend the alphanum_comp() function by traits to choose between decimal and hexadecimal.
namespace doj
{
// anonymous namespace for functions we use internally. But if you
// are coding in C, you can use alphanum_impl() directly, since it
// uses not C++ features.
namespace {
// if you want to honour the locale settings for detecting digit
// characters, you should define ALPHANUM_LOCALE
#ifdef ALPHANUM_LOCALE
/** wrapper function for ::isdigit() */
bool alphanum_isdigit(int c)
{
return isdigit(c);
}
#else
/** this function does not consider the current locale and only
works with ASCII digits.
@return true if c is a digit character
*/
bool alphanum_isdigit(const char c)
{
return c>='0' && c<='9';
}
#endif
/**
compare l and r with strcmp() semantics, but using
the "Alphanum Algorithm". This function is designed to read
through the l and r strings only one time, for
maximum performance. It does not allocate memory for
substrings. It can either use the C-library functions isdigit()
and atoi() to honour your locale settings, when recognizing
digit characters when you "#define ALPHANUM_LOCALE=1" or use
it's own digit character handling which only works with ASCII
digit characters, but provides better performance.
@param l NULL-terminated C-style string
@param r NULL-terminated C-style string
@return negative if l<r, 0 if l equals r, positive if l>r
*/
int alphanum_impl(const char *l, const char *r)
{
enum mode_t { STRING, NUMBER } mode=STRING;
while(*l && *r)
{
if(mode == STRING)
{
char l_char, r_char;
while((l_char=*l) && (r_char=*r))
{
// check if this are digit characters
const bool l_digit=alphanum_isdigit(l_char), r_digit=alphanum_isdigit(r_char);
// if both characters are digits, we continue in NUMBER mode
if(l_digit && r_digit)
{
mode=NUMBER;
break;
}
// if only the left character is a digit, we have a result
if(l_digit) return -1;
// if only the right character is a digit, we have a result
if(r_digit) return +1;
// compute the difference of both characters
const int diff=l_char - r_char;
// if they differ we have a result
if(diff != 0) return diff;
// otherwise process the next characters
++l;
++r;
}
}
else // mode==NUMBER
{
#ifdef ALPHANUM_LOCALE
// get the left number
char *end;
unsigned long l_int=strtoul(l, &end, 0);
l=end;
// get the right number
unsigned long r_int=strtoul(r, &end, 0);
r=end;
#else
// get the left number
unsigned long l_int=0;
while(*l && alphanum_isdigit(*l))
{
// TODO: this can overflow
l_int=l_int*10 + *l-'0';
++l;
}
// get the right number
unsigned long r_int=0;
while(*r && alphanum_isdigit(*r))
{
// TODO: this can overflow
r_int=r_int*10 + *r-'0';
++r;
}
#endif
// if the difference is not equal to zero, we have a comparison result
const long diff=l_int-r_int;
if(diff != 0)
return diff;
// otherwise we process the next substring in STRING mode
mode=STRING;
}
}
if(*r) return -1;
if(*l) return +1;
return 0;
}
}
/**
Compare left and right with the same semantics as strcmp(), but with the
"Alphanum Algorithm" which produces more human-friendly
results. The classes lT and rT must implement "std::ostream
operator<< (std::ostream&, const Ty&)".
@return negative if left<right, 0 if left==right, positive if left>right.
*/
template <typename lT, typename rT>
int alphanum_comp(const lT& left, const rT& right)
{
#ifdef DOJDEBUG
std::clog << "alphanum_comp<" << typeid(left).name() << "," << typeid(right).name() << "> " << left << "," << right << std::endl;
#endif
std::ostringstream l; l << left;
std::ostringstream r; r << right;
return alphanum_impl(l.str().c_str(), r.str().c_str());
}
/**
Compare l and r with the same semantics as strcmp(), but with
the "Alphanum Algorithm" which produces more human-friendly
results.
@return negative if l<r, 0 if l==r, positive if l>r.
*/
template <>
int alphanum_comp<std::string>(const std::string& l, const std::string& r)
{
#ifdef DOJDEBUG
std::clog << "alphanum_comp<std::string,std::string> " << l << "," << r << std::endl;
#endif
return alphanum_impl(l.c_str(), r.c_str());
}
////////////////////////////////////////////////////////////////////////////
// now follow a lot of overloaded alphanum_comp() functions to get a
// direct call to alphanum_impl() upon the various combinations of c
// and c++ strings.
/**
Compare l and r with the same semantics as strcmp(), but with
the "Alphanum Algorithm" which produces more human-friendly
results.
@return negative if l<r, 0 if l==r, positive if l>r.
*/
int alphanum_comp(char* l, char* r)
{
assert(l);
assert(r);
#ifdef DOJDEBUG
std::clog << "alphanum_comp<char*,char*> " << l << "," << r << std::endl;
#endif
return alphanum_impl(l, r);
}
int alphanum_comp(const char* l, const char* r)
{
assert(l);
assert(r);
#ifdef DOJDEBUG
std::clog << "alphanum_comp<const char*,const char*> " << l << "," << r << std::endl;
#endif
return alphanum_impl(l, r);
}
int alphanum_comp(char* l, const char* r)
{
assert(l);
assert(r);
#ifdef DOJDEBUG
std::clog << "alphanum_comp<char*,const char*> " << l << "," << r << std::endl;
#endif
return alphanum_impl(l, r);
}
int alphanum_comp(const char* l, char* r)
{
assert(l);
assert(r);
#ifdef DOJDEBUG
std::clog << "alphanum_comp<const char*,char*> " << l << "," << r << std::endl;
#endif
return alphanum_impl(l, r);
}
int alphanum_comp(const std::string& l, char* r)
{
assert(r);
#ifdef DOJDEBUG
std::clog << "alphanum_comp<std::string,char*> " << l << "," << r << std::endl;
#endif
return alphanum_impl(l.c_str(), r);
}
int alphanum_comp(char* l, const std::string& r)
{
assert(l);
#ifdef DOJDEBUG
std::clog << "alphanum_comp<char*,std::string> " << l << "," << r << std::endl;
#endif
return alphanum_impl(l, r.c_str());
}
int alphanum_comp(const std::string& l, const char* r)
{
assert(r);
#ifdef DOJDEBUG
std::clog << "alphanum_comp<std::string,const char*> " << l << "," << r << std::endl;
#endif
return alphanum_impl(l.c_str(), r);
}
int alphanum_comp(const char* l, const std::string& r)
{
assert(l);
#ifdef DOJDEBUG
std::clog << "alphanum_comp<const char*,std::string> " << l << "," << r << std::endl;
#endif
return alphanum_impl(l, r.c_str());
}
////////////////////////////////////////////////////////////////////////////
/**
Functor class to compare two objects with the "Alphanum
Algorithm". If the objects are no std::string, they must
implement "std::ostream operator<< (std::ostream&, const Ty&)".
*/
template<class Ty>
struct alphanum_less
{
bool operator()(const Ty& left, const Ty& right) const
{
return alphanum_comp(left, right) < 0;
}
};
}
#ifdef TESTMAIN
#include <algorithm>
#include <iostream>
#include <iterator>
#include <map>
#include <set>
#include <vector>
int main()
{
// testcases for the algorithm
assert(doj::alphanum_comp("","") == 0);
assert(doj::alphanum_comp("","a") < 0);
assert(doj::alphanum_comp("a","") > 0);
assert(doj::alphanum_comp("a","a") == 0);
assert(doj::alphanum_comp("","9") < 0);
assert(doj::alphanum_comp("9","") > 0);
assert(doj::alphanum_comp("1","1") == 0);
assert(doj::alphanum_comp("1","2") < 0);
assert(doj::alphanum_comp("3","2") > 0);
assert(doj::alphanum_comp("a1","a1") == 0);
assert(doj::alphanum_comp("a1","a2") < 0);
assert(doj::alphanum_comp("a2","a1") > 0);
assert(doj::alphanum_comp("a1a2","a1a3") < 0);
assert(doj::alphanum_comp("a1a2","a1a0") > 0);
assert(doj::alphanum_comp("134","122") > 0);
assert(doj::alphanum_comp("12a3","12a3") == 0);
assert(doj::alphanum_comp("12a1","12a0") > 0);
assert(doj::alphanum_comp("12a1","12a2") < 0);
assert(doj::alphanum_comp("a","aa") < 0);
assert(doj::alphanum_comp("aaa","aa") > 0);
assert(doj::alphanum_comp("Alpha 2","Alpha 2") == 0);
assert(doj::alphanum_comp("Alpha 2","Alpha 2A") < 0);
assert(doj::alphanum_comp("Alpha 2 B","Alpha 2") > 0);
assert(doj::alphanum_comp(1,1) == 0);
assert(doj::alphanum_comp(1,2) < 0);
assert(doj::alphanum_comp(2,1) > 0);
assert(doj::alphanum_comp(1.2,3.14) < 0);
assert(doj::alphanum_comp(3.14,2.71) > 0);
assert(doj::alphanum_comp(true,true) == 0);
assert(doj::alphanum_comp(true,false) > 0);
assert(doj::alphanum_comp(false,true) < 0);
std::string str("Alpha 2");
assert(doj::alphanum_comp(str,"Alpha 2") == 0);
assert(doj::alphanum_comp(str,"Alpha 2A") < 0);
assert(doj::alphanum_comp("Alpha 2 B",str) > 0);
assert(doj::alphanum_comp(str,strdup("Alpha 2")) == 0);
assert(doj::alphanum_comp(str,strdup("Alpha 2A")) < 0);
assert(doj::alphanum_comp(strdup("Alpha 2 B"),str) > 0);
#if 1
// show usage of the comparison functor with a set
std::set<std::string, doj::alphanum_less<std::string> > s;
s.insert("Xiph Xlater 58");
s.insert("Xiph Xlater 5000");
s.insert("Xiph Xlater 500");
s.insert("Xiph Xlater 50");
s.insert("Xiph Xlater 5");
s.insert("Xiph Xlater 40");
s.insert("Xiph Xlater 300");
s.insert("Xiph Xlater 2000");
s.insert("Xiph Xlater 10000");
s.insert("QRS-62F Intrinsia Machine");
s.insert("QRS-62 Intrinsia Machine");
s.insert("QRS-60F Intrinsia Machine");
s.insert("QRS-60 Intrinsia Machine");
s.insert("Callisto Morphamax 7000 SE2");
s.insert("Callisto Morphamax 7000 SE");
s.insert("Callisto Morphamax 7000");
s.insert("Callisto Morphamax 700");
s.insert("Callisto Morphamax 600");
s.insert("Callisto Morphamax 5000");
s.insert("Callisto Morphamax 500");
s.insert("Callisto Morphamax");
s.insert("Alpha 2A-900");
s.insert("Alpha 2A-8000");
s.insert("Alpha 2A");
s.insert("Alpha 200");
s.insert("Alpha 2");
s.insert("Alpha 100");
s.insert("Allegia 60 Clasteron");
s.insert("Allegia 52 Clasteron");
s.insert("Allegia 51B Clasteron");
s.insert("Allegia 51 Clasteron");
s.insert("Allegia 500 Clasteron");
s.insert("Allegia 50 Clasteron");
s.insert("40X Radonius");
s.insert("30X Radonius");
s.insert("20X Radonius Prime");
s.insert("20X Radonius");
s.insert("200X Radonius");
s.insert("10X Radonius");
s.insert("1000X Radonius Maximus");
// print sorted set to cout
std::copy(s.begin(), s.end(), std::ostream_iterator<std::string>(std::cout, "\n"));
// show usage of comparision functor with a map
typedef std::map<std::string, int, doj::alphanum_less<std::string> > m_t;
m_t m;
m["z1.doc"]=1;
m["z10.doc"]=2;
m["z100.doc"]=3;
m["z101.doc"]=4;
m["z102.doc"]=5;
m["z11.doc"]=6;
m["z12.doc"]=7;
m["z13.doc"]=8;
m["z14.doc"]=9;
m["z15.doc"]=10;
m["z16.doc"]=11;
m["z17.doc"]=12;
m["z18.doc"]=13;
m["z19.doc"]=14;
m["z2.doc"]=15;
m["z20.doc"]=16;
m["z3.doc"]=17;
m["z4.doc"]=18;
m["z5.doc"]=19;
m["z6.doc"]=20;
m["z7.doc"]=21;
m["z8.doc"]=22;
m["z9.doc"]=23;
// print sorted map to cout
for(m_t::iterator i=m.begin(); i!=m.end(); ++i)
std::cout << i->first << '\t' << i->second << std::endl;
// show usage of comparison functor with an STL algorithm on a vector
std::vector<std::string> v;
// vector contents are reversed sorted contents of the old set
std::copy(s.rbegin(), s.rend(), std::back_inserter(v));
// now sort the vector with the algorithm
std::sort(v.begin(), v.end(), doj::alphanum_less<std::string>());
// and print the vector to cout
std::copy(v.begin(), v.end(), std::ostream_iterator<std::string>(std::cout, "\n"));
#endif
return 0;
}
#endif
#endif

8
dep/alphanum/build.py Normal file
View File

@@ -0,0 +1,8 @@
from build.c import clibrary
clibrary(
name="alphanum",
srcs=[],
hdrs={"dep/alphanum/alphanum.h": "./alphanum.h"},
)

23
dep/lexy/LICENSE Normal file
View File

@@ -0,0 +1,23 @@
Boost Software License - Version 1.0 - August 17th, 2003
Permission is hereby granted, free of charge, to any person or organization
obtaining a copy of the software and accompanying documentation covered by
this license (the "Software") to use, reproduce, display, distribute,
execute, and transmit the Software, and to prepare derivative works of the
Software, and to permit third-parties to whom the Software is furnished to
do so, all subject to the following:
The copyright notices in the Software and this entire statement, including
the above license grant, this restriction and the following disclaimer,
must be included in all copies of the Software, in whole or in part, and
all derivative works of the Software, unless such copies or derivative
works are solely in the form of machine-executable object code generated by
a source language processor.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE, TITLE AND NON-INFRINGEMENT. IN NO EVENT
SHALL THE COPYRIGHT HOLDERS OR ANYONE DISTRIBUTING THE SOFTWARE BE LIABLE
FOR ANY DAMAGES OR OTHER LIABILITY, WHETHER IN CONTRACT, TORT OR OTHERWISE,
ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
DEALINGS IN THE SOFTWARE.

175
dep/lexy/README.adoc Normal file
View File

@@ -0,0 +1,175 @@
= lexy
ifdef::env-github[]
image:https://img.shields.io/endpoint?url=https%3A%2F%2Fwww.jonathanmueller.dev%2Fproject%2Flexy%2Findex.json[Project Status,link=https://www.jonathanmueller.dev/project/]
image:https://github.com/foonathan/lexy/workflows/Main%20CI/badge.svg[Build Status]
image:https://img.shields.io/badge/try_it_online-blue[Playground,link=https://lexy.foonathan.net/playground]
endif::[]
lexy is a parser combinator library for {cpp}17 and onwards.
It allows you to write a parser by specifying it in a convenient {cpp} DSL,
which gives you all the flexibility and control of a handwritten parser without any of the manual work.
ifdef::env-github[]
*Documentation*: https://lexy.foonathan.net/[lexy.foonathan.net]
endif::[]
.IPv4 address parser
--
ifndef::env-github[]
[.godbolt-example]
.+++<a href="https://godbolt.org/z/scvajjE17", title="Try it online">{{< svg "icons/play.svg" >}}</a>+++
endif::[]
[source,cpp]
----
namespace dsl = lexy::dsl;
// Parse an IPv4 address into a `std::uint32_t`.
struct ipv4_address
{
// What is being matched.
static constexpr auto rule = []{
// Match a sequence of (decimal) digits and convert it into a std::uint8_t.
auto octet = dsl::integer<std::uint8_t>;
// Match four of them separated by periods.
return dsl::times<4>(octet, dsl::sep(dsl::period)) + dsl::eof;
}();
// How the matched output is being stored.
static constexpr auto value
= lexy::callback<std::uint32_t>([](std::uint8_t a, std::uint8_t b, std::uint8_t c, std::uint8_t d) {
return (a << 24) | (b << 16) | (c << 8) | d;
});
};
----
--
== Features
Full control::
* *Describe the parser, not some abstract grammar*:
Unlike parser generators that use some table driven magic for parsing, lexy's grammar is just syntax sugar for a hand-written recursive descent parser.
The parsing algorithm does exactly what you've instructed it to do -- no more ambiguities or weird shift/reduce errors!
* *No implicit backtracking or lookahead*:
It will only backtrack when you say it should, and only lookahead when and how far you want it.
Don't worry about rules that have side-effects, they won't be executed unnecessarily thanks to the user-specified lookahead conditions.
https://lexy.foonathan.net/playground?example=peek[Try it online].
* *Escape hatch for manual parsing*:
Sometimes you want to parse something that can't be expressed easily with lexy's facilities.
Don't worry, you can integrate a hand-written parser into the grammar at any point.
https://lexy.foonathan.net/playground/?example=scan[Try it online].
* *Tracing*:
Figure out why the grammar isn't working the way you want it to.
https://lexy.foonathan.net/playground/?example=trace&mode=trace[Try it online].
Easily integrated::
* *A pure {cpp} DSL*:
No need to use an external grammar file; embed the grammar directly in your {cpp} project using operator overloading and functions.
* *Bring your own data structures*:
You can directly store results into your own types and have full control over all heap allocations.
* *Fully `constexpr` parsing*:
You want to parse a string literal at compile-time? You can do so.
* *Minimal standard library dependencies*:
The core parsing library only depends on fundamental headers such as `<type_traits>` or `<cstddef>`; no big includes like `<vector>` or `<algorithm>`.
* *Header-only core library* (by necessity, not by choice -- it's `constexpr` after all).
ifdef::env-github[Designed for text::]
ifndef::env-github[Designed for text (e.g. {{< github-example json >}}, {{< github-example xml >}}, {{< github-example email >}}) ::]
* *Unicode support*: parse UTF-8, UTF-16, or UTF-32, and access the Unicode character database to query char classes or perform case folding.
https://lexy.foonathan.net/playground?example=identifier-unicode[Try it online].
* *Convenience*:
Built-in rules for parsing nested structures, quotes and escape sequences.
https://lexy.foonathan.net/playground?example=parenthesized[Try it online].
* *Automatic whitespace skipping*:
No need to manually handle whitespace or comments.
https://lexy.foonathan.net/playground/?example=whitespace_comment[Try it online].
ifdef::env-github[Designed for programming languages::]
ifndef::env-github[Designed for programming languages (e.g. {{< github-example calculator >}}, {{< github-example shell >}})::]
* *Keyword and identifier parsing*:
Reserve a set of keywords that won't be matched as regular identifiers.
https://lexy.foonathan.net/playground/?example=reserved_identifier[Try it online].
* *Operator parsing*:
Parse unary/binary operators with different precedences and associativity, including chained comparisons `a < b < c`.
https://lexy.foonathan.net/playground/?example=expr[Try it online].
* *Automatic error recovery*:
Log an error, recover, and continue parsing!
https://lexy.foonathan.net/playground/?example=recover[Try it online].
ifdef::env-github[Designed for binary input::]
ifndef::env-github[Designed for binary input (e.g. {{< github-example protobuf >}})::]
* *Bytes*: Rules for parsing `N` bytes or Nbit big/little endian integer.
* *Bits*: Rules for parsing individual bit patterns.
* *Blobs*: Rules for parsing TLV formats.
== FAQ
Why should I use lexy over XYZ?::
lexy is closest to other PEG parsers.
However, they usually do more implicit backtracking, which can hurt performance and you need to be very careful with rules that have side-effects.
This is not the case for lexy, where backtracking is controlled using branch conditions.
lexy also gives you a lot of control over error reporting, supports error recovery, special support for operator precedence parsing, and other advanced features.
http://boost-spirit.com/home/[Boost.Spirit]:::
The main difference: it is not a Boost library.
In addition, Boost.Spirit is quite old and doesn't support e.g. non-common ranges as input.
Boost.Spirit also eagerly creates attributes from the rules, which can lead to nested tuples/variants while lexy uses callbacks which enables zero-copy parsing directly into your own data structure.
However, lexy's grammar is more verbose and designed to parser bigger grammars instead of the small one-off rules that Boost.Spirit is good at.
https://github.com/taocpp/PEGTL[PEGTL]:::
PEGTL is very similar and was a big inspiration.
The biggest difference is that lexy uses an operator based DSL instead of inheriting from templated classes as PEGTL does;
depending on your preference this can be an advantage or disadvantage.
Hand-written Parsers:::
Writing a handwritten parser is more manual work and error prone.
lexy automates that away without having to sacrifice control.
You can use it to quickly prototype a parser and then slowly replace more and more with a handwritten parser over time;
mixing a hand-written parser and a lexy grammar works seamlessly.
How bad are the compilation times?::
They're not as bad as you might expect (in debug mode, that is).
+
The example JSON parser compiles in about 2s on my machine.
If we remove all the lexy specific parts and just benchmark the time it takes for the compiler to process the datastructure (and stdlib includes),
that takes about 700ms.
If we validate JSON only instead of parsing it, so remove the data structures and keep only the lexy specific parts, we're looking at about 840ms.
+
Keep in mind, that you can fully isolate lexy in a single translation unit that only needs to be touched when you change the parser.
You can also split a lexy grammar into multiple translation units using the `dsl::subgrammar` rule.
How bad are the {cpp} error messages if you mess something up?::
They're certainly worse than the error message lexy gives you.
The big problem here is that the first line gives you the error, followed by dozens of template instantiations, which end at your `lexy::parse` call.
Besides providing an external tool to filter those error messages, there is nothing I can do about that.
How fast is it?::
Benchmarks are available in the `benchmarks/` directory.
A sample result of the JSON validator benchmark which compares the example JSON parser with various other implementations is available https://lexy.foonathan.net/benchmark_json/[here].
Why is it called lexy?::
I previously had a tokenizer library called foonathan/lex.
I've tried adding a parser to it, but found that the line between pure tokenization and parsing has become increasingly blurred.
lexy is a re-imagination on of the parser I've added to foonathan/lex, and I've simply kept a similar name.
ifdef::env-github[]
== Documentation
The documentation, including tutorials, reference documentation, and an interactive playground can be found at https://lexy.foonathan.net/[lexy.foonathan.net].
A minimal `CMakeLists.txt` that uses lexy can look like this:
.`CMakeLists.txt`
```cmake
project(lexy-example)
include(FetchContent)
FetchContent_Declare(lexy URL https://lexy.foonathan.net/download/lexy-src.zip)
FetchContent_MakeAvailable(lexy)
add_executable(lexy_example)
target_sources(lexy_example PRIVATE main.cpp)
target_link_libraries(lexy_example PRIVATE foonathan::lexy)
```
endif::[]

2
dep/lexy/UPSTREAM.md Normal file
View File

@@ -0,0 +1,2 @@
This is a heavily truncated copy of https://github.com/foonathan/lexy, commit
20926cf.

11
dep/lexy/build.py Normal file
View File

@@ -0,0 +1,11 @@
from build.c import cxxlibrary
from glob import glob
cxxlibrary(
name="lexy",
srcs=[],
hdrs={
h: f"./include/{h}"
for h in glob("**/*.hpp", root_dir="dep/lexy/include", recursive=True)
},
)

View File

@@ -0,0 +1,68 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_ANY_REF_HPP_INCLUDED
#define LEXY_DETAIL_ANY_REF_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
// Essentially a void*, but we can cast it in a constexpr context.
// The cost is an extra layer of indirection.
namespace lexy::_detail
{
template <typename T>
class any_holder;
// Store a pointer to this instead of a void*.
class any_base
{
public:
any_base(const any_base&) = delete;
any_base& operator=(const any_base&) = delete;
template <typename T>
constexpr T& get() noexcept
{
return static_cast<any_holder<T>*>(this)->get();
}
template <typename T>
constexpr const T& get() const noexcept
{
return static_cast<const any_holder<T>*>(this)->get();
}
private:
constexpr any_base() = default;
~any_base() = default;
template <typename T>
friend class any_holder;
};
using any_ref = any_base*;
using any_cref = const any_base*;
// Need to store the object in here.
template <typename T>
class any_holder : public any_base
{
public:
constexpr explicit any_holder(T&& obj) : _obj(LEXY_MOV(obj)) {}
constexpr T& get() noexcept
{
return _obj;
}
constexpr const T& get() const noexcept
{
return _obj;
}
private:
T _obj;
};
} // namespace lexy::_detail
#endif // LEXY_DETAIL_ANY_REF_HPP_INCLUDED

View File

@@ -0,0 +1,51 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_ASSERT_HPP_INCLUDED
#define LEXY_DETAIL_ASSERT_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
#ifndef LEXY_ENABLE_ASSERT
// By default, enable assertions if NDEBUG is not defined.
# if NDEBUG
# define LEXY_ENABLE_ASSERT 0
# else
# define LEXY_ENABLE_ASSERT 1
# endif
#endif
#if LEXY_ENABLE_ASSERT
// We want assertions: use assert() if that's available, otherwise abort.
// We don't use assert() directly as that's not constexpr.
# if NDEBUG
# include <cstdlib>
# define LEXY_PRECONDITION(Expr) ((Expr) ? void(0) : std::abort())
# define LEXY_ASSERT(Expr, Msg) ((Expr) ? void(0) : std::abort())
# else
# include <cassert>
# define LEXY_PRECONDITION(Expr) ((Expr) ? void(0) : assert(Expr))
# define LEXY_ASSERT(Expr, Msg) ((Expr) ? void(0) : assert((Expr) && (Msg)))
# endif
#else
// We don't want assertions.
# define LEXY_PRECONDITION(Expr) static_cast<void>(sizeof(Expr))
# define LEXY_ASSERT(Expr, Msg) static_cast<void>(sizeof(Expr))
#endif
#endif // LEXY_DETAIL_ASSERT_HPP_INCLUDED

View File

@@ -0,0 +1,160 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_BUFFER_BUILDER_HPP_INCLUDED
#define LEXY_DETAIL_BUFFER_BUILDER_HPP_INCLUDED
#include <cstring>
#include <lexy/_detail/assert.hpp>
#include <lexy/_detail/config.hpp>
#include <lexy/_detail/iterator.hpp>
#include <new>
namespace lexy::_detail
{
// Builds a buffer: it has a read are and a write area.
// The characters in the read area are already valid and can be read.
// The characters in the write area are not valid, but can be written too.
template <typename T>
class buffer_builder
{
static_assert(std::is_trivial_v<T>);
static constexpr std::size_t total_size_bytes = 1024;
static constexpr std::size_t stack_buffer_size
= (total_size_bytes - 3 * sizeof(T*)) / sizeof(T);
static constexpr auto growth_factor = 2;
public:
buffer_builder() noexcept : _data(_stack_buffer), _read_size(0), _write_size(stack_buffer_size)
{
static_assert(sizeof(*this) == total_size_bytes, "invalid buffer size calculation");
}
~buffer_builder() noexcept
{
// Free memory if we allocated any.
if (_data != _stack_buffer)
::operator delete(_data);
}
buffer_builder(const buffer_builder&) = delete;
buffer_builder& operator=(const buffer_builder&) = delete;
// The total capacity: read + write.
std::size_t capacity() const noexcept
{
return _read_size + _write_size;
}
// The read area.
const T* read_data() const noexcept
{
return _data;
}
std::size_t read_size() const noexcept
{
return _read_size;
}
// The write area.
T* write_data() noexcept
{
return _data + _read_size;
}
std::size_t write_size() const noexcept
{
return _write_size;
}
// Clears the read area.
void clear() noexcept
{
_write_size += _read_size;
_read_size = 0;
}
// Takes the first n characters of the write area and appends them to the read area.
void commit(std::size_t n) noexcept
{
LEXY_PRECONDITION(n <= _write_size);
_read_size += n;
_write_size -= n;
}
// Increases the write area, invalidates all pointers.
void grow()
{
const auto cur_cap = capacity();
const auto new_cap = growth_factor * cur_cap;
// Allocate new memory.
auto memory = static_cast<T*>(::operator new(new_cap * sizeof(T)));
// Copy the read area into the new memory.
std::memcpy(memory, _data, _read_size);
// Release the old memory, if there was any.
if (_data != _stack_buffer)
::operator delete(_data);
// Update for the new area.
_data = memory;
// _read_size hasn't been changed
_write_size = new_cap - _read_size;
}
//=== iterator ===//
// Stable iterator over the memory.
class stable_iterator : public forward_iterator_base<stable_iterator, const T>
{
public:
constexpr stable_iterator() = default;
explicit constexpr stable_iterator(const _detail::buffer_builder<T>& buffer,
std::size_t idx) noexcept
: _buffer(&buffer), _idx(idx)
{}
constexpr const T& deref() const noexcept
{
LEXY_PRECONDITION(_idx != _buffer->read_size());
return _buffer->read_data()[_idx];
}
constexpr void increment() noexcept
{
LEXY_PRECONDITION(_idx != _buffer->read_size());
++_idx;
}
constexpr bool equal(stable_iterator rhs) const noexcept
{
if (!_buffer || !rhs._buffer)
return !_buffer && !rhs._buffer;
else
{
LEXY_PRECONDITION(_buffer == rhs._buffer);
return _idx == rhs._idx;
}
}
constexpr std::size_t index() const noexcept
{
return _idx;
}
private:
const _detail::buffer_builder<T>* _buffer = nullptr;
std::size_t _idx = 0;
};
private:
T* _data;
std::size_t _read_size;
std::size_t _write_size;
T _stack_buffer[stack_buffer_size];
};
} // namespace lexy::_detail
#endif // LEXY_DETAIL_BUFFER_BUILDER_HPP_INCLUDED

View File

@@ -0,0 +1,368 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_CODE_POINT_HPP_INCLUDED
#define LEXY_DETAIL_CODE_POINT_HPP_INCLUDED
#include <lexy/input/base.hpp>
//=== encoding ===//
namespace lexy::_detail
{
template <typename Encoding>
constexpr std::size_t encode_code_point(char32_t cp, typename Encoding::char_type* buffer,
std::size_t size)
{
if constexpr (std::is_same_v<Encoding, lexy::ascii_encoding>)
{
LEXY_PRECONDITION(size >= 1);
*buffer = char(cp);
return 1;
}
else if constexpr (std::is_same_v<Encoding,
lexy::utf8_encoding> //
|| std::is_same_v<Encoding, lexy::utf8_char_encoding>)
{
using char_type = typename Encoding::char_type;
// Taken from http://www.herongyang.com/Unicode/UTF-8-UTF-8-Encoding-Algorithm.html.
if (cp <= 0x7F)
{
LEXY_PRECONDITION(size >= 1);
buffer[0] = char_type(cp);
return 1;
}
else if (cp <= 0x07'FF)
{
LEXY_PRECONDITION(size >= 2);
auto first = (cp >> 6) & 0x1F;
auto second = (cp >> 0) & 0x3F;
buffer[0] = char_type(0xC0 | first);
buffer[1] = char_type(0x80 | second);
return 2;
}
else if (cp <= 0xFF'FF)
{
LEXY_PRECONDITION(size >= 3);
auto first = (cp >> 12) & 0x0F;
auto second = (cp >> 6) & 0x3F;
auto third = (cp >> 0) & 0x3F;
buffer[0] = char_type(0xE0 | first);
buffer[1] = char_type(0x80 | second);
buffer[2] = char_type(0x80 | third);
return 3;
}
else
{
LEXY_PRECONDITION(size >= 4);
auto first = (cp >> 18) & 0x07;
auto second = (cp >> 12) & 0x3F;
auto third = (cp >> 6) & 0x3F;
auto fourth = (cp >> 0) & 0x3F;
buffer[0] = char_type(0xF0 | first);
buffer[1] = char_type(0x80 | second);
buffer[2] = char_type(0x80 | third);
buffer[3] = char_type(0x80 | fourth);
return 4;
}
}
else if constexpr (std::is_same_v<Encoding, lexy::utf16_encoding>)
{
if (cp <= 0xFF'FF)
{
LEXY_PRECONDITION(size >= 1);
buffer[0] = char16_t(cp);
return 1;
}
else
{
// Algorithm implemented from
// https://en.wikipedia.org/wiki/UTF-16#Code_points_from_U+010000_to_U+10FFFF.
LEXY_PRECONDITION(size >= 2);
auto u_prime = cp - 0x1'0000;
auto high_ten_bits = u_prime >> 10;
auto low_ten_bits = u_prime & 0b0000'0011'1111'1111;
buffer[0] = char16_t(0xD800 + high_ten_bits);
buffer[1] = char16_t(0xDC00 + low_ten_bits);
return 2;
}
}
else if constexpr (std::is_same_v<Encoding, lexy::utf32_encoding>)
{
LEXY_PRECONDITION(size >= 1);
*buffer = cp;
return 1;
}
else
{
static_assert(lexy::_detail::error<Encoding>,
"cannot encode a code point in this encoding");
(void)cp;
(void)buffer;
(void)size;
return 0;
}
}
} // namespace lexy::_detail
//=== parsing ===//
namespace lexy::_detail
{
enum class cp_error
{
success,
eof,
leads_with_trailing,
missing_trailing,
surrogate,
overlong_sequence,
out_of_range,
};
template <typename Reader>
struct cp_result
{
char32_t cp;
cp_error error;
typename Reader::marker end;
};
template <typename Reader>
constexpr cp_result<Reader> parse_code_point(Reader reader)
{
if constexpr (std::is_same_v<typename Reader::encoding, lexy::ascii_encoding>)
{
if (reader.peek() == Reader::encoding::eof())
return {{}, cp_error::eof, reader.current()};
auto cur = reader.peek();
reader.bump();
auto cp = static_cast<char32_t>(cur);
if (cp <= 0x7F)
return {cp, cp_error::success, reader.current()};
else
return {cp, cp_error::out_of_range, reader.current()};
}
else if constexpr (std::is_same_v<typename Reader::encoding, lexy::utf8_encoding> //
|| std::is_same_v<typename Reader::encoding, lexy::utf8_char_encoding>)
{
using uchar_t = unsigned char;
constexpr auto payload_lead1 = 0b0111'1111;
constexpr auto payload_lead2 = 0b0001'1111;
constexpr auto payload_lead3 = 0b0000'1111;
constexpr auto payload_lead4 = 0b0000'0111;
constexpr auto payload_cont = 0b0011'1111;
constexpr auto pattern_lead1 = 0b0 << 7;
constexpr auto pattern_lead2 = 0b110 << 5;
constexpr auto pattern_lead3 = 0b1110 << 4;
constexpr auto pattern_lead4 = 0b11110 << 3;
constexpr auto pattern_cont = 0b10 << 6;
auto first = uchar_t(reader.peek());
if ((first & ~payload_lead1) == pattern_lead1)
{
// ASCII character.
reader.bump();
return {first, cp_error::success, reader.current()};
}
else if ((first & ~payload_cont) == pattern_cont)
{
return {{}, cp_error::leads_with_trailing, reader.current()};
}
else if ((first & ~payload_lead2) == pattern_lead2)
{
reader.bump();
auto second = uchar_t(reader.peek());
if ((second & ~payload_cont) != pattern_cont)
return {{}, cp_error::missing_trailing, reader.current()};
reader.bump();
auto result = char32_t(first & payload_lead2);
result <<= 6;
result |= char32_t(second & payload_cont);
// C0 and C1 are overlong ASCII.
if (first == 0xC0 || first == 0xC1)
return {result, cp_error::overlong_sequence, reader.current()};
else
return {result, cp_error::success, reader.current()};
}
else if ((first & ~payload_lead3) == pattern_lead3)
{
reader.bump();
auto second = uchar_t(reader.peek());
if ((second & ~payload_cont) != pattern_cont)
return {{}, cp_error::missing_trailing, reader.current()};
reader.bump();
auto third = uchar_t(reader.peek());
if ((third & ~payload_cont) != pattern_cont)
return {{}, cp_error::missing_trailing, reader.current()};
reader.bump();
auto result = char32_t(first & payload_lead3);
result <<= 6;
result |= char32_t(second & payload_cont);
result <<= 6;
result |= char32_t(third & payload_cont);
auto cp = result;
if (0xD800 <= cp && cp <= 0xDFFF)
return {cp, cp_error::surrogate, reader.current()};
else if (first == 0xE0 && second < 0xA0)
return {cp, cp_error::overlong_sequence, reader.current()};
else
return {cp, cp_error::success, reader.current()};
}
else if ((first & ~payload_lead4) == pattern_lead4)
{
reader.bump();
auto second = uchar_t(reader.peek());
if ((second & ~payload_cont) != pattern_cont)
return {{}, cp_error::missing_trailing, reader.current()};
reader.bump();
auto third = uchar_t(reader.peek());
if ((third & ~payload_cont) != pattern_cont)
return {{}, cp_error::missing_trailing, reader.current()};
reader.bump();
auto fourth = uchar_t(reader.peek());
if ((fourth & ~payload_cont) != pattern_cont)
return {{}, cp_error::missing_trailing, reader.current()};
reader.bump();
auto result = char32_t(first & payload_lead4);
result <<= 6;
result |= char32_t(second & payload_cont);
result <<= 6;
result |= char32_t(third & payload_cont);
result <<= 6;
result |= char32_t(fourth & payload_cont);
auto cp = result;
if (cp > 0x10'FFFF)
return {cp, cp_error::out_of_range, reader.current()};
else if (first == 0xF0 && second < 0x90)
return {cp, cp_error::overlong_sequence, reader.current()};
else
return {cp, cp_error::success, reader.current()};
}
else // FE or FF
{
return {{}, cp_error::eof, reader.current()};
}
}
else if constexpr (std::is_same_v<typename Reader::encoding, lexy::utf16_encoding>)
{
constexpr auto payload1 = 0b0000'0011'1111'1111;
constexpr auto payload2 = payload1;
constexpr auto pattern1 = 0b110110 << 10;
constexpr auto pattern2 = 0b110111 << 10;
if (reader.peek() == Reader::encoding::eof())
return {{}, cp_error::eof, reader.current()};
auto first = char16_t(reader.peek());
if ((first & ~payload1) == pattern1)
{
reader.bump();
if (reader.peek() == Reader::encoding::eof())
return {{}, cp_error::missing_trailing, reader.current()};
auto second = char16_t(reader.peek());
if ((second & ~payload2) != pattern2)
return {{}, cp_error::missing_trailing, reader.current()};
reader.bump();
// We've got a valid code point.
auto result = char32_t(first & payload1);
result <<= 10;
result |= char32_t(second & payload2);
result |= 0x10000;
return {result, cp_error::success, reader.current()};
}
else if ((first & ~payload2) == pattern2)
{
return {{}, cp_error::leads_with_trailing, reader.current()};
}
else
{
// Single code unit code point; always valid.
reader.bump();
return {first, cp_error::success, reader.current()};
}
}
else if constexpr (std::is_same_v<typename Reader::encoding, lexy::utf32_encoding>)
{
if (reader.peek() == Reader::encoding::eof())
return {{}, cp_error::eof, reader.current()};
auto cur = reader.peek();
reader.bump();
auto cp = cur;
if (cp > 0x10'FFFF)
return {cp, cp_error::out_of_range, reader.current()};
else if (0xD800 <= cp && cp <= 0xDFFF)
return {cp, cp_error::surrogate, reader.current()};
else
return {cp, cp_error::success, reader.current()};
}
else
{
static_assert(lexy::_detail::error<typename Reader::encoding>,
"no known code point for this encoding");
return {};
}
}
template <typename Reader>
constexpr void recover_code_point(Reader& reader, cp_result<Reader> result)
{
switch (result.error)
{
case cp_error::success:
// Consume the entire code point.
reader.reset(result.end);
break;
case cp_error::eof:
// We don't need to do anything to "recover" from EOF.
break;
case cp_error::leads_with_trailing:
// Invalid code unit, consume to recover.
LEXY_PRECONDITION(result.end.position() == reader.position());
reader.bump();
break;
case cp_error::missing_trailing:
case cp_error::surrogate:
case cp_error::out_of_range:
case cp_error::overlong_sequence:
// Consume all the invalid code units to recover.
reader.reset(result.end);
break;
}
}
} // namespace lexy::_detail
#endif // LEXY_DETAIL_CODE_POINT_HPP_INCLUDED

View File

@@ -0,0 +1,199 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_CONFIG_HPP_INCLUDED
#define LEXY_DETAIL_CONFIG_HPP_INCLUDED
#include <cstddef>
#include <type_traits>
#if defined(LEXY_USER_CONFIG_HEADER)
# include LEXY_USER_CONFIG_HEADER
#elif defined(__has_include)
# if __has_include(<lexy_user_config.hpp>)
# include <lexy_user_config.hpp>
# elif __has_include("lexy_user_config.hpp")
# include "lexy_user_config.hpp"
# endif
#endif
#ifndef LEXY_HAS_UNICODE_DATABASE
# define LEXY_HAS_UNICODE_DATABASE 0
#endif
#ifndef LEXY_EXPERIMENTAL
# define LEXY_EXPERIMENTAL 0
#endif
//=== utility traits===//
#define LEXY_MOV(...) static_cast<std::remove_reference_t<decltype(__VA_ARGS__)>&&>(__VA_ARGS__)
#define LEXY_FWD(...) static_cast<decltype(__VA_ARGS__)>(__VA_ARGS__)
#define LEXY_DECLVAL(...) lexy::_detail::declval<__VA_ARGS__>()
#define LEXY_DECAY_DECLTYPE(...) std::decay_t<decltype(__VA_ARGS__)>
/// Creates a new type from the instantiation of a template.
/// This is used to shorten type names.
#define LEXY_INSTANTIATION_NEWTYPE(Name, Templ, ...) \
struct Name : Templ<__VA_ARGS__> \
{ \
using Templ<__VA_ARGS__>::Templ; \
}
namespace lexy::_detail
{
template <typename... T>
constexpr bool error = false;
template <typename T>
std::add_rvalue_reference_t<T> declval();
template <typename T>
constexpr void swap(T& lhs, T& rhs) noexcept
{
T tmp = LEXY_MOV(lhs);
lhs = LEXY_MOV(rhs);
rhs = LEXY_MOV(tmp);
}
template <typename T, typename U>
constexpr bool is_decayed_same = std::is_same_v<std::decay_t<T>, std::decay_t<U>>;
template <typename T, typename Fallback>
using type_or = std::conditional_t<std::is_void_v<T>, Fallback, T>;
} // namespace lexy::_detail
//=== NTTP ===//
#ifndef LEXY_HAS_NTTP
// See https://github.com/foonathan/lexy/issues/15.
# if __cpp_nontype_template_parameter_class >= 201806 || __cpp_nontype_template_args >= 201911
# define LEXY_HAS_NTTP 1
# else
# define LEXY_HAS_NTTP 0
# endif
#endif
#if LEXY_HAS_NTTP
# define LEXY_NTTP_PARAM auto
#else
# define LEXY_NTTP_PARAM const auto&
#endif
//=== consteval ===//
#ifndef LEXY_HAS_CONSTEVAL
# if defined(_MSC_VER) && !defined(__clang__)
// Currently can't handle returning strings from consteval, check back later.
# define LEXY_HAS_CONSTEVAL 0
# elif __cpp_consteval
# define LEXY_HAS_CONSTEVAL 1
# else
# define LEXY_HAS_CONSTEVAL 0
# endif
#endif
#if LEXY_HAS_CONSTEVAL
# define LEXY_CONSTEVAL consteval
#else
# define LEXY_CONSTEVAL constexpr
#endif
//=== constexpr ===//
#ifndef LEXY_HAS_CONSTEXPR_DTOR
# if __cpp_constexpr_dynamic_alloc
# define LEXY_HAS_CONSTEXPR_DTOR 1
# else
# define LEXY_HAS_CONSTEXPR_DTOR 0
# endif
#endif
#if LEXY_HAS_CONSTEXPR_DTOR
# define LEXY_CONSTEXPR_DTOR constexpr
#else
# define LEXY_CONSTEXPR_DTOR
#endif
//=== char8_t ===//
#ifndef LEXY_HAS_CHAR8_T
# if __cpp_char8_t
# define LEXY_HAS_CHAR8_T 1
# else
# define LEXY_HAS_CHAR8_T 0
# endif
#endif
#if LEXY_HAS_CHAR8_T
# define LEXY_CHAR_OF_u8 char8_t
# define LEXY_CHAR8_T char8_t
# define LEXY_CHAR8_STR(Str) u8##Str
#else
namespace lexy
{
using _char8_t = unsigned char;
} // namespace lexy
# define LEXY_CHAR_OF_u8 char
# define LEXY_CHAR8_T ::lexy::_char8_t
# define LEXY_CHAR8_STR(Str) \
LEXY_NTTP_STRING(::lexy::_detail::type_string, u8##Str)::c_str<LEXY_CHAR8_T>
#endif
//=== endianness ===//
#ifndef LEXY_IS_LITTLE_ENDIAN
# if defined(__BYTE_ORDER__)
# if __BYTE_ORDER__ == __ORDER_LITTLE_ENDIAN__
# define LEXY_IS_LITTLE_ENDIAN 1
# elif __BYTE_ORDER__ == __ORDER_BIG_ENDIAN__
# define LEXY_IS_LITTLE_ENDIAN 0
# else
# error "unsupported byte order"
# endif
# elif defined(_MSC_VER)
# define LEXY_IS_LITTLE_ENDIAN 1
# else
# error "unknown endianness"
# endif
#endif
//=== force inline ===//
#ifndef LEXY_FORCE_INLINE
# if defined(__has_cpp_attribute)
# if __has_cpp_attribute(gnu::always_inline)
# define LEXY_FORCE_INLINE [[gnu::always_inline]]
# endif
# endif
#
# ifndef LEXY_FORCE_INLINE
# define LEXY_FORCE_INLINE inline
# endif
#endif
//=== empty_member ===//
#ifndef LEXY_EMPTY_MEMBER
# if defined(__has_cpp_attribute)
# if defined(__GNUC__) && !defined(__clang__) && __GNUC__ <= 11
// GCC <= 11 has buggy support, see https://gcc.gnu.org/bugzilla/show_bug.cgi?id=101040
# define LEXY_HAS_EMPTY_MEMBER 0
# elif __has_cpp_attribute(no_unique_address)
# define LEXY_HAS_EMPTY_MEMBER 1
# endif
# endif
# ifndef LEXY_HAS_EMPTY_MEMBER
# define LEXY_HAS_EMPTY_MEMBER 0
# endif
# if LEXY_HAS_EMPTY_MEMBER
# define LEXY_EMPTY_MEMBER [[no_unique_address]]
# else
# define LEXY_EMPTY_MEMBER
# endif
#endif
#endif // LEXY_DETAIL_CONFIG_HPP_INCLUDED

View File

@@ -0,0 +1,35 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_DETECT_HPP_INCLUDED
#define LEXY_DETAIL_DETECT_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
namespace lexy::_detail
{
template <typename... Args>
using void_t = void;
template <template <typename...> typename Op, typename Void, typename... Args>
struct _detector : std::false_type
{
template <typename Fallback>
using type_or = Fallback;
};
template <template <typename...> typename Op, typename... Args>
struct _detector<Op, void_t<Op<Args...>>, Args...> : std::true_type
{
template <typename Fallback>
using type_or = Op<Args...>;
};
template <template <typename...> typename Op, typename... Args>
constexpr bool is_detected = _detector<Op, void, Args...>::value;
template <typename Fallback, template <typename...> typename Op, typename... Args>
using detected_or = typename _detector<Op, void, Args...>::template type_or<Fallback>;
} // namespace lexy::_detail
#endif // LEXY_DETAIL_DETECT_HPP_INCLUDED

View File

@@ -0,0 +1,64 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_INTEGER_SEQUENCE_HPP_INCLUDED
#define LEXY_DETAIL_INTEGER_SEQUENCE_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
namespace lexy::_detail
{
template <typename T, T... Indices>
struct integer_sequence
{
using type = integer_sequence<T, Indices...>;
};
template <std::size_t... Indices>
using index_sequence = integer_sequence<std::size_t, Indices...>;
#if defined(__clang__)
template <std::size_t Size>
using make_index_sequence = __make_integer_seq<integer_sequence, std::size_t, Size>;
#elif defined(__GNUC__) && __GNUC__ >= 8
template <std::size_t Size>
using make_index_sequence = index_sequence<__integer_pack(Size)...>;
#elif defined(_MSC_VER)
template <std::size_t Size>
using make_index_sequence = __make_integer_seq<integer_sequence, std::size_t, Size>;
#else
// Adapted from https://stackoverflow.com/a/32223343.
template <class Sequence1, class Sequence2>
struct concat_seq;
template <std::size_t... I1, std::size_t... I2>
struct concat_seq<index_sequence<I1...>, index_sequence<I2...>>
{
using type = index_sequence<I1..., (sizeof...(I1) + I2)...>;
};
template <size_t N>
struct _make_index_sequence : concat_seq<typename _make_index_sequence<N / 2>::type,
typename _make_index_sequence<N - N / 2>::type>
{};
template <>
struct _make_index_sequence<0>
{
using type = index_sequence<>;
};
template <>
struct _make_index_sequence<1>
{
using type = index_sequence<0>;
};
template <std::size_t Size>
using make_index_sequence = typename _make_index_sequence<Size>::type;
#endif
template <typename... T>
using index_sequence_for = make_index_sequence<sizeof...(T)>;
} // namespace lexy::_detail
#endif // LEXY_DETAIL_INTEGER_SEQUENCE_HPP_INCLUDED

View File

@@ -0,0 +1,70 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_INVOKE_HPP_INCLUDED
#define LEXY_DETAIL_INVOKE_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
namespace lexy::_detail
{
template <typename MemberPtr, bool = std::is_member_object_pointer_v<MemberPtr>>
struct _mem_invoker;
template <typename R, typename ClassT>
struct _mem_invoker<R ClassT::*, true>
{
static constexpr decltype(auto) invoke(R ClassT::*f, ClassT& object)
{
return object.*f;
}
static constexpr decltype(auto) invoke(R ClassT::*f, const ClassT& object)
{
return object.*f;
}
template <typename Ptr>
static constexpr auto invoke(R ClassT::*f, Ptr&& ptr) -> decltype((*LEXY_FWD(ptr)).*f)
{
return (*LEXY_FWD(ptr)).*f;
}
};
template <typename F, typename ClassT>
struct _mem_invoker<F ClassT::*, false>
{
template <typename ObjectT, typename... Args>
static constexpr auto _invoke(int, F ClassT::*f, ObjectT&& object, Args&&... args)
-> decltype((LEXY_FWD(object).*f)(LEXY_FWD(args)...))
{
return (LEXY_FWD(object).*f)(LEXY_FWD(args)...);
}
template <typename PtrT, typename... Args>
static constexpr auto _invoke(short, F ClassT::*f, PtrT&& ptr, Args&&... args)
-> decltype(((*LEXY_FWD(ptr)).*f)(LEXY_FWD(args)...))
{
return ((*LEXY_FWD(ptr)).*f)(LEXY_FWD(args)...);
}
template <typename... Args>
static constexpr auto invoke(F ClassT::*f,
Args&&... args) -> decltype(_invoke(0, f, LEXY_FWD(args)...))
{
return _invoke(0, f, LEXY_FWD(args)...);
}
};
template <typename ClassT, typename F, typename... Args>
constexpr auto invoke(F ClassT::*f, Args&&... args)
-> decltype(_mem_invoker<F ClassT::*>::invoke(f, LEXY_FWD(args)...))
{
return _mem_invoker<F ClassT::*>::invoke(f, LEXY_FWD(args)...);
}
template <typename F, typename... Args>
constexpr auto invoke(F&& f, Args&&... args) -> decltype(LEXY_FWD(f)(LEXY_FWD(args)...))
{
return LEXY_FWD(f)(LEXY_FWD(args)...);
}
} // namespace lexy::_detail
#endif // LEXY_DETAIL_INVOKE_HPP_INCLUDED

View File

@@ -0,0 +1,244 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_ITERATOR_HPP_INCLUDED
#define LEXY_DETAIL_ITERATOR_HPP_INCLUDED
#include <lexy/_detail/assert.hpp>
#include <lexy/_detail/config.hpp>
#include <lexy/_detail/detect.hpp>
#include <lexy/_detail/std.hpp>
//=== iterator algorithms ===//
namespace lexy::_detail
{
// Can't use std::is_base_of_v<std::random_access_iterator_tag, ...> without including <iterator>.
template <typename Iterator>
using _detect_random_access = decltype(LEXY_DECLVAL(Iterator) - LEXY_DECLVAL(Iterator));
template <typename Iterator>
constexpr auto is_random_access_iterator = is_detected<_detect_random_access, Iterator>;
template <typename Iterator, typename Sentinel>
constexpr std::size_t range_size(Iterator begin, Sentinel end)
{
if constexpr (std::is_same_v<Iterator, Sentinel> && is_random_access_iterator<Iterator>)
{
return static_cast<std::size_t>(end - begin);
}
else
{
std::size_t result = 0;
for (auto cur = begin; cur != end; ++cur)
++result;
return result;
}
}
template <typename Iterator>
constexpr Iterator next(Iterator iter)
{
return ++iter;
}
template <typename Iterator>
constexpr Iterator next(Iterator iter, std::size_t n)
{
if constexpr (is_random_access_iterator<Iterator>)
{
return iter + n;
}
else
{
for (auto i = 0u; i != n; ++i)
++iter;
return iter;
}
}
template <typename Iterator, typename Sentinel>
constexpr Iterator next_clamped(Iterator iter, std::size_t n, Sentinel end)
{
if constexpr (is_random_access_iterator<Iterator> && std::is_same_v<Iterator, Sentinel>)
{
auto remaining = std::size_t(end - iter);
if (remaining < n)
return end;
else
return iter + n;
}
else
{
for (auto i = 0u; i != n; ++i)
{
if (iter == end)
break;
++iter;
}
return iter;
}
}
// Used for assertions.
template <typename Iterator, typename Sentinel>
constexpr bool precedes([[maybe_unused]] Iterator first, [[maybe_unused]] Sentinel after)
{
if constexpr (is_random_access_iterator<Iterator> && std::is_same_v<Iterator, Sentinel>)
return first <= after;
else
return true;
}
// Requires: begin <= end_a && begin <= end_b.
// Returns min(end_a, end_b).
template <typename Iterator>
constexpr Iterator min_range_end(Iterator begin, Iterator end_a, Iterator end_b)
{
if constexpr (is_random_access_iterator<Iterator>)
{
LEXY_PRECONDITION(begin <= end_a && begin <= end_b);
if (end_a <= end_b)
return end_a;
else
return end_b;
}
else
{
auto cur = begin;
while (cur != end_a && cur != end_b)
++cur;
return cur;
}
}
// Requires: begin <= end_a && begin <= end_b.
// Returns max(end_a, end_b).
template <typename Iterator>
constexpr Iterator max_range_end(Iterator begin, Iterator end_a, Iterator end_b)
{
if constexpr (is_random_access_iterator<Iterator>)
{
LEXY_PRECONDITION(begin <= end_a && begin <= end_b);
if (end_a <= end_b)
return end_b;
else
return end_a;
}
else
{
auto cur = begin;
while (true)
{
if (cur == end_a)
return end_b;
else if (cur == end_b)
return end_a;
++cur;
}
return begin; // unreachable
}
}
} // namespace lexy::_detail
//=== facade classes ===//
namespace lexy::_detail
{
template <typename T>
struct _proxy_pointer
{
T value;
constexpr T* operator->() noexcept
{
return &value;
}
};
template <typename Derived, typename T, typename Reference = T&, typename Pointer = const T*>
struct forward_iterator_base
{
using value_type = std::remove_cv_t<T>;
using reference = Reference;
using pointer = lexy::_detail::type_or<Pointer, _proxy_pointer<value_type>>;
using difference_type = std::ptrdiff_t;
using iterator_category = std::forward_iterator_tag;
constexpr reference operator*() const noexcept
{
return static_cast<const Derived&>(*this).deref();
}
constexpr pointer operator->() const noexcept
{
if constexpr (std::is_void_v<Pointer>)
return pointer{**this};
else
return &**this;
}
constexpr Derived& operator++() noexcept
{
auto& derived = static_cast<Derived&>(*this);
derived.increment();
return derived;
}
constexpr Derived operator++(int) noexcept
{
auto& derived = static_cast<Derived&>(*this);
auto copy = derived;
derived.increment();
return copy;
}
friend constexpr bool operator==(const Derived& lhs, const Derived& rhs)
{
return lhs.equal(rhs);
}
friend constexpr bool operator!=(const Derived& lhs, const Derived& rhs)
{
return !lhs.equal(rhs);
}
};
template <typename Derived, typename T, typename Reference = T&, typename Pointer = const T*>
struct bidirectional_iterator_base : forward_iterator_base<Derived, T, Reference, Pointer>
{
using iterator_category = std::bidirectional_iterator_tag;
constexpr Derived& operator--() noexcept
{
auto& derived = static_cast<Derived&>(*this);
derived.decrement();
return derived;
}
constexpr Derived operator--(int) noexcept
{
auto& derived = static_cast<Derived&>(*this);
auto copy = derived;
derived.decrement();
return copy;
}
};
template <typename Derived, typename Iterator>
struct sentinel_base
{
friend constexpr bool operator==(const Iterator& lhs, Derived) noexcept
{
return lhs.is_end();
}
friend constexpr bool operator!=(const Iterator& lhs, Derived) noexcept
{
return !(lhs == Derived{});
}
friend constexpr bool operator==(Derived, const Iterator& rhs) noexcept
{
return rhs == Derived{};
}
friend constexpr bool operator!=(Derived, const Iterator& rhs) noexcept
{
return !(rhs == Derived{});
}
};
} // namespace lexy::_detail
#endif // LEXY_DETAIL_ITERATOR_HPP_INCLUDED

View File

@@ -0,0 +1,246 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_LAZY_INIT_HPP_INCLUDED
#define LEXY_DETAIL_LAZY_INIT_HPP_INCLUDED
#include <lexy/_detail/assert.hpp>
#include <lexy/_detail/config.hpp>
#include <lexy/_detail/std.hpp>
namespace lexy::_detail
{
template <typename T>
struct _lazy_init_storage_trivial
{
bool _init;
union
{
char _empty;
T _value;
};
constexpr _lazy_init_storage_trivial() noexcept : _init(false), _empty() {}
template <typename... Args>
constexpr _lazy_init_storage_trivial(int, Args&&... args)
: _init(true), _value(LEXY_FWD(args)...)
{}
template <typename... Args>
constexpr void _construct(Args&&... args)
{
*this = _lazy_init_storage_trivial(0, LEXY_FWD(args)...);
}
};
template <typename T>
struct _lazy_init_storage_non_trivial
{
bool _init;
union
{
char _empty;
T _value;
};
constexpr _lazy_init_storage_non_trivial() noexcept : _init(false), _empty() {}
template <typename... Args>
LEXY_CONSTEXPR_DTOR void _construct(Args&&... args)
{
_detail::construct_at(&_value, LEXY_FWD(args)...);
_init = true;
}
// Cannot add noexcept due to https://github.com/llvm/llvm-project/issues/59854.
LEXY_CONSTEXPR_DTOR ~_lazy_init_storage_non_trivial() /* noexcept */
{
if (_init)
_value.~T();
}
LEXY_CONSTEXPR_DTOR _lazy_init_storage_non_trivial(
_lazy_init_storage_non_trivial&& other) noexcept
: _init(other._init), _empty()
{
if (_init)
_detail::construct_at(&_value, LEXY_MOV(other._value));
}
LEXY_CONSTEXPR_DTOR _lazy_init_storage_non_trivial& operator=(
_lazy_init_storage_non_trivial&& other) noexcept
{
if (_init && other._init)
_value = LEXY_MOV(other._value);
else if (_init && !other._init)
{
_value.~T();
_init = false;
}
else if (!_init && other._init)
{
_detail::construct_at(&_value, LEXY_MOV(other._value));
_init = true;
}
else
{
// Both not initialized, nothing to do.
}
return *this;
}
};
template <typename T>
constexpr auto _lazy_init_trivial = [] {
// https://www.foonathan.net/2021/03/trivially-copyable/
return std::is_trivially_destructible_v<T> //
&& std::is_trivially_copy_constructible_v<T> //
&& std::is_trivially_copy_assignable_v<T> //
&& std::is_trivially_move_constructible_v<T> //
&& std::is_trivially_move_assignable_v<T>;
}();
template <typename T>
using _lazy_init_storage = std::conditional_t<_lazy_init_trivial<T>, _lazy_init_storage_trivial<T>,
_lazy_init_storage_non_trivial<T>>;
template <typename T>
class lazy_init : _lazy_init_storage<T>
{
public:
using value_type = T;
constexpr lazy_init() noexcept = default;
template <typename... Args>
constexpr T& emplace(Args&&... args)
{
if (*this)
this->_value = T(LEXY_FWD(args)...);
else
this->_construct(LEXY_FWD(args)...);
return this->_value;
}
template <typename Fn, typename... Args>
constexpr T& emplace_result(Fn&& fn, Args&&... args)
{
return emplace(LEXY_FWD(fn)(LEXY_FWD(args)...));
}
constexpr explicit operator bool() const noexcept
{
return this->_init;
}
constexpr T& operator*() & noexcept
{
LEXY_PRECONDITION(*this);
return this->_value;
}
constexpr const T& operator*() const& noexcept
{
LEXY_PRECONDITION(*this);
return this->_value;
}
constexpr T&& operator*() && noexcept
{
LEXY_PRECONDITION(*this);
return LEXY_MOV(this->_value);
}
constexpr const T&& operator*() const&& noexcept
{
LEXY_PRECONDITION(*this);
return LEXY_MOV(this->_value);
}
constexpr T* operator->() noexcept
{
LEXY_PRECONDITION(*this);
return &this->_value;
}
constexpr const T* operator->() const noexcept
{
LEXY_PRECONDITION(*this);
return &this->_value;
}
private:
template <typename... Args>
constexpr explicit lazy_init(int, Args&&... args) noexcept
: _lazy_init_storage<T>(0, LEXY_FWD(args)...)
{}
};
template <typename T>
class lazy_init<T&>
{
public:
using value_type = T&;
constexpr lazy_init() noexcept : _ptr(nullptr) {}
constexpr T& emplace(T& ref)
{
_ptr = &ref;
return ref;
}
template <typename Fn, typename... Args>
constexpr T& emplace_result(Fn&& fn, Args&&... args)
{
return emplace(LEXY_FWD(fn)(LEXY_FWD(args)...));
}
constexpr explicit operator bool() const noexcept
{
return _ptr != nullptr;
}
constexpr T& operator*() const noexcept
{
LEXY_PRECONDITION(*this);
return *_ptr;
}
constexpr T* operator->() const noexcept
{
LEXY_PRECONDITION(*this);
return _ptr;
}
private:
T* _ptr;
};
template <>
class lazy_init<void>
{
public:
using value_type = void;
constexpr lazy_init() noexcept : _init(false) {}
constexpr void emplace()
{
_init = true;
}
template <typename Fn, typename... Args>
constexpr void emplace_result(Fn&& fn, Args&&... args)
{
LEXY_FWD(fn)(LEXY_FWD(args)...);
_init = true;
}
constexpr explicit operator bool() const noexcept
{
return _init;
}
private:
bool _init;
};
} // namespace lexy::_detail
#endif // LEXY_DETAIL_LAZY_INIT_HPP_INCLUDED

View File

@@ -0,0 +1,152 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_MEMORY_RESOURCE_HPP_INCLUDED
#define LEXY_DETAIL_MEMORY_RESOURCE_HPP_INCLUDED
#include <cstring>
#include <lexy/_detail/assert.hpp>
#include <lexy/_detail/config.hpp>
#include <new>
#if 0 // NOLINT
// Subset of the interface of std::pmr::memory_resource.
class MemoryResource
{
public:
void* allocate(std::size_t bytes, std::size_t alignment);
void deallocate(void* ptr, std::size_t bytes, std::size_t alignment);
friend bool operator==(const MemoryResource& lhs, const MemoryResource& rhs);
};
#endif
namespace lexy::_detail
{
class default_memory_resource
{
public:
static void* allocate(std::size_t bytes, std::size_t alignment)
{
if (alignment > __STDCPP_DEFAULT_NEW_ALIGNMENT__)
return ::operator new(bytes, std::align_val_t{alignment});
else
return ::operator new(bytes);
}
static void deallocate(void* ptr, std::size_t bytes, std::size_t alignment) noexcept
{
#if LEXY_ENABLE_ASSERT
// In debug mode, we fill freed memory with 0xFF to detect dangling lexemes.
// For default, ASCII, bytes, this is just a noticable value.
// For UTF-8, this is the EOF integer value as its an invalid code unit.
// For UTF-16, this is the code point 0xFFFF, which is the replacement character.
// For UTF-32, this is an out of range code point.
std::memset(ptr, 0xFF, bytes);
#endif
#ifdef __cpp_sized_deallocation
if (alignment > __STDCPP_DEFAULT_NEW_ALIGNMENT__)
::operator delete(ptr, bytes, std::align_val_t{alignment});
else
::operator delete(ptr, bytes);
#else
(void)bytes;
if (alignment > __STDCPP_DEFAULT_NEW_ALIGNMENT__)
::operator delete(ptr, std::align_val_t{alignment});
else
::operator delete(ptr);
#endif
}
friend constexpr bool operator==(default_memory_resource, default_memory_resource) noexcept
{
return true;
}
};
} // namespace lexy::_detail
namespace lexy::_detail
{
template <typename MemoryResource>
class _memory_resource_ptr_empty
{
public:
constexpr explicit _memory_resource_ptr_empty(MemoryResource*) noexcept {}
constexpr explicit _memory_resource_ptr_empty(void*) noexcept {}
constexpr auto operator*() const noexcept
{
return MemoryResource{};
}
constexpr auto operator->() const noexcept
{
struct proxy
{
MemoryResource _resource;
constexpr MemoryResource* operator->() noexcept
{
return &_resource;
}
};
return proxy{};
}
constexpr MemoryResource* get() const noexcept
{
return nullptr;
}
};
template <typename MemoryResource>
class _memory_resource_ptr
{
public:
constexpr explicit _memory_resource_ptr(MemoryResource* resource) noexcept : _resource(resource)
{
LEXY_PRECONDITION(resource);
}
constexpr MemoryResource& operator*() const noexcept
{
return *_resource;
}
constexpr MemoryResource* operator->() const noexcept
{
return _resource;
}
constexpr MemoryResource* get() const noexcept
{
return _resource;
}
private:
MemoryResource* _resource;
};
// clang-format off
template <typename MemoryResource>
using memory_resource_ptr
= std::conditional_t<std::is_void_v<MemoryResource>,
_memory_resource_ptr_empty<default_memory_resource>,
std::conditional_t<std::is_empty_v<MemoryResource>,
_memory_resource_ptr_empty<MemoryResource>,
_memory_resource_ptr<MemoryResource>>>;
// clang-format on
template <typename MemoryResource, typename = std::enable_if_t<std::is_void_v<MemoryResource>
|| std::is_empty_v<MemoryResource>>>
constexpr MemoryResource* get_memory_resource()
{
return nullptr;
}
} // namespace lexy::_detail
#endif // LEXY_DETAIL_MEMORY_RESOURCE_HPP_INCLUDED

View File

@@ -0,0 +1,147 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_NTTP_STRING_HPP_INCLUDED
#define LEXY_DETAIL_NTTP_STRING_HPP_INCLUDED
#include <lexy/_detail/assert.hpp>
#include <lexy/_detail/config.hpp>
#include <lexy/encoding.hpp>
namespace lexy::_detail
{
// Note: we can't use type_string<auto...>, it doesn't work on older GCC.
template <typename CharT, CharT... Cs>
struct type_string
{
using char_type = CharT;
template <template <typename C, C...> typename T>
using rename = T<CharT, Cs...>;
static constexpr auto size = sizeof...(Cs);
template <typename T = char_type>
static constexpr T c_str[sizeof...(Cs) + 1] = {transcode_char<T>(Cs)..., T()};
};
} // namespace lexy::_detail
#if LEXY_HAS_NTTP // string NTTP implementation
# include <lexy/_detail/integer_sequence.hpp>
namespace lexy::_detail
{
template <std::size_t N, typename CharT>
struct string_literal
{
CharT data[N];
using char_type = CharT;
LEXY_CONSTEVAL string_literal(const CharT* str) : data{}
{
for (auto i = 0u; i != N; ++i)
data[i] = str[i];
}
LEXY_CONSTEVAL string_literal(CharT c) : data{}
{
data[0] = c;
}
static LEXY_CONSTEVAL auto size()
{
return N;
}
};
template <std::size_t N, typename CharT>
string_literal(const CharT (&)[N]) -> string_literal<N - 1, CharT>;
template <typename CharT>
string_literal(CharT) -> string_literal<1, CharT>;
template <template <typename C, C... Cs> typename T, string_literal Str, std::size_t... Idx>
auto _to_type_string(index_sequence<Idx...>)
{
return T<typename decltype(Str)::char_type, Str.data[Idx]...>{};
}
template <template <typename C, C... Cs> typename T, string_literal Str>
using to_type_string
= decltype(_to_type_string<T, Str>(make_index_sequence<decltype(Str)::size()>{}));
} // namespace lexy::_detail
# define LEXY_NTTP_STRING(T, Str) \
::lexy::_detail::to_type_string<T, ::lexy::_detail::string_literal(Str)>
#elif defined(__GNUC__) // literal implementation
# pragma GCC diagnostic push
# pragma GCC diagnostic ignored "-Wpedantic"
# ifdef __clang__
# pragma GCC diagnostic ignored "-Wgnu-string-literal-operator-template"
# endif
template <typename CharT, CharT... Cs>
constexpr ::lexy::_detail::type_string<CharT, Cs...> operator""_lexy_string_udl()
{
return {};
}
# define LEXY_NTTP_STRING(T, Str) decltype(Str##_lexy_string_udl)::rename<T>
# pragma GCC diagnostic pop
#else // string<Cs...> macro implementation
namespace lexy::_detail
{
template <typename A, typename B>
struct cat_;
template <typename CharT, CharT... C1, CharT... C2>
struct cat_<type_string<CharT, C1...>, type_string<CharT, C2...>>
{
using type = type_string<CharT, C1..., C2...>;
};
template <typename A, typename B>
using cat = typename cat_<A, B>::type;
template <template <typename CharT, CharT...> typename T, typename TypeString, std::size_t Size,
std::size_t MaxSize>
struct macro_type_string
{
static_assert(Size <= MaxSize, "string out of range");
using type = typename TypeString::template rename<T>;
};
} // namespace lexy::_detail
# define LEXY_NTTP_STRING_LENGTH(Str) (sizeof(Str) / sizeof(Str[0]) - 1)
// extract Ith character if not out of bounds
# define LEXY_NTTP_STRING1(Str, I) \
::std::conditional_t< \
(I < LEXY_NTTP_STRING_LENGTH(Str)), \
::lexy::_detail::type_string<::LEXY_DECAY_DECLTYPE(Str[0]), \
(I >= LEXY_NTTP_STRING_LENGTH(Str) ? Str[0] : Str[I])>, \
::lexy::_detail::type_string<::LEXY_DECAY_DECLTYPE(Str[0])>>
// recursively split the string in two
# define LEXY_NTTP_STRING2(Str, I) \
::lexy::_detail::cat<LEXY_NTTP_STRING1(Str, I), LEXY_NTTP_STRING1(Str, I + 1)>
# define LEXY_NTTP_STRING4(Str, I) \
::lexy::_detail::cat<LEXY_NTTP_STRING2(Str, I), LEXY_NTTP_STRING2(Str, I + 2)>
# define LEXY_NTTP_STRING8(Str, I) \
::lexy::_detail::cat<LEXY_NTTP_STRING4(Str, I), LEXY_NTTP_STRING4(Str, I + 4)>
# define LEXY_NTTP_STRING16(Str, I) \
::lexy::_detail::cat<LEXY_NTTP_STRING8(Str, I), LEXY_NTTP_STRING8(Str, I + 8)>
# define LEXY_NTTP_STRING32(Str, I) \
::lexy::_detail::cat<LEXY_NTTP_STRING16(Str, I), LEXY_NTTP_STRING16(Str, I + 16)>
// instantiate with overflow check
# define LEXY_NTTP_STRING(T, Str) \
::lexy::_detail::macro_type_string<T, LEXY_NTTP_STRING32(Str, 0), \
LEXY_NTTP_STRING_LENGTH(Str), 32>::type
#endif
#endif // LEXY_DETAIL_NTTP_STRING_HPP_INCLUDED

View File

@@ -0,0 +1,72 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_STATELESS_LAMBDA_HPP_INCLUDED
#define LEXY_DETAIL_STATELESS_LAMBDA_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
namespace lexy::_detail
{
template <typename Lambda>
struct stateless_lambda
{
static_assert(std::is_class_v<Lambda>);
static_assert(std::is_empty_v<Lambda>);
static constexpr Lambda get()
{
if constexpr (std::is_default_constructible_v<Lambda>)
{
// We're using C++20, lambdas are default constructible.
return Lambda();
}
else
{
// We're not having C++20; use a sequence of weird workarounds to legally construct a
// Lambda object without invoking any constructors.
// This works and is well-defined, but sadly not constexpr.
// Taken from: https://www.youtube.com/watch?v=yTb6xz_FSkY
// We're defining two standard layout types that have a char as a common initial
// sequence (as the Lambda is empty, it doesn't add anymore members to B).
struct A
{
char member;
};
struct B : Lambda
{
char member;
};
static_assert(std::is_standard_layout_v<A> && std::is_standard_layout_v<B>);
// We put the two types in a union and initialize the a member, which we can do.
union storage_t
{
A a;
B b;
} storage{};
// We can now take the address of member via b, as it is in the common initial sequence.
auto char_ptr = &storage.b.member;
// char_ptr is a pointer to the first member of B, so we can reinterpret_cast it to a
// pointer to B.
auto b_ptr = reinterpret_cast<B*>(char_ptr);
// Now we're having a pointer to a B object, which can we can cast to the base class
// Lambda.
auto lambda_ptr = static_cast<Lambda*>(b_ptr);
// Dereference the pointer to get the lambda object.
return *lambda_ptr;
}
}
template <typename... Args>
constexpr decltype(auto) operator()(Args&&... args) const
{
return get()(LEXY_FWD(args)...);
}
};
} // namespace lexy::_detail
#endif // LEXY_DETAIL_STATELESS_LAMBDA_HPP_INCLUDED

View File

@@ -0,0 +1,98 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_STD_HPP_INCLUDED
#define LEXY_DETAIL_STD_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
//=== iterator tags ===//
#if defined(__GLIBCXX__)
namespace std
{
_GLIBCXX_BEGIN_NAMESPACE_VERSION
struct forward_iterator_tag;
struct bidirectional_iterator_tag;
_GLIBCXX_END_NAMESPACE_VERSION
} // namespace std
#elif defined(_LIBCPP_VERSION)
_LIBCPP_BEGIN_NAMESPACE_STD
struct forward_iterator_tag;
struct bidirectional_iterator_tag;
_LIBCPP_END_NAMESPACE_STD
#else
// Forward declaring things in std is not allowed, but I'm willing to take the risk.
namespace std
{
struct forward_iterator_tag;
struct bidirectional_iterator_tag;
} // namespace std
#endif
//=== (constexpr) construct_at ===//
#if !LEXY_HAS_CONSTEXPR_DTOR
namespace lexy::_detail
{
// We don't have constexpr dtor's, so this is just a regular function.
template <typename T, typename... Args>
T* construct_at(T* ptr, Args&&... args)
{
return ::new ((void*)ptr) T(LEXY_FWD(args)...);
}
} // namespace lexy::_detail
#elif defined(_MSC_VER)
namespace lexy::_detail
{
// MSVC can make it constexpr if marked with an attribute given by a macro.
template <typename T, typename... Args>
constexpr T* construct_at(T* ptr, Args&&... args)
{
# if defined(_MSVC_CONSTEXPR)
_MSVC_CONSTEXPR
# endif
return ::new ((void*)ptr) T(LEXY_FWD(args)...);
}
} // namespace lexy::_detail
#else
namespace lexy::_detail
{
struct _construct_at_tag
{};
} // namespace lexy::_detail
namespace std
{
// GCC only allows constexpr placement new inside a function called `std::construct_at`.
// So we write our own.
template <typename T, typename... Args>
constexpr T* construct_at(lexy::_detail::_construct_at_tag, T* ptr, Args&&... args)
{
return ::new ((void*)ptr) T(LEXY_FWD(args)...);
}
} // namespace std
namespace lexy::_detail
{
template <typename T, typename... Args>
constexpr T* construct_at(T* ptr, Args&&... args)
{
return std::construct_at(lexy::_detail::_construct_at_tag{}, ptr, LEXY_FWD(args)...);
}
} // namespace lexy::_detail
#endif
#endif // LEXY_DETAIL_STD_HPP_INCLUDED

View File

@@ -0,0 +1,212 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_STRING_VIEW_HPP_INCLUDED
#define LEXY_DETAIL_STRING_VIEW_HPP_INCLUDED
#include <lexy/_detail/assert.hpp>
#include <lexy/_detail/config.hpp>
#include <lexy/_detail/integer_sequence.hpp>
namespace lexy::_detail
{
struct null_terminated
{};
template <typename CharT>
class basic_string_view
{
static constexpr CharT empty_string[] = {CharT()};
public:
using char_type = CharT;
//=== constructor ===//
constexpr basic_string_view() noexcept : _ptr(empty_string), _size(0u), _null_terminated(true)
{}
constexpr basic_string_view(const char_type* str) noexcept
: _ptr(str), _size(0u), _null_terminated(true)
{
while (*str++)
++_size;
}
constexpr basic_string_view(const char_type* ptr, std::size_t size) noexcept
: _ptr(ptr), _size(size), _null_terminated(false)
{}
constexpr basic_string_view(null_terminated, const char_type* ptr, std::size_t size) noexcept
: _ptr(ptr), _size(size), _null_terminated(true)
{
LEXY_PRECONDITION(_ptr[_size] == CharT());
}
constexpr basic_string_view(const char_type* begin, const char_type* end) noexcept
: _ptr(begin), _size(std::size_t(end - begin)), _null_terminated(false)
{
LEXY_PRECONDITION(begin <= end);
}
//=== access ===//
using iterator = const char_type*;
constexpr iterator begin() const noexcept
{
return _ptr;
}
constexpr iterator end() const noexcept
{
return _ptr + _size;
}
constexpr bool empty() const noexcept
{
return _size == 0u;
}
constexpr std::size_t size() const noexcept
{
return _size;
}
constexpr std::size_t length() const noexcept
{
return _size;
}
constexpr char_type operator[](std::size_t i) const noexcept
{
LEXY_PRECONDITION(i <= _size);
return _ptr[i];
}
constexpr char_type front() const noexcept
{
LEXY_PRECONDITION(!empty());
return *_ptr;
}
constexpr char_type back() const noexcept
{
LEXY_PRECONDITION(!empty());
return _ptr[_size - 1];
}
constexpr const char_type* data() const noexcept
{
return _ptr;
}
constexpr bool is_null_terminated() const noexcept
{
return _null_terminated;
}
constexpr const char_type* c_str() const noexcept
{
LEXY_PRECONDITION(is_null_terminated());
return _ptr;
}
//=== operations ===//
static constexpr std::size_t npos = std::size_t(-1);
constexpr void remove_prefix(std::size_t n) noexcept
{
LEXY_PRECONDITION(n <= _size);
_ptr += n;
_size -= n;
}
constexpr void remove_suffix(std::size_t n) noexcept
{
LEXY_PRECONDITION(n <= _size);
_size -= n;
_null_terminated = false;
}
constexpr basic_string_view substr(std::size_t pos, std::size_t length = npos) const noexcept
{
LEXY_PRECONDITION(pos < _size);
if (length >= _size - pos)
{
auto result = basic_string_view(_ptr + pos, end());
result._null_terminated = _null_terminated;
return result;
}
else
{
// Note that we're loosing null-terminated-ness.
return basic_string_view(_ptr + pos, length);
}
}
constexpr bool starts_with(basic_string_view prefix) const noexcept
{
return substr(0, prefix.size()) == prefix;
}
constexpr bool try_remove_prefix(basic_string_view prefix) noexcept
{
if (!starts_with(prefix))
return false;
remove_prefix(prefix.length());
return true;
}
constexpr std::size_t find(basic_string_view str, std::size_t pos = 0) const noexcept
{
for (auto i = pos; i < length(); ++i)
{
if (substr(i, str.length()) == str)
return i;
}
return npos;
}
constexpr std::size_t find(CharT c, std::size_t pos = 0) const noexcept
{
return find(basic_string_view(&c, 1), pos);
}
//=== comparison ===//
friend constexpr bool operator==(basic_string_view<CharT> lhs,
basic_string_view<CharT> rhs) noexcept
{
if (lhs.size() != rhs.size())
return false;
for (auto a = lhs.begin(), b = rhs.begin(); a != lhs.end(); ++a, ++b)
if (*a != *b)
return false;
return true;
}
friend constexpr bool operator!=(basic_string_view<CharT> lhs,
basic_string_view<CharT> rhs) noexcept
{
return !(lhs == rhs);
}
private:
const CharT* _ptr;
std::size_t _size;
bool _null_terminated;
};
using string_view = basic_string_view<char>;
} // namespace lexy::_detail
namespace lexy::_detail
{
template <auto FnPtr, typename Indices = make_index_sequence<FnPtr().size()>>
struct _string_view_holder;
template <auto FnPtr, std::size_t... Indices>
struct _string_view_holder<FnPtr, index_sequence<Indices...>>
{
static constexpr auto view = FnPtr();
static constexpr typename decltype(view)::char_type value[] = {view[Indices]..., {}};
};
template <auto FnPtr>
inline constexpr const auto* make_cstr = _string_view_holder<FnPtr>::value;
} // namespace lexy::_detail
#endif // LEXY_DETAIL_STRING_VIEW_HPP_INCLUDED

View File

@@ -0,0 +1,251 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_SWAR_HPP_INCLUDED
#define LEXY_DETAIL_SWAR_HPP_INCLUDED
#include <climits>
#include <cstdint>
#include <cstring>
#include <lexy/_detail/config.hpp>
#include <lexy/input/base.hpp>
#if defined(_MSC_VER)
# include <intrin.h>
#endif
namespace lexy::_detail
{
// Contains the chars in little endian order; rightmost bits are first char.
using swar_int = std::uintmax_t;
// The number of chars that can fit into one SWAR.
template <typename CharT>
constexpr auto swar_length = [] {
static_assert(sizeof(CharT) < sizeof(swar_int) && sizeof(swar_int) % sizeof(CharT) == 0);
return sizeof(swar_int) / sizeof(CharT);
}();
template <typename CharT>
constexpr auto char_bit_size = sizeof(CharT) * CHAR_BIT;
template <typename CharT>
constexpr auto make_uchar(CharT c)
{
if constexpr (std::is_same_v<CharT, LEXY_CHAR8_T>)
// Not all libstdc++ support char8_t and std::make_unsigned_t.
return c;
else
return std::make_unsigned_t<CharT>(c);
}
template <typename CharT>
using uchar_t = decltype(make_uchar(CharT()));
// Returns a swar_int filled with the specific char.
template <typename CharT>
constexpr swar_int swar_fill(CharT _c)
{
auto c = make_uchar(_c);
auto result = swar_int(0);
for (auto i = 0u; i != swar_length<CharT>; ++i)
{
result <<= char_bit_size<CharT>;
result |= c;
}
return result;
}
// Returns a swar_int filled with the complement of the specific char.
template <typename CharT>
constexpr swar_int swar_fill_compl(CharT _c)
{
auto c = uchar_t<CharT>(~uchar_t<CharT>(_c));
auto result = swar_int(0);
for (auto i = 0u; i != swar_length<CharT>; ++i)
{
result <<= char_bit_size<CharT>;
result |= c;
}
return result;
}
constexpr void _swar_pack(swar_int&, int) {}
template <typename H, typename... T>
constexpr void _swar_pack(swar_int& result, int index, H h, T... t)
{
if (std::size_t(index) == char_bit_size<swar_int>)
return;
if (index >= 0)
result |= swar_int(make_uchar(h)) << index;
_swar_pack(result, index + int(char_bit_size<H>), t...);
}
template <typename CharT>
struct swar_pack_result
{
swar_int value;
swar_int mask;
std::size_t count;
constexpr CharT operator[](std::size_t idx) const
{
constexpr auto mask = (swar_int(1) << char_bit_size<CharT>)-1;
return (value >> idx * char_bit_size<CharT>)&mask;
}
};
// Returns a swar_int containing the specified characters.
// If more are provided than fit, will only take the first couple ones.
template <int SkipFirstNChars = 0, typename... CharT>
constexpr auto swar_pack(CharT... cs)
{
using char_type = std::common_type_t<CharT...>;
swar_pack_result<char_type> result{0, 0, 0};
_swar_pack(result.value, -SkipFirstNChars * int(char_bit_size<char_type>), cs...);
auto count = int(sizeof...(CharT)) - SkipFirstNChars;
if (count <= 0)
{
result.mask = 0;
result.count = 0;
}
else if (count >= int(swar_length<char_type>))
{
result.mask = swar_int(-1);
result.count = swar_length<char_type>;
}
else
{
result.mask = swar_int(swar_int(1) << count * int(char_bit_size<char_type>)) - 1;
result.count = std::size_t(count);
}
return result;
}
// Returns the index of the char that is different between lhs and rhs.
template <typename CharT>
constexpr std::size_t swar_find_difference(swar_int lhs, swar_int rhs)
{
if (lhs == rhs)
return swar_length<CharT>;
auto mask = lhs ^ rhs;
#if defined(__GNUC__)
auto bit_idx = __builtin_ctzll(mask);
#elif defined(_MSC_VER) && defined(_WIN64)
unsigned long bit_idx;
_BitScanForward64(&bit_idx, mask);
#elif defined(_MSC_VER)
unsigned long bit_idx = 0;
if (!_BitScanForward(&bit_idx, static_cast<std::uint32_t>(mask))
&& _BitScanForward(&bit_idx, mask >> 32))
bit_idx += 32;
#else
# error "unsupported compiler; please file an issue"
#endif
return std::size_t(bit_idx) / char_bit_size<CharT>;
}
// Returns true if v has a char less than N.
template <typename CharT, CharT N>
constexpr bool swar_has_char_less(swar_int v)
{
// https://graphics.stanford.edu/~seander/bithacks.html#HasLessInWord
constexpr auto offset = swar_fill(CharT(N));
auto zero_or_msb = v - offset;
constexpr auto msb_mask = swar_fill(CharT(1 << (char_bit_size<CharT> - 1)));
auto not_msb = ~v & msb_mask;
return zero_or_msb & not_msb;
}
// Returns true if v has a zero char.
template <typename CharT>
constexpr bool swar_has_zero(swar_int v)
{
return swar_has_char_less<CharT, 1>(v);
}
// Returns true if v contains the specified char.
template <typename CharT, CharT C>
constexpr bool swar_has_char(swar_int v)
{
if constexpr (C == 0)
{
return swar_has_zero<CharT>(v);
}
else
{
constexpr auto mask = swar_fill(C);
return swar_has_zero<CharT>(v ^ mask);
}
}
} // namespace lexy::_detail
namespace lexy::_detail
{
struct _swar_base
{};
template <typename Reader>
constexpr auto is_swar_reader = std::is_base_of_v<_swar_base, Reader>;
template <typename Derived>
class swar_reader_base : _swar_base
{
public:
swar_int peek_swar() const
{
auto ptr = static_cast<const Derived&>(*this).position();
swar_int result;
#if LEXY_IS_LITTLE_ENDIAN
std::memcpy(&result, ptr, sizeof(swar_int));
#else
using char_type = typename Derived::encoding::char_type;
auto dst = reinterpret_cast<char*>(&result);
auto length = sizeof(swar_int) / sizeof(char_type);
for (auto i = 0u; i != length; ++i)
{
std::memcpy(dst + i, ptr + length - i - 1, sizeof(char_type));
}
#endif
return result;
}
void bump_swar()
{
auto ptr = static_cast<Derived&>(*this).position();
ptr += swar_length<typename Derived::encoding::char_type>;
static_cast<Derived&>(*this).reset({ptr});
}
void bump_swar(std::size_t char_count)
{
auto ptr = static_cast<Derived&>(*this).position();
ptr += char_count;
static_cast<Derived&>(*this).reset({ptr});
}
};
constexpr std::size_t round_size_for_swar(std::size_t size_in_bytes)
{
// We round up to the next multiple.
if (auto remainder = size_in_bytes % sizeof(swar_int); remainder > 0)
size_in_bytes += sizeof(swar_int) - remainder;
// Then add one extra space of padding on top.
size_in_bytes += sizeof(swar_int);
return size_in_bytes;
}
} // namespace lexy::_detail
#endif // LEXY_DETAIL_SWAR_HPP_INCLUDED

View File

@@ -0,0 +1,119 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_TUPLE_HPP_INCLUDED
#define LEXY_DETAIL_TUPLE_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
#include <lexy/_detail/integer_sequence.hpp>
namespace lexy::_detail
{
template <std::size_t Idx, typename T>
struct _tuple_holder
{
#if !defined(__GNUC__) || defined(__clang__)
// https://gcc.gnu.org/bugzilla/show_bug.cgi?id=105795
LEXY_EMPTY_MEMBER
#endif
T value;
};
template <std::size_t Idx, typename... T>
struct _nth_type;
template <std::size_t Idx, typename H, typename... T>
struct _nth_type<Idx, H, T...>
{
using type = typename _nth_type<Idx - 1, T...>::type;
};
template <typename H, typename... T>
struct _nth_type<0, H, T...>
{
using type = H;
};
template <typename T>
struct _tuple_get_type
{
using type = T&;
};
template <typename T>
struct _tuple_get_type<T&&>
{
using type = T&&;
};
template <typename Indices, typename... T>
class _tuple;
template <std::size_t... Idx, typename... T>
class _tuple<index_sequence<Idx...>, T...> : public _tuple_holder<Idx, T>...
{
public:
constexpr _tuple() = default;
template <typename... Args>
constexpr _tuple(Args&&... args) : _tuple_holder<Idx, T>{LEXY_FWD(args)}...
{}
};
template <typename... T>
struct tuple : _tuple<index_sequence_for<T...>, T...>
{
constexpr tuple() = default;
template <typename... Args>
constexpr explicit tuple(Args&&... args)
: _tuple<index_sequence_for<T...>, T...>(LEXY_FWD(args)...)
{}
template <std::size_t N>
using element_type = typename _nth_type<N, T...>::type;
template <std::size_t N>
constexpr decltype(auto) get() noexcept
{
// NOLINTNEXTLINE: this is fine.
auto&& holder = static_cast<_tuple_holder<N, element_type<N>>&>(*this);
// NOLINTNEXTLINE
return static_cast<typename _tuple_get_type<element_type<N>>::type>(holder.value);
}
template <std::size_t N>
constexpr decltype(auto) get() const noexcept
{
// NOLINTNEXTLINE: this is fine.
auto&& holder = static_cast<const _tuple_holder<N, element_type<N>>&>(*this);
// NOLINTNEXTLINE
return static_cast<typename _tuple_get_type<const element_type<N>>::type>(holder.value);
}
static constexpr auto index_sequence()
{
return index_sequence_for<T...>{};
}
};
template <>
struct tuple<>
{
constexpr tuple() = default;
static constexpr auto index_sequence()
{
return index_sequence_for<>{};
}
};
template <typename... Args>
constexpr auto make_tuple(Args&&... args)
{
return tuple<std::decay_t<Args>...>(LEXY_FWD(args)...);
}
template <typename... Args>
constexpr auto forward_as_tuple(Args&&... args)
{
return tuple<Args&&...>(LEXY_FWD(args)...);
}
} // namespace lexy::_detail
#endif // LEXY_DETAIL_TUPLE_HPP_INCLUDED

View File

@@ -0,0 +1,130 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DETAIL_TYPE_NAME_HPP_INCLUDED
#define LEXY_DETAIL_TYPE_NAME_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
#include <lexy/_detail/detect.hpp>
#include <lexy/_detail/integer_sequence.hpp>
#include <lexy/_detail/string_view.hpp>
namespace lexy::_detail
{
template <typename T>
using _detect_name_f = std::enable_if_t<std::is_convertible_v<decltype(T::name()), string_view>>;
template <typename T>
using _detect_name_v = decltype(T::name);
template <typename T>
constexpr auto _full_type_name()
{
#if defined(__clang__)
# define LEXY_HAS_AUTOMATIC_TYPE_NAME 1
# define LEXY_HAS_CONSTEXPR_AUTOMATIC_TYPE_NAME 1
constexpr auto prefix = string_view("auto lexy::_detail::_full_type_name() [T = ");
constexpr auto suffix = string_view("]");
auto function = string_view(__PRETTY_FUNCTION__);
function.remove_prefix(prefix.length());
function.remove_suffix(suffix.length());
function.try_remove_prefix("(anonymous namespace)::");
return function;
#elif defined(__GNUC__)
# define LEXY_HAS_AUTOMATIC_TYPE_NAME 1
# if __GNUC__ > 8
# define LEXY_HAS_CONSTEXPR_AUTOMATIC_TYPE_NAME 1
# else
# define LEXY_HAS_CONSTEXPR_AUTOMATIC_TYPE_NAME 0
# endif
constexpr auto prefix
= string_view("constexpr auto lexy::_detail::_full_type_name() [with T = ");
constexpr auto suffix = string_view("]");
auto function = string_view(__PRETTY_FUNCTION__);
function.remove_prefix(prefix.length());
function.remove_suffix(suffix.length());
function.try_remove_prefix("{anonymous}::");
return function;
#elif defined(_MSC_VER)
# define LEXY_HAS_AUTOMATIC_TYPE_NAME 1
# define LEXY_HAS_CONSTEXPR_AUTOMATIC_TYPE_NAME 1
constexpr auto prefix = string_view("auto __cdecl lexy::_detail::_full_type_name<");
constexpr auto suffix = string_view(">(void)");
auto function = string_view(__FUNCSIG__);
function.remove_prefix(prefix.length());
function.remove_suffix(suffix.length());
function.try_remove_prefix("struct ") || function.try_remove_prefix("class ");
function.try_remove_prefix("`anonymous-namespace'::");
return function;
#else
# define LEXY_HAS_AUTOMATIC_TYPE_NAME 0
# define LEXY_HAS_CONSTEXPR_AUTOMATIC_TYPE_NAME 0
return string_view("unknown-type");
#endif
}
template <typename T, int NsCount>
constexpr string_view _type_name()
{
auto name = _full_type_name<T>();
if (name.find('<') != string_view::npos && NsCount != 0)
return name;
for (auto namespace_count = NsCount; namespace_count > 0; --namespace_count)
{
auto pos = name.find("::");
if (pos == string_view::npos)
break;
name.remove_prefix(pos + 2);
}
return name;
}
template <typename T, int NsCount = 1>
constexpr const char* type_name()
{
if constexpr (_detail::is_detected<_detect_name_f, T>)
return T::name();
else if constexpr (_detail::is_detected<_detect_name_v, T>)
return T::name;
else if constexpr (LEXY_HAS_CONSTEXPR_AUTOMATIC_TYPE_NAME)
return make_cstr<_type_name<T, NsCount>>;
else
return "unknown-type";
}
template <typename T, int NsCount>
inline constexpr const char* _type_id_holder = type_name<T, NsCount>();
// Returns a unique address for each type.
// For implementation reasons, it also doubles as the pointer to the name.
template <typename T, int NsCount = 1>
constexpr const char* const* type_id()
{
if constexpr (_detail::is_detected<_detect_name_v, T> //
&& !_detail::is_detected<_detect_name_f, T>)
{
// We can use the address of the static constexpr directly.
return &T::name;
}
else
{
// We instantiate a variable template with a function unique by type.
// As the variable is inline, there should be a single address only.
return &_type_id_holder<T, NsCount>;
}
}
} // namespace lexy::_detail
#endif // LEXY_DETAIL_TYPE_NAME_HPP_INCLUDED

View File

@@ -0,0 +1,83 @@
// AUTOGENERATED FILE --- DO NOT EDIT!
// Generated by `support/generate-unicode-db.py`.
#define LEXY_UNICODE_DATABASE_VERSION "15.0.0"
namespace lexy::_unicode_db
{
constexpr const char* block_starts = "\000\001\002\003\004\005\006\007\010\011\012\013\014\015\016\017\020\021\022\023\024\021\025\026\027\030\031\032\033\034\035\036\037 !\"#$%&'(!)*+,-./0'\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\0211\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\0212\021\021\0213\021456789\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021:;;;;;;;;<<<<<<<<<<<<<<<<<<<<<<<<<\021=>?@ABCDEFGH\021IJKLMNOPQRSTUVWXYZ[\\]^_`a\021\021\021bcdeeeeeeeeef\021\021\021\021geeeeeeeeeeeeeee\021\021heeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee\021\021ijeekl\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021m\021\021\021\021noeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeep\021qreeeeeeeeeseeeeeeeeeeeeeeeeeetuvwxyz{|''}eeee~\177\200\201e\202ee\203\204\205ee\206\207\210e\211\212\213\214''\215\216\217'\220\221eeee\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\222\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\223\224\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\225\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\226eeeeeeeeeeee\021\021\227eeeee\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\230\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\231eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
"eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
"eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee"
"eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee\232\233eeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeeee<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\234"
"<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<\234";
constexpr const char* blocks = "\000\000\000\000\000\000\000\000\000\001\001\001\001\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\002\003\003\003\004\003\003\003\005\006\003\007\003\010\003\003\011\011\011\011\011\011\011\011\011\011\003\003\007\007\007\003\003\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\005\003\006\013\014\013\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\005\007\006\007\000\000\000\000\000\000\001\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\000\002\003\004\004\004\004\016\003\013\016\017\020\007\021\016\013\016\007\022\022\013\023\003\024\013\022\017\025\022\022\022\003\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\007\012\012\012\012\012\012\012\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\007\015\015\015\015\015\015\015\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\027\015\026\015\026\015\026\015\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\030\026\015\026\015\026\015\031\015\032\026\015\026\015\033\026\015\034\034\026\015\015\035\036\037\026\015\034 \015!\"\026\015\015\015!#\015$\026\015\026\015\026\015%\026\015%\015\015\026\015%\026\015&&\026\015\026\015'\026\015\015(\026\015\015\015(((()*\015)*\015)*\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\015)*\015\026\015+,\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015-\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\015\015\015\015\015\015.\026\015/0\015\015\026\015123\026\015\026\015\026\015\026\015\026\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015(\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015444444444555555544\013\013\013\013555555555555\013\013\013\013\013\013\013\013\013\013\013\013\013\01344444\013\013\013\013\013\013\0135\0135\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\0136666666666666666666666666666666666666666666666666666666666666666666667666666666666666666666666666666666666666666\026\015\026\0155\013\026\015889\015\015\015\003:8888\013\013;\024<<<8=8>>\015\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\0128\012\012\012\012\012\012\012\012\012\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015?\015\015\015\015\015\015\015\015\015\015\015\015@AB\027\027\027CD\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015EF\015\015GH\007\026\015I\026\015\015---"
"JJJJJJJJJJJJJJJJ\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\01666666KK\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015L\026\015\026\015\026\015\026\015\026\015\026\015\026\015\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\0158MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM885\003\003\003\003\003\003\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\003\01088\016\016\00486666666666666666666666666666666NNNNNNNNNNNNNN\010N\003NN\003NN\003N88888888(((((((((((((((((((((((((((8888((((\003\00388888888888\021\021\021\021\021\021\007\007\007\003\003\004\003\003\016\016NNNNNNNNNNN\003\021\003\003\003((((((((((((((((((((((((((((((((5((((((((((NNNNNNNNNNNNN6NNNNNNN\011\011\011\011\011\011\011\011\011\011\003\003\003\003((N(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((\003(NNNNNNN\021\01666NNNN55NN\016666N((\011\011\011\011\011\011\011\011\011\011(((\016\016(\003\003\003\003\003\003\003\003\003\003\003\003\003\0038\021(N((((((((((((((((((((((((((((((NNNNNNNNNNNNNNNN6666666666688(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((NNNNNNNNNNN(88888888888888\011\011\011\011\011\011\011\011\011\011(((((((((((((((((((((((((((((((((66666666655\016\003\003\0035886\004\004"
"((((((((((((((((((((((NN665NNNNNNNNN5NNN5NNNN688\003\003\003\003\003\003\003\003\003\003\003\003\003\003\0038(((((((((((((((((((((((((66688\0038(((((((((((88888((((((((((((((((((((((((\013((((((8\021\02188888866666666(((((((((((((((((((((((((((((((((((((((((56666666666NNNNNNNNNNNN66\021NNNNNNN666666NNNNNNNNNNNNNNNNNNNO((((((((((((((((((((((((((((((((((((((((((((((((((((((NO6(OOONNNNNNNNOOOO6OO(6666NNN((((((((((NN\003\003\011\011\011\011\011\011\011\011\011\011\0035(((((((((((((((NOO8((((((((88((88((((((((((((((((((((((8(((((((8(888((((886(OOONNNN88OO88OO6(88888888O8888((8(((NN88\011\011\011\011\011\011\011\011\011\011((\004\004\022\022\022\022\022\022\016\004(\003688NNO8((((((8888((88((((((((((((((((((((((8(((((((8((8((8((8868OOONN8888NN88NN6888N8888888((((8(8888888\011\011\011\011\011\011\011\011\011\011NN(((N\0038888888888NNO8(((((((((8(((8((((((((((((((((((((((8(((((((8((8(((((886(OOONNNNN8NNO8OO688(888888888888888((NN88\011\011\011\011\011\011\011\011\011\011\003\0048888888(NNN6668NOO8((((((((88((88((((((((((((((((((((((8(((((((8((8(((((886(ONONNNN88OO88OO688888886NO8888((8(((NN88\011\011\011\011\011\011\011\011\011\011\016(\022\022\022\022\022\0228888888888N(8((((((888(((8((((888((8(8((888((888(((888((((((((((((8888OONOO888OOO8OOO688(888888O88888888888888\011\011\011\011\011\011\011\011\011\011\022\022\022\016\016\016\016\016\016\004\01688888"
"NOOON((((((((8(((8(((((((((((((((((((((((8((((((((((((((((886(NNNOOOO8NNN8NNN68888888NN8(((88(88((NN88\011\011\011\011\011\011\011\011\011\0118888888\003\022\022\022\022\022\022\022\016(NOO\003((((((((8(((8(((((((((((((((((((((((8((((((((((8(((((886(ONOOOOO8NOO8OON68888888OO888888((8((NN88\011\011\011\011\011\011\011\011\011\0118((O888888888888NNOO(((((((((8(((8(((((((((((((((((((((((((((((((((((((((((66(OOONNNN8OOO8OOO6(\0168888(((O\022\022\022\022\022\022\022(((NN88\011\011\011\011\011\011\011\011\011\011\022\022\022\022\022\022\022\022\022\016((((((8NOO8((((((((((((((((((888((((((((((((((((((((((((8(((((((((8(88(((((((88868888OOONNN8N8OOOOOOOO888888\011\011\011\011\011\011\011\011\011\01188OO\003888888888888((((((((((((((((((((((((((((((((((((((((((((((((N(PNNNNNNN8888\004((((((5666666N6\003\011\011\011\011\011\011\011\011\011\011\003\0038888888888888888888888888888888888888((8(8(((((8((((((((((((((((((((((((8(8((((((((((N(PNNNNNN6NN(88(((((85866666N68\011\011\011\011\011\011\011\011\011\01188((((88888888888888888888888888888888(\016\016\016\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003\016\003\016\016\01666\016\016\016\016\016\016\011\011\011\011\011\011\011\011\011\011\022\022\022\022\022\022\022\022\022\022\0166\0166\0166\005\006\005\006QQ((((((((8((((((((((((((((((((((((((((((((((((8888NNNNNNNNNNNNNNONNNN6\00366(((((NNNNNNNNNNN8NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN8\016\016\016\016\016\016\016\0166\016\016\016\016\016\0168\016\016\003\003\003\003\003\016\016\016\016\003\0038888888888888888888888888888888888888"
"(((((((((((((((((((((((((((((((((((((((((((OONNNNONNNNN6O66OONN(\011\011\011\011\011\011\011\011\011\011\003\003\003\003\003\003((((((OONN((((NNN(OOO((OOOOOOO(((NNNN(((((((((((((NOONNOOOOOON(O\011\011\011\011\011\011\011\011\011\011OOON\016\016RRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRRR8R88888R88\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\0034\015\015\015(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8((((88(((((((8(8((((88(((((((((((((((((((((((((((((((((((((((((8((((88(((((((((((((((((((((((((((((((((8((((88(((((((8(8((((88(((((((((((((((8(((((((((((((((((((((((((((((((((((((((((((((((((((((((((8((((88(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88666\003\003\003\003\003\003\003\003\003SSSSSSSSS\022\022\022\022\022\022\022\022\022\022\022888((((((((((((((((\016\016\016\016\016\016\016\016\016\016888888\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\02788TTTTTT88"
"\010((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((\016\003(((((((((((((((((\002((((((((((((((((((((((((((\005\006888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((\003\003\003UUU((((((((8888888((((((((((((((((((NN6Q888888888(((((((((((((((((((NNQ\003\003888888888((((((((((((((((((NN888888888888(((((((((((((8(((8NN888888888888((((((((((((((((((((((((((((((((((((((((((((((((((((66ONNNNNNNOOOOOOOONOO66666666666\003\003\0035\003\003\003\004(688\011\011\011\011\011\011\011\011\011\011888888\022\022\022\022\022\022\022\022\022\022888888\003\003\003\003\003\003\010\003\003\003\003666\0216\011\011\011\011\011\011\011\011\011\011888888(((((((((((((((((((((((((((((((((((5(((((((((((((((((((((((((((((((((((((((((((((((((((((8888888(((((VV((((((((((((((((((((((((((((((((((N(88888((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8888888888"
"(((((((((((((((((((((((((((((((8NNNOOOONNOOO8888OONOOOOOO6668888\016888\003\003\011\011\011\011\011\011\011\011\011\011((((((((((((((((((((((((((((((88(((((88888888888((((((((((((((((((((((((((((((((((((((((((((8888((((((((((((((((((((((((((888888\011\011\011\011\011\011\011\011\011\011S888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016(((((((((((((((((((((((NNOON88\003\003(((((((((((((((((((((((((((((((((((((((((((((((((((((ONONNNNNNN86ONOONNNNNNNNOOOOOONN66666666886\011\011\011\011\011\011\011\011\011\011888888\011\011\011\011\011\011\011\011\011\011888888\003\003\003\003\003\003\0035\003\003\003\003\003\0038866666666666666KNN66666666666NNN8888888888888888888888888888888888888888888888888NNNNO(((((((((((((((((((((((((((((((((((((((((((((((6ONNNNNONOOOOONOQ((((((((888\011\011\011\011\011\011\011\011\011\011\003\003\003\003\003\003\003\016\016\016\016\016\016\016\016\016\016666666666\016\016\016\016\016\016\016\016\016\003\0038NNO((((((((((((((((((((((((((((((ONNNNOONNQ6NN((\011\011\011\011\011\011\011\011\011\011((((((((((((((((((((((((((((((((((((((((((((6ONNOOONONNNQQ88888888\003\003\003\003((((((((((((((((((((((((((((((((((((OOOOOOOONNNNNNNNOON6888\003\003\003\003\003\011\011\011\011\011\011\011\011\011\011888(((\011\011\011\011\011\011\011\011\011\011((((((((((((((((((((((((((((((555555\003\003WXYZZ[\\]^8888888___________________________________________88___\003\003\003\003\003\003\003\00388888888666\0036666666666666Q6666666((((6((((((6((Q66(88888"
"\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015444444444444444444444444444444444444444444444444444444444444444\015\015\015\015\015\015\015\015\015\015\015\015\0154\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\0154444444444444444444444444444444444444666666666666666666666666666666666666666NNNNNNNNNNNNNN66666666666\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\015\015\015\015\015`\015\015a\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\015\015\015\015\015\015\015\015bbbbbbbb\015\015\015\015\015\01588bbbbbb88\015\015\015\015\015\015\015\015bbbbbbbb\015\015\015\015\015\015\015\015bbbbbbbb\015\015\015\015\015\01588bbbbbb88\015\015\015\015\015\015\015\0158b8b8b8b\015\015\015\015\015\015\015\015bbbbbbbb\015\015\015\015\015\015\015\015\015\015\015\015\015\01588\015\015\015\015\015\015\015\015cccccccc\015\015\015\015\015\015\015\015cccccccc\015\015\015\015\015\015\015\015cccccccc\015\015\015\015\0158\015\015bbdde\013f\013\013\013\015\015\0158\015\015gggge\013\013\013\015\015\015\01588\015\015bbhh8\013\013\013\015\015\015\015\015\015\015\015bbiiI\013\013\01388\015\015\0158\015\015jjkke\013\0138\002\002\002\002\002\002\002\002\002\002\002\021ll\021\021\010\010\010\010\010\010\003\003\020\025\005\020\020\025\005\020\003\003\003\003\003\003\003\003mn\021\021\021\021\021\002\003\003\003\003\003\003\003\003\003\020\025\003\003\003\003\014\014\003\003\003\007\005\006\003\003\003\003\003\003\003\003\003\003\003\007\003\014\003\003\003\003\003\003\003\003\003\003\002\021\021\021\021\0218\021\021\021\021\021\021\021\021\021\021\022488\022\022\022\022\022\022\007\007\007\005\0064\022\022\022\022\022\022\022\022\022\022\007\007\007\005\00684444444444444888\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\004\0048888888888888886666666666666KKKK6KKK666666666666888888888888888"
"\016\016\027\016\016\016\016\027\016\016\015\027\027\027\015\015\027\027\027\015\016\027\016\016o\027\027\027\027\027\016\016\016\016\016\016\027\016p\016\027\016qr\027\027s\015\027\027t\027\015((((\015\016\016\015\015\027\027\007\007\007\007\007\027\015\015\015\015\016\007\016\016\015\016\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022uuuuuuuuuuuuuuuuvvvvvvvvvvvvvvvvUUU\026\015UUUU\022\016\0168888\007\007\007\007\007\016\016\016\016\016\007\007\016\016\016\016\007\016\016\007\016\016\007\016\016\016\016\016\016\016\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\007\016\016\007\016\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\016\016\016\016\016\016\016\016\005\006\005\006\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\007\016\016\016\016\016\016\016\005\006\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\007\007\007\007\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888888888888888888888888\016\016\016\016\016\016\016\016\016\016\016888888888888888888888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016wwwwwwwwwwwwwwwwwwwwwwwwwwxxxxxxxxxxxxxxxxxxxxxxxxxx\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022"
"\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\016\016\016\016\016\016\016\016\016\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\007\007\007\007\007\007\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\005\006\005\006\005\006\005\006\005\006\005\006\005\006\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\007\007\007\007\005\006\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\005\006\005\006\005\006\005\006\005\006\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016"
"\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\005\006\005\006\005\006\005\006\005\006\005\006\005\006\005\006\005\006\005\006\005\006\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\005\006\005\006\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\005\006\007\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\007\016\016\007\007\007\007\007\007\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\01688\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016MMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMMM\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\026\015yz{\015\015\026\015\026\015\026\015|}~\177\015\026\015\015\026\015\015\015\015\015\01544\200\200\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\015\016\016\016\016\016\016\026\015\026\015666\026\01588888\003\003\003\003\022\003\003\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\0158\01588888\01588((((((((((((((((((((((((((((((((((((((((((((((((((((((((88888885\003888888888888886(((((((((((((((((((((((888888888(((((((8(((((((8(((((((8(((((((8(((((((8(((((((8(((((((8(((((((8NNNNNNNNNNNNNNNNNNNNNNNNNNNNNNNN"
"\003\003\020\025\020\025\003\003\003\020\025\003\020\025\003\003\003\003\003\003\003\003\003\010\003\003\010\003\020\025\003\003\020\025\005\006\005\006\005\006\005\006\003\003\003\003\003\201\003\003\003\003\003\003\003\003\003\003\010\010\003\003\003\003\010\003\005\003\003\003\003\003\003\003\003\003\003\003\003\003\016\016\003\003\003\005\006\005\006\005\006\005\006\0108888888888888888888888888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\01688888888888888888888888888\016\016\016\016\016\016\016\016\016\016\016\0168888\002\003\003\003\0165(U\005\006\005\006\005\006\005\006\005\006\016\016\005\006\005\006\005\006\005\006\010\005\006\006\016UUUUUUUUU6666QQ\01055555\016\016UUU5(\003\016\0168((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8866\013\01355(\010((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((\003555(88888(((((((((((((((((((((((((((((((((((((((((((8((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8\016\016\022\022\022\022\016\016\016\016\016\016\016\016\016\016((((((((((((((((((((((((((((((((\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888888888888(((((((((((((((("
"\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168\022\022\022\022\022\022\022\022\022\022\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\022\022\022\022\022\022\022\022\016\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\022\022\022\022\022\022\022\022\022\022\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016(((((((((((((((((((((5(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888888888((((((((((((((((((((((((((((((((((((((((555555\003\003"
"((((((((((((5\003\003\003((((((((((((((((\011\011\011\011\011\011\011\011\011\011((88888888888888888888\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015(6KKK\003NNNNNNNN66\0035\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\01544NN((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((UUUUUUUUUU66\003\003\003\003\003\00388888888\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013555555555\013\013\026\015\026\015\026\015\026\015\026\015\026\015\026\015\015\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\0154\015\015\015\015\015\015\015\015\026\015\026\015\202\026\015\026\015\026\015\026\015\026\0155\013\013\026\015\203\015(\026\015\026\015\015\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\204\205\206\207\204\015\210\211\212\213\026\015\026\015\026\015\026\015\026\015\026\015\026\015\026\015\214\215\216\026\015\026\01588888\026\0158\0158\015\026\015\026\015888888888888888888888888444\026\015(44\015(((((((N(((6((((N(((((((((((((((((((((((OONNO\016\016\016\0166888\022\022\022\022\022\022\016\016\004\016888888((((((((((((((((((((((((((((((((((((((((((((((((((((\003\003\003\00388888888OO((((((((((((((((((((((((((((((((((((((((((((((((((OOOOOOOOOOOOOOOO6N88888888\003\003\011\011\011\011\011\011\011\011\011\011888888666666666666666666((((((\003\003\003(\003((N\011\011\011\011\011\011\011\011\011\011((((((((((((((((((((((((((((NNNNN666\003\003(((((((((((((((((((((((NNNNNNNNNNNOQ88888888888\003(((((((((((((((((((((((((((((888NNNO(((((((((((((((((((((((((((((((((((((((((((((((6OONNNNOONNOOQ\003\003\003\003\003\003\003\003\003\003\003\003\00385\011\011\011\011\011\011\011\011\011\0118888\003\003(((((N5(((((((((\011\011\011\011\011\011\011\011\011\011(((((8"
"(((((((((((((((((((((((((((((((((((((((((NNNNNNOONNOONN888888888(((N((((((((NO88\011\011\011\011\011\011\011\011\011\01188\003\003\003\003((((((((((((((((5((((((\016\016\016(ONO((((((((((((((((((((((((((((((((((((((((((((((((((N(NNN((NN(((((N6(6(888888888888888888888888((5\003\003(((((((((((ONNOO\003\003(55O68888888888((((((88((((((88((((((888888888(((((((8(((((((8\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\0134444\015\015\015\015\015\015\015\015\0154\013\0138888\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217\217(((((((((((((((((((((((((((((((((((OONOONOO\003Q688\011\011\011\011\011\011\011\011\011\011888888((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((888888888888(((((((((((((((((((((((8888(((((((((((((((((((((((((((((((((((((((((((((((((8888\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220\220"
"\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88888888888888888888888888888888888888\015\015\015\015\015\015\015888888888888\015\015\015\015\01588888(N((((((((((\007(((((((((((((8(((((8(8((8((8((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\013\0138888888888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((\222\222\222\222\222\222(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((("
"((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((\006\005\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88((((((((((((((((((((((((((((((((((((((((((((((((((((((8888888\01688888888888888888888888888888888((((((((((\222\222\004\016\016\0166666666666666666\003\003\003\003\003\003\003\005\006\0038888886666666666666666\003\010\010\014\014\005\006\005\006\005\006\005\006\005\006\005\006\005\006\005\006\003\003\005\006\003\003\003\003\014\014\014\003\003\0038\003\003\003\003\010\005\006\005\006\005\006\003\003\003\007\010\007\007\0078\003\004\003\0038888\222(\222(\2228\222(\222(\222(\222(\222((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88\0218\003\003\003\004\003\003\003\005\006\003\007\003\010\003\003\011\011\011\011\011\011\011\011\011\011\003\003\007\007\007\003\003\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\005\003\006\013\014\013\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\005\007\006\007\005\006\003\005\006\003\003((((((((((5(((((((((((((((((((((((((((((((((((((((((((((\223\223(((((((((((((((((((((((((((((((888((((((88((((((88((((((88(((888\004\004\007\013\016\004\0048\016\007\007\007\007\016\0168888888888\021\021\021\016\01688((((((((((((8((((((((((((((((((((((((((8(((((((((((((((((((8((8(((((((((((((((88((((((((((((((8888888888888888888888888888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88888"
"\003\003\0038888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022888\016\016\016\016\016\016\016\016\016UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU\022\022\022\022\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\022\022\016\016\0168\016\016\016\016\016\016\016\016\016\016\016\016\016888\01688888888888888888888888888888888888888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\01668888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((((((((((((((((((((((((888(((((((((((((((((((((((((((((((((((((((((((((((((8888888888888886\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\0228888((((((((((((((((((((((((((((((((\022\022\022\022888888888((((((((((((((((((((U((((((((U88888((((((((((((((((((((((((((((((((((((((NNNNN88888((((((((((((((((((((((((((((((8\003((((((((((((((((((((((((((((((((((((8888((((((((\003UUUUU888888888888888888888888888888888888888888\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88\011\011\011\011\011\011\011\011\011\011888888\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\224\2248888\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\0158888"
"((((((((((((((((((((((((((((((((((((((((88888888((((((((((((((((((((((((((((((((((((((((((((((((((((88888888888\003\225\225\225\225\225\225\225\225\225\225\2258\225\225\225\225\225\225\225\225\225\225\225\225\225\225\2258\225\225\225\225\225\225\2258\225\2258\015\015\015\015\015\015\015\015\015\015\0158\015\015\015\015\015\015\015\015\015\015\015\015\015\015\0158\015\015\015\015\015\015\0158\015\0158888888888888888888888888888888888888888888888888888888888888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((888888888((((((((((((((((((((((8888888888((((((((88888888888888888888888845544484444444444444444444444444444444444444444448444444444888888888888888888888888888888888888888888888888888888888888888888888((((((88(8((((((((((((((((((((((((((((((((((((((((((((8((888(88(((((((((((((((((((((((8\003\022\022\022\022\022\022\022\022(((((((((((((((((((((((\016\016\022\022\022\022\022\022\022(((((((((((((((((((((((((((((((88888888\022\022\022\022\022\022\022\022\022888888888888888888888888888888888888888888888888(((((((((((((((((((8((88888\022\022\022\022\022((((((((((((((((((((((\022\022\022\022\022\022888\003((((((((((((((((((((((((((88888\0038888888888888888888888888888888888888888888888888888888888888888((((((((((((((((((((((((((((((((((((((((((((((((((((((((8888\022\022((\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\02288\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022"
"(NNN8NN88888NNNN((((8(((8(((((((((((((((((((((((((((((8866688886\022\022\022\022\022\022\022\022\0228888888\003\003\003\003\003\003\003\003\0038888888(((((((((((((((((((((((((((((\022\022\003(((((((((((((((((((((((((((((\022\022\02288888888888888888888888888888888((((((((\016((((((((((((((((((((((((((((668888\022\022\022\022\022\003\003\003\003\003\003\003888888888((((((((((((((((((((((((((((((((((((((((((((((((((((((888\003\003\003\003\003\003\003((((((((((((((((((((((88\022\022\022\022\022\022\022\022(((((((((((((((((((88888\022\022\022\022\022\022\022\022((((((((((((((((((8888888\003\003\003\003888888888888\022\022\022\022\022\022\02288888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8888888888888888888888888888888888888888888888888888888===================================================8888888888888\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\0158888888\022\022\022\022\022\022((((((((((((((((((((((((((((((((((((NNNN88888888\011\011\011\011\011\011\011\011\011\011888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\0228((((((((((((((((((((((((((((((((((((((((((8NN\01088((888888888888888888888888888888888888888888888888888888888888888888888888888666(((((((((((((((((((((((((((((\022\022\022\022\022\022\022\022\022\022(88888888((((((((((((((((((((((66666666666\022\022\022\022\003\003\003\003\0038888888888888888888888((((((((((((((((((6666\003\003\003\00388888888888888888888888888888888888888(((((((((((((((((((((\022\022\022\022\022\022\02288888888888888888888(((((((((((((((((((((((888888888ONO(((((((((((((((((((((((((((((((((((((((((((((((((((((NNNNNNNNNNNNNN6\003\003\003\003\003\003\0038888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\011\011\011\011\011\011\011\011\011\0116((NN(8888888886NNO(((((((((((((((((((((((((((((((((((((((((((((OOONNNNOO66\003\003\021\003\003\003\003N8888888888\02188(((((((((((((((((((((((((8888888\011\011\011\011\011\011\011\011\011\011888888NNN((((((((((((((((((((((((((((((((((((NNNNNONNNNNN668\011\011\011\011\011\011\011\011\011\011\003\003\003\003(OO(88888888(((((((((((((((((((((((((((((((((((6\003\003(888888888NNO((((((((((((((((((((((((((((((((((((((((((((((((OOONNNNNNNNNOQ((((\003\003\003\0036666\003ON\011\011\011\011\011\011\011\011\011\011(\003(\003\003\0038\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\02288888888888"
"((((((((((((((((((8(((((((((((((((((((((((((OOONNNOONQ6N\003\003\003\003\003\003N((N88888888888888888888888888888888888888888888888888888888888888(((((((8(8((((8(((((((((((((((8((((((((((\003888888(((((((((((((((((((((((((((((((((((((((((((((((NOOONNNNNN6688888\011\011\011\011\011\011\011\011\011\011888888NNOO8((((((((88((88((((((((((((((((((((((8(((((((8((8(((((866(OONOOOO88OO88OOQ88(888888O88888(((((OO886666666888666668888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((OOONNNNNNNNOO6NNO6((((\003\003\003\003\003\011\011\011\011\011\011\011\011\011\011\003\0038\0036(((888888888888888888888888888888((((((((((((((((((((((((((((((((((((((((((((((((OOONNNNNNONOOOONNO66((\003(88888888\011\011\011\011\011\011\011\011\011\0118888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((((((((((((((((((((((((((((((((((((((((((OOONNNN88OOOONNO66\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003\003((((NN8888888888888888888888888888888888"
"((((((((((((((((((((((((((((((((((((((((((((((((OOONNNNNNNNOONO6N\003\003\003(88888888888\011\011\011\011\011\011\011\011\011\011888888\003\003\003\003\003\003\003\003\003\003\003\003\0038888888888888888888(((((((((((((((((((((((((((((((((((((((((((NONOONNNNNNQ6(\003888888\011\011\011\011\011\011\011\011\011\011888888888888888888888888888888888888888888888888888888(((((((((((((((((((((((((((88NNNOONNNNONNNN68888\011\011\011\011\011\011\011\011\011\011\022\022\003\003\003\016(((((((88888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888((((((((((((((((((((((((((((((((((((((((((((OOONNNNNNNNNO66\0038888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\011\011\011\011\011\011\011\011\011\011\022\022\022\022\022\022\022\022\022888888888888((((((((88(88((((((((8((8((((((((((((((((((((((((OOOOOO8OO88NNQ6(O(O6\003\003\003888888888\011\011\011\011\011\011\011\011\011\0118888888888888888888888888888888888888888888888888888888888888888888888((((((((88(((((((((((((((((((((((((((((((((((((((OOONNNN88NNOOOO6(\003(O888888888888888888888888888"
"(NNNNNNNNNN((((((((((((((((((((((((((((((((((((((((66NNNNO(NNNN\003\003\003\003\003\003\003\003688888888(NNNNNNOONNN((((((((((((((((((((((((((((((((((((((((((((((NNNNNNNNNNNNNO66\003\003\003(\003\003\003\003\0038888888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8888888\003\003\003\003\003\003\003\003\003\003888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((((8(((((((((((((((((((((((((((((((((((((ONNNNNNN8NNNNNNO6(\003\003\003\003\0038888888888\011\011\011\011\011\011\011\011\011\011\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022888\003\003((((((((((((((((((((((((((((((88NNNNNNNNNNNNNNNNNNNNNN8ONNNNNNNONNONN8888888888888888888888888888888888888888888888888888888888888888888888888(((((((8((8((((((((((((((((((((((((((((((((((((((NNNNNN888N8NN8NNN6N66(N88888888\011\011\011\011\011\011\011\011\011\011888888((((((8((8((((((((((((((((((((((((((((((((OOOOO8NN8OONO6(8888888\011\011\011\011\011\011\011\011\011\01188888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"88888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((((((((((((((NNOO\003\0038888888NN(O(((((((((((((8((((((((((((((((((((((((((((((((((OONNNNN888OONQ6\003\003\003\003\003\003\003\003\003\003\003\003\003\011\011\011\011\011\011\011\011\011\01188888888888888888888888888888888888888888888888888888888888888888888888888888888888888(888888888888888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\016\016\016\016\016\016\016\016\004\004\004\004\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888888888888\003((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888UUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUUU8\003\003\003\003\00388888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((("
"((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((\003\0038888888888888((((((((((((((((((((((((((((((((((((((((((((((((\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\0216((((((66666666666666688888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((8888888(((((((((((((((((((((((((((((((8\011\011\011\011\011\011\011\011\011\0118888\003\003(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8\011\011\011\011\011\011\011\011\011\011888888((((((((((((((((((((((((((((((8866666\0038888888888((((((((((((((((((((((((((((((((((((((((((((((((6666666\003\003\003\003\003\016\016\016\0165555\003\0168888888888\011\011\011\011\011\011\011\011\011\0118\022\022\022\022\022\022\0228(((((((((((((((((((((88888(((((((((((((((((((88888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\012\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\003\003\003\00388888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8888N(OOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO8888888NNNN5555555555555888888888888888888888888888888888888888888888888888888888888888855\0035688888888888OO88888888888888((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88888888((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((888888888888888888888888888888888888888888(((((((((8888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"8888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888885555855555558558(((((((((((((((((((((((((((((((((((888888888888888(88888888888888888888888888888(((88(88888888888888((((88888888((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88888(((((((((((((888(((((((((8888888((((((((((88\0166N\003\021\021\021\02188888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"66666666666666666666666666666666666666666666668866666666666666666666666888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888888888888888888888888888888888888888888888888888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\01688\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016QQ666\016\016\016QQQQQQ\021\021\021\021\021\021\021\02166666666\016\0166666666\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0166666\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888888888888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016666\01688888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022888888888888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022888888888888"
"\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888888888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\0158\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\0278\027\02788\02788\027\02788\027\027\027\0278\027\027\027\027\027\027\027\027\015\015\015\0158\0158\015\015\015\015\015\015\0158\015\015\015\015\015\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\0278\027\027\027\02788\027\027\027\027\027\027\027\0278\027\027\027\027\027\027\0278\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\0278\027\027\027\0278\027\027\027\027\0278\027888\027\027\027\027\027\027\0278\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\01588\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\007\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\007\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\007\015\015\015\015"
"\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\007\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\007\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\007\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\007\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\007\015\015\015\015\015\015\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\027\007\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\007\015\015\015\015\015\015\027\01588\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\011\0116666666666666666666666666666666666666666666666666666666\016\016\016\01666666666666666666666666666666666666666666666666666\016\016\016\016\016\016\016\0166\016\016\016\016\016\016\016\016\016\016\016\016\016\0166\016\016\003\003\003\003\00388888888888888866666866666666666666688888888888888888888888888888888888888888888888888888888888888888888888888888888\015\015\015\015\015\015\015\015\015\015(\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015888888\015\015\015\015\015\015888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888NNNNNNN8NNNNNNNNNNNNNNNNN88NNNNNNN8NN8NNNNN8888844444444444444444444444444444444444444444444444444444444444444888888888888888888888888888888888N8888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"(((((((((((((((((((((((((((((((((((((((((((((8886666666555555588\011\011\011\011\011\011\011\011\011\0118888(\01688888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888((((((((((((((((((((((((((((((688888888888888888((((((((((((((((((((((((((((((((((((((((((((6666\011\011\011\011\011\011\011\011\011\01188888\0048888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((((((((((((((((((((((56666\011\011\011\011\011\011\011\011\011\01188888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888(((((((8((((8((8(((((((((((((((8"
"(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88\022\022\022\022\022\022\022\022\022666666688888888888888888888888888888888888888888\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\226\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015\015666N66658888\011\011\011\011\011\011\011\011\011\0118888\003\003888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\016\022\022\022\004\022\022\022\0228888888888888888888888888888888888888888888888888888888888888888888888888888\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\022\016\022\022\022\022\022\022\022\022\022\022\022\022\022\022\02288888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"((((8(((((((((((((((((((((((((((8((8(88(8((((((((((8((((8(8(888888(8888(8(8(8(((8((8(88(8(8(8(8(8((8(88((((8(((((((8((((8((((8(8((((((((((8(((((((((((((((((88888(((8(((((8(((((((((((((((((8888888888888888888888888888888888888888888888888888\007\00788888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\01688\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888888888\022\022\022\022\022\022\022\022\022\022\022\022\022\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\016\016\016\016\016\016\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\016\016\016\016\016\016\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\227\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\01688888888888888888888888888888888888888888888888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888\016\016\016\016\016\016\016\016\0168888888\016\01688888888888888\016\016\016\016\016\0168888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\013\013\013\013\013\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888\016\016\016\016\016\016\016\016\016\016\016\016\016888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888888\016\016\016\016\016\016\016\016\016\016\016\0168888\016888888888888888\016\016\016\016\016\016\016\016\016\016\016\0168888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\01688888888\016\016\016\016\016\016\016\016\016\016888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\01688888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\01688\016\016888888888888888888888888888888888888888888888888888888888888888888888888888888"
"\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016888888888888\016\016\016\016\016\016\016\016\016\016\016\016\016\01688\016\016\016\016\016\016\016\016\016\016\016\016\016888\016\016\016\016\016\016\016\016\0168888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168\016\016\016\016\016\016\01688888888\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888\016\016\016\016\016\016\016\016\0168888888\016\016\016\016\016\016\016\016\0168888888\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\016\0168888888888888888888888888888888888888\011\011\011\011\011\011\011\011\011\011888888((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88888888888888888888888888888888((((((((((((((((((((((((((((((((((((((((((((((((((((((((((888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((("
"((((((((((((((((((((((((((((((88((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88888888888888(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((8888888888888888888888888888888((((((((((((((((((((((((((((((8888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888"
"(((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((88888((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((((888888888888888888888888888888888888888888888888888888888888888888888888888888888\021888888888888888888888888888888\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021\021888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888888886666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666666668888888888888888"
"\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\221\22188";
constexpr std::size_t property_index(char32_t code_point)
{
LEXY_PRECONDITION(code_point <= 0x10FFFF);
auto high = (code_point >> 8) & 0xFFFF;
auto low = code_point & 0xFF;
auto block_start = static_cast<unsigned char>(block_starts[high]);
auto block_index = block_start * 256u + low;
return static_cast<unsigned char>(blocks[block_index]);
}
constexpr lexy::code_point::general_category_t category[] = {lexy::code_point::Cc,lexy::code_point::Cc,lexy::code_point::Zs,lexy::code_point::Po,lexy::code_point::Sc,lexy::code_point::Ps,lexy::code_point::Pe,lexy::code_point::Sm,lexy::code_point::Pd,lexy::code_point::Nd,lexy::code_point::Lu,lexy::code_point::Sk,lexy::code_point::Pc,lexy::code_point::Ll,lexy::code_point::So,lexy::code_point::Lo,lexy::code_point::Pi,lexy::code_point::Cf,lexy::code_point::No,lexy::code_point::Ll,lexy::code_point::Po,lexy::code_point::Pf,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Ll,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lo,lexy::code_point::Lu,lexy::code_point::Lt,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lm,lexy::code_point::Lm,lexy::code_point::Mn,lexy::code_point::Mn,lexy::code_point::Cn,lexy::code_point::Lm,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Ll,lexy::code_point::Lu,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Lu,lexy::code_point::Ll,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Me,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Mn,lexy::code_point::Mc,lexy::code_point::Lo,lexy::code_point::Mc,lexy::code_point::Lu,lexy::code_point::No,lexy::code_point::Ll,lexy::code_point::Nl,lexy::code_point::Mn,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Ll,lexy::code_point::Lu,lexy::code_point::Ll,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lt,lexy::code_point::Lu,lexy::code_point::Lt,lexy::code_point::Ll,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Cf,lexy::code_point::Zl,lexy::code_point::Zp,lexy::code_point::Sm,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::So,lexy::code_point::Lu,lexy::code_point::Nl,lexy::code_point::Nl,lexy::code_point::So,lexy::code_point::So,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lm,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Ll,lexy::code_point::Cs,lexy::code_point::Co,lexy::code_point::Lo,lexy::code_point::Lm,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::Lu,lexy::code_point::So,};
enum binary_properties_t
{
whitespace,
join_control,
alphabetic,
uppercase,
lowercase,
xid_start,
xid_continue,
_property_count,
};
static_assert(static_cast<int>(_property_count) <= 8);
constexpr std::uint_least8_t binary_properties[] = {0,1,1,0,0,0,0,0,0,64,108,0,64,116,0,116,0,0,0,116,64,0,108,108,108,116,108,108,108,108,108,108,108,108,108,108,108,108,108,108,100,108,100,108,108,108,108,108,108,108,108,108,116,100,64,84,0,20,108,108,108,108,108,116,108,116,116,116,116,116,116,108,116,108,108,0,108,108,68,68,68,64,108,64,116,100,100,116,116,116,116,116,116,116,116,108,116,108,108,100,108,100,116,108,108,108,108,108,2,1,1,96,108,108,108,96,108,108,116,12,20,108,108,108,108,108,108,108,108,4,108,108,108,108,108,108,108,108,108,108,108,108,108,116,0,0,4,68,108,108,108,12,};
constexpr std::int_least32_t case_folding_offset[] = {0,0,0,0,0,0,0,0,0,0,32,0,0,0,0,0,0,0,0,775,0,0,1,0,-121,-268,210,206,205,79,202,203,207,211,209,213,214,218,217,219,0,2,1,-97,-56,-130,10795,-163,10792,-195,69,71,0,0,0,116,0,0,116,38,37,64,63,1,8,-30,-25,-15,-22,-54,-48,-60,-64,-7,80,0,15,48,0,0,0,0,7264,0,-8,0,0,-6222,-6221,-6212,-6210,-6211,-6204,-6180,35267,-3008,-58,-7615,-8,-8,-74,-9,-7173,-86,-100,-112,-128,-126,0,0,0,0,-7517,-8383,-8262,0,28,16,0,26,0,-10743,-3814,-10727,-10780,-10749,-10783,-10782,-10815,0,-35332,-42280,-42308,-42319,-42315,-42305,-42258,-42282,-42261,928,-48,-42307,-35384,-38864,0,0,0,0,40,39,34,0,};
} // namespace lexy::_unicode_db

View File

@@ -0,0 +1,275 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_ACTION_BASE_HPP_INCLUDED
#define LEXY_ACTION_BASE_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
#include <lexy/_detail/lazy_init.hpp>
#include <lexy/_detail/type_name.hpp>
#include <lexy/callback/noop.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/grammar.hpp>
//=== parse_context ===//
namespace lexy
{
namespace _detail
{
struct parse_context_var_base
{
const void* id;
parse_context_var_base* next;
constexpr parse_context_var_base(const void* id) : id(id), next(nullptr) {}
template <typename Context>
constexpr void link(Context& context)
{
auto cb = context.control_block;
next = cb->vars;
cb->vars = this;
}
template <typename Context>
constexpr void unlink(Context& context)
{
auto cb = context.control_block;
cb->vars = next;
}
};
template <typename Id, typename T>
struct parse_context_var : parse_context_var_base
{
static constexpr auto type_id = lexy::_detail::type_id<Id>();
T value;
explicit constexpr parse_context_var(T&& value)
: parse_context_var_base(static_cast<const void*>(&type_id) /* NOLINT */),
value(LEXY_MOV(value))
{}
template <typename ControlBlock>
static constexpr T& get(const ControlBlock* cb)
{
for (auto cur = cb->vars; cur; cur = cur->next)
if (cur->id == static_cast<const void*>(&type_id) /* NOLINT */)
return static_cast<parse_context_var*>(cur)->value;
LEXY_ASSERT(false, "context variable hasn't been created");
return static_cast<parse_context_var*>(cb->vars)->value;
}
};
template <typename Handler, typename State = void>
struct parse_context_control_block
{
using handler_type = Handler;
using state_type = State;
LEXY_EMPTY_MEMBER Handler parse_handler;
State* parse_state;
parse_context_var_base* vars;
int cur_depth, max_depth;
bool enable_whitespace_skipping;
constexpr parse_context_control_block(Handler&& handler, State* state,
std::size_t max_depth)
: parse_handler(LEXY_MOV(handler)), parse_state(state), //
vars(nullptr), //
cur_depth(0), max_depth(static_cast<int>(max_depth)), enable_whitespace_skipping(true)
{}
template <typename OtherHandler>
constexpr parse_context_control_block(Handler&& handler,
parse_context_control_block<OtherHandler, State>* cb)
: parse_handler(LEXY_MOV(handler)), parse_state(cb->parse_state), //
vars(cb->vars), cur_depth(cb->cur_depth), max_depth(cb->max_depth),
enable_whitespace_skipping(cb->enable_whitespace_skipping)
{}
template <typename OtherHandler>
constexpr void copy_vars_from(parse_context_control_block<OtherHandler, State>* cb)
{
vars = cb->vars;
cur_depth = cb->cur_depth;
max_depth = cb->max_depth;
enable_whitespace_skipping = cb->enable_whitespace_skipping;
}
};
} // namespace _detail
// If a production doesn't define whitespace, we don't need to pass it and can shorten the template
// name.
template <typename Production>
using _whitespace_production_of
= std::conditional_t<_production_defines_whitespace<Production>, Production, void>;
template <typename Handler, typename State, typename Production>
using _production_value_type =
typename Handler::template value_callback<Production, State>::return_type;
template <typename Handler, typename State, typename Production,
typename WhitespaceProduction = _whitespace_production_of<Production>>
struct _pc
{
using handler_type = Handler;
using state_type = State;
using production = Production;
using whitespace_production = WhitespaceProduction;
using value_type = _production_value_type<Handler, State, Production>;
typename Handler::event_handler handler;
_detail::parse_context_control_block<Handler, State>* control_block;
_detail::lazy_init<value_type> value;
constexpr explicit _pc(_detail::parse_context_control_block<Handler, State>* cb)
: handler(Production{}), control_block(cb)
{}
template <typename ChildProduction>
constexpr auto sub_context(ChildProduction)
{
// Update the whitespace production if necessary.
// If the current production is a token or defines whitespace,
// we change it to the current production (or void), otherwise keep it.
using new_whitespace_production
= std::conditional_t<is_token_production<ChildProduction> //
|| _production_defines_whitespace<ChildProduction>,
_whitespace_production_of<ChildProduction>, WhitespaceProduction>;
return _pc<Handler, State, ChildProduction, new_whitespace_production>(control_block);
}
constexpr auto value_callback()
{
using callback = typename Handler::template value_callback<Production, State>;
return callback(control_block->parse_state);
}
template <typename Event, typename... Args>
constexpr auto on(Event ev, Args&&... args)
{
return handler.on(control_block->parse_handler, ev, LEXY_FWD(args)...);
}
};
} // namespace lexy
//=== do_action ===//
namespace lexy::_detail
{
struct final_parser
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader&, Args&&... args)
{
context.value.emplace_result(context.value_callback(), LEXY_FWD(args)...);
return true;
}
};
template <typename NextParser>
struct context_finish_parser
{
template <typename Context, typename Reader, typename SubContext, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, SubContext& sub_context,
Args&&... args)
{
// Might need to skip whitespace, according to the original context.
using continuation
= std::conditional_t<lexy::is_token_production<typename SubContext::production>,
lexy::whitespace_parser<Context, NextParser>, NextParser>;
// Pass the produced value to the next parser.
if constexpr (std::is_void_v<typename SubContext::value_type>)
return continuation::parse(context, reader, LEXY_FWD(args)...);
else
return continuation::parse(context, reader, LEXY_FWD(args)...,
LEXY_MOV(*sub_context.value));
}
};
} // namespace lexy::_detail
namespace lexy
{
constexpr void* no_parse_state = nullptr;
template <typename Handler, typename State, typename Production, typename Reader>
constexpr auto _do_action(_pc<Handler, State, Production>& context, Reader& reader)
{
context.on(parse_events::grammar_start{}, reader.position());
context.on(parse_events::production_start{}, reader.position());
// We parse whitespace, theen the rule, then finish.
using parser = lexy::whitespace_parser<
LEXY_DECAY_DECLTYPE(context),
lexy::parser_for<lexy::production_rule<Production>, _detail::final_parser>>;
auto rule_result = parser::parse(context, reader);
if (rule_result)
{
context.on(parse_events::production_finish{}, reader.position());
context.on(parse_events::grammar_finish{}, reader);
}
else
{
context.on(parse_events::production_cancel{}, reader.position());
context.on(parse_events::grammar_cancel{}, reader);
}
return rule_result;
}
template <typename Production, template <typename> typename Result, typename Handler,
typename State, typename Reader>
constexpr auto do_action(Handler&& handler, State* state, Reader& reader)
{
static_assert(!std::is_reference_v<Handler>, "need to move handler in");
_detail::parse_context_control_block control_block(LEXY_MOV(handler), state,
max_recursion_depth<Production>());
_pc<Handler, State, Production> context(&control_block);
auto rule_result = _do_action(context, reader);
using value_type = typename decltype(context)::value_type;
if constexpr (std::is_void_v<value_type>)
return LEXY_MOV(control_block.parse_handler).template get_result<Result<void>>(rule_result);
else if (context.value)
return LEXY_MOV(control_block.parse_handler)
.template get_result<Result<value_type>>(rule_result, LEXY_MOV(*context.value));
else
return LEXY_MOV(control_block.parse_handler)
.template get_result<Result<value_type>>(rule_result);
}
} // namespace lexy
//=== value callback ===//
namespace lexy::_detail
{
struct void_value_callback
{
constexpr void_value_callback() = default;
template <typename State>
constexpr explicit void_value_callback(State*)
{}
using return_type = void;
constexpr auto sink() const
{
return lexy::noop.sink();
}
template <typename... Args>
constexpr void operator()(Args&&...) const
{}
};
} // namespace lexy::_detail
#endif // LEXY_ACTION_BASE_HPP_INCLUDED

View File

@@ -0,0 +1,90 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_ACTION_MATCH_HPP_INCLUDED
#define LEXY_ACTION_MATCH_HPP_INCLUDED
#include <lexy/action/base.hpp>
namespace lexy
{
class _mh
{
public:
constexpr _mh() : _failed(false) {}
class event_handler
{
public:
constexpr event_handler(production_info) {}
template <typename Error>
constexpr void on(_mh& handler, parse_events::error, Error&&)
{
handler._failed = true;
}
template <typename Event, typename... Args>
constexpr int on(_mh&, Event, const Args&...)
{
return 0; // operation_chain_start needs to return something
}
};
template <typename Production, typename State>
using value_callback = _detail::void_value_callback;
template <typename>
constexpr bool get_result(bool rule_parse_result) &&
{
return rule_parse_result && !_failed;
}
private:
bool _failed;
};
template <typename State, typename Input>
struct match_action
{
State* _state = nullptr;
using handler = _mh;
using state = State;
using input = Input;
template <typename>
using result_type = bool;
constexpr match_action() = default;
template <typename U = State>
constexpr explicit match_action(U& state) : _state(&state)
{}
template <typename Production>
constexpr auto operator()(Production, const Input& input) const
{
auto reader = input.reader();
return lexy::do_action<Production, result_type>(handler(), _state, reader);
}
};
template <typename Production, typename Input>
constexpr bool match(const Input& input)
{
return match_action<void, Input>()(Production{}, input);
}
template <typename Production, typename Input, typename State>
constexpr bool match(const Input& input, State& state)
{
return match_action<State, Input>(state)(Production{}, input);
}
template <typename Production, typename Input, typename State>
constexpr bool match(const Input& input, const State& state)
{
return match_action<const State, Input>(state)(Production{}, input);
}
} // namespace lexy
#endif // LEXY_ACTION_MATCH_HPP_INCLUDED

View File

@@ -0,0 +1,191 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_ACTION_PARSE_HPP_INCLUDED
#define LEXY_ACTION_PARSE_HPP_INCLUDED
#include <lexy/_detail/invoke.hpp>
#include <lexy/action/base.hpp>
#include <lexy/action/validate.hpp>
#include <lexy/callback/base.hpp>
#include <lexy/callback/bind.hpp>
namespace lexy
{
template <typename T, typename ErrorCallback>
class parse_result
{
using _impl_t = lexy::validate_result<ErrorCallback>;
public:
using value_type = T;
using error_callback = ErrorCallback;
using error_type = typename _impl_t::error_type;
//=== status ===//
constexpr explicit operator bool() const noexcept
{
return _impl.is_success();
}
constexpr bool is_success() const noexcept
{
return _impl.is_success();
}
constexpr bool is_error() const noexcept
{
return _impl.is_error();
}
constexpr bool is_recovered_error() const noexcept
{
return _impl.is_recovered_error();
}
constexpr bool is_fatal_error() const noexcept
{
return _impl.is_fatal_error();
}
//=== value ===//
constexpr bool has_value() const noexcept
{
return static_cast<bool>(_value);
}
constexpr decltype(auto) value() const& noexcept
{
return *_value;
}
constexpr decltype(auto) value() && noexcept
{
return LEXY_MOV(*_value);
}
//=== error ===//
constexpr std::size_t error_count() const noexcept
{
return _impl.error_count();
}
constexpr const auto& errors() const& noexcept
{
return _impl.errors();
}
constexpr auto&& errors() && noexcept
{
return LEXY_MOV(_impl).errors();
}
private:
constexpr explicit parse_result(_impl_t&& impl) noexcept : _impl(LEXY_MOV(impl)), _value() {}
template <typename U>
constexpr explicit parse_result(_impl_t&& impl, U&& v) noexcept : _impl(LEXY_MOV(impl))
{
LEXY_PRECONDITION(impl.is_success() || impl.is_recovered_error());
_value.emplace(LEXY_FWD(v));
}
// In principle we could do a space optimization, as we can reconstruct the impl's status from
// the state of _value and error. Feel free to implement it.
_impl_t _impl;
lexy::_detail::lazy_init<T> _value;
template <typename Reader>
friend class _ph;
};
} // namespace lexy
namespace lexy
{
template <typename Reader>
class _ph
{
using iterator = typename Reader::iterator;
public:
template <typename Input, typename Sink>
constexpr explicit _ph(const _detail::any_holder<const Input*>& input,
_detail::any_holder<Sink>& sink)
: _validate(input, sink)
{}
using event_handler = typename _vh<Reader>::event_handler;
constexpr operator _vh<Reader>&()
{
return _validate;
}
template <typename Production, typename State>
using value_callback = production_value_callback<Production, State>;
template <typename Result, typename T>
constexpr auto get_result(bool rule_parse_result, T&& result) &&
{
using validate_result = lexy::validate_result<typename Result::error_callback>;
return Result(LEXY_MOV(_validate).template get_result<validate_result>(rule_parse_result),
LEXY_MOV(result));
}
template <typename Result>
constexpr auto get_result(bool rule_parse_result) &&
{
using validate_result = lexy::validate_result<typename Result::error_callback>;
return Result(LEXY_MOV(_validate).template get_result<validate_result>(rule_parse_result));
}
private:
_vh<Reader> _validate;
};
template <typename State, typename Input, typename ErrorCallback>
struct parse_action
{
const ErrorCallback* _callback;
State* _state = nullptr;
using handler = _ph<lexy::input_reader<Input>>;
using state = State;
using input = Input;
template <typename T>
using result_type = parse_result<T, ErrorCallback>;
constexpr explicit parse_action(const ErrorCallback& callback) : _callback(&callback) {}
template <typename U = State>
constexpr explicit parse_action(U& state, const ErrorCallback& callback)
: _callback(&callback), _state(&state)
{}
template <typename Production>
constexpr auto operator()(Production, const Input& input) const
{
_detail::any_holder input_holder(&input);
_detail::any_holder sink(_get_error_sink(*_callback));
auto reader = input.reader();
return lexy::do_action<Production, result_type>(handler(input_holder, sink), _state,
reader);
}
};
/// Parses the production into a value, invoking the callback on error.
template <typename Production, typename Input, typename ErrorCallback>
constexpr auto parse(const Input& input, const ErrorCallback& callback)
{
return parse_action<void, Input, ErrorCallback>(callback)(Production{}, input);
}
/// Parses the production into a value, invoking the callback on error.
/// All callbacks gain access to the specified parse state.
template <typename Production, typename Input, typename State, typename ErrorCallback>
constexpr auto parse(const Input& input, State& state, const ErrorCallback& callback)
{
return parse_action<State, Input, ErrorCallback>(state, callback)(Production{}, input);
}
template <typename Production, typename Input, typename State, typename ErrorCallback>
constexpr auto parse(const Input& input, const State& state, const ErrorCallback& callback)
{
return parse_action<const State, Input, ErrorCallback>(state, callback)(Production{}, input);
}
} // namespace lexy
#endif // LEXY_ACTION_PARSE_HPP_INCLUDED

View File

@@ -0,0 +1,217 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_ACTION_PARSE_AS_TREE_HPP_INCLUDED
#define LEXY_ACTION_PARSE_AS_TREE_HPP_INCLUDED
#include <lexy/action/base.hpp>
#include <lexy/action/validate.hpp>
#include <lexy/dsl/any.hpp>
#include <lexy/parse_tree.hpp>
namespace lexy
{
template <typename Tree, typename Reader>
class _pth
{
public:
template <typename Input, typename Sink>
explicit _pth(Tree& tree, const _detail::any_holder<const Input*>& input,
_detail::any_holder<Sink>& sink)
: _tree(&tree), _depth(0), _validate(input, sink)
{}
class event_handler
{
using iterator = typename Reader::iterator;
public:
event_handler(production_info info) : _validate(info) {}
void on(_pth& handler, parse_events::grammar_start, iterator)
{
LEXY_PRECONDITION(handler._depth == 0);
handler._builder.emplace(LEXY_MOV(*handler._tree), _validate.get_info());
}
void on(_pth& handler, parse_events::grammar_finish, Reader& reader)
{
LEXY_PRECONDITION(handler._depth == 0);
auto begin = reader.position();
lexy::try_match_token(dsl::any, reader);
auto end = reader.position();
*handler._tree = LEXY_MOV(*handler._builder).finish({begin, end});
}
void on(_pth& handler, parse_events::grammar_cancel, Reader&)
{
LEXY_PRECONDITION(handler._depth == 0);
handler._tree->clear();
}
void on(_pth& handler, parse_events::production_start ev, iterator pos)
{
if (handler._depth++ > 0)
_marker = handler._builder->start_production(_validate.get_info());
_validate.on(handler._validate, ev, pos);
}
void on(_pth& handler, parse_events::production_finish ev, iterator pos)
{
if (--handler._depth > 0)
{
if (handler._builder->current_child_count() == 0)
handler._builder->token(lexy::position_token_kind, _validate.production_begin(),
_validate.production_begin());
handler._builder->finish_production(LEXY_MOV(_marker));
}
_validate.on(handler._validate, ev, pos);
}
void on(_pth& handler, parse_events::production_cancel ev, iterator pos)
{
if (--handler._depth > 0)
{
// Cancelling the production removes all nodes from the tree.
// To ensure that the parse tree remains lossless, we add everything consumed by it
// as an error token.
handler._builder->cancel_production(LEXY_MOV(_marker));
handler._builder->token(lexy::error_token_kind, _validate.production_begin(), pos);
}
_validate.on(handler._validate, ev, pos);
}
auto on(_pth& handler, lexy::parse_events::operation_chain_start, iterator)
{
// As we don't know the production yet (or whether it is actually an operation),
// we create a container node to decide later.
return handler._builder->start_container();
}
template <typename Operation>
void on(_pth& handler, lexy::parse_events::operation_chain_op, Operation op, iterator)
{
// We set the production of the current container.
// This will do a "left rotation" on the parse tree, making a new container the parent.
handler._builder->set_container_production(op);
}
template <typename Marker>
void on(_pth& handler, lexy::parse_events::operation_chain_finish, Marker&& marker,
iterator)
{
handler._builder->finish_container(LEXY_MOV(marker));
}
template <typename TokenKind>
void on(_pth& handler, parse_events::token, TokenKind kind, iterator begin, iterator end)
{
handler._builder->token(kind, begin, end);
}
template <typename Error>
void on(_pth& handler, parse_events::error ev, Error&& error)
{
_validate.on(handler._validate, ev, LEXY_FWD(error));
}
template <typename Event, typename... Args>
auto on(_pth& handler, Event ev, Args&&... args)
{
return _validate.on(handler._validate, ev, LEXY_FWD(args)...);
}
private:
typename Tree::builder::marker _marker;
typename _vh<Reader>::event_handler _validate;
};
template <typename Production, typename State>
using value_callback = _detail::void_value_callback;
template <typename T>
constexpr auto get_result(bool rule_parse_result) &&
{
LEXY_PRECONDITION(_depth == 0);
return LEXY_MOV(_validate).template get_result<T>(rule_parse_result);
}
private:
lexy::_detail::lazy_init<typename Tree::builder> _builder;
Tree* _tree;
int _depth;
_vh<Reader> _validate;
};
template <typename State, typename Input, typename ErrorCallback, typename TokenKind = void,
typename MemoryResource = void>
struct parse_as_tree_action
{
using tree_type = lexy::parse_tree_for<Input, TokenKind, MemoryResource>;
tree_type* _tree;
const ErrorCallback* _callback;
State* _state = nullptr;
using handler = _pth<tree_type, lexy::input_reader<Input>>;
using state = State;
using input = Input;
template <typename>
using result_type = validate_result<ErrorCallback>;
constexpr explicit parse_as_tree_action(tree_type& tree, const ErrorCallback& callback)
: _tree(&tree), _callback(&callback)
{}
template <typename U = State>
constexpr explicit parse_as_tree_action(U& state, tree_type& tree,
const ErrorCallback& callback)
: _tree(&tree), _callback(&callback), _state(&state)
{}
template <typename Production>
constexpr auto operator()(Production, const Input& input) const
{
_detail::any_holder input_holder(&input);
_detail::any_holder sink(_get_error_sink(*_callback));
auto reader = input.reader();
return lexy::do_action<Production, result_type>(handler(*_tree, input_holder, sink), _state,
reader);
}
};
template <typename Production, typename TokenKind, typename MemoryResource, typename Input,
typename ErrorCallback>
auto parse_as_tree(parse_tree<lexy::input_reader<Input>, TokenKind, MemoryResource>& tree,
const Input& input,
const ErrorCallback& callback) -> validate_result<ErrorCallback>
{
return parse_as_tree_action<void, Input, ErrorCallback, TokenKind,
MemoryResource>(tree, callback)(Production{}, input);
}
template <typename Production, typename TokenKind, typename MemoryResource, typename Input,
typename State, typename ErrorCallback>
auto parse_as_tree(parse_tree<lexy::input_reader<Input>, TokenKind, MemoryResource>& tree,
const Input& input, State& state,
const ErrorCallback& callback) -> validate_result<ErrorCallback>
{
return parse_as_tree_action<State, Input, ErrorCallback, TokenKind,
MemoryResource>(state, tree, callback)(Production{}, input);
}
template <typename Production, typename TokenKind, typename MemoryResource, typename Input,
typename State, typename ErrorCallback>
auto parse_as_tree(parse_tree<lexy::input_reader<Input>, TokenKind, MemoryResource>& tree,
const Input& input, const State& state,
const ErrorCallback& callback) -> validate_result<ErrorCallback>
{
return parse_as_tree_action<const State, Input, ErrorCallback, TokenKind,
MemoryResource>(state, tree, callback)(Production{}, input);
}
} // namespace lexy
#endif // LEXY_ACTION_PARSE_AS_TREE_HPP_INCLUDED

View File

@@ -0,0 +1,98 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_ACTION_SCAN_HPP_INCLUDED
#define LEXY_ACTION_SCAN_HPP_INCLUDED
#include <lexy/action/base.hpp>
#include <lexy/action/validate.hpp>
#include <lexy/dsl/scan.hpp>
namespace lexy
{
template <typename ControlProduction>
struct _scp;
template <>
struct _scp<void>
{
static LEXY_CONSTEVAL auto name()
{
return "scanner control production";
}
static constexpr auto rule = dsl::scan;
static constexpr auto value = lexy::noop;
};
template <typename ControlProduction>
struct _scp : ControlProduction, _scp<void>
{};
template <typename ControlProduction, typename Input, typename State, typename ErrorCallback>
class scanner : public _detail::scanner<scanner<ControlProduction, Input, State, ErrorCallback>,
lexy::input_reader<Input>>
{
using _impl = _detail::scanner<scanner<ControlProduction, Input, State, ErrorCallback>,
lexy::input_reader<Input>>;
using _handler = lexy::_vh<lexy::input_reader<Input>>;
using _production = _scp<ControlProduction>;
public:
constexpr explicit scanner(const Input& input, State* state, const ErrorCallback& callback)
: _impl(input.reader()), _input(&input), _sink(_get_error_sink(callback)),
_cb(_handler(_input, _sink), state, max_recursion_depth<_production>()), _context(&_cb)
{
_context.on(parse_events::production_start{}, this->position());
}
constexpr const auto& parse_state() const
{
return *_context.control_block->parse_state;
}
constexpr auto finish() && -> lexy::validate_result<ErrorCallback>
{
auto parse_result = static_cast<bool>(*this);
if (parse_result)
_context.on(parse_events::production_finish{}, this->position());
else
_context.on(parse_events::production_cancel{}, this->position());
return LEXY_MOV(_cb.parse_handler)
.template get_result<validate_result<ErrorCallback>>(parse_result);
}
private:
auto& context() noexcept
{
return _context;
}
_detail::any_holder<const Input*> _input;
_detail::any_holder<_error_sink_t<ErrorCallback>> _sink;
_detail::parse_context_control_block<_handler, State> _cb;
_pc<_handler, State, _production> _context;
friend _impl;
};
template <typename ControlProduction = void, typename Input, typename ErrorCallback>
constexpr auto scan(const Input& input, const ErrorCallback& callback)
{
return scanner<ControlProduction, Input, void, ErrorCallback>(input, no_parse_state, callback);
}
template <typename ControlProduction = void, typename Input, typename State, typename ErrorCallback>
constexpr auto scan(const Input& input, State& state, const ErrorCallback& callback)
{
return scanner<ControlProduction, Input, State, ErrorCallback>(input, &state, callback);
}
template <typename ControlProduction = void, typename Input, typename State, typename ErrorCallback>
constexpr auto scan(const Input& input, const State& state, const ErrorCallback& callback)
{
return scanner<ControlProduction, Input, const State, ErrorCallback>(input, &state, callback);
}
} // namespace lexy
#endif // LEXY_ACTION_SCAN_HPP_INCLUDED

View File

@@ -0,0 +1,506 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_ACTION_TRACE_HPP_INCLUDED
#define LEXY_ACTION_TRACE_HPP_INCLUDED
#include <lexy/_detail/nttp_string.hpp>
#include <lexy/action/base.hpp>
#include <lexy/input_location.hpp>
#include <lexy/token.hpp>
#include <lexy/visualize.hpp>
//=== debug event ===//
namespace lexy::parse_events
{
/// Debug event was triggered.
/// Arguments: pos, str
struct debug
{};
} // namespace lexy::parse_events
namespace lexyd
{
template <typename CharT, CharT... C>
struct _debug : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
constexpr auto str = lexy::_detail::type_string<CharT, C...>::template c_str<>;
context.on(_ev::debug{}, reader.position(), str);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
};
#if LEXY_HAS_NTTP
template <lexy::_detail::string_literal Str>
constexpr auto debug = lexy::_detail::to_type_string<_debug, Str>{};
#endif
#define LEXY_DEBUG(Str) \
LEXY_NTTP_STRING(::lexyd::_debug, Str) {}
} // namespace lexyd
//=== trace ====//
namespace lexy::_detail
{
template <typename OutputIt, typename TokenKind>
class trace_writer
{
public:
explicit trace_writer(OutputIt out, visualization_options opts)
: _out(out), _opts(opts), _cur_depth(0)
{}
template <typename Location>
void write_production_start(const Location& loc, const char* name)
{
if (_cur_depth <= _opts.max_tree_depth)
{
write_prefix(loc, prefix::event);
_out = _detail::write_color<_detail::color::bold>(_out, _opts);
_out = _detail::write_str(_out, name);
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
if (_cur_depth == _opts.max_tree_depth)
{
// Print an ellipsis instead of children.
_out = _detail::write_str(_out, ": ");
_out = _detail::write_ellipsis(_out, _opts);
}
else
{
// Prepare for children.
_out = _detail::write_str(_out, ":");
}
}
++_cur_depth;
}
template <typename Location, typename Reader>
void write_token(const Location& loc, lexy::token_kind<TokenKind> kind,
lexy::lexeme<Reader> lexeme)
{
if (_cur_depth > _opts.max_tree_depth || (kind.ignore_if_empty() && lexeme.empty()))
return;
write_prefix(loc, prefix::event);
_out = _detail::write_color<_detail::color::bold>(_out, _opts);
_out = _detail::write_str(_out, kind.name());
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
if (!lexeme.empty())
{
_out = _detail::write_str(_out, ": ");
_out = visualize_to(_out, lexeme, _opts | visualize_space);
}
}
template <typename Location, typename Reader>
void write_backtrack(const Location& loc, lexy::lexeme<Reader> lexeme)
{
if (_cur_depth > _opts.max_tree_depth || lexeme.empty())
return;
write_prefix(loc, prefix::event);
_out = _detail::write_color<_detail::color::yellow, _detail::color::bold>(_out, _opts);
_out = _detail::write_str(_out, "backtracked");
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
_out = _detail::write_str(_out, ": ");
_out = _detail::write_color<_detail::color::yellow>(_out, _opts);
_out = visualize_to(_out, lexeme, _opts.reset(visualize_use_color) | visualize_space);
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
}
template <typename Location, typename Reader, typename Tag>
void write_error(const Location& loc, const lexy::error<Reader, Tag>& error)
{
if (_cur_depth > _opts.max_tree_depth)
return;
write_prefix(loc, prefix::event);
_out = _detail::write_color<_detail::color::red, _detail::color::bold>(_out, _opts);
_out = _detail::write_str(_out, "error");
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
_out = _detail::write_color<_detail::color::red>(_out, _opts);
_out = _detail::write_str(_out, ": ");
if constexpr (std::is_same_v<Tag, lexy::expected_literal>)
{
auto string = _detail::make_literal_lexeme<typename Reader::encoding>(error.string(),
error.length());
_out = _detail::write_str(_out, "expected '");
_out = visualize_to(_out, string, _opts);
_out = _detail::write_str(_out, "'");
}
else if constexpr (std::is_same_v<Tag, lexy::expected_keyword>)
{
auto string = _detail::make_literal_lexeme<typename Reader::encoding>(error.string(),
error.length());
_out = _detail::write_str(_out, "expected keyword '");
_out = visualize_to(_out, string, _opts);
_out = _detail::write_str(_out, "'");
}
else if constexpr (std::is_same_v<Tag, lexy::expected_char_class>)
{
_out = _detail::write_str(_out, "expected ");
_out = _detail::write_str(_out, error.name());
}
else
{
_out = _detail::write_str(_out, error.message());
}
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
}
template <typename Location>
void write_recovery_start(const Location& loc)
{
if (_cur_depth <= _opts.max_tree_depth)
{
write_prefix(loc, prefix::event);
_out = _detail::write_color<_detail::color::yellow, _detail::color::bold>(_out, _opts);
_out = _detail::write_str(_out, "error recovery");
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
_out = _detail::write_color<_detail::color::yellow>(_out, _opts);
_out = _detail::write_str(_out, ":");
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
if (_cur_depth == _opts.max_tree_depth)
{
// Print an ellipsis instead of children.
_out = _detail::write_str(_out, " ");
_out = _detail::write_ellipsis(_out, _opts);
}
}
++_cur_depth;
}
template <typename Location>
void write_operation(const Location& loc, const char* name)
{
if (_cur_depth > _opts.max_tree_depth)
return;
write_prefix(loc, prefix::event);
_out = _detail::write_color<_detail::color::bold>(_out, _opts);
_out = _detail::write_str(_out, "operation");
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
_out = _detail::write_str(_out, ": ");
_out = _detail::write_str(_out, name);
}
template <typename Location>
void write_debug(const Location& loc, const char* str)
{
if (_cur_depth > _opts.max_tree_depth)
return;
write_prefix(loc, prefix::event);
_out = _detail::write_color<_detail::color::blue, _detail::color::bold>(_out, _opts);
_out = _detail::write_str(_out, "debug");
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
_out = _detail::write_color<_detail::color::blue>(_out, _opts);
_out = _detail::write_str(_out, ": ");
_out = _detail::write_str(_out, str);
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
}
template <typename Location>
void write_finish(const Location& loc)
{
if (_cur_depth <= _opts.max_tree_depth)
write_prefix(loc, prefix::finish);
--_cur_depth;
}
template <typename Location>
void write_cancel(const Location& loc)
{
if (_cur_depth <= _opts.max_tree_depth)
write_prefix(loc, prefix::cancel);
--_cur_depth;
}
OutputIt finish() &&
{
*_out++ = '\n';
return _out;
}
private:
enum class prefix
{
event,
cancel,
finish,
};
template <typename Location>
void write_prefix(const Location& loc, prefix p)
{
const auto use_unicode = _opts.is_set(visualize_use_unicode);
if (_cur_depth > 0)
*_out++ = '\n';
_out = _detail::write_color<_detail::color::faint>(_out, _opts);
_out = _detail::write_format(_out, "%2u:%3u", loc.line_nr(), loc.column_nr());
_out = _detail::write_str(_out, ": ");
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
if (_cur_depth > 0)
{
for (auto i = 0u; i != _cur_depth - 1; ++i)
_out = _detail::write_str(_out, use_unicode ? u8"" : u8" ");
switch (p)
{
case prefix::event:
_out = _detail::write_str(_out, use_unicode ? u8"├──" : u8"- ");
break;
case prefix::cancel:
_out = _detail::write_str(_out, use_unicode ? u8"" : u8"-");
_out = _detail::write_color<_detail::color::yellow>(_out, _opts);
_out = _detail::write_str(_out, use_unicode ? u8"" : u8"x");
_out = _detail::write_color<_detail::color::reset>(_out, _opts);
break;
case prefix::finish:
_out = _detail::write_str(_out, use_unicode ? u8"" : u8"- finish");
break;
}
}
}
OutputIt _out;
visualization_options _opts;
std::size_t _cur_depth;
};
} // namespace lexy::_detail
namespace lexy
{
template <typename OutputIt, typename Input, typename TokenKind = void>
class _th
{
public:
explicit _th(OutputIt out, const Input& input, visualization_options opts = {}) noexcept
: _writer(out, opts), _input(&input), _anchor(input)
{
LEXY_PRECONDITION(opts.max_tree_depth <= visualization_options::max_tree_depth_limit);
}
class event_handler
{
using iterator = typename lexy::input_reader<Input>::iterator;
public:
constexpr event_handler(production_info info) : _info(info) {}
void on(_th&, parse_events::grammar_start, iterator) {}
void on(_th&, parse_events::grammar_finish, lexy::input_reader<Input>&) {}
void on(_th&, parse_events::grammar_cancel, lexy::input_reader<Input>&) {}
void on(_th& handler, parse_events::production_start, iterator pos)
{
auto loc = handler.get_location(pos);
handler._writer.write_production_start(loc, _info.name);
// All events for the production are after the initial event.
_previous_anchor.emplace(handler._anchor);
handler._anchor = loc.anchor();
}
void on(_th& handler, parse_events::production_finish, iterator pos)
{
auto loc = handler.get_location(pos);
handler._writer.write_finish(loc);
}
void on(_th& handler, parse_events::production_cancel, iterator pos)
{
auto loc = handler.get_location(pos);
handler._writer.write_cancel(loc);
// We've backtracked, so we need to restore the anchor.
handler._anchor = *_previous_anchor;
}
int on(_th& handler, parse_events::operation_chain_start, iterator pos)
{
auto loc = handler.get_location(pos);
handler._writer.write_production_start(loc, "operation chain");
return 0; // need to return something
}
template <typename Operation>
void on(_th& handler, parse_events::operation_chain_op, Operation, iterator pos)
{
auto loc = handler.get_location(pos);
handler._writer.write_operation(loc, lexy::production_name<Operation>());
}
void on(_th& handler, parse_events::operation_chain_finish, int, iterator pos)
{
auto loc = handler.get_location(pos);
handler._writer.write_finish(loc);
}
template <typename TK>
void on(_th& handler, parse_events::token, TK kind, iterator begin, iterator end)
{
auto loc = handler.get_location(begin);
handler._writer.write_token(loc, token_kind<TokenKind>(kind),
lexeme_for<Input>(begin, end));
}
void on(_th& handler, parse_events::backtracked, iterator begin, iterator end)
{
auto loc = handler.get_location(begin);
handler._writer.write_backtrack(loc, lexeme_for<Input>(begin, end));
}
template <typename Error>
void on(_th& handler, parse_events::error, const Error& error)
{
auto loc = handler.get_location(error.position());
handler._writer.write_error(loc, error);
}
void on(_th& handler, parse_events::recovery_start, iterator pos)
{
auto loc = handler.get_location(pos);
handler._writer.write_recovery_start(loc);
}
void on(_th& handler, parse_events::recovery_finish, iterator pos)
{
auto loc = handler.get_location(pos);
handler._writer.write_finish(loc);
}
void on(_th& handler, parse_events::recovery_cancel, iterator pos)
{
auto loc = handler.get_location(pos);
handler._writer.write_cancel(loc);
}
void on(_th& handler, parse_events::debug, iterator pos, const char* str)
{
auto loc = handler.get_location(pos);
handler._writer.write_debug(loc, str);
}
private:
production_info _info;
// The beginning of the previous production.
// If the current production gets canceled, it needs to be restored.
_detail::lazy_init<input_location_anchor<Input>> _previous_anchor;
};
template <typename Production, typename State>
using value_callback = _detail::void_value_callback;
template <typename>
constexpr OutputIt get_result(bool) &&
{
return LEXY_MOV(_writer).finish();
}
private:
input_location<Input> get_location(typename lexy::input_reader<Input>::iterator pos)
{
return get_input_location(*_input, pos, _anchor);
}
_detail::trace_writer<OutputIt, TokenKind> _writer;
const Input* _input;
input_location_anchor<Input> _anchor;
};
template <typename State, typename Input, typename OutputIt, typename TokenKind = void>
struct trace_action
{
OutputIt _out;
visualization_options _opts;
State* _state = nullptr;
using handler = _th<OutputIt, Input>;
using state = State;
using input = Input;
template <typename>
using result_type = OutputIt;
constexpr explicit trace_action(OutputIt out, visualization_options opts = {})
: _out(out), _opts(opts)
{}
template <typename U = State>
constexpr explicit trace_action(U& state, OutputIt out, visualization_options opts = {})
: _out(out), _opts(opts), _state(&state)
{}
template <typename Production>
constexpr auto operator()(Production, const Input& input) const
{
auto reader = input.reader();
return lexy::do_action<Production, result_type>(handler(_out, input, _opts), _state,
reader);
}
};
template <typename Production, typename TokenKind = void, typename OutputIt, typename Input>
OutputIt trace_to(OutputIt out, const Input& input, visualization_options opts = {})
{
return trace_action<void, Input, OutputIt, TokenKind>(out, opts)(Production{}, input);
}
template <typename Production, typename TokenKind = void, typename OutputIt, typename Input,
typename State>
OutputIt trace_to(OutputIt out, const Input& input, State& state, visualization_options opts = {})
{
return trace_action<State, Input, OutputIt, TokenKind>(state, out, opts)(Production{}, input);
}
template <typename Production, typename TokenKind = void, typename OutputIt, typename Input,
typename State>
OutputIt trace_to(OutputIt out, const Input& input, const State& state,
visualization_options opts = {})
{
return trace_action<const State, Input, OutputIt, TokenKind>(state, out, opts)(Production{},
input);
}
template <typename Production, typename TokenKind = void, typename Input>
void trace(std::FILE* file, const Input& input, visualization_options opts = {})
{
trace_to<Production, TokenKind>(cfile_output_iterator{file}, input, opts);
}
template <typename Production, typename TokenKind = void, typename Input, typename State>
void trace(std::FILE* file, const Input& input, State& state, visualization_options opts = {})
{
trace_to<Production, TokenKind>(cfile_output_iterator{file}, input, state, opts);
}
template <typename Production, typename TokenKind = void, typename Input, typename State>
void trace(std::FILE* file, const Input& input, const State& state, visualization_options opts = {})
{
trace_to<Production, TokenKind>(cfile_output_iterator{file}, input, state, opts);
}
} // namespace lexy
#endif // LEXY_ACTION_TRACE_HPP_INCLUDED

View File

@@ -0,0 +1,320 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_ACTION_VALIDATE_HPP_INCLUDED
#define LEXY_ACTION_VALIDATE_HPP_INCLUDED
#include <lexy/_detail/any_ref.hpp>
#include <lexy/_detail/lazy_init.hpp>
#include <lexy/action/base.hpp>
#include <lexy/callback/base.hpp>
#include <lexy/callback/container.hpp>
#include <lexy/callback/noop.hpp>
#include <lexy/error.hpp>
namespace lexy
{
// Convert the callback into an appropriate sink.
template <typename Callback>
constexpr auto _get_error_sink(const Callback& callback)
{
if constexpr (std::is_same_v<Callback, lexy::_noop>)
{
// We collect noop instead, which counts the errors.
return lexy::collect(callback).sink();
}
else if constexpr (lexy::is_sink<Callback>)
{
// It already is a sink.
return callback.sink();
}
else
{
static_assert(
std::is_void_v<typename Callback::return_type>,
"need to use `lexy::collect()` to create an error callback that can handle multiple errors");
// We need to collect the errors.
return lexy::collect(callback).sink();
}
}
template <typename Callback>
using _error_sink_t = decltype(_get_error_sink(LEXY_DECLVAL(Callback)));
template <typename ErrorCallback>
class validate_result
{
using _sink_t = _error_sink_t<ErrorCallback>;
public:
using error_callback = ErrorCallback;
using error_type = typename _sink_t::return_type;
static_assert(!std::is_void_v<error_type>, "ErrorCallback must not be a void returning sink");
constexpr explicit operator bool() const noexcept
{
return is_success();
}
constexpr bool is_success() const noexcept
{
return _status == _status_success;
}
constexpr bool is_error() const noexcept
{
return !is_success();
}
constexpr bool is_recovered_error() const noexcept
{
return _status == _status_recovered;
}
constexpr bool is_fatal_error() const noexcept
{
return _status == _status_fatal;
}
constexpr std::size_t error_count() const noexcept
{
if constexpr (std::is_same_v<error_type, std::size_t>)
// void-returning callback yields the count only.
return _error;
else
// We assume it's some sort of container otherwise.
return _error.size();
}
constexpr const auto& errors() const& noexcept
{
return _error;
}
constexpr auto&& errors() && noexcept
{
return LEXY_MOV(_error);
}
private:
constexpr explicit validate_result(bool did_recover, error_type&& error)
: _error(LEXY_MOV(error)), _status()
{
if (error_count() == 0u)
_status = _status_success;
else if (did_recover)
_status = _status_recovered;
else
_status = _status_fatal;
}
error_type _error;
enum
{
_status_success,
_status_recovered,
_status_fatal,
} _status;
template <typename Reader>
friend class _vh;
};
} // namespace lexy
namespace lexy
{
template <typename Reader>
struct _validate_callbacks
{
_detail::any_ref sink;
_detail::any_cref input;
void (*generic)(_detail::any_ref sink, production_info info, _detail::any_cref input,
typename Reader::iterator begin, const error<Reader, void>& error);
void (*literal)(_detail::any_ref sink, production_info info, _detail::any_cref input,
typename Reader::iterator begin, const error<Reader, expected_literal>& error);
void (*keyword)(_detail::any_ref sink, production_info info, _detail::any_cref input,
typename Reader::iterator begin, const error<Reader, expected_keyword>& error);
void (*char_class)(_detail::any_ref sink, production_info info, _detail::any_cref input,
typename Reader::iterator begin,
const error<Reader, expected_char_class>& error);
template <typename Input, typename Sink>
constexpr _validate_callbacks(const _detail::any_holder<const Input*>& input,
_detail::any_holder<Sink>& sink)
: sink(&sink), input(&input),
generic([](_detail::any_ref sink, production_info info, _detail::any_cref input,
typename Reader::iterator begin, const error<Reader, void>& error) {
lexy::error_context err_ctx(info, *input->template get<const Input*>(), begin);
sink->template get<Sink>()(err_ctx, LEXY_FWD(error));
}),
literal([](_detail::any_ref sink, production_info info, _detail::any_cref input,
typename Reader::iterator begin, const error<Reader, expected_literal>& error) {
lexy::error_context err_ctx(info, *input->template get<const Input*>(), begin);
sink->template get<Sink>()(err_ctx, LEXY_FWD(error));
}),
keyword([](_detail::any_ref sink, production_info info, _detail::any_cref input,
typename Reader::iterator begin, const error<Reader, expected_keyword>& error) {
lexy::error_context err_ctx(info, *input->template get<const Input*>(), begin);
sink->template get<Sink>()(err_ctx, LEXY_FWD(error));
}),
char_class([](_detail::any_ref sink, production_info info, _detail::any_cref input,
typename Reader::iterator begin,
const error<Reader, expected_char_class>& error) {
lexy::error_context err_ctx(info, *input->template get<const Input*>(), begin);
sink->template get<Sink>()(err_ctx, LEXY_FWD(error));
})
{}
};
template <typename Reader>
class _vh
{
public:
template <typename Input, typename Sink>
constexpr explicit _vh(const _detail::any_holder<const Input*>& input,
_detail::any_holder<Sink>& sink)
: _cb(input, sink)
{}
class event_handler
{
using iterator = typename Reader::iterator;
public:
constexpr event_handler(production_info info) : _begin(), _info(info) {}
constexpr void on(_vh& handler, parse_events::production_start, iterator pos)
{
_begin = pos;
_prev = handler._top;
handler._top = this;
}
constexpr void on(_vh& handler, parse_events::production_finish, iterator)
{
handler._top = _prev;
}
constexpr void on(_vh& handler, parse_events::production_cancel, iterator)
{
handler._top = _prev;
}
template <typename R, typename Tag>
constexpr void on(_vh& handler, parse_events::error, const error<R, Tag>& error)
{
handler._cb.generic(handler._cb.sink, get_info(), handler._cb.input, _begin, error);
}
template <typename R>
constexpr void on(_vh& handler, parse_events::error, const error<R, void>& error)
{
handler._cb.generic(handler._cb.sink, get_info(), handler._cb.input, _begin, error);
}
template <typename R>
constexpr void on(_vh& handler, parse_events::error,
const error<R, expected_literal>& error)
{
handler._cb.literal(handler._cb.sink, get_info(), handler._cb.input, _begin, error);
}
template <typename R>
constexpr void on(_vh& handler, parse_events::error,
const error<R, expected_keyword>& error)
{
handler._cb.keyword(handler._cb.sink, get_info(), handler._cb.input, _begin, error);
}
template <typename R>
constexpr void on(_vh& handler, parse_events::error,
const error<R, expected_char_class>& error)
{
handler._cb.char_class(handler._cb.sink, get_info(), handler._cb.input, _begin, error);
}
template <typename Event, typename... Args>
constexpr auto on(_vh&, Event, const Args&...)
{
return 0; // operation_chain_start must return something
}
constexpr iterator production_begin() const
{
return _begin;
}
constexpr production_info get_info() const
{
auto cur = this;
while (cur->_info.is_transparent && cur->_prev != nullptr)
cur = cur->_prev;
return cur->_info;
}
private:
iterator _begin;
production_info _info;
event_handler* _prev = nullptr;
};
template <typename Production, typename State>
using value_callback = _detail::void_value_callback;
template <typename Result>
constexpr auto get_result(bool rule_parse_result) &&
{
using sink_t = _error_sink_t<typename Result::error_callback>;
return Result(rule_parse_result, LEXY_MOV(_cb.sink->template get<sink_t>()).finish());
}
private:
_validate_callbacks<Reader> _cb;
event_handler* _top = nullptr;
};
template <typename State, typename Input, typename ErrorCallback>
struct validate_action
{
const ErrorCallback* _callback;
State* _state = nullptr;
using handler = _vh<lexy::input_reader<Input>>;
using state = State;
using input = Input;
template <typename>
using result_type = validate_result<ErrorCallback>;
constexpr explicit validate_action(const ErrorCallback& callback) : _callback(&callback) {}
template <typename U = State>
constexpr explicit validate_action(U& state, const ErrorCallback& callback)
: _callback(&callback), _state(&state)
{}
template <typename Production>
constexpr auto operator()(Production, const Input& input) const
{
_detail::any_holder input_holder(&input);
_detail::any_holder sink(_get_error_sink(*_callback));
auto reader = input.reader();
return lexy::do_action<Production, result_type>(handler(input_holder, sink), _state,
reader);
}
};
template <typename Production, typename Input, typename ErrorCallback>
constexpr auto validate(const Input& input,
const ErrorCallback& callback) -> validate_result<ErrorCallback>
{
return validate_action<void, Input, ErrorCallback>(callback)(Production{}, input);
}
template <typename Production, typename Input, typename State, typename ErrorCallback>
constexpr auto validate(const Input& input, State& state,
const ErrorCallback& callback) -> validate_result<ErrorCallback>
{
return validate_action<State, Input, ErrorCallback>(state, callback)(Production{}, input);
}
template <typename Production, typename Input, typename State, typename ErrorCallback>
constexpr auto validate(const Input& input, const State& state,
const ErrorCallback& callback) -> validate_result<ErrorCallback>
{
return validate_action<const State, Input, ErrorCallback>(state, callback)(Production{}, input);
}
} // namespace lexy
#endif // LEXY_ACTION_VALIDATE_HPP_INCLUDED

View File

@@ -0,0 +1,23 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_HPP_INCLUDED
#define LEXY_CALLBACK_HPP_INCLUDED
#include <lexy/callback/adapter.hpp>
#include <lexy/callback/aggregate.hpp>
#include <lexy/callback/base.hpp>
#include <lexy/callback/bind.hpp>
#include <lexy/callback/bit_cast.hpp>
#include <lexy/callback/composition.hpp>
#include <lexy/callback/constant.hpp>
#include <lexy/callback/container.hpp>
#include <lexy/callback/fold.hpp>
#include <lexy/callback/forward.hpp>
#include <lexy/callback/integer.hpp>
#include <lexy/callback/noop.hpp>
#include <lexy/callback/object.hpp>
#include <lexy/callback/string.hpp>
#endif // LEXY_CALLBACK_HPP_INCLUDED

View File

@@ -0,0 +1,168 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_ADAPTER_HPP_INCLUDED
#define LEXY_CALLBACK_ADAPTER_HPP_INCLUDED
#include <lexy/callback/base.hpp>
namespace lexy
{
template <typename ReturnType, typename... Fns>
struct _callback : _overloaded<Fns...>
{
using return_type = ReturnType;
constexpr explicit _callback(Fns... fns) : _overloaded<Fns...>(LEXY_MOV(fns)...) {}
};
template <typename ReturnType, typename... Fns>
struct _callback_with_state : _overloaded<Fns...>
{
using return_type = ReturnType;
template <typename State>
struct _with_state
{
const _callback_with_state& _cb;
State& _state;
template <typename... Args>
constexpr return_type operator()(Args&&... args) const&&
{
return _cb(_state, LEXY_FWD(args)...);
}
};
constexpr explicit _callback_with_state(Fns... fns) : _overloaded<Fns...>(LEXY_MOV(fns)...) {}
template <typename State>
constexpr auto operator[](State& state) const
{
return _with_state<State>{*this, state};
}
};
/// Creates a callback.
template <typename... Fns>
constexpr auto callback(Fns&&... fns)
{
if constexpr ((lexy::is_callback<std::decay_t<Fns>> && ...))
return _callback<std::common_type_t<typename std::decay_t<Fns>::return_type...>,
std::decay_t<Fns>...>(LEXY_FWD(fns)...);
else
return _callback<void, std::decay_t<Fns>...>(LEXY_FWD(fns)...);
}
template <typename ReturnType, typename... Fns>
constexpr auto callback(Fns&&... fns)
{
return _callback<ReturnType, std::decay_t<Fns>...>(LEXY_FWD(fns)...);
}
/// Creates a callback that also receives the parse state.
template <typename... Fns>
constexpr auto callback_with_state(Fns&&... fns)
{
if constexpr ((lexy::is_callback<std::decay_t<Fns>> && ...))
return _callback_with_state<std::common_type_t<typename std::decay_t<Fns>::return_type...>,
std::decay_t<Fns>...>(LEXY_FWD(fns)...);
else
return _callback_with_state<void, std::decay_t<Fns>...>(LEXY_FWD(fns)...);
}
template <typename ReturnType, typename... Fns>
constexpr auto callback_with_state(Fns&&... fns)
{
return _callback_with_state<ReturnType, std::decay_t<Fns>...>(LEXY_FWD(fns)...);
}
template <typename Sink>
struct _cb_from_sink
{
Sink _sink;
using _cb = lexy::sink_callback<Sink>;
using return_type = typename _cb::return_type;
template <typename... Args>
constexpr auto operator()(Args&&... args) const -> decltype((LEXY_DECLVAL(_cb&)(LEXY_FWD(args)),
..., LEXY_DECLVAL(_cb&&).finish()))
{
auto cb = _sink.sink();
(cb(LEXY_FWD(args)), ...);
return LEXY_MOV(cb).finish();
}
};
/// Creates a callback that forwards all arguments to the sink.
template <typename Sink, typename = lexy::sink_callback<Sink>>
constexpr auto callback(Sink&& sink)
{
return _cb_from_sink<std::decay_t<Sink>>{LEXY_FWD(sink)};
}
} // namespace lexy
namespace lexy
{
template <typename MemFn>
struct _mem_fn_traits // MemFn is member data
{
using return_type = MemFn;
};
#define LEXY_MAKE_MEM_FN_TRAITS(...) \
template <typename ReturnType, typename... Args> \
struct _mem_fn_traits<ReturnType(Args...) __VA_ARGS__> \
{ \
using return_type = ReturnType; \
}; \
template <typename ReturnType, typename... Args> \
struct _mem_fn_traits<ReturnType(Args..., ...) __VA_ARGS__> \
{ \
using return_type = ReturnType; \
};
#define LEXY_MAKE_MEM_FN_TRAITS_CV(...) \
LEXY_MAKE_MEM_FN_TRAITS(__VA_ARGS__) \
LEXY_MAKE_MEM_FN_TRAITS(const __VA_ARGS__) \
LEXY_MAKE_MEM_FN_TRAITS(volatile __VA_ARGS__) \
LEXY_MAKE_MEM_FN_TRAITS(const volatile __VA_ARGS__)
#define LEXY_MAKE_MEM_FN_TRAITS_CV_REF(...) \
LEXY_MAKE_MEM_FN_TRAITS_CV(__VA_ARGS__) \
LEXY_MAKE_MEM_FN_TRAITS_CV(&__VA_ARGS__) \
LEXY_MAKE_MEM_FN_TRAITS_CV(&&__VA_ARGS__)
LEXY_MAKE_MEM_FN_TRAITS_CV_REF()
LEXY_MAKE_MEM_FN_TRAITS_CV_REF(noexcept)
#undef LEXY_MAKE_MEM_FN_TRAITS_CV_REF
#undef LEXY_MAKE_MEM_FN_TRAITS_CV
#undef LEXY_MAKE_MEM_FN_TRAITS
template <typename Fn>
struct _mem_fn;
template <typename MemFn, typename T>
struct _mem_fn<MemFn T::*>
{
MemFn T::*_fn;
using return_type = typename _mem_fn_traits<MemFn>::return_type;
template <typename... Args>
constexpr auto operator()(Args&&... args) const
-> decltype(_detail::_mem_invoker<MemFn T::*>::invoke(_fn, LEXY_FWD(args)...))
{
return _detail::_mem_invoker<MemFn T::*>::invoke(_fn, LEXY_FWD(args)...);
}
};
/// Creates a callback from a member function.
template <typename MemFn, typename T>
constexpr auto mem_fn(MemFn T::*fn)
{
return _mem_fn<MemFn T::*>{fn};
}
} // namespace lexy
#endif // LEXY_CALLBACK_ADAPTER_HPP_INCLUDED

View File

@@ -0,0 +1,72 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_AGGREGATE_HPP_INCLUDED
#define LEXY_CALLBACK_AGGREGATE_HPP_INCLUDED
#include <lexy/callback/base.hpp>
#include <lexy/dsl/member.hpp>
namespace lexy
{
struct nullopt;
template <typename T>
struct _as_aggregate
{
using return_type = T;
static_assert(std::is_aggregate_v<return_type>);
constexpr T operator()(lexy::nullopt&&) const
{
return {};
}
constexpr T operator()(T&& result) const
{
return LEXY_MOV(result);
}
template <typename Fn, typename Value, typename... Tail>
constexpr T operator()(lexy::member<Fn>, Value&& value, Tail&&... tail) const
{
T result{};
Fn{}(result, LEXY_FWD(value));
return (*this)(LEXY_MOV(result), LEXY_FWD(tail)...);
}
template <typename Fn, typename Value, typename... Tail>
constexpr T operator()(T&& result, lexy::member<Fn>, Value&& value, Tail&&... tail) const
{
Fn{}(result, LEXY_FWD(value));
return (*this)(LEXY_MOV(result), LEXY_FWD(tail)...);
}
struct _sink
{
T _result{};
using return_type = T;
template <typename Fn, typename Value>
constexpr void operator()(lexy::member<Fn>, Value&& value)
{
Fn()(_result, LEXY_FWD(value));
}
constexpr auto&& finish() &&
{
return LEXY_MOV(_result);
}
};
constexpr auto sink() const
{
return _sink{};
}
};
/// A callback with sink that creates an aggregate.
template <typename T>
constexpr auto as_aggregate = _as_aggregate<T>{};
} // namespace lexy
#endif // LEXY_CALLBACK_AGGREGATE_HPP_INCLUDED

View File

@@ -0,0 +1,91 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_BASE_HPP_INCLUDED
#define LEXY_CALLBACK_BASE_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
#include <lexy/_detail/detect.hpp>
#include <lexy/_detail/invoke.hpp>
namespace lexy
{
template <typename T>
using _detect_callback = typename T::return_type;
template <typename T>
constexpr bool is_callback = _detail::is_detected<_detect_callback, T>;
template <typename T, typename... Args>
using _detect_callback_for = decltype(LEXY_DECLVAL(const T)(LEXY_DECLVAL(Args)...));
template <typename T, typename... Args>
constexpr bool is_callback_for
= _detail::is_detected<_detect_callback_for, std::decay_t<T>, Args...>;
template <typename T, typename State>
using _detect_callback_state = decltype(LEXY_DECLVAL(const T)[LEXY_DECLVAL(State&)]);
template <typename T, typename State>
constexpr bool is_callback_state
= _detail::is_detected<_detect_callback_state, T, std::decay_t<State>>;
template <typename T, typename State, typename... Args>
using _detect_callback_with_state_for
= decltype(LEXY_DECLVAL(const T)[LEXY_DECLVAL(State&)](LEXY_DECLVAL(Args)...));
template <typename T, typename State, typename... Args>
constexpr bool is_callback_with_state_for
= _detail::is_detected<_detect_callback_with_state_for, std::decay_t<T>, State, Args...>;
/// Returns the type of the `.sink()` function.
template <typename Sink, typename... Args>
using sink_callback = decltype(LEXY_DECLVAL(Sink).sink(LEXY_DECLVAL(Args)...));
template <typename T, typename... Args>
using _detect_sink_callback_for = decltype(LEXY_DECLVAL(T&)(LEXY_DECLVAL(Args)...));
template <typename T, typename... Args>
constexpr bool is_sink_callback_for
= _detail::is_detected<_detect_sink_callback_for, std::decay_t<T>, Args...>;
template <typename T, typename... Args>
using _detect_sink = decltype(LEXY_DECLVAL(const T).sink(LEXY_DECLVAL(Args)...).finish());
template <typename T, typename... Args>
constexpr bool is_sink = _detail::is_detected<_detect_sink, T, Args...>;
} // namespace lexy
namespace lexy
{
template <typename Fn>
struct _fn_holder
{
Fn fn;
constexpr explicit _fn_holder(Fn fn) : fn(fn) {}
template <typename... Args>
constexpr auto operator()(Args&&... args) const -> decltype(_detail::invoke(fn,
LEXY_FWD(args)...))
{
return _detail::invoke(fn, LEXY_FWD(args)...);
}
};
template <typename Fn>
using _fn_as_base = std::conditional_t<std::is_class_v<Fn>, Fn, _fn_holder<Fn>>;
template <typename... Fns>
struct _overloaded : _fn_as_base<Fns>...
{
constexpr explicit _overloaded(Fns... fns) : _fn_as_base<Fns>(LEXY_MOV(fns))... {}
using _fn_as_base<Fns>::operator()...;
};
template <typename... Op>
constexpr auto _make_overloaded(Op&&... op)
{
if constexpr (sizeof...(Op) == 1)
return (LEXY_FWD(op), ...);
else
return _overloaded(LEXY_FWD(op)...);
}
} // namespace lexy
#endif // LEXY_CALLBACK_BASE_HPP_INCLUDED

View File

@@ -0,0 +1,385 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_BIND_HPP_INCLUDED
#define LEXY_CALLBACK_BIND_HPP_INCLUDED
#include <lexy/_detail/tuple.hpp>
#include <lexy/callback/base.hpp>
//=== placeholder details ===//
namespace lexy::_detail
{
struct placeholder_base
{};
template <typename T>
constexpr auto is_placeholder = std::is_base_of_v<placeholder_base, T>;
template <typename BoundArg, typename State, typename... Args>
constexpr decltype(auto) expand_bound_arg(const BoundArg& bound, State& state,
const _detail::tuple<Args...>& actual_args)
{
if constexpr (is_placeholder<std::decay_t<BoundArg>>)
return bound(state, actual_args);
else
return bound;
}
struct all_values_placeholder
{};
struct no_bind_state
{};
template <std::size_t Idx, typename Fn, typename... BoundArgs, typename State,
typename... ActualArgs, std::size_t... ActualIdx, typename... ProducedArgs>
constexpr decltype(auto) _invoke_bound(Fn&& fn, const _detail::tuple<BoundArgs...>& bound_args,
State& state,
const _detail::tuple<ActualArgs...>& actual_args,
_detail::index_sequence<ActualIdx...>,
ProducedArgs&&... produced_args)
{
if constexpr (Idx == sizeof...(BoundArgs))
{
(void)bound_args;
(void)state;
(void)actual_args;
return LEXY_FWD(fn)(LEXY_FWD(produced_args)...);
}
else
{
using bound_arg_t
= std::decay_t<typename _detail::tuple<BoundArgs...>::template element_type<Idx>>;
if constexpr (std::is_same_v<bound_arg_t, all_values_placeholder>)
{
return _invoke_bound<Idx + 1>(LEXY_FWD(fn), bound_args, state, actual_args,
actual_args.index_sequence(), LEXY_FWD(produced_args)...,
// Expand to all actual arguments.
LEXY_FWD(actual_args.template get<ActualIdx>())...);
}
else
{
return _invoke_bound<Idx + 1>(LEXY_FWD(fn), bound_args, state, actual_args,
actual_args.index_sequence(), LEXY_FWD(produced_args)...,
// Expand the currently bound argument
expand_bound_arg(bound_args.template get<Idx>(), state,
actual_args));
}
}
}
template <typename Fn, typename... BoundArgs, std::size_t... Idx, typename State,
typename... Args>
constexpr decltype(auto) invoke_bound(Fn&& fn, //
const _detail::tuple<BoundArgs...>& bound_args,
_detail::index_sequence<Idx...>, //
State& state, Args&&... args)
{
auto actual_args = _detail::forward_as_tuple(LEXY_FWD(args)...);
if constexpr ((_detail::is_decayed_same<BoundArgs, _detail::all_values_placeholder> || ...))
{
// If we're having the placeholder we need to recursively expand every argument.
return _invoke_bound<0>(LEXY_FWD(fn), bound_args, state, actual_args,
actual_args.index_sequence());
}
else
{
// If we're not having the all_values_placeholder, every placeholder expands to a single
// argument. We can thus expand it easily by mapping each value of bound args to the actual
// argument.
return LEXY_FWD(fn)(
expand_bound_arg(bound_args.template get<Idx>(), state, actual_args)...);
}
}
} // namespace lexy::_detail
//=== placeholders ===//
namespace lexy
{
/// Placeholder for bind that expands to all values produced by the rule.
constexpr auto values = _detail::all_values_placeholder{};
struct nullopt;
struct _default
{
template <typename T, typename = std::enable_if_t<std::is_default_constructible_v<T>>>
constexpr operator T() const noexcept
{
return T();
}
};
template <std::size_t N, typename T, typename Fn>
struct _nth_value : _detail::placeholder_base // fallback + map
{
LEXY_EMPTY_MEMBER T _fallback;
LEXY_EMPTY_MEMBER Fn _fn;
template <typename State, typename... Args>
LEXY_FORCE_INLINE constexpr decltype(auto) operator()(State&,
const _detail::tuple<Args...>& args) const
{
if constexpr (N > sizeof...(Args))
return _fallback; // Argument is missing.
else
{
using arg_t = typename _detail::tuple<Args...>::template element_type<N - 1>;
if constexpr (_detail::is_decayed_same<arg_t, nullopt>)
return _fallback; // Argument is nullopt.
else
return _detail::invoke(_fn, args.template get<N - 1>());
}
}
};
template <std::size_t N, typename T>
struct _nth_value<N, T, void> : _detail::placeholder_base // fallback only
{
LEXY_EMPTY_MEMBER T _fallback;
template <typename State, typename... Args>
LEXY_FORCE_INLINE constexpr decltype(auto) operator()(State&,
const _detail::tuple<Args...>& args) const
{
if constexpr (N > sizeof...(Args))
return _fallback; // Argument is missing.
else
{
using arg_t = typename _detail::tuple<Args...>::template element_type<N - 1>;
if constexpr (_detail::is_decayed_same<arg_t, nullopt>)
return _fallback; // Argument is nullopt.
else
return args.template get<N - 1>();
}
}
template <typename Fn>
constexpr auto map(Fn&& fn) const
{
return _nth_value<N, T, std::decay_t<Fn>>{{}, _fallback, LEXY_FWD(fn)};
}
};
template <std::size_t N, typename Fn>
struct _nth_value<N, void, Fn> : _detail::placeholder_base // map only
{
LEXY_EMPTY_MEMBER Fn _fn;
template <typename State, typename... Args>
LEXY_FORCE_INLINE constexpr decltype(auto) operator()(State&,
const _detail::tuple<Args...>& args) const
{
static_assert(N <= sizeof...(Args), "not enough arguments for nth_value<N>");
return _detail::invoke(_fn, args.template get<N - 1>());
}
template <typename Arg>
constexpr auto operator||(Arg&& fallback) const
{
return _nth_value<N, std::decay_t<Arg>, Fn>{{}, LEXY_FWD(fallback), _fn};
}
template <typename Arg>
constexpr auto or_(Arg&& fallback) const
{
return *this || LEXY_FWD(fallback);
}
constexpr auto or_default() const
{
return *this || _default{};
}
};
template <std::size_t N>
struct _nth_value<N, void, void> : _detail::placeholder_base
{
// I'm sorry, but this is for consistency with std::bind.
static_assert(N > 0, "values are 1-indexed");
template <typename State, typename... Args>
LEXY_FORCE_INLINE constexpr decltype(auto) operator()(State&,
const _detail::tuple<Args...>& args) const
{
static_assert(N <= sizeof...(Args), "not enough arguments for nth_value<N>");
return args.template get<N - 1>();
}
template <typename Arg>
constexpr auto operator||(Arg&& fallback) const
{
return _nth_value<N, std::decay_t<Arg>, void>{{}, LEXY_FWD(fallback)};
}
template <typename Arg>
constexpr auto or_(Arg&& fallback) const
{
return *this || LEXY_FWD(fallback);
}
constexpr auto or_default() const
{
return *this || _default{};
}
template <typename Fn>
constexpr auto map(Fn&& fn) const
{
return _nth_value<N, void, std::decay_t<Fn>>{{}, LEXY_FWD(fn)};
}
};
/// Placeholder for bind that expands to the nth value produced by the rule.
template <std::size_t N>
constexpr auto nth_value = _nth_value<N, void, void>{};
inline namespace placeholders
{
constexpr auto _1 = nth_value<1>;
constexpr auto _2 = nth_value<2>;
constexpr auto _3 = nth_value<3>;
constexpr auto _4 = nth_value<4>;
constexpr auto _5 = nth_value<5>;
constexpr auto _6 = nth_value<6>;
constexpr auto _7 = nth_value<7>;
constexpr auto _8 = nth_value<8>;
} // namespace placeholders
template <typename Fn>
struct _parse_state : _detail::placeholder_base
{
LEXY_EMPTY_MEMBER Fn _fn;
template <typename State, typename... Args>
constexpr decltype(auto) operator()(State& state, const _detail::tuple<Args...>&) const
{
static_assert(!std::is_same_v<State, _detail::no_bind_state>,
"lexy::parse_state requires that a state is passed to lexy::parse()");
return _detail::invoke(_fn, state);
}
};
template <>
struct _parse_state<void> : _detail::placeholder_base
{
template <typename State, typename... Args>
constexpr decltype(auto) operator()(State& state, const _detail::tuple<Args...>&) const
{
static_assert(!std::is_same_v<State, _detail::no_bind_state>,
"lexy::parse_state requires that a state is passed to lexy::parse()");
return state;
}
template <typename Fn>
constexpr auto map(Fn&& fn) const
{
return _parse_state<std::decay_t<Fn>>{{}, LEXY_FWD(fn)};
}
};
constexpr auto parse_state = _parse_state<void>{};
} // namespace lexy
//=== bind ===//
namespace lexy
{
template <typename Callback, typename... BoundArgs>
struct _bound_cb
{
LEXY_EMPTY_MEMBER Callback _callback;
LEXY_EMPTY_MEMBER _detail::tuple<BoundArgs...> _bound_args;
using return_type = typename Callback::return_type;
template <typename State>
struct _with_state
{
const _bound_cb& _bound;
State& _state;
template <typename... Args>
constexpr return_type operator()(Args&&... args) const&&
{
return _detail::invoke_bound(_bound._callback, _bound._bound_args,
_bound._bound_args.index_sequence(), _state,
LEXY_FWD(args)...);
}
};
template <typename State>
constexpr auto operator[](State& state) const
{
return _with_state<State>{*this, state};
}
template <typename... Args>
constexpr return_type operator()(Args&&... args) const
{
auto state = _detail::no_bind_state{};
return _detail::invoke_bound(_callback, _bound_args, _bound_args.index_sequence(), state,
LEXY_FWD(args)...);
}
};
/// Binds the `operator()` of the callback with pre-defined/remapped values.
template <typename Callback, typename... BoundArgs>
constexpr auto bind(Callback&& callback, BoundArgs&&... args)
{
using bound = _bound_cb<std::decay_t<Callback>, std::decay_t<BoundArgs>...>;
return bound{LEXY_FWD(callback), _detail::make_tuple(LEXY_FWD(args)...)};
}
} // namespace lexy
namespace lexy
{
template <typename Sink>
struct _sink_wrapper
{
const Sink& _sink;
template <typename... Args>
constexpr auto operator()(Args&&... args)
{
return _sink.sink(LEXY_FWD(args)...);
}
};
template <typename Sink, typename... BoundArgs>
struct _bound_sink
{
LEXY_EMPTY_MEMBER Sink _sink;
LEXY_EMPTY_MEMBER _detail::tuple<BoundArgs...> _bound;
template <typename... Args>
constexpr auto operator()(Args&&... args) const -> decltype(_sink(LEXY_FWD(args)...))
{
return _sink(LEXY_FWD(args)...);
}
template <bool Dummy = true,
typename = std::enable_if_t<(!_detail::is_placeholder<BoundArgs> && ... && Dummy)>>
constexpr auto sink() const
{
auto state = _detail::no_bind_state{};
return _detail::invoke_bound(_sink_wrapper<Sink>{_sink}, _bound, _bound.index_sequence(),
state);
}
template <typename State>
constexpr auto sink(State& state) const
{
return _detail::invoke_bound(_sink_wrapper<Sink>{_sink}, _bound, _bound.index_sequence(),
state);
}
};
/// Binds the `.sink()` function of a sink.
/// The result has a `.sink()` function that accepts the state (i.e. the parse state), but no
/// additional values.
template <typename Sink, typename... BoundArgs>
constexpr auto bind_sink(Sink&& sink, BoundArgs&&... args)
{
static_assert(
(!_detail::is_decayed_same<BoundArgs, _detail::all_values_placeholder> && ...),
"lexy::values as a placeholder for bind_sink() doesn't make sense - there won't be any values");
using bound = _bound_sink<std::decay_t<Sink>, std::decay_t<BoundArgs>...>;
return bound{LEXY_FWD(sink), _detail::make_tuple(LEXY_FWD(args)...)};
}
} // namespace lexy
#endif // LEXY_CALLBACK_BIND_HPP_INCLUDED

View File

@@ -0,0 +1,80 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_BIT_CAST_HPP_INCLUDED
#define LEXY_CALLBACK_BIT_CAST_HPP_INCLUDED
#include <lexy/callback/base.hpp>
#ifndef LEXY_HAS_BITCAST
# if defined(__has_builtin)
# if __has_builtin(__builtin_bit_cast)
# define LEXY_HAS_BITCAST 2
# endif
# endif
#endif
#ifndef LEXY_HAS_BITCAST
# if defined(__has_include)
# if __has_include(<bit>) && __cplusplus >= 202002L
# include <bit>
# ifdef __cpp_lib_bit_cast
# define LEXY_HAS_BITCAST 1
# endif
# endif
# endif
#endif
#ifndef LEXY_HAS_BITCAST
# define LEXY_HAS_BITCAST 0
#endif
#if LEXY_HAS_BITCAST == 2
# define LEXY_BITCAST_CONSTEXPR constexpr
#elif LEXY_HAS_BITCAST == 1
# define LEXY_BITCAST_CONSTEXPR constexpr
#else
# include <cstring>
# define LEXY_BITCAST_CONSTEXPR
#endif
namespace lexy
{
template <typename T>
struct _bit_cast
{
static_assert(std::is_trivially_copyable_v<T>);
using return_type = T;
template <typename Arg, typename = std::enable_if_t<sizeof(T) == sizeof(Arg)
&& std::is_trivially_copyable_v<Arg>>>
LEXY_BITCAST_CONSTEXPR T operator()(const Arg& arg) const
{
#if LEXY_HAS_BITCAST == 2
return __builtin_bit_cast(T, arg);
#elif LEXY_HAS_BITCAST == 1
return std::bit_cast<T>(arg);
#else
static_assert(std::is_default_constructible_v<T>, "sorry, get a better standard library");
T to;
std::memcpy(&to, &arg, sizeof(T));
return to;
#endif
}
};
/// std::bit_cast as a callback.
template <typename T>
constexpr auto bit_cast = _bit_cast<T>{};
} // namespace lexy
#endif // LEXY_CALLBACK_BIT_CAST_HPP_INCLUDED

View File

@@ -0,0 +1,120 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_COMPOSITION_HPP_INCLUDED
#define LEXY_CALLBACK_COMPOSITION_HPP_INCLUDED
#include <lexy/callback/base.hpp>
namespace lexy
{
template <typename Cb, typename State, typename = void>
struct _compose_state
{
const Cb& _cb;
State& _state;
using return_type = typename Cb::return_type;
template <typename... Args>
constexpr auto operator()(Args&&... args) const -> decltype(_cb(LEXY_FWD(args)...))
{
return _cb(LEXY_FWD(args)...);
}
};
template <typename Cb, typename State>
struct _compose_state<Cb, State, std::enable_if_t<lexy::is_callback_state<Cb, State>>>
{
const Cb& _cb;
State& _state;
using return_type = typename Cb::return_type;
template <typename... Args>
constexpr auto operator()(Args&&... args) const -> decltype(_cb[_state](LEXY_FWD(args)...))
{
return _cb[_state](LEXY_FWD(args)...);
}
};
template <typename First, typename Second>
struct _compose_cb
{
LEXY_EMPTY_MEMBER First _first;
LEXY_EMPTY_MEMBER Second _second;
constexpr explicit _compose_cb(First&& first, Second&& second)
: _first(LEXY_MOV(first)), _second(LEXY_MOV(second))
{}
using return_type = typename Second::return_type;
template <typename State,
typename = std::enable_if_t<lexy::is_callback_state<First, State> //
|| lexy::is_callback_state<Second, State>>>
constexpr auto operator[](State& state) const
{
auto first = _compose_state<First, State>{_first, state};
auto second = _compose_state<Second, State>{_second, state};
return lexy::_compose_cb(LEXY_MOV(first), LEXY_MOV(second));
}
template <typename... Args>
constexpr auto operator()(Args&&... args) const
-> LEXY_DECAY_DECLTYPE(_first(LEXY_FWD(args)...), LEXY_DECLVAL(return_type))
{
return _second(_first(LEXY_FWD(args)...));
}
};
template <typename Sink, typename Callback>
struct _compose_s
{
LEXY_EMPTY_MEMBER Sink _sink;
LEXY_EMPTY_MEMBER Callback _callback;
using return_type = typename Callback::return_type;
template <typename... Args>
constexpr auto sink(Args&&... args) const -> decltype(_sink.sink(LEXY_FWD(args)...))
{
return _sink.sink(LEXY_FWD(args)...);
}
template <typename State, typename = std::enable_if_t<lexy::is_callback_state<Callback, State>>>
constexpr auto operator[](State& state) const
{
return _compose_state<Callback, State>{_callback, state};
}
template <typename... Args>
constexpr auto operator()(Args&&... args) const -> decltype(_callback(LEXY_FWD(args)...))
{
return _callback(LEXY_FWD(args)...);
}
};
/// Composes two callbacks.
template <typename First, typename Second, typename = _detect_callback<First>,
typename = _detect_callback<Second>>
constexpr auto operator|(First first, Second second)
{
return _compose_cb(LEXY_MOV(first), LEXY_MOV(second));
}
template <typename S, typename Cb, typename Second>
constexpr auto operator|(_compose_s<S, Cb> composed, Second second)
{
auto cb = LEXY_MOV(composed._callback) | LEXY_MOV(second);
return _compose_s<S, decltype(cb)>{LEXY_MOV(composed._sink), LEXY_MOV(cb)};
}
/// Composes a sink with a callback.
template <typename Sink, typename Callback, typename = _detect_callback<Callback>>
constexpr auto operator>>(Sink sink, Callback cb)
{
return _compose_s<Sink, Callback>{LEXY_MOV(sink), LEXY_MOV(cb)};
}
} // namespace lexy
#endif // LEXY_CALLBACK_COMPOSITION_HPP_INCLUDED

View File

@@ -0,0 +1,33 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_CONSTANT_HPP_INCLUDED
#define LEXY_CALLBACK_CONSTANT_HPP_INCLUDED
#include <lexy/callback/base.hpp>
namespace lexy
{
template <typename T>
struct _constant
{
T _value;
using return_type = T;
constexpr const T& operator()() const
{
return _value;
}
};
/// Creates a callback that produces the given value without accepting arguments.
template <typename Arg>
LEXY_CONSTEVAL auto constant(Arg&& value)
{
return _constant<std::decay_t<Arg>>{LEXY_FWD(value)};
}
} // namespace lexy
#endif // LEXY_CALLBACK_CONSTANT_HPP_INCLUDED

View File

@@ -0,0 +1,511 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_CONTAINER_HPP_INCLUDED
#define LEXY_CALLBACK_CONTAINER_HPP_INCLUDED
#include <lexy/callback/base.hpp>
namespace lexy
{
struct nullopt;
template <typename Container>
using _detect_reserve = decltype(LEXY_DECLVAL(Container&).reserve(std::size_t()));
template <typename Container>
constexpr auto _has_reserve = _detail::is_detected<_detect_reserve, Container>;
template <typename Container>
using _detect_append = decltype(LEXY_DECLVAL(Container&).append(LEXY_DECLVAL(Container&&)));
template <typename Container>
constexpr auto _has_append = _detail::is_detected<_detect_append, Container>;
} // namespace lexy
//=== as_list ===//
namespace lexy
{
template <typename Container>
struct _list_sink
{
Container _result;
using return_type = Container;
template <typename C = Container, typename U>
constexpr auto operator()(U&& obj) -> decltype(LEXY_DECLVAL(C&).push_back(LEXY_FWD(obj)))
{
return _result.push_back(LEXY_FWD(obj));
}
template <typename C = Container, typename... Args>
constexpr auto operator()(Args&&... args)
-> decltype(LEXY_DECLVAL(C&).emplace_back(LEXY_FWD(args)...))
{
return _result.emplace_back(LEXY_FWD(args)...);
}
constexpr Container&& finish() &&
{
return LEXY_MOV(_result);
}
};
template <typename Container, typename AllocFn>
struct _list_alloc
{
AllocFn _alloc;
using return_type = Container;
template <typename State>
struct _with_state
{
State& _state;
const AllocFn& _alloc;
constexpr Container operator()(Container&& container) const
{
return LEXY_MOV(container);
}
constexpr Container operator()(nullopt&&) const
{
return Container(_detail::invoke(_alloc, _state));
}
template <typename... Args>
constexpr auto operator()(Args&&... args) const
-> LEXY_DECAY_DECLTYPE((LEXY_DECLVAL(Container&).push_back(LEXY_FWD(args)), ...),
LEXY_DECLVAL(Container))
{
Container result(_detail::invoke(_alloc, _state));
if constexpr (_has_reserve<Container>)
result.reserve(sizeof...(args));
(result.emplace_back(LEXY_FWD(args)), ...);
return result;
}
};
template <typename State>
constexpr auto operator[](State& state) const
{
return _with_state<State>{state, _alloc};
}
template <typename State>
constexpr auto sink(State& state) const
{
return _list_sink<Container>{Container(_detail::invoke(_alloc, state))};
}
};
template <typename Container>
struct _list
{
using return_type = Container;
constexpr Container operator()(Container&& container) const
{
return LEXY_MOV(container);
}
constexpr Container operator()(nullopt&&) const
{
return Container();
}
template <typename... Args>
constexpr auto operator()(Args&&... args) const
-> LEXY_DECAY_DECLTYPE((LEXY_DECLVAL(Container&).push_back(LEXY_FWD(args)), ...),
LEXY_DECLVAL(Container))
{
Container result;
if constexpr (_has_reserve<Container>)
result.reserve(sizeof...(args));
(result.emplace_back(LEXY_FWD(args)), ...);
return result;
}
template <typename C = Container, typename... Args>
constexpr auto operator()(const typename C::allocator_type& allocator, Args&&... args) const
-> decltype((LEXY_DECLVAL(C&).push_back(LEXY_FWD(args)), ...), C(allocator))
{
Container result(allocator);
if constexpr (_has_reserve<Container>)
result.reserve(sizeof...(args));
(result.emplace_back(LEXY_FWD(args)), ...);
return result;
}
constexpr auto sink() const
{
return _list_sink<Container>{Container()};
}
template <typename C = Container>
constexpr auto sink(const typename C::allocator_type& allocator) const
{
return _list_sink<Container>{Container(allocator)};
}
template <typename AllocFn>
constexpr auto allocator(AllocFn alloc_fn) const
{
return _list_alloc<Container, AllocFn>{alloc_fn};
}
constexpr auto allocator() const
{
return allocator([](const auto& alloc) { return alloc; });
}
};
/// A callback with sink that creates a list of things (e.g. a `std::vector`, `std::list`, etc.).
/// It repeatedly calls `push_back()` and `emplace_back()`.
template <typename Container>
constexpr auto as_list = _list<Container>{};
} // namespace lexy
//=== as_collection ===//
namespace lexy
{
template <typename Container>
struct _collection_sink
{
Container _result;
using return_type = Container;
template <typename C = Container, typename U>
constexpr auto operator()(U&& obj) -> decltype(LEXY_DECLVAL(C&).insert(LEXY_FWD(obj)))
{
return _result.insert(LEXY_FWD(obj));
}
template <typename C = Container, typename... Args>
constexpr auto operator()(Args&&... args)
-> decltype(LEXY_DECLVAL(C&).emplace(LEXY_FWD(args)...))
{
return _result.emplace(LEXY_FWD(args)...);
}
constexpr Container&& finish() &&
{
return LEXY_MOV(_result);
}
};
template <typename Container, typename AllocFn>
struct _collection_alloc
{
AllocFn _alloc;
using return_type = Container;
template <typename State>
struct _with_state
{
State& _state;
const AllocFn& _alloc;
constexpr Container operator()(Container&& container) const
{
return LEXY_MOV(container);
}
constexpr Container operator()(nullopt&&) const
{
return Container(_detail::invoke(_alloc, _state));
}
template <typename... Args>
constexpr auto operator()(Args&&... args) const
-> LEXY_DECAY_DECLTYPE((LEXY_DECLVAL(Container&).insert(LEXY_FWD(args)), ...),
LEXY_DECLVAL(Container))
{
Container result(_detail::invoke(_alloc, _state));
if constexpr (_has_reserve<Container>)
result.reserve(sizeof...(args));
(result.emplace(LEXY_FWD(args)), ...);
return result;
}
};
template <typename State>
constexpr auto operator[](State& state) const
{
return _with_state<State>{state, _alloc};
}
template <typename State>
constexpr auto sink(State& state) const
{
return _collection_sink<Container>{Container(_detail::invoke(_alloc, state))};
}
};
template <typename Container>
struct _collection
{
using return_type = Container;
constexpr Container operator()(Container&& container) const
{
return LEXY_MOV(container);
}
constexpr Container operator()(nullopt&&) const
{
return Container();
}
template <typename... Args>
constexpr auto operator()(Args&&... args) const
-> LEXY_DECAY_DECLTYPE((LEXY_DECLVAL(Container&).insert(LEXY_FWD(args)), ...),
LEXY_DECLVAL(Container))
{
Container result;
if constexpr (_has_reserve<Container>)
result.reserve(sizeof...(args));
(result.emplace(LEXY_FWD(args)), ...);
return result;
}
template <typename C = Container, typename... Args>
constexpr auto operator()(const typename C::allocator_type& allocator, Args&&... args) const
-> decltype((LEXY_DECLVAL(C&).insert(LEXY_FWD(args)), ...), C(allocator))
{
Container result(allocator);
if constexpr (_has_reserve<Container>)
result.reserve(sizeof...(args));
(result.emplace(LEXY_FWD(args)), ...);
return result;
}
constexpr auto sink() const
{
return _collection_sink<Container>{Container()};
}
template <typename C = Container>
constexpr auto sink(const typename C::allocator_type& allocator) const
{
return _collection_sink<Container>{Container(allocator)};
}
template <typename AllocFn>
constexpr auto allocator(AllocFn alloc_fn) const
{
return _collection_alloc<Container, AllocFn>{alloc_fn};
}
constexpr auto allocator() const
{
return allocator([](const auto& alloc) { return alloc; });
}
};
/// A callback with sink that creates an unordered collection of things (e.g. a `std::set`,
/// `std::unordered_map`, etc.). It repeatedly calls `insert()` and `emplace()`.
template <typename T>
constexpr auto as_collection = _collection<T>{};
} // namespace lexy
//=== concat ===//
namespace lexy
{
template <typename Container>
struct _concat
{
using return_type = Container;
constexpr Container operator()(nullopt&&) const
{
return Container();
}
template <typename... Tail>
constexpr Container _call(Container&& head, Tail&&... tail) const
{
if constexpr (sizeof...(Tail) == 0)
return LEXY_MOV(head);
else
{
if constexpr (_has_reserve<Container>)
{
auto total_size = (head.size() + ... + tail.size());
head.reserve(total_size);
}
auto append = [&head](Container&& container) {
if constexpr (_has_append<Container>)
{
head.append(LEXY_MOV(container));
}
else
{
for (auto& elem : container)
head.push_back(LEXY_MOV(elem));
}
};
(append(LEXY_MOV(tail)), ...);
return LEXY_MOV(head);
}
}
template <typename... Args>
constexpr auto operator()(Args&&... args) const -> decltype(_call(Container(LEXY_FWD(args))...))
{
return _call(Container(LEXY_FWD(args))...);
}
struct _sink
{
Container _result;
using return_type = Container;
constexpr void operator()(Container&& container)
{
if (_result.empty())
{
// We assign until we have items.
// That way we get the existing allocator.
_result = LEXY_MOV(container);
}
else if constexpr (_has_append<Container>)
{
_result.append(LEXY_MOV(container));
}
else
{
if constexpr (_has_reserve<Container>)
{
auto capacity = _result.capacity();
auto total_size = _result.size() + container.size();
if (total_size > capacity)
{
// If we need more space we reserve at least twice as much.
auto exp_capacity = 2 * capacity;
if (total_size > exp_capacity)
_result.reserve(total_size);
else
_result.reserve(exp_capacity);
}
}
for (auto& elem : container)
_result.push_back(LEXY_MOV(elem));
}
}
constexpr Container&& finish() &&
{
return LEXY_MOV(_result);
}
};
constexpr auto sink() const
{
return _sink{};
}
};
template <typename Container>
constexpr auto concat = _concat<Container>{};
} // namespace lexy
//=== collect ===//
namespace lexy
{
template <typename Container, typename Callback>
class _collect_sink
{
public:
constexpr explicit _collect_sink(Callback callback) : _callback(LEXY_MOV(callback)) {}
template <typename C = Container>
constexpr explicit _collect_sink(Callback callback, const typename C::allocator_type& allocator)
: _result(allocator), _callback(LEXY_MOV(callback))
{}
using return_type = Container;
template <typename... Args>
constexpr auto operator()(Args&&... args)
-> decltype(void(LEXY_DECLVAL(Callback)(LEXY_FWD(args)...)))
{
_result.push_back(_callback(LEXY_FWD(args)...));
}
constexpr auto finish() &&
{
return LEXY_MOV(_result);
}
private:
Container _result;
LEXY_EMPTY_MEMBER Callback _callback;
};
template <typename Callback>
class _collect_sink<void, Callback>
{
public:
constexpr explicit _collect_sink(Callback callback) : _count(0), _callback(LEXY_MOV(callback))
{}
using return_type = std::size_t;
template <typename... Args>
constexpr auto operator()(Args&&... args)
-> decltype(void(LEXY_DECLVAL(Callback)(LEXY_FWD(args)...)))
{
_callback(LEXY_FWD(args)...);
++_count;
}
constexpr auto finish() &&
{
return _count;
}
private:
std::size_t _count;
LEXY_EMPTY_MEMBER Callback _callback;
};
template <typename Container, typename Callback>
class _collect
{
public:
constexpr explicit _collect(Callback callback) : _callback(LEXY_MOV(callback)) {}
constexpr auto sink() const
{
return _collect_sink<Container, Callback>(_callback);
}
template <typename C = Container>
constexpr auto sink(const typename C::allocator_type& allocator) const
{
return _collect_sink<Container, Callback>(_callback, allocator);
}
private:
LEXY_EMPTY_MEMBER Callback _callback;
};
/// Returns a sink that invokes the void-returning callback multiple times, resulting in the number
/// of times it was invoked.
template <typename Callback>
constexpr auto collect(Callback&& callback)
{
using callback_t = std::decay_t<Callback>;
static_assert(std::is_void_v<typename callback_t::return_type>,
"need to specify a container to collect into for non-void callbacks");
return _collect<void, callback_t>(LEXY_FWD(callback));
}
/// Returns a sink that invokes the callback multiple times, storing each result in the container.
template <typename Container, typename Callback>
constexpr auto collect(Callback&& callback)
{
using callback_t = std::decay_t<Callback>;
static_assert(!std::is_void_v<typename callback_t::return_type>,
"cannot collect a void callback into a container");
return _collect<Container, callback_t>(LEXY_FWD(callback));
}
} // namespace lexy
#endif // LEXY_CALLBACK_CONTAINER_HPP_INCLUDED

View File

@@ -0,0 +1,92 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_FOLD_HPP_INCLUDED
#define LEXY_CALLBACK_FOLD_HPP_INCLUDED
#include <lexy/callback/base.hpp>
namespace lexy
{
template <bool Inplace>
struct _fold_sfinae;
template <>
struct _fold_sfinae<true>
{
template <typename Op, typename T, typename... Args>
using type = decltype(void(
_detail::invoke(LEXY_DECLVAL(Op), LEXY_DECLVAL(T&), LEXY_DECLVAL(Args)...)));
};
template <>
struct _fold_sfinae<false>
{
template <typename Op, typename T, typename... Args>
using type = decltype(void(
_detail::invoke(LEXY_DECLVAL(Op), LEXY_DECLVAL(T&&), LEXY_DECLVAL(Args)...)));
};
template <typename T, typename Arg, bool Inplace, typename Op>
struct _fold
{
Arg _init;
LEXY_EMPTY_MEMBER Op _op;
struct _sink_callback
{
T _result;
Op _op;
using return_type = T;
template <typename... Args>
constexpr auto operator()(Args&&... args) ->
typename _fold_sfinae<Inplace>::template type<Op, T, Args&&...>
{
if constexpr (Inplace)
_detail::invoke(_op, _result, LEXY_FWD(args)...);
else
_result = _detail::invoke(_op, LEXY_MOV(_result), LEXY_FWD(args)...);
}
constexpr T finish() &&
{
return LEXY_MOV(_result);
}
};
constexpr auto sink() const
{
if constexpr (std::is_constructible_v<T, Arg>)
return _sink_callback{T(_init), _op};
else
return _sink_callback{_init(), _op};
}
};
/// Sink that folds all the arguments with the binary operation op.
template <typename T, typename Arg = T, typename... Op>
constexpr auto fold(Arg&& init, Op&&... op)
{
auto fn = _make_overloaded(LEXY_FWD(op)...);
return _fold<T, std::decay_t<Arg>, false, decltype(fn)>{LEXY_FWD(init), LEXY_MOV(fn)};
}
/// Sink that folds all the arguments with the binary operation op that modifies the
/// result in-place.
template <typename T, typename Arg = T, typename... Op>
constexpr auto fold_inplace(Arg&& init, Op&&... op)
{
auto fn = _make_overloaded(LEXY_FWD(op)...);
return _fold<T, std::decay_t<Arg>, true, decltype(fn)>{LEXY_FWD(init), LEXY_MOV(fn)};
}
} // namespace lexy
namespace lexy
{
/// Sink that counts all arguments.
constexpr auto count
= fold_inplace<std::size_t>(0u, [](std::size_t& result, auto&&...) { ++result; });
} // namespace lexy
#endif // LEXY_CALLBACK_FOLD_HPP_INCLUDED

View File

@@ -0,0 +1,51 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_FORWARD_HPP_INCLUDED
#define LEXY_CALLBACK_FORWARD_HPP_INCLUDED
#include <lexy/callback/base.hpp>
namespace lexy
{
struct nullopt;
template <typename T>
struct _fwd
{
using return_type = T;
constexpr T operator()(T&& t) const
{
return LEXY_MOV(t);
}
constexpr T operator()(const T& t) const
{
return t;
}
};
template <>
struct _fwd<void>
{
using return_type = void;
template <typename... Args>
constexpr auto sink(const Args&...) const
{
// We don't need a separate type, forward itself can have the required functions.
return *this;
}
constexpr void operator()() const {}
constexpr void operator()(const lexy::nullopt&) const {}
constexpr void finish() && {}
};
/// A callback that just forwards an existing object.
template <typename T>
constexpr auto forward = _fwd<T>{};
} // namespace lexy
#endif // LEXY_CALLBACK_FORWARD_HPP_INCLUDED

View File

@@ -0,0 +1,44 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_INTEGER_HPP_INCLUDED
#define LEXY_CALLBACK_INTEGER_HPP_INCLUDED
#include <lexy/callback/base.hpp>
#include <lexy/dsl/sign.hpp>
namespace lexy
{
template <typename T>
struct _int
{
using return_type = T;
// You don't actually produce an integer value.
constexpr T operator()(lexy::plus_sign) const = delete;
constexpr T operator()(lexy::minus_sign) const = delete;
template <typename Integer>
constexpr T operator()(const Integer& value) const
{
return T(value);
}
template <typename Integer>
constexpr T operator()(lexy::plus_sign, const Integer& value) const
{
return T(value);
}
template <typename Integer>
constexpr T operator()(lexy::minus_sign, const Integer& value) const
{
return T(-value);
}
};
// A callback that takes an optional sign and an integer and produces the signed integer.
template <typename T>
constexpr auto as_integer = _int<T>{};
} // namespace lexy
#endif // LEXY_CALLBACK_INTEGER_HPP_INCLUDED

View File

@@ -0,0 +1,34 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_NOOP_HPP_INCLUDED
#define LEXY_CALLBACK_NOOP_HPP_INCLUDED
#include <lexy/callback/base.hpp>
namespace lexy
{
struct _noop
{
using return_type = void;
template <typename... Args>
constexpr auto sink(const Args&...) const
{
// We don't need a separate type, noop itself can have the required functions.
return *this;
}
template <typename... Args>
constexpr void operator()(const Args&...) const
{}
constexpr void finish() && {}
};
/// A callback with sink that does nothing.
inline constexpr auto noop = _noop{};
} // namespace lexy
#endif // LEXY_CALLBACK_NOOP_HPP_INCLUDED

View File

@@ -0,0 +1,98 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_OBJECT_HPP_INCLUDED
#define LEXY_CALLBACK_OBJECT_HPP_INCLUDED
#include <lexy/callback/base.hpp>
namespace lexy::_detail
{
template <typename T, typename... Args>
using _detect_brace_construct = decltype(T{LEXY_DECLVAL(Args)...});
template <typename T, typename... Args>
constexpr auto is_brace_constructible = _detail::is_detected<_detect_brace_construct, T, Args...>;
template <typename T, typename... Args>
constexpr auto is_constructible
= std::is_constructible_v<T, Args...> || is_brace_constructible<T, Args...>;
} // namespace lexy::_detail
namespace lexy
{
template <typename T>
struct _construct
{
using return_type = T;
constexpr T operator()(T&& t) const
{
return LEXY_MOV(t);
}
constexpr T operator()(const T& t) const
{
return t;
}
template <typename... Args>
constexpr auto operator()(Args&&... args) const
-> std::enable_if_t<_detail::is_constructible<T, Args&&...>, T>
{
if constexpr (std::is_constructible_v<T, Args&&...>)
return T(LEXY_FWD(args)...);
else
return T{LEXY_FWD(args)...};
}
};
template <>
struct _construct<void>
{
using return_type = void;
constexpr void operator()() const {}
};
/// A callback that constructs an object of type T by forwarding the arguments.
template <typename T>
constexpr auto construct = _construct<T>{};
template <typename T, typename PtrT>
struct _new
{
using return_type = PtrT;
constexpr PtrT operator()(T&& t) const
{
auto ptr = new T(LEXY_MOV(t));
return PtrT(ptr);
}
constexpr PtrT operator()(const T& t) const
{
auto ptr = new T(t);
return PtrT(ptr);
}
template <typename... Args>
constexpr auto operator()(Args&&... args) const
-> std::enable_if_t<_detail::is_constructible<T, Args&&...>, PtrT>
{
if constexpr (std::is_constructible_v<T, Args&&...>)
{
auto ptr = new T(LEXY_FWD(args)...);
return PtrT(ptr);
}
else
{
auto ptr = new T{LEXY_FWD(args)...};
return PtrT(ptr);
}
}
};
/// A callback that constructs an object of type T on the heap by forwarding the arguments.
template <typename T, typename PtrT = T*>
constexpr auto new_ = _new<T, PtrT>{};
} // namespace lexy
#endif // LEXY_CALLBACK_OBJECT_HPP_INCLUDED

View File

@@ -0,0 +1,210 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CALLBACK_STRING_HPP_INCLUDED
#define LEXY_CALLBACK_STRING_HPP_INCLUDED
#include <lexy/_detail/code_point.hpp>
#include <lexy/callback/base.hpp>
#include <lexy/code_point.hpp>
#include <lexy/encoding.hpp>
#include <lexy/input/base.hpp>
#include <lexy/lexeme.hpp>
namespace lexy
{
struct nullopt;
template <typename String>
using _string_char_type = LEXY_DECAY_DECLTYPE(LEXY_DECLVAL(String)[0]);
template <typename String, typename Encoding, typename CaseFoldingDSL = void>
struct _as_string
{
using return_type = String;
using _char_type = _string_char_type<String>;
static_assert(lexy::_detail::is_compatible_char_type<Encoding, _char_type>,
"invalid character type/encoding combination");
static constexpr String&& _case_folding(String&& str)
{
if constexpr (std::is_void_v<CaseFoldingDSL>)
{
return LEXY_MOV(str);
}
else if constexpr (CaseFoldingDSL::template is_inplace<Encoding>)
{
// We can change the string in place.
auto original_reader = lexy::_range_reader<Encoding>(str.begin(), str.end());
auto reader = typename CaseFoldingDSL::template case_folding<decltype(original_reader)>{
original_reader};
for (auto ptr = str.data(); true; ++ptr)
{
auto cur = reader.peek();
if (cur == Encoding::eof())
break;
reader.bump();
// Once we've bumped it, we're not looking at it again.
*ptr = static_cast<_char_type>(cur);
}
return LEXY_MOV(str);
}
else
{
// We store the existing string somewhere else and clear it.
// Then we can read the case folded string and append each code unit.
auto original = LEXY_MOV(str);
str = String();
str.reserve(original.size());
auto original_reader = lexy::_range_reader<Encoding>(original.begin(), original.end());
auto reader = typename CaseFoldingDSL::template case_folding<decltype(original_reader)>{
original_reader};
while (true)
{
auto cur = reader.peek();
if (cur == Encoding::eof())
break;
str.push_back(static_cast<_char_type>(cur));
reader.bump();
}
return LEXY_MOV(str);
}
}
template <typename NewCaseFoldingDSL>
constexpr auto case_folding(NewCaseFoldingDSL) const
{
return _as_string<String, Encoding, NewCaseFoldingDSL>{};
}
constexpr String operator()(nullopt&&) const
{
return String();
}
constexpr String&& operator()(String&& str) const
{
return _case_folding(LEXY_MOV(str));
}
template <typename Iterator>
constexpr auto operator()(Iterator begin, Iterator end) const -> decltype(String(begin, end))
{
return _case_folding(String(begin, end));
}
template <typename Str = String, typename Iterator>
constexpr auto operator()(const typename Str::allocator_type& allocator, Iterator begin,
Iterator end) const -> decltype(String(begin, end, allocator))
{
return _case_folding(String(begin, end, allocator));
}
template <typename Reader>
constexpr String operator()(lexeme<Reader> lex) const
{
static_assert(lexy::char_type_compatible_with_reader<Reader, _char_type>,
"cannot convert lexeme to this string type");
using iterator = typename lexeme<Reader>::iterator;
if constexpr (std::is_convertible_v<iterator, const _char_type*>)
return _case_folding(String(lex.data(), lex.size()));
else
return _case_folding(String(lex.begin(), lex.end()));
}
template <typename Str = String, typename Reader>
constexpr String operator()(const typename Str::allocator_type& allocator,
lexeme<Reader> lex) const
{
static_assert(lexy::char_type_compatible_with_reader<Reader, _char_type>,
"cannot convert lexeme to this string type");
using iterator = typename lexeme<Reader>::iterator;
if constexpr (std::is_convertible_v<iterator, const _char_type*>)
return _case_folding(String(lex.data(), lex.size(), allocator));
else
return _case_folding(String(lex.begin(), lex.end(), allocator));
}
constexpr String operator()(code_point cp) const
{
typename Encoding::char_type buffer[4] = {};
auto size = _detail::encode_code_point<Encoding>(cp.value(), buffer, 4);
return _case_folding(String(buffer, buffer + size));
}
template <typename Str = String>
constexpr String operator()(const typename Str::allocator_type& allocator, code_point cp) const
{
typename Encoding::char_type buffer[4] = {};
auto size = _detail::encode_code_point<Encoding>(cp.value(), buffer, 4);
return _case_folding(String(buffer, buffer + size, allocator));
}
struct _sink
{
String _result;
using return_type = String;
template <typename CharT, typename = decltype(LEXY_DECLVAL(String).push_back(CharT()))>
constexpr void operator()(CharT c)
{
_result.push_back(c);
}
constexpr void operator()(String&& str)
{
_result.append(LEXY_MOV(str));
}
template <typename Str = String, typename Iterator>
constexpr auto operator()(Iterator begin, Iterator end)
-> decltype(void(LEXY_DECLVAL(Str).append(begin, end)))
{
_result.append(begin, end);
}
template <typename Reader>
constexpr void operator()(lexeme<Reader> lex)
{
static_assert(lexy::char_type_compatible_with_reader<Reader, _char_type>,
"cannot convert lexeme to this string type");
_result.append(lex.begin(), lex.end());
}
constexpr void operator()(code_point cp)
{
typename Encoding::char_type buffer[4] = {};
auto size = _detail::encode_code_point<Encoding>(cp.value(), buffer, 4);
_result.append(buffer, buffer + size);
}
constexpr String&& finish() &&
{
return _case_folding(LEXY_MOV(_result));
}
};
constexpr auto sink() const
{
return _sink{String()};
}
template <typename S = String>
constexpr auto sink(const typename S::allocator_type& allocator) const
{
return _sink{String(allocator)};
}
};
/// A callback with sink that creates a string (e.g. `std::string`).
/// As a callback, it converts a lexeme into the string.
/// As a sink, it repeatedly calls `.push_back()` for individual characters,
/// or `.append()` for lexemes or other strings.
template <typename String, typename Encoding = deduce_encoding<_string_char_type<String>>>
constexpr auto as_string = _as_string<String, Encoding>{};
} // namespace lexy
#endif // LEXY_CALLBACK_STRING_HPP_INCLUDED

View File

@@ -0,0 +1,306 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_CODE_POINT_HPP_INCLUDED
#define LEXY_CODE_POINT_HPP_INCLUDED
#include <cstdint>
#include <lexy/_detail/assert.hpp>
#include <lexy/_detail/config.hpp>
#if LEXY_HAS_UNICODE_DATABASE
# define LEXY_UNICODE_CONSTEXPR constexpr
#else
# define LEXY_UNICODE_CONSTEXPR
#endif
namespace lexy
{
/// A unicode code point.
class code_point
{
public:
constexpr code_point() noexcept : _value(0xFFFF'FFFF) {}
constexpr explicit code_point(char32_t value) noexcept : _value(value) {}
constexpr auto value() const noexcept
{
return _value;
}
//=== classification ===//
constexpr bool is_ascii() const noexcept
{
return _value <= 0x7F;
}
constexpr bool is_bmp() const noexcept
{
return _value <= 0xFFFF;
}
constexpr bool is_valid() const noexcept
{
return _value <= 0x10'FFFF;
}
constexpr bool is_control() const noexcept
{
return _value <= 0x1F || (0x7F <= _value && _value <= 0x9F);
}
constexpr bool is_surrogate() const noexcept
{
return 0xD800 <= _value && _value <= 0xDFFF;
}
constexpr bool is_private_use() const noexcept
{
return (0xE000 <= _value && _value <= 0xF8FF)
|| (0x0F'0000 <= _value && _value <= 0x0F'FFFD)
|| (0x10'0000 <= _value && _value <= 0x10'FFFD);
}
constexpr bool is_noncharacter() const noexcept
{
// Contiguous range of 32 non-characters.
if (0xFDD0 <= _value && _value <= 0xFDEF)
return true;
// Last two code points of every plane.
auto in_plane = _value & 0xFFFF;
return in_plane == 0xFFFE || in_plane == 0xFFFF;
}
constexpr bool is_scalar() const noexcept
{
return is_valid() && !is_surrogate();
}
//=== general category ===//
enum general_category_t
{
// NOLINTNEXTLINE: can't use parentheses here
#define LEXY_UNICODE_CATEGORY(Short, Long) Short, Long = Short
LEXY_UNICODE_CATEGORY(Lu, uppercase_letter),
LEXY_UNICODE_CATEGORY(Ll, lowercase_letter),
LEXY_UNICODE_CATEGORY(Lt, titlecase_letter),
LEXY_UNICODE_CATEGORY(Lm, modifier_letter),
LEXY_UNICODE_CATEGORY(Lo, other_letter),
LEXY_UNICODE_CATEGORY(Mn, nonspacing_mark),
LEXY_UNICODE_CATEGORY(Mc, spacing_mark),
LEXY_UNICODE_CATEGORY(Me, enclosing_mark),
LEXY_UNICODE_CATEGORY(Nd, decimal_number),
LEXY_UNICODE_CATEGORY(Nl, letter_number),
LEXY_UNICODE_CATEGORY(No, other_number),
LEXY_UNICODE_CATEGORY(Pc, connector_punctuation),
LEXY_UNICODE_CATEGORY(Pd, dash_punctuation),
LEXY_UNICODE_CATEGORY(Ps, open_punctuation),
LEXY_UNICODE_CATEGORY(Pe, closing_punctuation),
LEXY_UNICODE_CATEGORY(Pi, initial_puncutation),
LEXY_UNICODE_CATEGORY(Pf, final_puncutation),
LEXY_UNICODE_CATEGORY(Po, other_punctuation),
LEXY_UNICODE_CATEGORY(Sm, math_symbol),
LEXY_UNICODE_CATEGORY(Sc, currency_symbol),
LEXY_UNICODE_CATEGORY(Sk, modifier_symbol),
LEXY_UNICODE_CATEGORY(So, other_symbol),
LEXY_UNICODE_CATEGORY(Zs, space_separator),
LEXY_UNICODE_CATEGORY(Zl, line_separator),
LEXY_UNICODE_CATEGORY(Zp, paragraph_separator),
LEXY_UNICODE_CATEGORY(Cc, control),
LEXY_UNICODE_CATEGORY(Cf, format),
LEXY_UNICODE_CATEGORY(Cs, surrogate),
LEXY_UNICODE_CATEGORY(Co, private_use),
LEXY_UNICODE_CATEGORY(Cn, unassigned),
#undef LEXY_UNICODE_CATEGORY
};
template <general_category_t... Cats>
struct _gc_group
{
const char* name;
friend constexpr bool operator==(_gc_group, general_category_t cat)
{
return ((cat == Cats) || ...);
}
friend constexpr bool operator==(general_category_t cat, _gc_group)
{
return ((cat == Cats) || ...);
}
friend constexpr bool operator!=(_gc_group, general_category_t cat)
{
return !(_gc_group{} == cat);
}
friend constexpr bool operator!=(general_category_t cat, _gc_group)
{
return !(_gc_group{} == cat);
}
};
#define LEXY_UNICODE_CATEGORY_GROUP(Name, Short, Long, ...) \
static constexpr _gc_group<__VA_ARGS__> Short{"code-point." Name}; \
static constexpr _gc_group<__VA_ARGS__> Long = Short
LEXY_UNICODE_CATEGORY_GROUP("cased-letter", LC, cased_letter, Lu, Ll, Lt);
LEXY_UNICODE_CATEGORY_GROUP("letter", L, letter, Lu, Ll, Lt, Lm, Lo);
LEXY_UNICODE_CATEGORY_GROUP("mark", M, mark, Mn, Mc, Me);
LEXY_UNICODE_CATEGORY_GROUP("number", N, number, Nd, Nl, No);
LEXY_UNICODE_CATEGORY_GROUP("punctuation", P, punctuation, Pc, Pd, Ps, Pe, Pi, Pf, Po);
LEXY_UNICODE_CATEGORY_GROUP("symbol", S, symbol, Sm, Sc, Sk, So);
LEXY_UNICODE_CATEGORY_GROUP("separator", Z, separator, Zs, Zl, Zp);
LEXY_UNICODE_CATEGORY_GROUP("other", C, other, Cc, Cf, Cs, Co, Cn);
#undef LEXY_UNICODE_CATEGORY_GROUP
LEXY_UNICODE_CONSTEXPR general_category_t general_category() const noexcept;
//=== comparision ===//
friend constexpr bool operator==(code_point lhs, code_point rhs) noexcept
{
return lhs._value == rhs._value;
}
friend constexpr bool operator!=(code_point lhs, code_point rhs) noexcept
{
return lhs._value != rhs._value;
}
private:
char32_t _value;
};
LEXY_UNICODE_CONSTEXPR code_point simple_case_fold(code_point cp) noexcept;
} // namespace lexy
namespace lexy::_detail
{
constexpr const char* general_category_name(lexy::code_point::general_category_t category)
{
switch (category)
{
case lexy::code_point::Lu:
return "code-point.uppercase-letter";
case lexy::code_point::Ll:
return "code-point.lowercase-letter";
case lexy::code_point::Lt:
return "code-point.titlecase-letter";
case lexy::code_point::Lm:
return "code-point.modifier-letter";
case lexy::code_point::Lo:
return "code-point.other-letter";
case lexy::code_point::Mn:
return "code-point.nonspacing-mark";
case lexy::code_point::Mc:
return "code-point.combining-mark";
case lexy::code_point::Me:
return "code-point.enclosing-mark";
case lexy::code_point::Nd:
return "code-point.decimal-number";
case lexy::code_point::Nl:
return "code-point.letter-number";
case lexy::code_point::No:
return "code-point.other-number";
case lexy::code_point::Pc:
return "code-point.connector-punctuation";
case lexy::code_point::Pd:
return "code-point.dash-punctuation";
case lexy::code_point::Ps:
return "code-point.open-punctuation";
case lexy::code_point::Pe:
return "code-point.close-punctuation";
case lexy::code_point::Pi:
return "code-point.initial-quote-punctuation";
case lexy::code_point::Pf:
return "code-point.final-quote-punctuation";
case lexy::code_point::Po:
return "code-point.other-punctuation";
case lexy::code_point::Sm:
return "code-point.math-symbol";
case lexy::code_point::Sc:
return "code-point.currency-symbol";
case lexy::code_point::Sk:
return "code-point.modifier-symbol";
case lexy::code_point::So:
return "code-point.other-symbol";
case lexy::code_point::Zs:
return "code-point.space-separator";
case lexy::code_point::Zl:
return "code-point.line-separator";
case lexy::code_point::Zp:
return "code-point.paragraph-separator";
case lexy::code_point::Cc:
return "code-point.control";
case lexy::code_point::Cf:
return "code-point.format";
case lexy::code_point::Cs:
return "code-point.surrogate";
case lexy::code_point::Co:
return "code-point.private-use";
case lexy::code_point::Cn:
return "code-point.not-assigned";
}
return nullptr; // unreachable
}
} // namespace lexy::_detail
#if LEXY_HAS_UNICODE_DATABASE
# include <lexy/_detail/unicode_database.hpp>
constexpr lexy::code_point::general_category_t lexy::code_point::general_category() const noexcept
{
if (!is_valid())
return general_category_t::unassigned;
auto idx = _unicode_db::property_index(_value);
return _unicode_db::category[idx];
}
constexpr lexy::code_point lexy::simple_case_fold(code_point cp) noexcept
{
if (!cp.is_valid())
return cp;
auto idx = _unicode_db::property_index(cp.value());
auto offset = _unicode_db::case_folding_offset[idx];
return code_point(char32_t(std::int_least32_t(cp.value()) + offset));
}
namespace lexy::_detail
{
template <lexy::_unicode_db::binary_properties_t... Props>
LEXY_FORCE_INLINE constexpr bool code_point_has_properties(char32_t cp)
{
constexpr auto mask = ((1 << Props) | ...);
auto idx = _unicode_db::property_index(cp);
auto props = _unicode_db::binary_properties[idx];
return (props & mask) != 0;
}
} // namespace lexy::_detail
# define LEXY_UNICODE_PROPERTY(Name) ::lexy::_unicode_db::Name
#else
namespace lexy::_detail
{
template <int... Props>
bool code_point_has_properties(char32_t cp); // not implemented
} // namespace lexy::_detail
# define LEXY_UNICODE_PROPERTY(Name) 0
#endif
#endif // LEXY_CODE_POINT_HPP_INCLUDED

View File

@@ -0,0 +1,68 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_HPP_INCLUDED
#define LEXY_DSL_HPP_INCLUDED
#include <lexy/dsl/any.hpp>
#include <lexy/dsl/ascii.hpp>
#include <lexy/dsl/bits.hpp>
#include <lexy/dsl/bom.hpp>
#include <lexy/dsl/brackets.hpp>
#include <lexy/dsl/branch.hpp>
#include <lexy/dsl/byte.hpp>
#include <lexy/dsl/capture.hpp>
#include <lexy/dsl/case_folding.hpp>
#include <lexy/dsl/char_class.hpp>
#include <lexy/dsl/choice.hpp>
#include <lexy/dsl/code_point.hpp>
#include <lexy/dsl/combination.hpp>
#include <lexy/dsl/context_counter.hpp>
#include <lexy/dsl/context_flag.hpp>
#include <lexy/dsl/context_identifier.hpp>
#include <lexy/dsl/delimited.hpp>
#include <lexy/dsl/digit.hpp>
#include <lexy/dsl/effect.hpp>
#include <lexy/dsl/eof.hpp>
#include <lexy/dsl/error.hpp>
#include <lexy/dsl/expression.hpp>
#include <lexy/dsl/flags.hpp>
#include <lexy/dsl/follow.hpp>
#include <lexy/dsl/identifier.hpp>
#include <lexy/dsl/if.hpp>
#include <lexy/dsl/integer.hpp>
#include <lexy/dsl/list.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/dsl/lookahead.hpp>
#include <lexy/dsl/loop.hpp>
#include <lexy/dsl/member.hpp>
#include <lexy/dsl/newline.hpp>
#include <lexy/dsl/operator.hpp>
#include <lexy/dsl/option.hpp>
#include <lexy/dsl/parse_as.hpp>
#include <lexy/dsl/peek.hpp>
#include <lexy/dsl/position.hpp>
#include <lexy/dsl/production.hpp>
#include <lexy/dsl/punctuator.hpp>
#include <lexy/dsl/recover.hpp>
#include <lexy/dsl/repeat.hpp>
#include <lexy/dsl/return.hpp>
#include <lexy/dsl/scan.hpp>
#include <lexy/dsl/separator.hpp>
#include <lexy/dsl/sequence.hpp>
#include <lexy/dsl/sign.hpp>
#include <lexy/dsl/subgrammar.hpp>
#include <lexy/dsl/symbol.hpp>
#include <lexy/dsl/terminator.hpp>
#include <lexy/dsl/times.hpp>
#include <lexy/dsl/token.hpp>
#include <lexy/dsl/unicode.hpp>
#include <lexy/dsl/until.hpp>
#include <lexy/dsl/whitespace.hpp>
#if LEXY_EXPERIMENTAL
# include <lexy/dsl/parse_tree_node.hpp>
#endif
#endif // LEXY_DSL_HPP_INCLUDED

View File

@@ -0,0 +1,52 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_ANY_HPP_INCLUDED
#define LEXY_DSL_ANY_HPP_INCLUDED
#include <lexy/_detail/swar.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/token.hpp>
namespace lexyd
{
struct _any : token_base<_any, unconditional_branch_base>
{
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr std::true_type try_parse(Reader reader)
{
using encoding = typename Reader::encoding;
if constexpr (lexy::_detail::is_swar_reader<Reader>)
{
while (!lexy::_detail::swar_has_char<typename encoding::char_type, encoding::eof()>(
reader.peek_swar()))
reader.bump_swar();
}
while (reader.peek() != encoding::eof())
reader.bump();
end = reader.current();
return {};
}
};
};
/// Matches anything and consumes all remaining characters.
constexpr auto any = _any{};
} // namespace lexyd
namespace lexy
{
template <>
inline constexpr auto token_kind_of<lexy::dsl::_any> = lexy::any_token_kind;
}
#endif // LEXY_DSL_ANY_HPP_INCLUDED

View File

@@ -0,0 +1,514 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_ASCII_HPP_INCLUDED
#define LEXY_DSL_ASCII_HPP_INCLUDED
#include <lexy/_detail/nttp_string.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/char_class.hpp>
// SWAR tricks inspired by https://garbagecollected.org/2017/01/31/four-column-ascii/.
namespace lexyd::ascii
{
//=== control ===//
struct _control : char_class_base<_control>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.control";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert(0x00, 0x1F);
result.insert(0x7F);
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
using char_type = typename Encoding::char_type;
constexpr auto mask = lexy::_detail::swar_fill_compl(char_type(0b11111));
constexpr auto expected = lexy::_detail::swar_fill(char_type(0b00'00000));
// We're only checking for 0x00-0x1F, and allow a false negative for 0x7F.
return (c & mask) == expected;
}
};
inline constexpr auto control = _control{};
//=== whitespace ===//
struct _blank : char_class_base<_blank>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.blank";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert(' ');
result.insert('\t');
return result;
}
};
inline constexpr auto blank = _blank{};
struct _newline : char_class_base<_newline>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.newline";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('\n');
result.insert('\r');
return result;
}
};
inline constexpr auto newline = _newline{};
struct _other_space : char_class_base<_other_space>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.other-space";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('\f');
result.insert('\v');
return result;
}
};
inline constexpr auto other_space = _other_space{};
struct _space : char_class_base<_space>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.space";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert(_blank::char_class_ascii());
result.insert(_newline::char_class_ascii());
result.insert(_other_space::char_class_ascii());
return result;
}
};
inline constexpr auto space = _space{};
//=== alpha ===//
struct _lower : char_class_base<_lower>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.lower";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('a', 'z');
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
using char_type = typename Encoding::char_type;
// All interesting characters are in column 4.
constexpr auto mask = lexy::_detail::swar_fill_compl(char_type(0b11111));
constexpr auto expected = lexy::_detail::swar_fill(char_type(0b11'00000));
// But we need to eliminate ~ at the beginning and {|}~\x7F at the end.
constexpr auto offset_low = lexy::_detail::swar_fill(char_type(1));
constexpr auto offset_high = lexy::_detail::swar_fill(char_type(5));
return ((c - offset_low) & mask) == expected && ((c + offset_high) & mask) == expected;
}
};
inline constexpr auto lower = _lower{};
struct _upper : char_class_base<_upper>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.upper";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('A', 'Z');
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
using char_type = typename Encoding::char_type;
// All interesting characters are in column 3.
constexpr auto mask = lexy::_detail::swar_fill_compl(char_type(0b11111));
constexpr auto expected = lexy::_detail::swar_fill(char_type(0b10'00000));
// But we need to eliminate @ at the beginning and [\]^_ at the end.
constexpr auto offset_low = lexy::_detail::swar_fill(char_type(1));
constexpr auto offset_high = lexy::_detail::swar_fill(char_type(5));
return ((c - offset_low) & mask) == expected && ((c + offset_high) & mask) == expected;
}
};
inline constexpr auto upper = _upper{};
struct _alpha : char_class_base<_alpha>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.alpha";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('a', 'z');
result.insert('A', 'Z');
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
// We're assuming lower characters are more common, so do the efficient check only for them.
return _lower::template char_class_match_swar<Encoding>(c);
}
};
inline constexpr auto alpha = _alpha{};
struct _alphau : char_class_base<_alphau>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.alpha-underscore";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('a', 'z');
result.insert('A', 'Z');
result.insert('_');
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
// We're assuming alpha characters are more common, so do the efficient check only for them.
return _alpha::template char_class_match_swar<Encoding>(c);
}
};
inline constexpr auto alpha_underscore = _alphau{};
//=== digit ===//
struct _digit : char_class_base<_digit>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.digit";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('0', '9');
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
using char_type = typename Encoding::char_type;
// All interesting characters are in the second half of column 1.
constexpr auto mask = lexy::_detail::swar_fill_compl(char_type(0b01111));
constexpr auto expected = lexy::_detail::swar_fill(char_type(0b01'10000));
// But we need to eliminate :;<=>? at the end.
constexpr auto offset_high = lexy::_detail::swar_fill(char_type(6));
return (c & mask) == expected && ((c + offset_high) & mask) == expected;
}
};
inline constexpr auto digit = _digit{};
struct _alnum : char_class_base<_alnum>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.alpha-digit";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert(_alpha::char_class_ascii());
result.insert(_digit::char_class_ascii());
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
// We're assuming alpha characters are more common, so do the efficient check only for them.
return _alpha::template char_class_match_swar<Encoding>(c);
}
};
inline constexpr auto alnum = _alnum{};
inline constexpr auto alpha_digit = _alnum{};
struct _word : char_class_base<_word>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.word";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert(_alphau::char_class_ascii());
result.insert(_digit::char_class_ascii());
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
// We're assuming alphau characters are more common, so do the efficient check only for
// them.
return _alphau::template char_class_match_swar<Encoding>(c);
}
};
inline constexpr auto word = _word{};
inline constexpr auto alpha_digit_underscore = _word{};
//=== punct ===//
struct _punct : char_class_base<_punct>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.punct";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('!');
result.insert('"');
result.insert('#');
result.insert('$');
result.insert('%');
result.insert('&');
result.insert('\'');
result.insert('(');
result.insert(')');
result.insert('*');
result.insert('+');
result.insert(',');
result.insert('-');
result.insert('.');
result.insert('/');
result.insert(':');
result.insert(';');
result.insert('<');
result.insert('=');
result.insert('>');
result.insert('?');
result.insert('@');
result.insert('[');
result.insert('\\');
result.insert(']');
result.insert('^');
result.insert('_');
result.insert('`');
result.insert('{');
result.insert('|');
result.insert('}');
result.insert('~');
return result;
}
};
inline constexpr auto punct = _punct{};
//=== categories ===//
struct _graph : char_class_base<_graph>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.graph";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert(0x21, 0x7E);
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
using char_type = typename Encoding::char_type;
// First check that we have only ASCII, but shifted by one, so we also exclude 0x7F.
constexpr auto ascii_mask = lexy::_detail::swar_fill_compl(char_type(0b11'11111));
constexpr auto ascii_offset = lexy::_detail::swar_fill(char_type(1));
constexpr auto ascii_expected = lexy::_detail::swar_fill(char_type(0));
if (((c + ascii_offset) & ascii_mask) != ascii_expected)
return false;
// The above check also included 0xFF for single byte encodings where it overflowed,
// so do a separate check in those cases.
if constexpr (sizeof(char_type) == 1)
{
if ((c & ascii_mask) != ascii_expected)
return false;
}
// Then we must not have a character in column 0, or space.
// If we subtract one we turn 0x21-0x01 into column 0 and 0x00 to a value definitely not in
// column 0, so need to check both.
constexpr auto mask = lexy::_detail::swar_fill_compl(char_type(0b11111));
constexpr auto offset_low = lexy::_detail::swar_fill(char_type(1));
return !lexy::_detail::swar_has_zero<char_type>(c & mask)
&& !lexy::_detail::swar_has_zero<char_type>((c - offset_low) & mask);
}
};
inline constexpr auto graph = _graph{};
struct _print : char_class_base<_print>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII.print";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert(0x20, 0x7E);
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
using char_type = typename Encoding::char_type;
// First check that we have only ASCII, but shifted by one, so we also exclude 0x7F.
constexpr auto ascii_mask = lexy::_detail::swar_fill_compl(char_type(0b11'11111));
constexpr auto ascii_offset = lexy::_detail::swar_fill(char_type(1));
constexpr auto ascii_expected = lexy::_detail::swar_fill(char_type(0));
if (((c + ascii_offset) & ascii_mask) != ascii_expected)
return false;
// The above check also included 0xFF for single byte encodings where it overflowed,
// so do a separate check in those cases.
if constexpr (sizeof(char_type) == 1)
{
if ((c & ascii_mask) != ascii_expected)
return false;
}
// Then we must not have a character in column 0.
constexpr auto mask = lexy::_detail::swar_fill_compl(char_type(0b11111));
return !lexy::_detail::swar_has_zero<char_type>(c & mask);
}
};
inline constexpr auto print = _print{};
struct _char : char_class_base<_char>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "ASCII";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert(0x00, 0x7F);
return result;
}
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int c)
{
using char_type = typename Encoding::char_type;
constexpr auto mask = lexy::_detail::swar_fill_compl(char_type(0b11'11111));
constexpr auto expected = lexy::_detail::swar_fill(char_type(0));
return (c & mask) == expected;
}
};
inline constexpr auto character = _char{};
} // namespace lexyd::ascii
namespace lexyd::ascii
{
template <char... C>
struct _alt : char_class_base<_alt<C...>>
{
static_assert(sizeof...(C) > 0);
static LEXY_CONSTEVAL auto char_class_name()
{
return lexy::_detail::type_string<char, C...>::template c_str<char>;
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
(result.insert(C), ...);
return result;
}
};
template <typename CharT, CharT... C>
struct _one_of
{
static_assert((std::is_same_v<CharT, char> && ... && lexy::_detail::is_ascii(C)),
"only ASCII characters are supported");
using rule = _alt<C...>;
};
#if LEXY_HAS_NTTP
/// Matches one of the ASCII characters.
template <lexy::_detail::string_literal Str>
constexpr auto one_of = typename lexy::_detail::to_type_string<_one_of, Str>::rule{};
#endif
#define LEXY_ASCII_ONE_OF(Str) \
LEXY_NTTP_STRING(::lexyd::ascii::_one_of, Str)::rule {}
} // namespace lexyd::ascii
#endif // LEXY_DSL_ASCII_HPP_INCLUDED

View File

@@ -0,0 +1,273 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_BASE_HPP_INCLUDED
#define LEXY_DSL_BASE_HPP_INCLUDED
#include <lexy/_detail/config.hpp>
#include <lexy/_detail/lazy_init.hpp>
#include <lexy/grammar.hpp>
#include <lexy/input/base.hpp>
//=== parse_events ===//
namespace lexy::parse_events
{
/// Parsing started.
/// Arguments: position
struct grammar_start
{};
/// Parsing finished succesfully.
/// Arguments: the reader at the final parse position.
struct grammar_finish
{};
/// Parsing finished unsuccesfully.
/// Arguments: the reader at the final parse position.
struct grammar_cancel
{};
/// Start of the current production.
/// Arguments: position
struct production_start
{};
/// End of the current production.
/// Arguments: position
struct production_finish
{};
/// Production is canceled.
/// Arguments: position
struct production_cancel
{};
/// Start of a chain of left-associative operations.
/// Arguments: position
/// Returns: a handle that needs to be passed to finish.
struct operation_chain_start
{};
/// Operation inside a chain.
/// Arguments: operation, position
struct operation_chain_op
{};
/// End of a chain of operations.
/// Arguments: handle, position
struct operation_chain_finish
{};
/// A token was consumed.
/// Arguments: kind, begin, end
struct token
{};
/// The input backtracked from end to begin.
/// Only meaningful for begin != end.
/// Arguments: begin, end
struct backtracked
{};
/// A parse error occurrs.
/// Arguments: error object
struct error
{};
/// Non-trivial error recovery started,
/// i.e. it is currently discarding input.
/// Arguments: position
struct recovery_start
{};
/// Non-trivial error recovery succeeded.
/// It will now continue with normal parsing.
/// Arguments: position
struct recovery_finish
{};
/// Non-trivial error recovery failed because it reaches the limit.
/// It will now cancel until the next recovery point.
/// Arguments: position
struct recovery_cancel
{};
} // namespace lexy::parse_events
namespace lexyd
{
namespace _ev = lexy::parse_events;
// Does not copy token tags.
template <typename Rule>
auto _copy_base_impl()
{
if constexpr (lexy::is_unconditional_branch_rule<Rule>)
return unconditional_branch_base{};
else if constexpr (lexy::is_branch_rule<Rule>)
return branch_base{};
else
return rule_base{};
}
template <typename Rule>
using _copy_base = decltype(_copy_base_impl<Rule>());
} // namespace lexyd
//=== parser ===//
#define LEXY_PARSER_FUNC LEXY_FORCE_INLINE constexpr
namespace lexy
{
template <typename Rule, typename NextParser>
using parser_for = typename Rule::template p<NextParser>;
template <typename BranchRule, typename Reader>
using branch_parser_for = typename BranchRule::template bp<Reader>;
template <typename Production, typename Reader>
struct _pb : lexy::branch_parser_for<lexy::production_rule<Production>, Reader>
{};
// We create a new type here to shorten its name.
template <typename Production, typename Reader>
using production_branch_parser = _pb<Production, Reader>;
/// A branch parser that takes a branch unconditionally and forwards to the regular parser.
template <typename Rule, typename Reader>
struct unconditional_branch_parser
{
constexpr std::true_type try_parse(const void*, const Reader&)
{
return {};
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
return parser_for<Rule, NextParser>::parse(context, reader, LEXY_FWD(args)...);
}
};
/// A branch parser that parses a branch rule but with a special continuation.
template <typename BranchRule, typename Reader, template <typename> typename Continuation>
struct continuation_branch_parser
{
branch_parser_for<BranchRule, Reader> impl;
template <typename ControlBlock>
constexpr auto try_parse(const ControlBlock* cb, const Reader& reader)
{
return impl.try_parse(cb, reader);
}
template <typename Context>
constexpr void cancel(Context& context)
{
impl.cancel(context);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
return impl.template finish<Continuation<NextParser>>(context, reader, LEXY_FWD(args)...);
}
};
/// A parser that does not support any arguments.
template <typename... PrevArgs>
struct pattern_parser
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static std::true_type parse(Context&, Reader&, const PrevArgs&..., Args&&...)
{
// A rule is used inside a loop or similar situation, where it must not produce values, but
// it did.
static_assert(sizeof...(Args) == 0, "pattern rule must not produce any values");
return {};
}
};
/// A parser that forwards all arguments to a sink, which is the first argument.
struct sink_parser
{
template <typename Context, typename Reader, typename Sink, typename... Args>
LEXY_PARSER_FUNC static std::true_type parse(Context&, Reader&, Sink& sink, Args&&... args)
{
if constexpr (sizeof...(Args) > 0)
sink(LEXY_FWD(args)...);
return {};
}
};
/// A parser that finishes a sink and continues with the next one.
template <typename NextParser>
struct sink_finish_parser
{
template <typename Context, typename Reader, typename Sink, typename... Args>
LEXY_PARSER_FUNC static auto parse(Context& context, Reader& reader, Sink& sink, Args&&... args)
{
if constexpr (std::is_same_v<typename Sink::return_type, void>)
{
LEXY_MOV(sink).finish();
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
else
{
return NextParser::parse(context, reader, LEXY_FWD(args)..., LEXY_MOV(sink).finish());
}
}
};
} // namespace lexy
//=== whitespace ===//
namespace lexy::_detail
{
template <typename NextParser>
struct automatic_ws_parser;
} // namespace lexy::_detail
namespace lexy
{
template <typename Context, typename NextParser,
typename = lexy::production_whitespace<typename Context::production,
typename Context::whitespace_production>>
struct whitespace_parser : _detail::automatic_ws_parser<NextParser>
{};
// If we know the whitespace rule is void, go to NextParser immediately.
// This is both an optimization and also doesn't require inclusion of whitespace.hpp.
template <typename Context, typename NextParser>
struct whitespace_parser<Context, NextParser, void> : NextParser
{};
} // namespace lexy
//=== token parser ===//
namespace lexy
{
template <typename TokenRule, typename Reader>
using token_parser_for = typename TokenRule::template tp<Reader>;
template <typename TokenRule, typename Reader>
LEXY_FORCE_INLINE constexpr auto try_match_token(TokenRule, Reader& reader)
{
lexy::token_parser_for<TokenRule, Reader> parser(reader);
using try_parse_result = decltype(parser.try_parse(reader));
if constexpr (std::is_same_v<try_parse_result, std::true_type>)
{
parser.try_parse(reader);
reader.reset(parser.end);
return std::true_type{};
}
else if constexpr (std::is_same_v<try_parse_result, std::false_type>)
{
// try_parse() is pure and we don't want to advance the reader, so no need to call it.
return std::false_type{};
}
else
{
if (!parser.try_parse(reader))
return false;
reader.reset(parser.end);
return true;
}
}
} // namespace lexy
#endif // LEXY_DSL_BASE_HPP_INCLUDED

View File

@@ -0,0 +1,152 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_BITS_HPP_INCLUDED
#define LEXY_DSL_BITS_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/token.hpp>
//=== bit rules ===//
namespace lexyd::bit
{
struct _bit_pattern
{
unsigned mask;
unsigned value;
};
struct _bit_rule
{};
struct _b0 : _bit_rule
{
static constexpr auto size = 1u;
static constexpr void apply(_bit_pattern& p)
{
p.mask <<= 1;
p.value <<= 1;
p.mask |= 1;
}
};
/// Matches a 0 bit.
inline constexpr auto _0 = _b0{};
struct _b1 : _bit_rule
{
static constexpr auto size = 1u;
static constexpr void apply(_bit_pattern& p)
{
p.mask <<= 1;
p.value <<= 1;
p.mask |= 1;
p.value |= 1;
}
};
/// Matches a 1 bit.
inline constexpr auto _1 = _b1{};
template <unsigned Value>
struct _n : _bit_rule
{
static_assert(Value <= 0xF);
static constexpr auto size = 4u;
static constexpr void apply(_bit_pattern& p)
{
p.mask <<= 4;
p.value <<= 4;
p.mask |= 0b1111;
p.value |= Value;
}
};
/// Matches a specific nibble, i.e. four bits.
template <unsigned Value>
constexpr auto nibble = _n<Value>{};
template <unsigned N>
struct _b : _bit_rule
{
static_assert(N > 0);
static constexpr auto size = N;
static constexpr void apply(_bit_pattern& p)
{
p.mask <<= N;
p.value <<= N;
}
};
/// Matches any bit.
inline constexpr auto _ = _b<1>{};
/// Matches N arbitrary bits.
template <unsigned N>
constexpr auto any = _b<N>{};
} // namespace lexyd::bit
//=== bits ===//
namespace lexyd
{
template <unsigned Mask, unsigned Value>
struct _bits : token_base<_bits<Mask, Value>>
{
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
static_assert(lexy::is_byte_encoding<typename Reader::encoding>);
auto byte = reader.peek();
if (byte == Reader::encoding::eof()
|| ((static_cast<unsigned char>(byte) & Mask) != Value))
return false;
reader.bump();
end = reader.current();
return true;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader&)
{
auto err = lexy::error<Reader, lexy::expected_char_class>(end.position(), "bits");
context.on(_ev::error{}, err);
}
};
};
/// Matches the specific bit pattern.
template <typename... Bits>
constexpr auto bits(Bits...)
{
static_assert((std::is_base_of_v<bit::_bit_rule, Bits> && ...), "bits() requires bit rules");
static_assert((0 + ... + Bits::size) == 8, "must specify 8 bit at a time");
constexpr auto pattern = [] {
bit::_bit_pattern result{0, 0};
(Bits::apply(result), ...);
return result;
}();
return _bits<pattern.mask, pattern.value>{};
}
} // namespace lexyd
#endif // LEXY_DSL_BITS_HPP_INCLUDED

View File

@@ -0,0 +1,47 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_BOM_HPP_INCLUDED
#define LEXY_DSL_BOM_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/dsl/token.hpp>
namespace lexyd
{
template <typename Encoding, lexy::encoding_endianness Endianness>
struct _bom : _lit<unsigned char>
{};
template <lexy::encoding_endianness DontCare>
struct _bom<lexy::utf8_encoding, DontCare> //
: _lit<unsigned char, 0xEF, 0xBB, 0xBF>
{};
template <lexy::encoding_endianness DontCare>
struct _bom<lexy::utf8_char_encoding, DontCare> //
: _lit<unsigned char, 0xEF, 0xBB, 0xBF>
{};
template <>
struct _bom<lexy::utf16_encoding, lexy::encoding_endianness::little>
: _lit<unsigned char, 0xFF, 0xFE>
{};
template <>
struct _bom<lexy::utf16_encoding, lexy::encoding_endianness::big> //
: _lit<unsigned char, 0xFE, 0xFF>
{};
template <>
struct _bom<lexy::utf32_encoding, lexy::encoding_endianness::little>
: _lit<unsigned char, 0xFF, 0xFE, 0x00, 0x00>
{};
template <>
struct _bom<lexy::utf32_encoding, lexy::encoding_endianness::big>
: _lit<unsigned char, 0x00, 0x00, 0xFE, 0xFF>
{};
/// The BOM for that particular encoding.
template <typename Encoding, lexy::encoding_endianness Endianness>
inline constexpr auto bom = _bom<Encoding, Endianness>{};
} // namespace lexyd
#endif // LEXY_DSL_BOM_HPP_INCLUDED

View File

@@ -0,0 +1,117 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_BRACKETS_HPP_INCLUDED
#define LEXY_DSL_BRACKETS_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/dsl/terminator.hpp>
namespace lexyd
{
template <typename Open, typename Close, typename RecoveryLimit = void>
struct _brackets
{
/// Adds the literal tokens to the recovery limit.
template <typename... Literals>
constexpr auto limit(Literals... literals) const
{
static_assert(sizeof...(Literals) > 0);
auto l = (recovery_rule().get_limit() / ... / literals);
return _brackets<Open, Close, decltype(l)>{};
}
//=== rules ===//
/// Matches the rule surrounded by brackets.
template <typename R>
constexpr auto operator()(R r) const
{
return open() >> as_terminator()(r);
}
/// Matches the rule surrounded by brackets, recovering on error.
template <typename R>
constexpr auto try_(R r) const
{
return open() >> as_terminator().try_(r);
}
/// Matches `opt(r)` surrounded by brackets.
/// The rule does not require a condition.
template <typename R>
constexpr auto opt(R r) const
{
return open() >> as_terminator().opt(r);
}
/// Matches `list(r, sep)` surrounded by brackets.
/// The rule does not require a condition.
template <typename R>
constexpr auto list(R r) const
{
return open() >> as_terminator().list(r);
}
template <typename R, typename S>
constexpr auto list(R r, S sep) const
{
return open() >> as_terminator().list(r, sep);
}
/// Matches `opt_list(r, sep)` surrounded by brackets.
/// The rule does not require a condition.
template <typename R>
constexpr auto opt_list(R r) const
{
return open() >> as_terminator().opt_list(r);
}
template <typename R, typename S>
constexpr auto opt_list(R r, S sep) const
{
return open() >> as_terminator().opt_list(r, sep);
}
//=== access ===//
/// Matches the open bracket.
constexpr auto open() const
{
return Open{};
}
/// Matches the closing bracket.
constexpr auto close() const
{
return Close{};
}
/// Returns an equivalent terminator.
constexpr auto as_terminator() const
{
return _term<Close, RecoveryLimit>{};
}
constexpr auto recovery_rule() const
{
return as_terminator().recovery_rule();
}
};
/// Defines open and close brackets.
template <typename Open, typename Close>
constexpr auto brackets(Open, Close)
{
LEXY_REQUIRE_BRANCH_RULE(Open, "brackets()");
LEXY_REQUIRE_BRANCH_RULE(Close, "brackets()");
return _brackets<Open, Close>{};
}
constexpr auto round_bracketed = brackets(lit_c<'('>, lit_c<')'>);
constexpr auto square_bracketed = brackets(lit_c<'['>, lit_c<']'>);
constexpr auto curly_bracketed = brackets(lit_c<'{'>, lit_c<'}'>);
constexpr auto angle_bracketed = brackets(lit_c<'<'>, lit_c<'>'>);
constexpr auto parenthesized = round_bracketed;
} // namespace lexyd
#endif // LEXY_DSL_BRACKETS_HPP_INCLUDED

View File

@@ -0,0 +1,147 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_BRANCH_HPP_INCLUDED
#define LEXY_DSL_BRANCH_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/sequence.hpp>
namespace lexyd
{
template <typename Condition, typename... R>
struct _br : _copy_base<Condition>
{
static_assert(sizeof...(R) >= 0);
template <typename NextParser>
using _pc = lexy::parser_for<_seq_impl<R...>, NextParser>;
// We parse Condition and then seq<R...>.
// Condition's try_parse() checks the branch condition, which is what we want.
template <typename Reader>
using bp = lexy::continuation_branch_parser<Condition, Reader, _pc>;
template <typename NextParser>
using p = lexy::parser_for<_seq_impl<Condition, R...>, NextParser>;
};
//=== operator>> ===//
/// Parses `Then` only after `Condition` has matched.
template <typename Condition, typename Then>
constexpr auto operator>>(Condition, Then)
{
LEXY_REQUIRE_BRANCH_RULE(Condition, "Left-hand-side of >>");
return _br<Condition, Then>{};
}
template <typename Condition, typename... R>
constexpr auto operator>>(Condition, _seq<R...>)
{
LEXY_REQUIRE_BRANCH_RULE(Condition, "Left-hand-side of >>");
return _br<Condition, R...>{};
}
template <typename Condition, typename C, typename... R>
constexpr auto operator>>(Condition, _br<C, R...>)
{
LEXY_REQUIRE_BRANCH_RULE(Condition, "Left-hand-side of >>");
return _br<Condition, C, R...>{};
}
// Prevent nested branches in `_br`'s condition.
template <typename C, typename... R, typename Then>
constexpr auto operator>>(_br<C, R...>, Then)
{
return C{} >> _seq<R..., Then>{};
}
template <typename C, typename... R, typename... S>
constexpr auto operator>>(_br<C, R...>, _seq<S...>)
{
return C{} >> _seq<R..., S...>{};
}
// Disambiguation.
template <typename C1, typename... R, typename C2, typename... S>
constexpr auto operator>>(_br<C1, R...>, _br<C2, S...>)
{
return _br<C1, R..., C2, S...>{};
}
//=== operator+ ===//
// If we add something on the left to a branch, we loose the branchy-ness.
template <typename Rule, typename Condition, typename... R>
constexpr auto operator+(Rule rule, _br<Condition, R...>)
{
return rule + _seq<Condition, R...>{};
}
// Disambiguation.
template <typename... R, typename Condition, typename... S>
constexpr auto operator+(_seq<R...>, _br<Condition, S...>)
{
return _seq<R...>{} + _seq<Condition, S...>{};
}
// If we add something on the right to a branch, we extend the then.
template <typename Condition, typename... R, typename Rule>
constexpr auto operator+(_br<Condition, R...>, Rule)
{
return _br<Condition, R..., Rule>{};
}
// Disambiguation.
template <typename Condition, typename... R, typename... S>
constexpr auto operator+(_br<Condition, R...>, _seq<S...>)
{
return _br<Condition, R..., S...>{};
}
// If we add two branches, we use the condition of the first one and treat the second as sequence.
template <typename C1, typename... R, typename C2, typename... S>
constexpr auto operator+(_br<C1, R...>, _br<C2, S...>)
{
return _br<C1, R..., C2, S...>{};
}
template <typename Condition, typename Then>
constexpr auto _maybe_branch(Condition condition, Then then)
{
if constexpr (lexy::is_branch_rule<Condition>)
return condition >> then;
else
return condition + then;
}
} // namespace lexyd
namespace lexyd
{
struct _else : unconditional_branch_base
{
template <typename NextParser>
using p = NextParser;
template <typename Reader>
using bp = lexy::unconditional_branch_parser<_else, Reader>;
};
struct _else_dsl
{
template <typename R>
friend constexpr auto operator>>(_else_dsl, R rule)
{
return _else{} >> rule;
}
template <typename... R>
friend constexpr auto operator>>(_else_dsl, _seq<R...> rule)
{
return _else{} >> rule;
}
template <typename C, typename... R>
friend constexpr auto operator>>(_else_dsl, _br<C, R...> rule)
{
return _else{} >> rule;
}
};
/// Takes the branch unconditionally.
inline constexpr auto else_ = _else_dsl{};
} // namespace lexyd
#endif // LEXY_DSL_BRANCH_HPP_INCLUDED

View File

@@ -0,0 +1,399 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_BYTE_HPP_INCLUDED
#define LEXY_DSL_BYTE_HPP_INCLUDED
#include <cstdint>
#include <lexy/_detail/integer_sequence.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/char_class.hpp>
#include <lexy/dsl/token.hpp>
//=== byte ===//
namespace lexyd
{
template <std::size_t N, typename Predicate>
struct _b : token_base<_b<N, Predicate>>
{
static_assert(N > 0);
static constexpr bool _match(lexy::byte_encoding::int_type cur)
{
if (cur == lexy::byte_encoding::eof())
return false;
if constexpr (!std::is_void_v<Predicate>)
{
constexpr auto predicate = Predicate{};
return predicate(static_cast<lexy::byte_encoding::char_type>(cur));
}
else
{
return true;
}
}
template <typename Reader, typename Indices = lexy::_detail::make_index_sequence<N>>
struct tp;
template <typename Reader, std::size_t... Idx>
struct tp<Reader, lexy::_detail::index_sequence<Idx...>>
{
static_assert(lexy::is_byte_encoding<typename Reader::encoding>);
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
// Bump N times.
auto result
= ((_match(reader.peek()) ? (reader.bump(), true) : ((void)Idx, false)) && ...);
end = reader.current();
return result;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader&)
{
constexpr auto name
= std::is_void_v<Predicate> ? "byte" : lexy::_detail::type_name<Predicate>();
auto err = lexy::error<Reader, lexy::expected_char_class>(end.position(), name);
context.on(_ev::error{}, err);
}
};
//=== dsl ===//
template <typename P>
constexpr auto if_() const
{
static_assert(std::is_void_v<Predicate>);
return _b<N, P>{};
}
template <unsigned char Low, unsigned char High>
constexpr auto range() const
{
struct predicate
{
static LEXY_CONSTEVAL auto name()
{
return "byte.range";
}
constexpr bool operator()(unsigned char byte) const
{
return Low <= byte && byte <= High;
}
};
return if_<predicate>();
}
template <unsigned char... Bytes>
constexpr auto set() const
{
struct predicate
{
static LEXY_CONSTEVAL auto name()
{
return "byte.set";
}
constexpr bool operator()(unsigned char byte) const
{
return ((byte == Bytes) || ...);
}
};
return if_<predicate>();
}
constexpr auto ascii() const
{
struct predicate
{
static LEXY_CONSTEVAL auto name()
{
return "byte.ASCII";
}
constexpr bool operator()(unsigned char byte) const
{
return byte <= 0x7F;
}
};
return if_<predicate>();
}
};
/// Matches an arbitrary byte.
constexpr auto byte = _b<1, void>{};
/// Matches N arbitrary bytes.
template <std::size_t N>
constexpr auto bytes = _b<N, void>{};
} // namespace lexyd
namespace lexy
{
template <std::size_t N>
constexpr auto token_kind_of<lexy::dsl::_b<N, void>> = lexy::any_token_kind;
} // namespace lexy
//=== padding bytes ===//
namespace lexyd
{
template <std::size_t N, unsigned char Padding = 0>
struct _pb : branch_base
{
template <typename Context, typename Reader, typename Iterator>
static constexpr void _validate(Context& context, const Reader&, Iterator begin, Iterator end)
{
constexpr unsigned char str[] = {Padding, 0};
for (auto iter = begin; iter != end; ++iter)
{
if (*iter != Padding)
{
auto err = lexy::error<Reader, lexy::expected_literal>(iter, str, 0, 1);
context.on(_ev::error{}, err);
}
}
}
template <typename Reader>
struct bp
{
static_assert(lexy::is_byte_encoding<typename Reader::encoding>);
typename Reader::marker end;
constexpr auto try_parse(const void*, const Reader& reader)
{
lexy::token_parser_for<_b<N, void>, Reader> parser(reader);
auto result = parser.try_parse(reader);
end = parser.end;
return result;
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC auto finish(Context& context, Reader& reader, Args&&... args)
{
auto begin = reader.position();
context.on(_ev::token{}, lexy::any_token_kind, begin, end.position());
reader.reset(end);
_validate(context, reader, begin, end.position());
return lexy::whitespace_parser<Context, NextParser>::parse(context, reader,
LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
static_assert(lexy::is_byte_encoding<typename Reader::encoding>);
auto begin = reader.position();
if (!_b<N, void>::token_parse(context, reader))
return false;
auto end = reader.position();
_validate(context, reader, begin, end);
return lexy::whitespace_parser<Context, NextParser>::parse(context, reader,
LEXY_FWD(args)...);
}
};
};
/// Matches N bytes set to the padding value.
/// It is a recoverable error if the byte doesn't have that value.
template <std::size_t N, unsigned char Padding = 0>
constexpr auto padding_bytes = _pb<N, Padding>{};
} // namespace lexyd
//=== bint ===//
namespace lexy::_detail
{
enum bint_endianness
{
bint_little,
bint_big,
#if LEXY_IS_LITTLE_ENDIAN
bint_native = bint_little,
#else
bint_native = bint_big,
#endif
};
template <std::size_t N>
auto _bint()
{
if constexpr (N == 1)
return std::uint_least8_t(0);
else if constexpr (N == 2)
return std::uint_least16_t(0);
else if constexpr (N == 4)
return std::uint_least32_t(0);
else if constexpr (N == 8)
return std::uint_least64_t(0);
else
{
static_assert(N == 1, "invalid byte integer size");
return 0;
}
}
template <std::size_t N>
using bint = decltype(_bint<N>());
} // namespace lexy::_detail
namespace lexy
{
struct mismatched_byte_count
{
static LEXY_CONSTEVAL auto name()
{
return "mismatched byte count";
}
};
} // namespace lexy
namespace lexyd
{
template <std::size_t N, int Endianness, typename Rule = void>
struct _bint : branch_base
{
using _rule = lexy::_detail::type_or<Rule, _b<N, void>>;
template <typename NextParser, typename Indices = lexy::_detail::make_index_sequence<N>>
struct _pc;
template <typename NextParser, std::size_t... Idx>
struct _pc<NextParser, lexy::_detail::index_sequence<Idx...>>
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader,
typename Reader::iterator begin,
typename Reader::iterator end, Args&&... args)
{
if (lexy::_detail::range_size(begin, end) != N)
{
auto err = lexy::error<Reader, lexy::mismatched_byte_count>(begin, end);
context.on(_ev::error{}, err);
return false;
}
lexy::_detail::bint<N> result = 0;
if constexpr (N == 1)
{
// For a single byte, endianness doesn't matter.
result = static_cast<unsigned char>(*begin);
}
else if constexpr (Endianness == lexy::_detail::bint_big)
{
// In big endian, the first byte is shifted the most,
// so read in order.
((result = static_cast<decltype(result)>(result << 8),
result = static_cast<decltype(result)>(result | *begin++), (void)Idx),
...);
}
else
{
// In little endian, we need to reverse the order,
// so copy into a buffer to do that.
unsigned char buffer[N] = {((void)Idx, *begin++)...};
((result = static_cast<decltype(result)>(result << 8),
result = static_cast<decltype(result)>(result | buffer[N - Idx - 1])),
...);
}
return lexy::whitespace_parser<Context, NextParser>::parse(context, reader,
LEXY_FWD(args)..., result);
}
};
template <typename Reader>
struct bp
{
static_assert(lexy::is_byte_encoding<typename Reader::encoding>);
typename Reader::marker end;
constexpr auto try_parse(const void*, const Reader& reader)
{
lexy::token_parser_for<_rule, Reader> parser(reader);
auto result = parser.try_parse(reader);
end = parser.end;
return result;
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC auto finish(Context& context, Reader& reader, Args&&... args)
{
auto begin = reader.position();
context.on(_ev::token{}, _rule{}, begin, end.position());
reader.reset(end);
return _pc<NextParser>::parse(context, reader, begin, end.position(),
LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
static_assert(lexy::is_byte_encoding<typename Reader::encoding>);
auto begin = reader.position();
if (!_rule::token_parse(context, reader))
return false;
auto end = reader.position();
return _pc<NextParser>::parse(context, reader, begin, end, LEXY_FWD(args)...);
}
};
//=== dsl ===//
/// Matches a specific token rule instead of arbitrary bytes.
template <typename Token>
constexpr auto operator()(Token) const
{
static_assert(lexy::is_token_rule<Token>);
static_assert(std::is_void_v<Rule>);
return _bint<N, Endianness, Token>{};
}
};
/// Matches one byte and converts it into an 8-bit integer.
inline constexpr auto bint8 = _bint<1, lexy::_detail::bint_native>{};
/// Matches two bytes and converts it into an 16-bit integer.
inline constexpr auto bint16 = _bint<2, lexy::_detail::bint_native>{};
inline constexpr auto little_bint16 = _bint<2, lexy::_detail::bint_little>{};
inline constexpr auto big_bint16 = _bint<2, lexy::_detail::bint_big>{};
/// Matches four bytes and converts it into an 32-bit integer.
inline constexpr auto bint32 = _bint<4, lexy::_detail::bint_native>{};
inline constexpr auto little_bint32 = _bint<4, lexy::_detail::bint_little>{};
inline constexpr auto big_bint32 = _bint<4, lexy::_detail::bint_big>{};
/// Matches eight bytes and converts it into an 64-bit integer.
inline constexpr auto bint64 = _bint<8, lexy::_detail::bint_native>{};
inline constexpr auto little_bint64 = _bint<8, lexy::_detail::bint_little>{};
inline constexpr auto big_bint64 = _bint<8, lexy::_detail::bint_big>{};
} // namespace lexyd
#endif // LEXY_DSL_BYTE_HPP_INCLUDED

View File

@@ -0,0 +1,143 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_CAPTURE_HPP_INCLUDED
#define LEXY_DSL_CAPTURE_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/whitespace.hpp>
#include <lexy/lexeme.hpp>
namespace lexyd
{
template <typename Token>
struct _cap : _copy_base<Token>
{
template <typename Reader>
struct bp
{
typename Reader::marker end;
constexpr auto try_parse(const void*, const Reader& reader)
{
lexy::token_parser_for<Token, Reader> parser(reader);
auto result = parser.try_parse(reader);
end = parser.end;
return result;
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC auto finish(Context& context, Reader& reader, Args&&... args)
{
auto begin = reader.position();
context.on(_ev::token{}, Token{}, begin, end.position());
reader.reset(end);
using continuation = lexy::whitespace_parser<Context, NextParser>;
return continuation::parse(context, reader, LEXY_FWD(args)...,
lexy::lexeme<Reader>(begin, end.position()));
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
auto begin = reader.position();
if (!Token::token_parse(context, reader))
return false;
auto end = reader.position();
using continuation = lexy::whitespace_parser<Context, NextParser>;
return continuation::parse(context, reader, LEXY_FWD(args)...,
lexy::lexeme<Reader>(begin, end));
}
};
};
template <typename Rule>
struct _capr : _copy_base<Rule>
{
template <typename NextParser, typename... PrevArgs>
struct _pc : lexy::_detail::disable_whitespace_skipping
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader,
PrevArgs&&... prev_args, typename Reader::iterator begin,
Args&&... args)
{
using continuation = lexy::whitespace_parser<Context, NextParser>;
return continuation::parse(context, reader, LEXY_FWD(prev_args)...,
lexy::lexeme(reader, begin), LEXY_FWD(args)...);
}
};
template <typename Reader>
struct bp
{
lexy::branch_parser_for<Rule, Reader> rule;
template <typename ControlBlock>
constexpr auto try_parse(const ControlBlock* cb, const Reader& reader)
{
return rule.try_parse(cb, reader);
}
template <typename Context>
constexpr void cancel(Context& context)
{
rule.cancel(context);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC auto finish(Context& context, Reader& reader, Args&&... args)
{
// Forward to the rule, but remember the current reader position.
using continuation = _pc<NextParser, Args...>;
return rule.template finish<continuation>(context, reader, LEXY_FWD(args)...,
reader.position());
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
// Forward to the rule, but remember the current reader position.
using parser = lexy::parser_for<Rule, _pc<NextParser, Args...>>;
return parser::parse(context, reader, LEXY_FWD(args)..., reader.position());
}
};
};
template <typename Production>
struct _prd;
/// Captures whatever the token matches as a lexeme; does not include trailing whitespace.
template <typename Token>
constexpr auto capture(Token)
{
static_assert(lexy::is_token_rule<Token>);
return _cap<Token>{};
}
/// Captures whatever the token production matches as lexeme; does not include trailing whitespace.
template <typename Production>
constexpr auto capture(_prd<Production>)
{
static_assert(lexy::is_token_production<Production>);
return _capr<_prd<Production>>{};
}
} // namespace lexyd
#endif // LEXY_DSL_CAPTURE_HPP_INCLUDED

View File

@@ -0,0 +1,303 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_CASE_FOLDING_HPP_INCLUDED
#define LEXY_DSL_CASE_FOLDING_HPP_INCLUDED
#include <lexy/_detail/code_point.hpp>
#include <lexy/code_point.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/literal.hpp>
//=== generic rule impl ===//
namespace lexyd
{
template <template <typename> typename CaseFolding>
struct _cfl_folding
{
template <typename Reader>
using reader = CaseFolding<Reader>;
};
template <typename Literal, template <typename> typename CaseFolding>
struct _cfl : token_base<_cfl<Literal, CaseFolding>>, _lit_base
{
static constexpr auto lit_max_char_count = Literal::lit_max_char_count;
static constexpr auto lit_char_classes = Literal::lit_char_classes;
using lit_case_folding = _cfl_folding<CaseFolding>;
template <typename Encoding>
static constexpr auto lit_first_char() -> typename Encoding::char_type
{
return Literal::template lit_first_char<Encoding>();
}
template <typename Trie>
static LEXY_CONSTEVAL std::size_t lit_insert(Trie& trie, std::size_t pos,
std::size_t char_class)
{
return Literal::lit_insert(trie, pos, char_class);
}
template <typename Reader>
struct tp
{
lexy::token_parser_for<Literal, CaseFolding<Reader>> impl;
typename Reader::marker end;
constexpr explicit tp(const Reader& reader)
: impl(CaseFolding<Reader>{reader}), end(reader.current())
{}
constexpr bool try_parse(Reader _reader)
{
CaseFolding<Reader> reader{_reader};
auto result = impl.try_parse(reader);
end = impl.end;
return result;
}
template <typename Context>
constexpr void report_error(Context& context, Reader reader)
{
impl.report_error(context, CaseFolding<Reader>{reader});
}
};
};
} // namespace lexyd
namespace lexy
{
template <typename Literal, template <typename> typename CaseFolding>
constexpr auto token_kind_of<lexy::dsl::_cfl<Literal, CaseFolding>> = lexy::literal_token_kind;
} // namespace lexy
//=== ASCII ===//
namespace lexy
{
template <typename Reader>
struct _acfr // ascii case folding reader
{
Reader _impl;
using encoding = typename Reader::encoding;
using iterator = typename Reader::iterator;
using marker = typename Reader::marker;
constexpr auto peek() const -> typename encoding::int_type
{
auto c = _impl.peek();
if (encoding::to_int_type('A') <= c && c <= encoding::to_int_type('Z'))
return typename encoding::int_type(c + encoding::to_int_type('a' - 'A'));
else
return c;
}
constexpr void bump()
{
_impl.bump();
}
constexpr iterator position() const
{
return _impl.position();
}
constexpr marker current() const noexcept
{
return _impl.current();
}
constexpr void reset(marker m) noexcept
{
_impl.reset(m);
}
};
} // namespace lexy
namespace lexyd::ascii
{
struct _cf_dsl
{
template <typename Encoding>
static constexpr auto is_inplace = true;
template <typename Reader>
using case_folding = lexy::_acfr<Reader>;
template <typename Literal>
constexpr auto operator()(Literal) const
{
static_assert(lexy::is_literal_rule<Literal>);
static_assert(std::is_void_v<typename Literal::lit_case_folding>, "cannot case fold twice");
return _cfl<Literal, case_folding>{};
}
};
/// Matches Literal with case insensitive ASCII characters.
inline constexpr auto case_folding = _cf_dsl{};
} // namespace lexyd::ascii
//=== Unicode ===//
namespace lexy
{
template <typename Reader>
struct _sucfr32 // simple unicode case folding reader, UTF-32
{
Reader _impl;
constexpr explicit _sucfr32(Reader impl) : _impl(impl) {}
using encoding = typename Reader::encoding;
using iterator = typename Reader::iterator;
using marker = typename Reader::marker;
constexpr auto peek() const -> typename encoding::int_type
{
auto c = _impl.peek();
return lexy::simple_case_fold(lexy::code_point(c)).value();
}
constexpr void bump()
{
_impl.bump();
}
constexpr iterator position() const
{
return _impl.position();
}
constexpr marker current() const noexcept
{
return _impl.current();
}
constexpr void reset(marker m) noexcept
{
_impl.reset(m);
}
};
template <typename Reader>
struct _sucfrm // simple unicode case folding reader, UTF-8 and UTF-16
{
using encoding = typename Reader::encoding;
using iterator = typename Reader::iterator;
using marker = typename Reader::marker;
Reader _impl;
typename Reader::marker _cur_pos;
typename encoding::char_type _buffer[4];
unsigned char _buffer_size;
unsigned char _buffer_cur;
constexpr explicit _sucfrm(Reader impl)
: _impl(impl), _cur_pos(_impl.current()), _buffer{}, _buffer_size(0), _buffer_cur(0)
{
_fill();
}
constexpr void _fill()
{
_cur_pos = _impl.current();
// We need to read the next code point at this point.
auto result = lexy::_detail::parse_code_point(_impl);
if (result.error == lexy::_detail::cp_error::success)
{
// Fill the buffer with the folded code point.
auto folded = lexy::simple_case_fold(lexy::code_point(result.cp));
_buffer_size = static_cast<unsigned char>(
lexy::_detail::encode_code_point<encoding>(folded.value(), _buffer, 4));
_buffer_cur = 0;
_impl.reset(result.end);
}
else
{
// Fill the buffer with the partial code point.
_buffer_cur = _buffer_size = 0;
while (_impl.position() != result.end.position())
{
_buffer[_buffer_size] = static_cast<typename encoding::char_type>(_impl.peek());
++_buffer_size;
_impl.bump();
}
}
}
constexpr auto peek() const -> typename encoding::int_type
{
if (_buffer_cur == _buffer_size)
return encoding::eof();
auto cur = _buffer[_buffer_cur];
return encoding::to_int_type(cur);
}
constexpr void bump()
{
++_buffer_cur;
if (_buffer_cur == _buffer_size)
_fill();
}
constexpr iterator position() const
{
return current().position();
}
constexpr marker current() const noexcept
{
// We only report a marker at a code point boundary.
// This has two consequences:
// 1. If we don't match a rule, the error token does not include any common start code
// units. That's actually nice, and makes it unnecessary to handle that situation in the
// error reporting. The only relevant difference is in the error token.
// 2. If the user wants to match partial code unit sequences, the behavior can become buggy.
// However, that's not really something we should worry about.
return _cur_pos;
}
constexpr void reset(marker m) noexcept
{
_impl.reset(m);
}
};
template <typename Reader>
using _sucfr_for
= std::conditional_t<std::is_same_v<typename Reader::encoding, lexy::utf32_encoding>,
_sucfr32<Reader>, _sucfrm<Reader>>;
template <typename Reader>
struct _sucfr : _sucfr_for<Reader>
{
using _sucfr_for<Reader>::_sucfr_for;
};
} // namespace lexy
namespace lexyd::unicode
{
struct _scf_dsl
{
template <typename Encoding>
static constexpr auto is_inplace = std::is_same_v<Encoding, lexy::utf32_encoding>;
template <typename Reader>
using case_folding = lexy::_sucfr<Reader>;
template <typename Literal>
constexpr auto operator()(Literal) const
{
static_assert(lexy::is_literal_rule<Literal>);
static_assert(std::is_void_v<typename Literal::lit_case_folding>, "cannot case fold twice");
return _cfl<Literal, case_folding>{};
}
};
/// Matches Literal with case insensitive Unicode characters (simple case folding).
inline constexpr auto simple_case_folding = _scf_dsl{};
} // namespace lexyd::unicode
#endif // LEXY_DSL_CASE_FOLDING_HPP_INCLUDED

View File

@@ -0,0 +1,611 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_CHAR_CLASS_HPP_INCLUDED
#define LEXY_DSL_CHAR_CLASS_HPP_INCLUDED
#include <lexy/_detail/code_point.hpp>
#include <lexy/_detail/swar.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/token.hpp>
namespace lexy::_detail
{
struct ascii_set
{
bool contains[128];
constexpr ascii_set() : contains{} {}
template <typename Fn>
constexpr void visit(Fn fn) const
{
for (auto i = 0; i != 128; ++i)
if (contains[i])
fn(i);
}
template <typename Fn>
constexpr void visit_range(Fn fn) const
{
auto range_begin = -1;
auto last_char = range_begin;
visit([&](int c) {
if (range_begin == -1)
{
range_begin = c;
last_char = c;
}
else if (last_char + 1 == c)
{
last_char = c;
}
else
{
fn(range_begin, last_char);
range_begin = c;
last_char = c;
}
});
if (range_begin != -1)
fn(range_begin, last_char);
}
constexpr void insert(int c)
{
contains[c] = true;
}
constexpr void insert(int lower, int upper)
{
for (auto i = lower; i <= upper; ++i)
contains[i] = true;
}
constexpr void insert(const ascii_set& other)
{
other.visit([&](int c) { contains[c] = true; });
}
constexpr void remove(const ascii_set& other)
{
other.visit([&](int c) { contains[c] = false; });
}
};
template <std::size_t RangeCount, std::size_t SingleCount>
struct compressed_ascii_set
{
char range_lower[RangeCount == 0 ? 1 : RangeCount];
char range_upper[RangeCount == 0 ? 1 : RangeCount];
char singles[SingleCount == 0 ? 1 : SingleCount];
static constexpr std::size_t range_count()
{
return RangeCount;
}
static constexpr std::size_t single_count()
{
return SingleCount;
}
};
template <typename T>
constexpr auto compress_ascii_set()
{
constexpr auto set = T::char_class_ascii();
constexpr auto count = [&set] {
struct result_t
{
std::size_t range_count;
std::size_t single_count;
} result{0, 0};
set.visit_range([&](int lower, int upper) {
if (lower != upper)
++result.range_count;
else
++result.single_count;
});
return result;
}();
compressed_ascii_set<count.range_count, count.single_count> result{};
auto cur_range = 0u;
auto cur_single = 0u;
set.visit_range([&](int lower, int upper) {
if (lower != upper)
{
result.range_lower[cur_range] = char(lower);
result.range_upper[cur_range] = char(upper);
++cur_range;
}
else
{
result.singles[cur_single] = char(lower);
++cur_single;
}
});
return result;
}
} // namespace lexy::_detail
namespace lexy::_detail
{
template <const auto& CompressedAsciiSet,
typename = make_index_sequence<CompressedAsciiSet.range_count()>,
typename = make_index_sequence<CompressedAsciiSet.single_count()>>
struct ascii_set_matcher;
template <const auto& CompressedAsciiSet, std::size_t... RangeIdx, std::size_t... SingleIdx>
struct ascii_set_matcher<CompressedAsciiSet, index_sequence<RangeIdx...>,
index_sequence<SingleIdx...>>
{
template <typename Encoding>
static LEXY_CONSTEVAL auto to_int_type(char c)
{
return Encoding::to_int_type(static_cast<typename Encoding::char_type>(c));
}
template <typename Encoding>
LEXY_FORCE_INLINE static constexpr bool match([[maybe_unused]] typename Encoding::int_type cur)
{
return
// It must be in one of the ranges...
((to_int_type<Encoding>(CompressedAsciiSet.range_lower[RangeIdx]) <= cur
&& cur <= to_int_type<Encoding>(CompressedAsciiSet.range_upper[RangeIdx]))
|| ...)
// or one of the single characters.
|| ((cur == to_int_type<Encoding>(CompressedAsciiSet.singles[SingleIdx])) || ...);
}
};
} // namespace lexy::_detail
namespace lexyd
{
template <typename CharSet>
constexpr auto _cas = lexy::_detail::compress_ascii_set<CharSet>();
template <typename Derived>
struct char_class_base : token_base<Derived>, _char_class_base
{
//=== "virtual" functions ===//
// static const char* char_class_name();
// static ascii_set char_class_ascii();
static constexpr bool char_class_unicode()
{
return true;
}
static constexpr std::false_type char_class_match_cp(char32_t)
{
return {};
}
template <typename Reader, typename Context>
static constexpr void char_class_report_error(Context& context,
typename Reader::iterator position)
{
constexpr auto name = Derived::char_class_name();
auto err = lexy::error<Reader, lexy::expected_char_class>(position, name);
context.on(_ev::error{}, err);
}
/// Returns true if c contains only characters from the char class.
/// If it returns false, it may still be valid, it just couldn't be detected.
template <typename Encoding>
static constexpr auto char_class_match_swar(lexy::_detail::swar_int)
{
return std::false_type{};
}
//=== provided functions ===//
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
static_assert(lexy::is_char_encoding<typename Reader::encoding>);
using matcher = lexy::_detail::ascii_set_matcher<_cas<Derived>>;
if (matcher::template match<typename Reader::encoding>(reader.peek()))
{
reader.bump();
end = reader.current();
return true;
}
if constexpr (std::is_same_v<decltype(Derived::char_class_match_cp(char32_t())),
std::false_type>)
{
return false;
}
else if constexpr (lexy::is_unicode_encoding<typename Reader::encoding>)
{
static_assert(Derived::char_class_unicode(),
"cannot use this character class with Unicode encoding");
// Parse one code point.
auto result = lexy::_detail::parse_code_point(reader);
if (result.error != lexy::_detail::cp_error::success)
return false;
if (!Derived::char_class_match_cp(result.cp))
return false;
end = result.end;
return true;
}
else
{
static_assert(!Derived::char_class_unicode(),
"cannot use this character class with non-Unicode char encodings");
if (reader.peek() == Reader::encoding::eof())
return false;
auto cp = static_cast<char32_t>(reader.peek());
reader.bump();
if (!Derived::char_class_match_cp(cp))
return false;
end = reader.current();
return true;
}
}
template <typename Context>
constexpr void report_error(Context& context, const Reader& reader)
{
Derived::template char_class_report_error<Reader>(context, reader.position());
}
};
};
} // namespace lexyd
#define LEXY_CHAR_CLASS(Name, Rule) \
[] { \
static_assert(::lexy::is_char_class_rule<LEXY_DECAY_DECLTYPE(Rule)>); \
struct c : ::lexyd::char_class_base<c> \
{ \
static constexpr auto char_class_unicode() \
{ \
return (Rule).char_class_unicode(); \
} \
static LEXY_CONSTEVAL auto char_class_name() \
{ \
return Name; \
} \
static LEXY_CONSTEVAL auto char_class_ascii() \
{ \
return (Rule).char_class_ascii(); \
} \
static constexpr auto char_class_match_cp(char32_t cp) \
{ \
return (Rule).char_class_match_cp(cp); \
} \
}; \
return c{}; \
}()
namespace lexyd
{
template <typename CharT, CharT... C>
struct _lit;
template <char32_t... Cp>
struct _lcp;
// Implementation helper for the literal overloads.
template <char32_t Cp>
struct _ccp : char_class_base<_ccp<Cp>>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "code-point";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
if constexpr (Cp <= 0x7F)
result.insert(Cp);
return result;
}
static constexpr auto char_class_match_cp([[maybe_unused]] char32_t cp)
{
if constexpr (Cp <= 0x7F)
return std::false_type{};
else
return cp == Cp;
}
};
template <unsigned char Byte>
struct _cb : char_class_base<_cb<Byte>>
{
static constexpr auto char_class_unicode()
{
return Byte <= 0x7F;
}
static LEXY_CONSTEVAL auto char_class_name()
{
return "byte";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
if constexpr (Byte <= 0x7F)
result.insert(Byte);
return result;
}
static constexpr auto char_class_match_cp([[maybe_unused]] char32_t cp)
{
if constexpr (Byte <= 0x7F)
return std::false_type{};
else
return cp == Byte;
}
};
template <typename C, typename = std::enable_if_t<lexy::is_char_class_rule<C>>>
constexpr auto _make_char_class(C c)
{
return c;
}
template <typename CharT, CharT C,
typename = std::enable_if_t<C <= 0x7F || std::is_same_v<CharT, char32_t>
|| std::is_same_v<CharT, unsigned char>>>
constexpr auto _make_char_class(_lit<CharT, C>)
{
if constexpr (std::is_same_v<CharT, unsigned char>)
return _cb<C>{};
else
return _ccp<static_cast<char32_t>(C)>{};
}
template <char32_t CP>
constexpr auto _make_char_class(_lcp<CP>)
{
return _ccp<CP>{};
}
} // namespace lexyd
namespace lexyd
{
template <typename... Cs>
struct _calt : char_class_base<_calt<Cs...>>
{
static_assert(sizeof...(Cs) > 1);
static constexpr auto char_class_unicode()
{
constexpr auto non_unicode = (!Cs::char_class_unicode() || ...);
static_assert(!non_unicode
// If at least one is non-Unicode, either they all must be non-Unicode or
// only match ASCII.
|| ((!Cs::char_class_unicode()
|| std::is_same_v<decltype(Cs::char_class_match_cp(0)),
std::false_type>)
&& ...),
"cannot mix bytes and Unicode char classes");
return !non_unicode;
}
static LEXY_CONSTEVAL auto char_class_name()
{
return "union";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
(result.insert(Cs::char_class_ascii()), ...);
return result;
}
static constexpr auto char_class_match_cp(char32_t cp)
{
if constexpr ((std::is_same_v<decltype(Cs::char_class_match_cp(cp)), std::false_type>
&& ...))
return std::false_type{};
else
return (Cs::char_class_match_cp(cp) || ...);
}
};
template <typename R1, typename R2>
constexpr auto operator/(R1 r1, R2 r2)
-> _calt<decltype(_make_char_class(r1)), decltype(_make_char_class(r2))>
{
return {};
}
template <typename... Cs, typename C>
constexpr auto operator/(_calt<Cs...>, C c) -> _calt<Cs..., decltype(_make_char_class(c))>
{
return {};
}
template <typename C, typename... Cs>
constexpr auto operator/(C c, _calt<Cs...>) -> _calt<decltype(_make_char_class(c)), Cs...>
{
return {};
}
template <typename... Cs, typename... Ds>
constexpr auto operator/(_calt<Cs...>, _calt<Ds...>) -> _calt<Cs..., Ds...>
{
return {};
}
} // namespace lexyd
namespace lexyd
{
template <typename C>
struct _ccomp : char_class_base<_ccomp<C>>
{
static constexpr auto char_class_unicode()
{
return C::char_class_unicode();
}
static LEXY_CONSTEVAL auto char_class_name()
{
return "complement";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert(0x00, 0x7F);
result.remove(C::char_class_ascii());
return result;
}
static constexpr auto char_class_match_cp(char32_t cp)
{
if (cp <= 0x7F)
// If we haven't matched an ASCII character so far, this was intentional.
return false;
if constexpr (std::is_same_v<decltype(C::char_class_match_cp(cp)), std::false_type>)
return true;
else
return !C::char_class_match_cp(cp);
}
};
template <typename C>
constexpr auto operator-(C c) -> _ccomp<decltype(_make_char_class(c))>
{
return {};
}
template <typename C>
constexpr auto operator-(_ccomp<C>) -> C
{
return {};
}
} // namespace lexyd
namespace lexyd
{
template <typename Set, typename Minus>
struct _cminus : char_class_base<_cminus<Set, Minus>>
{
// calt does the correct logic as well, so re-use it.
static constexpr auto char_class_unicode()
{
return _calt<Set, Minus>::char_class_unicode();
}
static LEXY_CONSTEVAL auto char_class_name()
{
return "minus";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
auto result = Set::char_class_ascii();
result.remove(Minus::char_class_ascii());
return result;
}
static constexpr auto char_class_match_cp(char32_t cp)
{
if constexpr (std::is_same_v<decltype(Set::char_class_match_cp(cp)), std::false_type>)
return std::false_type{};
else if constexpr (std::is_same_v<decltype(Minus::char_class_match_cp(cp)),
std::false_type>)
// We don't match ASCII at this point: they only reach this point if the ASCII table
// failed.
return cp > 0x7F && Set::char_class_match_cp(cp);
else
// Same as above, no ASCII.
return cp > 0x7F && Set::char_class_match_cp(cp) && !Minus::char_class_match_cp(cp);
}
};
template <typename Set, typename Minus>
constexpr auto operator-(Set, Minus minus)
{
return _cminus<Set, decltype(_make_char_class(minus))>{};
}
template <typename Set, typename Minus, typename OtherMinus>
constexpr auto operator-(_cminus<Set, Minus>, OtherMinus other)
{
return Set{} - _calt<Minus, decltype(_make_char_class(other))>{};
}
} // namespace lexyd
namespace lexyd
{
template <typename... Cs>
struct _cand : char_class_base<_cand<Cs...>>
{
static_assert(sizeof...(Cs) > 1);
// calt does the correct logic as well, so re-use it.
static constexpr auto char_class_unicode()
{
return _calt<Cs...>::char_class_unicode();
}
static LEXY_CONSTEVAL auto char_class_name()
{
return "intersection";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
for (auto c = 0; c <= 0x7F; ++c)
if ((Cs::char_class_ascii().contains[c] && ...))
result.insert(c);
return result;
}
static constexpr auto char_class_match_cp(char32_t cp)
{
if constexpr ((std::is_same_v<decltype(Cs::char_class_match_cp(cp)), std::false_type>
&& ...))
return std::false_type{};
else
return (Cs::char_class_match_cp(cp) && ...);
}
};
template <typename C1, typename C2>
constexpr auto operator&(C1 c1, C2 c2)
-> _cand<decltype(_make_char_class(c1)), decltype(_make_char_class(c2))>
{
return {};
}
template <typename... Cs, typename C>
constexpr auto operator&(_cand<Cs...>, C c) -> _cand<Cs..., decltype(_make_char_class(c))>
{
return {};
}
template <typename C, typename... Cs>
constexpr auto operator&(C c, _cand<Cs...>) -> _cand<decltype(_make_char_class(c)), Cs...>
{
return {};
}
template <typename... Cs, typename... Ds>
constexpr auto operator&(_cand<Cs...>, _cand<Ds...>) -> _cand<Cs..., Ds...>
{
return {};
}
} // namespace lexyd
#endif // LEXY_DSL_CHAR_CLASS_HPP_INCLUDED

View File

@@ -0,0 +1,161 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_CHOICE_HPP_INCLUDED
#define LEXY_DSL_CHOICE_HPP_INCLUDED
#include <lexy/_detail/tuple.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/error.hpp>
namespace lexy
{
struct exhausted_choice
{
static LEXY_CONSTEVAL auto name()
{
return "exhausted choice";
}
};
} // namespace lexy
namespace lexyd
{
template <typename... R>
struct _chc
// Only make it a branch rule if it doesn't have an unconditional branch.
// A choice rule with an unconditional branch is itself an unconditional branch, which is most
// likely a bug.
: std::conditional_t<(lexy::is_unconditional_branch_rule<R> || ...), rule_base, branch_base>
{
static constexpr auto _any_unconditional = (lexy::is_unconditional_branch_rule<R> || ...);
template <typename Reader, typename Indices = lexy::_detail::make_index_sequence<sizeof...(R)>>
struct bp;
template <typename Reader, std::size_t... Idx>
struct bp<Reader, lexy::_detail::index_sequence<Idx...>>
{
template <typename Rule>
using rp = lexy::branch_parser_for<Rule, Reader>;
lexy::_detail::tuple<rp<R>...> r_parsers;
std::size_t branch_idx;
template <typename ControlBlock>
constexpr auto try_parse(const ControlBlock* cb, const Reader& reader)
-> std::conditional_t<_any_unconditional, std::true_type, bool>
{
auto try_r = [&](std::size_t idx, auto& parser) {
if (!parser.try_parse(cb, reader))
return false;
branch_idx = idx;
return true;
};
// Need to try each possible branch.
auto found_branch = (try_r(Idx, r_parsers.template get<Idx>()) || ...);
if constexpr (_any_unconditional)
{
LEXY_ASSERT(found_branch,
"it is unconditional, but we still haven't found a rule?!");
return {};
}
else
{
return found_branch;
}
}
template <typename Context>
constexpr void cancel(Context& context)
{
// Need to cancel all branches.
(r_parsers.template get<Idx>().cancel(context), ...);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
// Need to call finish on the selected branch, and cancel on all others before that.
auto result = false;
(void)((Idx == branch_idx
? (result
= r_parsers.template get<Idx>()
.template finish<NextParser>(context, reader, LEXY_FWD(args)...),
true)
: (r_parsers.template get<Idx>().cancel(context), false))
|| ...);
return result;
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
auto result = false;
auto try_r = [&](auto&& parser) {
if (!parser.try_parse(context.control_block, reader))
{
parser.cancel(context);
return false;
}
// LEXY_FWD(args) will break MSVC builds targeting C++17.
result = parser.template finish<NextParser>(context, reader,
static_cast<Args&&>(args)...);
return true;
};
// Try to parse each branch in order.
auto found_branch = (try_r(lexy::branch_parser_for<R, Reader>{}) || ...);
if constexpr (_any_unconditional)
{
LEXY_ASSERT(found_branch,
"it is unconditional, but we still haven't found a rule?!");
return result;
}
else
{
if (found_branch)
return result;
auto err = lexy::error<Reader, lexy::exhausted_choice>(reader.position());
context.on(_ev::error{}, err);
return false;
}
}
};
};
template <typename R, typename S>
constexpr auto operator|(R, S)
{
LEXY_REQUIRE_BRANCH_RULE(R, "choice");
LEXY_REQUIRE_BRANCH_RULE(S, "choice");
return _chc<R, S>{};
}
template <typename... R, typename S>
constexpr auto operator|(_chc<R...>, S)
{
LEXY_REQUIRE_BRANCH_RULE(S, "choice");
return _chc<R..., S>{};
}
template <typename R, typename... S>
constexpr auto operator|(R, _chc<S...>)
{
LEXY_REQUIRE_BRANCH_RULE(R, "choice");
return _chc<R, S...>{};
}
template <typename... R, typename... S>
constexpr auto operator|(_chc<R...>, _chc<S...>)
{
return _chc<R..., S...>{};
}
} // namespace lexyd
#endif // LEXY_DSL_CHOICE_HPP_INCLUDED

View File

@@ -0,0 +1,212 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_CODE_POINT_HPP_INCLUDED
#define LEXY_DSL_CODE_POINT_HPP_INCLUDED
#include <lexy/code_point.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/char_class.hpp>
namespace lexyd
{
template <typename Predicate>
struct _cp : char_class_base<_cp<Predicate>>
{
static LEXY_CONSTEVAL auto char_class_name()
{
if constexpr (std::is_void_v<Predicate>)
return "code-point";
else
return lexy::_detail::type_name<Predicate>();
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
if constexpr (std::is_void_v<Predicate>)
{
lexy::_detail::ascii_set result;
result.insert(0x00, 0x7F);
return result;
}
else
{
lexy::_detail::ascii_set result;
for (auto c = 0; c <= 0x7F; ++c)
if (Predicate{}(lexy::code_point(char32_t(c))))
result.insert(c);
return result;
}
}
static constexpr bool char_class_match_cp([[maybe_unused]] char32_t cp)
{
if constexpr (std::is_void_v<Predicate>)
return true;
else
return Predicate{}(lexy::code_point(cp));
}
//=== dsl ===//
template <typename P>
constexpr auto if_() const
{
static_assert(std::is_void_v<Predicate>);
return _cp<P>{};
}
template <char32_t Low, char32_t High>
constexpr auto range() const
{
struct predicate
{
static LEXY_CONSTEVAL auto name()
{
return "code-point.range";
}
constexpr bool operator()(lexy::code_point cp) const
{
return Low <= cp.value() && cp.value() <= High;
}
};
return if_<predicate>();
}
template <char32_t... CPs>
constexpr auto set() const
{
struct predicate
{
static LEXY_CONSTEVAL auto name()
{
return "code-point.set";
}
constexpr bool operator()(lexy::code_point cp) const
{
return ((cp.value() == CPs) || ...);
}
};
return if_<predicate>();
}
constexpr auto ascii() const
{
struct predicate
{
static LEXY_CONSTEVAL auto name()
{
return "code-point.ASCII";
}
constexpr bool operator()(lexy::code_point cp) const
{
return cp.is_ascii();
}
};
return if_<predicate>();
}
constexpr auto bmp() const
{
struct predicate
{
static LEXY_CONSTEVAL auto name()
{
return "code-point.BMP";
}
constexpr bool operator()(lexy::code_point cp) const
{
return cp.is_bmp();
}
};
return if_<predicate>();
}
constexpr auto noncharacter() const
{
struct predicate
{
static LEXY_CONSTEVAL auto name()
{
return "code-point.non-character";
}
constexpr bool operator()(lexy::code_point cp) const
{
return cp.is_noncharacter();
}
};
return if_<predicate>();
}
template <lexy::code_point::general_category_t Category>
constexpr auto general_category() const
{
struct predicate
{
static LEXY_CONSTEVAL auto name()
{
return lexy::_detail::general_category_name(Category);
}
constexpr bool operator()(lexy::code_point cp) const
{
// Note: can't use `cp.is_noncharacter()` for `Cn` as `Cn` also includes all code
// points that are currently unassigned.
if constexpr (Category == lexy::code_point::Cc)
return cp.is_control();
else if constexpr (Category == lexy::code_point::Cs)
return cp.is_surrogate();
else if constexpr (Category == lexy::code_point::Co)
return cp.is_private_use();
else
return cp.general_category() == Category;
}
};
return if_<predicate>();
}
template <const auto& GcGroup>
struct _group_pred;
template <lexy::code_point::general_category_t... Cats,
const lexy::code_point::_gc_group<Cats...>& GcGroup>
struct _group_pred<GcGroup>
{
static LEXY_CONSTEVAL auto name()
{
return GcGroup.name;
}
constexpr bool operator()(lexy::code_point cp) const
{
return cp.general_category() == GcGroup;
}
};
template <const auto& GcGroup>
constexpr auto general_category() const
{
return if_<_group_pred<GcGroup>>();
}
};
/// Matches a single unicode code point in the current unicode encoding.
constexpr auto code_point = _cp<void>{};
} // namespace lexyd
namespace lexy
{
// The void-version without predicate logically matches any input (modulo encoding errors, of
// course).
template <>
inline constexpr auto token_kind_of<lexy::dsl::_cp<void>> = lexy::any_token_kind;
} // namespace lexy
#endif // LEXY_DSL_CODE_POINT_HPP_INCLUDED

View File

@@ -0,0 +1,153 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_COMBINATION_HPP_INCLUDED
#define LEXY_DSL_COMBINATION_HPP_INCLUDED
#include <lexy/_detail/integer_sequence.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/choice.hpp>
#include <lexy/dsl/error.hpp>
#include <lexy/dsl/loop.hpp>
#include <lexy/dsl/sequence.hpp>
namespace lexy
{
struct combination_duplicate
{
static LEXY_CONSTEVAL auto name()
{
return "combination duplicate";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Sink>
struct _comb_control
{
// The sink to store values of the item.
Sink& sink;
// Whether or not the state has already been handled.
const bool* handled;
// Write the index of the item in here.
std::size_t idx = 0;
// Whether or not we should break.
bool loop_break = false;
};
// Final rule for one item in the combination.
template <std::size_t Idx>
struct _comb_it : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename Sink, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context&, Reader&, _comb_control<Sink>& ctrl,
Args&&... args)
{
ctrl.idx = Idx;
if constexpr (sizeof...(Args) > 0)
{
if (!ctrl.handled[Idx])
// Only call the sink if it is not a duplicate.
ctrl.sink(LEXY_FWD(args)...);
}
return true;
}
};
};
template <typename DuplicateError, typename ElseRule, typename... R>
struct _comb : rule_base
{
template <std::size_t... Idx>
static auto _comb_choice_(lexy::_detail::index_sequence<Idx...>)
{
if constexpr (std::is_void_v<ElseRule>)
return ((R{} >> _comb_it<Idx>{}) | ...);
else
return ((R{} >> _comb_it<Idx>{}) | ... | ElseRule{});
}
using _comb_choice = decltype(_comb_choice_(lexy::_detail::index_sequence_for<R...>{}));
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
constexpr auto N = sizeof...(R);
auto sink = context.value_callback().sink();
bool handled[N] = {};
_comb_control<decltype(sink)> control{sink, handled};
// Parse all iterations of the choice.
for (auto count = 0; count < int(N); ++count)
{
auto begin = reader.position();
using parser
= lexy::parser_for<_comb_choice, lexy::pattern_parser<decltype(control)>>;
if (!parser::parse(context, reader, control))
return false;
else if (control.loop_break)
break; // Partial combination and we're done.
if (handled[control.idx])
{
using tag = lexy::_detail::type_or<DuplicateError, lexy::combination_duplicate>;
auto err = lexy::error<Reader, tag>(begin, reader.position());
context.on(_ev::error{}, err);
// We can trivially recover, but need to do another iteration.
--count;
}
else
{
handled[control.idx] = true;
}
}
// Obtain the final result and continue.
return lexy::sink_finish_parser<NextParser>::parse(context, reader, sink,
LEXY_FWD(args)...);
}
};
//=== dsl ===//
template <typename Tag>
static constexpr _comb<Tag, ElseRule, R...> duplicate_error = {};
template <typename Tag>
static constexpr _comb<DuplicateError, _err<Tag, void>, R...> missing_error = {};
};
/// Matches each of the rules in an arbitrary order.
/// Only matches each rule exactly once.
template <typename... R>
constexpr auto combination(R...)
{
LEXY_REQUIRE_BRANCH_RULE(R..., "combination()");
static_assert((!lexy::is_unconditional_branch_rule<R> && ...),
"combination() does not support unconditional branches");
return _comb<void, void, R...>{};
}
/// Matches some of the rules in an arbitrary order.
/// Only matches a rule at most once.
template <typename... R>
constexpr auto partial_combination(R...)
{
LEXY_REQUIRE_BRANCH_RULE(R..., "partial_combination()");
static_assert((!lexy::is_unconditional_branch_rule<R> && ...),
"partial_combination() does not support unconditional branches");
// If the choice no longer matches, we just break.
return _comb<void, decltype(break_), R...>{};
}
} // namespace lexyd
#endif // LEXY_DSL_COMBINATION_HPP_INCLUDED

View File

@@ -0,0 +1,304 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_CONTEXT_COUNTER_HPP_INCLUDED
#define LEXY_DSL_CONTEXT_COUNTER_HPP_INCLUDED
#include <lexy/_detail/iterator.hpp>
#include <lexy/action/base.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/error.hpp>
namespace lexy
{
struct unequal_counts
{
static LEXY_CONSTEVAL auto name()
{
return "unequal counts";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Id>
using _ctx_counter = lexy::_detail::parse_context_var<Id, int>;
template <typename Id, int InitialValue>
struct _ctx_ccreate : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
_ctx_counter<Id> var(InitialValue);
var.link(context);
auto result = NextParser::parse(context, reader, LEXY_FWD(args)...);
var.unlink(context);
return result;
}
};
};
template <typename Id, int Delta>
struct _ctx_cadd : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
_ctx_counter<Id>::get(context.control_block) += Delta;
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
};
template <typename Id, typename Rule, int Sign>
struct _ctx_cpush : _copy_base<Rule>
{
template <typename NextParser>
struct _pc
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader,
typename Reader::iterator begin, Args&&... args)
{
auto end = reader.position();
auto length = lexy::_detail::range_size(begin, end);
_ctx_counter<Id>::get(context.control_block) += int(length) * Sign;
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename Reader>
struct bp
{
lexy::branch_parser_for<Rule, Reader> rule;
template <typename ControlBlock>
constexpr auto try_parse(const ControlBlock* cb, const Reader& reader)
{
return rule.try_parse(cb, reader);
}
template <typename Context>
constexpr void cancel(Context& context)
{
rule.cancel(context);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC auto finish(Context& context, Reader& reader, Args&&... args)
{
// Forward to the rule, but remember the current reader position.
return rule.template finish<_pc<NextParser>>(context, reader, reader.position(),
LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
// Forward to the rule, but remember the current reader position.
using parser = lexy::parser_for<Rule, _pc<NextParser>>;
return parser::parse(context, reader, reader.position(), LEXY_FWD(args)...);
}
};
};
template <typename Id, typename Pred>
struct _ctx_cis : branch_base
{
template <typename Reader>
struct bp
{
template <typename ControlBlock>
constexpr bool try_parse(const ControlBlock* cb, const Reader&)
{
return Pred{}(_ctx_counter<Id>::get(cb));
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename NextParser>
using p = NextParser;
};
template <typename Id>
struct _ctx_cvalue : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
return NextParser::parse(context, reader, LEXY_FWD(args)...,
_ctx_counter<Id>::get(context.control_block));
}
};
};
template <typename... Ids>
struct _ctx_ceq;
template <typename H, typename... T>
struct _ctx_ceq<H, T...> : branch_base
{
template <typename Reader>
struct bp
{
template <typename ControlBlock>
constexpr bool try_parse(const ControlBlock* cb, const Reader&)
{
auto value = _ctx_counter<H>::get(cb);
return ((value == _ctx_counter<T>::get(cb)) && ...);
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
auto value = _ctx_counter<H>::get(context.control_block);
if (((value != _ctx_counter<T>::get(context.control_block)) || ...))
{
auto err = lexy::error<Reader, lexy::unequal_counts>(reader.position());
context.on(_ev::error{}, err);
// Trivially recover.
}
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
};
} // namespace lexyd
namespace lexyd
{
template <int Value>
struct _eq
{
template <typename T>
constexpr bool operator()(T id) const noexcept
{
return id == Value;
}
};
template <int Value>
struct _neq
{
template <typename T>
constexpr bool operator()(T id) const noexcept
{
return id != Value;
}
};
template <typename Id>
struct _ctx_counter_dsl
{
template <int InitialValue = 0>
constexpr auto create() const
{
return _ctx_ccreate<Id, InitialValue>{};
}
constexpr auto inc() const
{
return _ctx_cadd<Id, +1>{};
}
constexpr auto dec() const
{
return _ctx_cadd<Id, -1>{};
}
template <typename Rule>
constexpr auto push(Rule) const
{
return _ctx_cpush<Id, Rule, +1>{};
}
template <typename Rule>
constexpr auto pop(Rule) const
{
return _ctx_cpush<Id, Rule, -1>{};
}
template <typename Pred>
constexpr auto is() const
{
return _ctx_cis<Id, Pred>{};
}
template <int Value>
constexpr auto is() const
{
return _ctx_cis<Id, _eq<Value>>{};
}
constexpr auto is_zero() const
{
return is<0>();
}
template <int Value>
constexpr auto is_not() const
{
return _ctx_cis<Id, _neq<Value>>{};
}
constexpr auto is_not_zero() const
{
return is_not<0>();
}
constexpr auto value() const
{
return _ctx_cvalue<Id>{};
}
};
/// Declares an integer counter that is added to the parsing context.
template <typename Id>
constexpr auto context_counter = _ctx_counter_dsl<Id>{};
/// Takes a branch only if all counters are equal.
template <typename... Ids>
constexpr auto equal_counts(_ctx_counter_dsl<Ids>...)
{
static_assert(sizeof...(Ids) > 1);
return _ctx_ceq<Ids...>{};
}
} // namespace lexyd
#endif // LEXY_DSL_CONTEXT_COUNTER_HPP_INCLUDED

View File

@@ -0,0 +1,153 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_CONTEXT_FLAG_HPP_INCLUDED
#define LEXY_DSL_CONTEXT_FLAG_HPP_INCLUDED
#include <lexy/action/base.hpp>
#include <lexy/dsl/base.hpp>
namespace lexyd
{
template <typename Id>
using _ctx_flag = lexy::_detail::parse_context_var<Id, bool>;
template <typename Id, bool InitialValue>
struct _ctx_fcreate : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
_ctx_flag<Id> var(InitialValue);
var.link(context);
auto result = NextParser::parse(context, reader, LEXY_FWD(args)...);
var.unlink(context);
return result;
}
};
};
template <typename Id, bool Value>
struct _ctx_fset : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
_ctx_flag<Id>::get(context.control_block) = Value;
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
};
template <typename Id>
struct _ctx_ftoggle : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
auto& flag = _ctx_flag<Id>::get(context.control_block);
flag = !flag;
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
};
template <typename Id, bool Value>
struct _ctx_fis : branch_base
{
template <typename Reader>
struct bp
{
template <typename ControlBlock>
constexpr bool try_parse(const ControlBlock* cb, const Reader&)
{
return _ctx_flag<Id>::get(cb) == Value;
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename NextParser>
using p = NextParser;
};
template <typename Id>
struct _ctx_fvalue : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
return NextParser::parse(context, reader, LEXY_FWD(args)...,
_ctx_flag<Id>::get(context.control_block));
}
};
};
} // namespace lexyd
namespace lexyd
{
template <typename Id>
struct _ctx_flag_dsl
{
template <bool InitialValue = false>
constexpr auto create() const
{
return _ctx_fcreate<Id, InitialValue>{};
}
constexpr auto set() const
{
return _ctx_fset<Id, true>{};
}
constexpr auto reset() const
{
return _ctx_fset<Id, false>{};
}
constexpr auto toggle() const
{
return _ctx_ftoggle<Id>{};
}
constexpr auto is_set() const
{
return _ctx_fis<Id, true>{};
}
constexpr auto is_reset() const
{
return _ctx_fis<Id, false>{};
}
constexpr auto value() const
{
return _ctx_fvalue<Id>{};
}
};
/// Declares a flag.
template <typename Id>
constexpr auto context_flag = _ctx_flag_dsl<Id>{};
} // namespace lexyd
#endif // LEXY_DSL_CONTEXT_FLAG_HPP_INCLUDED

View File

@@ -0,0 +1,178 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_CONTEXT_IDENTIFIER_HPP_INCLUDED
#define LEXY_DSL_CONTEXT_IDENTIFIER_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/capture.hpp>
#include <lexy/dsl/identifier.hpp>
namespace lexy
{
struct different_identifier
{
static LEXY_CONSTEVAL auto name()
{
return "different identifier";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Id, typename Reader>
using _ctx_id = lexy::_detail::parse_context_var<Id, lexy::lexeme<Reader>>;
template <typename Id>
struct _ctx_icreate : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
_ctx_id<Id, Reader> var({});
var.link(context);
auto result = NextParser::parse(context, reader, LEXY_FWD(args)...);
var.unlink(context);
return result;
}
};
};
template <typename Id, typename Identifier>
struct _ctx_icap : branch_base
{
template <typename NextParser>
struct _pc
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
// The last argument will be a lexeme.
_ctx_id<Id, Reader>::get(context.control_block) = (args, ...);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename Reader>
using bp = lexy::continuation_branch_parser<Identifier, Reader, _pc>;
template <typename NextParser>
using p = lexy::parser_for<Identifier, _pc<NextParser>>;
};
template <typename Id, typename Identifier, typename Tag>
struct _ctx_irem : branch_base
{
// We only need the pattern:
// We don't want a value and don't need to check for reserved identifiers,
// as it needs to match a previously parsed identifier, which wasn't reserved.
using _pattern = decltype(Identifier{}.pattern());
template <typename Reader>
struct bp
{
typename Reader::marker end;
template <typename ControlBlock>
constexpr bool try_parse(const ControlBlock* cb, const Reader& reader)
{
// Parse the pattern.
lexy::token_parser_for<_pattern, Reader> parser(reader);
if (!parser.try_parse(reader))
return false;
end = parser.end;
// The two lexemes need to be equal.
auto lexeme = lexy::lexeme<Reader>(reader.position(), end.position());
return lexy::_detail::equal_lexemes(_ctx_id<Id, Reader>::get(cb), lexeme);
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
// Finish parsing the token.
context.on(_ev::token{}, lexy::identifier_token_kind, reader.position(),
end.position());
reader.reset(end);
return lexy::whitespace_parser<Context, NextParser>::parse(context, reader,
LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename... PrevArgs>
struct _cont
{
template <typename Context, typename Reader>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, PrevArgs&&... args,
lexy::lexeme<Reader> lexeme)
{
if (!lexy::_detail::equal_lexemes(_ctx_id<Id, Reader>::get(context.control_block),
lexeme))
{
// The lexemes weren't equal.
using tag = lexy::_detail::type_or<Tag, lexy::different_identifier>;
auto err = lexy::error<Reader, tag>(lexeme.begin(), lexeme.end());
context.on(_ev::error{}, err);
// But we can trivially recover.
}
// Continue parsing with the symbol value.
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
// Capture the pattern and continue with special continuation.
return lexy::parser_for<_cap<_pattern>, _cont<Args...>>::parse(context, reader,
LEXY_FWD(args)...);
}
};
template <typename Error>
static constexpr _ctx_irem<Id, Identifier, Error> error = {};
};
} // namespace lexyd
namespace lexyd
{
template <typename Id, typename Identifier>
struct _ctx_id_dsl
{
constexpr auto create() const
{
return _ctx_icreate<Id>{};
}
constexpr auto capture() const
{
return _ctx_icap<Id, Identifier>{};
}
constexpr auto rematch() const
{
return _ctx_irem<Id, Identifier, void>{};
}
};
/// Declares a context variable that stores one instance of the given identifier.
template <typename Id, typename Leading, typename Trailing, typename... Reserved>
constexpr auto context_identifier(_id<Leading, Trailing, Reserved...>)
{
return _ctx_id_dsl<Id, _id<Leading, Trailing, Reserved...>>{};
}
} // namespace lexyd
#endif // LEXY_DSL_CONTEXT_IDENTIFIER_HPP_INCLUDED

View File

@@ -0,0 +1,471 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_DELIMITED_HPP_INCLUDED
#define LEXY_DSL_DELIMITED_HPP_INCLUDED
#include <lexy/_detail/swar.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/capture.hpp>
#include <lexy/dsl/char_class.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/dsl/symbol.hpp>
#include <lexy/dsl/whitespace.hpp>
namespace lexy
{
/// The reader ends before the closing delimiter was found.
struct missing_delimiter
{
static LEXY_CONSTEVAL auto name()
{
return "missing delimiter";
}
};
} // namespace lexy
namespace lexyd
{
template <typename CharClass, typename Reader>
struct _del_chars
{
typename Reader::iterator begin;
constexpr _del_chars(const Reader& reader) : begin(reader.position()) {}
template <typename Context>
constexpr void _recover(Context& context, typename Reader::iterator recover_begin,
typename Reader::iterator recover_end)
{
CharClass::template char_class_report_error<Reader>(context, recover_begin);
// We recovery by discarding the ASCII character.
// (We've checked for EOF before, so that's not the error.)
context.on(_ev::recovery_start{}, recover_begin);
context.on(_ev::token{}, lexy::error_token_kind, recover_begin, recover_end);
context.on(_ev::recovery_finish{}, recover_end);
// Restart the next character here.
begin = recover_end;
}
template <typename Close, typename... Escs>
constexpr void parse_swar(Reader& reader, Close, Escs...)
{
using encoding = typename Reader::encoding;
// If we have a SWAR reader and the Close and Escape chars are literal rules,
// we can munch as much content as possible in a fast loop.
// We also need to efficiently check for the CharClass for it to make sense.
if constexpr (lexy::_detail::is_swar_reader<Reader> //
&& (lexy::is_literal_rule<Close> && ... && Escs::esc_is_literal)
&& !std::is_same_v<
decltype(CharClass::template char_class_match_swar<encoding>({})),
std::false_type>)
{
using char_type = typename encoding::char_type;
using lexy::_detail::swar_has_char;
while (true)
{
auto cur = reader.peek_swar();
// If we have an EOF or the initial character of the closing delimiter, we exit as
// we have no more content.
if (swar_has_char<char_type, encoding::eof()>(cur)
|| swar_has_char<char_type, Close::template lit_first_char<encoding>()>(cur))
break;
// The same is true if we have the escape character.
if constexpr (sizeof...(Escs) > 0)
{
if ((swar_has_char<char_type, Escs::template esc_first_char<encoding>()>(cur)
|| ...))
break;
}
// We definitely don't have the end of the delimited content in the current SWAR,
// check if they all follow the char class.
if (!CharClass::template char_class_match_swar<encoding>(cur))
// They don't or we need to look closer, exit the loop.
break;
reader.bump_swar();
}
}
}
// Precondition: the next code unit definitely belongs to the content, not the delimiter.
template <typename Context, typename Sink>
constexpr void parse_one(Context& context, Reader& reader, Sink& sink)
{
using encoding = typename Reader::encoding;
// First try to match the ASCII characters.
using matcher = lexy::_detail::ascii_set_matcher<_cas<CharClass>>;
if (matcher::template match<encoding>(reader.peek()))
{
reader.bump();
}
else if constexpr (!std::is_same_v<decltype(CharClass::char_class_match_cp(char32_t())),
std::false_type>)
{
if constexpr (lexy::is_unicode_encoding<encoding>)
{
static_assert(CharClass::char_class_unicode(),
"cannot use this character class with Unicode encoding");
auto result = lexy::_detail::parse_code_point(reader);
if (result.error == lexy::_detail::cp_error::success
&& CharClass::char_class_match_cp(result.cp))
{
reader.reset(result.end);
}
else
{
finish(context, sink, reader.position());
auto recover_begin = reader.position();
if (recover_begin == result.end.position())
reader.bump();
else
reader.reset(result.end);
_recover(context, recover_begin, reader.position());
}
}
else
{
static_assert(!CharClass::char_class_unicode(),
"cannot use this character class with non-Unicode char encoding");
LEXY_ASSERT(reader.peek() != encoding::eof(),
"EOF should be checked before calling this");
auto recover_begin = reader.position();
auto cp = static_cast<char32_t>(reader.peek());
reader.bump();
if (!CharClass::char_class_match_cp(cp))
{
finish(context, sink, recover_begin);
_recover(context, recover_begin, reader.position());
}
}
}
// It doesn't match Unicode characters.
else
{
// We can just discard the invalid ASCII character.
LEXY_ASSERT(reader.peek() != encoding::eof(),
"EOF should be checked before calling this");
auto recover_begin = reader.position();
reader.bump();
auto recover_end = reader.position();
finish(context, sink, recover_begin);
_recover(context, recover_begin, recover_end);
}
}
template <typename Context, typename Sink>
constexpr void finish(Context& context, Sink& sink, typename Reader::iterator end)
{
if (begin == end)
return;
context.on(_ev::token{}, typename CharClass::token_type{}, begin, end);
sink(lexy::lexeme<Reader>(begin, end));
}
};
template <typename Token, typename Error = lexy::missing_delimiter>
struct _del_limit
{
using error = Error;
template <typename Reader>
static constexpr bool peek(Reader reader)
{
return lexy::try_match_token(Token{}, reader) || reader.peek() == Reader::encoding::eof();
}
};
template <typename Error>
struct _del_limit<void, Error>
{
using error = Error;
template <typename Reader>
static constexpr bool peek(Reader reader)
{
return reader.peek() == Reader::encoding::eof();
}
};
template <typename Close, typename Char, typename Limit, typename... Escapes>
struct _del : rule_base
{
using _limit = std::conditional_t<std::is_void_v<Limit> || lexy::is_token_rule<Limit>,
_del_limit<Limit>, Limit>;
template <typename CloseParser, typename Context, typename Reader, typename Sink>
LEXY_PARSER_FUNC static bool _loop(CloseParser& close, Context& context, Reader& reader,
Sink& sink)
{
auto del_begin = reader.position();
_del_chars<Char, Reader> cur_chars(reader);
while (true)
{
// Parse as many content chars as possible.
// If it returns, we need to look closer at the next char.
cur_chars.parse_swar(reader, Close{}, Escapes{}...);
// Check for closing delimiter.
if (close.try_parse(context.control_block, reader))
break;
close.cancel(context);
// Check for missing delimiter.
if (_limit::peek(reader))
{
// We're done, so finish the current characters.
auto end = reader.position();
cur_chars.finish(context, sink, end);
auto err = lexy::error<Reader, typename _limit::error>(del_begin, end);
context.on(_ev::error{}, err);
return false;
}
// Check for escape sequences.
if ((Escapes::esc_try_parse(context, reader, sink, cur_chars) || ...))
// We had an escape sequence, so do nothing in this iteration.
continue;
// It is actually a content char, consume it.
cur_chars.parse_one(context, reader, sink);
}
// Finish the currently active character sequence.
cur_chars.finish(context, sink, reader.position());
return true;
}
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
static_assert(lexy::is_char_encoding<typename Reader::encoding>);
auto sink = context.value_callback().sink();
// Parse characters until we have the closing delimiter.
lexy::branch_parser_for<Close, Reader> close{};
if (!_loop(close, context, reader, sink))
return false;
// We're done, finish the sink and then the closing delimiter.
if constexpr (std::is_same_v<typename decltype(sink)::return_type, void>)
{
LEXY_MOV(sink).finish();
return close.template finish<NextParser>(context, reader, LEXY_FWD(args)...);
}
else
{
return close.template finish<NextParser>(context, reader, LEXY_FWD(args)...,
LEXY_MOV(sink).finish());
}
}
};
};
struct _escape_base
{};
template <typename Open, typename Close, typename Limit = void>
struct _delim_dsl
{
/// Add char classes that will limit the delimited to detect a missing terminator.
template <typename LimitCharClass>
constexpr auto limit(LimitCharClass) const
{
static_assert(std::is_void_v<Limit> && lexy::is_char_class_rule<LimitCharClass>);
return _delim_dsl<Open, Close, LimitCharClass>{};
}
/// Add char classes that will limit the delimited and specify the error.
template <typename Error, typename LimitCharClass>
constexpr auto limit(LimitCharClass) const
{
static_assert(std::is_void_v<Limit> && lexy::is_char_class_rule<LimitCharClass>);
return _delim_dsl<Open, Close, _del_limit<LimitCharClass, Error>>{};
}
//=== rules ===//
/// Sets the content.
template <typename Char, typename... Escapes>
constexpr auto operator()(Char, Escapes...) const
{
static_assert(lexy::is_char_class_rule<Char>);
static_assert((std::is_base_of_v<_escape_base, Escapes> && ...));
return no_whitespace(open() >> _del<Close, Char, Limit, Escapes...>{});
}
//=== access ===//
/// Matches the open delimiter.
constexpr auto open() const
{
return Open{};
}
/// Matches the closing delimiter.
constexpr auto close() const
{
// Close never has any whitespace.
return Close{};
}
};
/// Parses everything between the two delimiters and captures it.
template <typename Open, typename Close>
constexpr auto delimited(Open, Close)
{
LEXY_REQUIRE_BRANCH_RULE(Open, "delimited()");
LEXY_REQUIRE_BRANCH_RULE(Close, "delimited()");
return _delim_dsl<Open, Close>{};
}
/// Parses everything between a paired delimiter.
template <typename Delim>
constexpr auto delimited(Delim)
{
LEXY_REQUIRE_BRANCH_RULE(Delim, "delimited()");
return _delim_dsl<Delim, Delim>{};
}
constexpr auto quoted = delimited(LEXY_LIT("\""));
constexpr auto triple_quoted = delimited(LEXY_LIT("\"\"\""));
constexpr auto single_quoted = delimited(LEXY_LIT("'"));
constexpr auto backticked = delimited(LEXY_LIT("`"));
constexpr auto double_backticked = delimited(LEXY_LIT("``"));
constexpr auto triple_backticked = delimited(LEXY_LIT("```"));
} // namespace lexyd
namespace lexy
{
struct invalid_escape_sequence
{
static LEXY_CONSTEVAL auto name()
{
return "invalid escape sequence";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Escape, typename... Branches>
struct _escape : _escape_base
{
static constexpr bool esc_is_literal = lexy::is_literal_rule<Escape>;
template <typename Encoding>
static constexpr auto esc_first_char() -> typename Encoding::char_type
{
return Escape::template lit_first_char<Encoding>();
}
template <typename Context, typename Reader, typename Sink, typename Char>
static constexpr bool esc_try_parse(Context& context, Reader& reader, Sink& sink,
_del_chars<Char, Reader>& cur_chars)
{
auto begin = reader.position();
// Check whether we're having the initial escape character.
lexy::branch_parser_for<Escape, Reader> token{};
if (!token.try_parse(context.control_block, reader))
// No need to call `.cancel()`; it's a token.
return false;
// We do, so finish current character sequence and consume the escape token.
cur_chars.finish(context, sink, begin);
// It's a token, so this can't fail.
token.template finish<lexy::pattern_parser<>>(context, reader);
// Try to parse the correct branch.
auto try_parse_branch = [&](auto branch) {
lexy::branch_parser_for<decltype(branch), Reader> parser{};
if (!parser.try_parse(context.control_block, reader))
{
parser.cancel(context);
return false;
}
// This might fail, but we don't care:
// it will definitely consume the escape token, and everything that is a valid prefix.
// The remaining stuff is then just treated as part of the delimited.
parser.template finish<lexy::sink_parser>(context, reader, sink);
return true;
};
auto found = (try_parse_branch(Branches{}) || ...);
if constexpr ((lexy::is_unconditional_branch_rule<Branches> || ...))
{
LEXY_ASSERT(found, "there is an unconditional branch");
}
else if (!found)
{
// We haven't found any branch of the escape sequence.
auto err = lexy::error<Reader, lexy::invalid_escape_sequence>(begin, reader.position());
context.on(_ev::error{}, err);
}
// Restart the current character sequence after the escape sequence.
cur_chars.begin = reader.position();
return true;
}
/// Adds a generic escape rule.
template <typename Branch>
constexpr auto rule(Branch) const
{
LEXY_REQUIRE_BRANCH_RULE(Branch, "escape()");
return _escape<Escape, Branches..., Branch>{};
}
/// Adds an escape rule that captures the branch.
template <typename Branch>
constexpr auto capture(Branch branch) const
{
LEXY_REQUIRE_BRANCH_RULE(Branch, "escape()");
return this->rule(lexy::dsl::capture(branch));
}
/// Adds an escape rule that parses the symbol.
template <const auto& Table, typename Rule>
constexpr auto symbol(Rule rule) const
{
return this->rule(lexyd::symbol<Table>(rule));
}
template <const auto& Table>
constexpr auto symbol() const
{
return this->rule(lexyd::symbol<Table>);
}
};
/// Creates an escape rule.
/// The token is the initial rule to begin,
/// and then you can add rules that match after it.
template <typename EscapeToken>
constexpr auto escape(EscapeToken)
{
static_assert(lexy::is_token_rule<EscapeToken>);
return _escape<EscapeToken>{};
}
constexpr auto backslash_escape = escape(lit_c<'\\'>);
constexpr auto dollar_escape = escape(lit_c<'$'>);
} // namespace lexyd
#endif // LEXY_DSL_DELIMITED_HPP_INCLUDED

View File

@@ -0,0 +1,652 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_DIGIT_HPP_INCLUDED
#define LEXY_DSL_DIGIT_HPP_INCLUDED
#include <lexy/_detail/swar.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/char_class.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/dsl/token.hpp>
//=== bases ===//
// SWAR matching code adapted from:
// https://lemire.me/blog/2018/09/30/quickly-identifying-a-sequence-of-digits-in-a-string-of-characters/
namespace lexyd
{
template <int Radix>
struct _d;
template <>
struct _d<2> : char_class_base<_d<2>>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "digit.binary";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('0', '1');
return result;
}
static constexpr unsigned digit_radix = 2;
template <typename CharT>
static constexpr unsigned digit_value(CharT c)
{
return static_cast<unsigned>(c) - '0';
}
template <typename CharT>
static constexpr bool swar_matches(lexy::_detail::swar_int c)
{
constexpr auto mask = lexy::_detail::swar_fill_compl(CharT(0xF));
constexpr auto expected = lexy::_detail::swar_fill(CharT(0x30));
constexpr auto offset = lexy::_detail::swar_fill(CharT(0x0E));
return (c & mask) == expected && ((c + offset) & mask) == expected;
}
};
using binary = _d<2>;
template <>
struct _d<8> : char_class_base<_d<8>>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "digit.octal";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('0', '7');
return result;
}
static constexpr unsigned digit_radix = 8;
template <typename CharT>
static constexpr unsigned digit_value(CharT c)
{
return static_cast<unsigned>(c) - '0';
}
template <typename CharT>
static constexpr bool swar_matches(lexy::_detail::swar_int c)
{
constexpr auto mask = lexy::_detail::swar_fill_compl(CharT(0xF));
constexpr auto expected = lexy::_detail::swar_fill(CharT(0x30));
constexpr auto offset = lexy::_detail::swar_fill(CharT(0x08));
return (c & mask) == expected && ((c + offset) & mask) == expected;
}
};
using octal = _d<8>;
template <>
struct _d<10> : char_class_base<_d<10>>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "digit.decimal";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('0', '9');
return result;
}
static constexpr unsigned digit_radix = 10;
template <typename CharT>
static constexpr unsigned digit_value(CharT c)
{
return static_cast<unsigned>(c) - '0';
}
template <typename CharT>
static constexpr bool swar_matches(lexy::_detail::swar_int c)
{
constexpr auto mask = lexy::_detail::swar_fill_compl(CharT(0xF));
constexpr auto expected = lexy::_detail::swar_fill(CharT(0x30));
constexpr auto offset = lexy::_detail::swar_fill(CharT(0x06));
return (c & mask) == expected && ((c + offset) & mask) == expected;
}
};
using decimal = _d<10>;
struct hex_lower : char_class_base<hex_lower>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "digit.hex-lower";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('0', '9');
result.insert('a', 'f');
return result;
}
static constexpr unsigned digit_radix = 16;
template <typename CharT>
static constexpr unsigned digit_value(CharT c)
{
if (c >= 'a')
return static_cast<unsigned>(c) - 'a' + 10;
else if (c <= '9')
return static_cast<unsigned>(c) - '0';
else
return unsigned(-1);
}
template <typename CharT>
static constexpr bool swar_matches(lexy::_detail::swar_int c)
{
// False negative for hex digits, but that's okay.
return _d<10>::swar_matches<CharT>(c);
}
};
struct hex_upper : char_class_base<hex_upper>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "digit.hex-upper";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('0', '9');
result.insert('A', 'F');
return result;
}
static constexpr unsigned digit_radix = 16;
template <typename CharT>
static constexpr unsigned digit_value(CharT c)
{
if (c >= 'A')
return static_cast<unsigned>(c) - 'A' + 10;
else if (c <= '9')
return static_cast<unsigned>(c) - '0';
else
return unsigned(-1);
}
template <typename CharT>
static constexpr bool swar_matches(lexy::_detail::swar_int c)
{
// False negative for hex digits, but that's okay.
return _d<10>::swar_matches<CharT>(c);
}
};
template <>
struct _d<16> : char_class_base<_d<16>>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "digit.hex";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('0', '9');
result.insert('a', 'f');
result.insert('A', 'F');
return result;
}
static constexpr unsigned digit_radix = 16;
template <typename CharT>
static constexpr unsigned digit_value(CharT c)
{
if (c >= 'a')
return static_cast<unsigned>(c) - 'a' + 10;
else if (c >= 'A')
return static_cast<unsigned>(c) - 'A' + 10;
else if (c <= '9')
return static_cast<unsigned>(c) - '0';
else
return unsigned(-1);
}
template <typename CharT>
static constexpr bool swar_matches(lexy::_detail::swar_int c)
{
// False negative for hex digits, but that's okay.
return _d<10>::swar_matches<CharT>(c);
}
};
using hex = _d<16>;
} // namespace lexyd
//=== digit ===//
namespace lexyd
{
struct _zero : char_class_base<_zero>
{
static LEXY_CONSTEVAL auto char_class_name()
{
return "digit.zero";
}
static LEXY_CONSTEVAL auto char_class_ascii()
{
lexy::_detail::ascii_set result;
result.insert('0');
return result;
}
};
/// Matches the zero digit.
constexpr auto zero = _zero{};
/// Matches a single digit.
template <typename Base = decimal, int = Base::digit_radix>
constexpr auto digit = Base{};
} // namespace lexyd
namespace lexy
{
template <>
inline constexpr auto token_kind_of<lexy::dsl::_zero> = lexy::digits_token_kind;
template <int Radix>
constexpr auto token_kind_of<lexy::dsl::_d<Radix>> = lexy::digits_token_kind;
template <>
inline constexpr auto token_kind_of<lexy::dsl::hex_lower> = lexy::digits_token_kind;
template <>
inline constexpr auto token_kind_of<lexy::dsl::hex_upper> = lexy::digits_token_kind;
} // namespace lexy
//=== digits ===//
namespace lexy
{
struct forbidden_leading_zero
{
static LEXY_CONSTEVAL auto name()
{
return "forbidden leading zero";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Base, typename Reader>
constexpr bool _match_digits(Reader& reader)
{
// Need at least one digit.
// Checking for a single digit is also cheaper than doing a SWAR comparison,
// so we do that manually in either case.
if (!lexy::try_match_token(digit<Base>, reader))
return false;
// Now we consume as many digits as possible.
// First using SWAR...
if constexpr (lexy::_detail::is_swar_reader<Reader>)
{
using char_type = typename Reader::encoding::char_type;
while (Base::template swar_matches<char_type>(reader.peek_swar()))
reader.bump_swar();
}
// ... then manually to get any trailing digits.
while (lexy::try_match_token(digit<Base>, reader))
{
}
return true;
}
template <typename Base, typename Sep, typename Reader>
constexpr bool _match_digits_sep(Reader& reader)
{
// Need at least one digit.
if (!lexy::try_match_token(digit<Base>, reader))
return false;
// Might have following digits.
while (true)
{
if (lexy::try_match_token(Sep{}, reader))
{
// Need a digit after a separator.
if (!lexy::try_match_token(digit<Base>, reader))
return false;
}
else
{
// Attempt to consume as many digits as possible.
if constexpr (lexy::_detail::is_swar_reader<Reader>)
{
using char_type = typename Reader::encoding::char_type;
while (Base::template swar_matches<char_type>(reader.peek_swar()))
reader.bump_swar();
}
if (!lexy::try_match_token(digit<Base>, reader))
// If we're not having a digit, we're done.
break;
}
}
return true;
}
template <typename Base, typename Sep>
struct _digits_st : token_base<_digits_st<Base, Sep>>
{
template <typename Reader>
struct tp
{
typename Reader::marker end;
bool forbidden_leading_zero;
constexpr explicit tp(const Reader& reader)
: end(reader.current()), forbidden_leading_zero(false)
{}
constexpr bool try_parse(Reader reader)
{
using char_type = typename Reader::encoding::char_type;
auto begin = reader.current();
auto result = _match_digits_sep<Base, Sep>(reader);
end = reader.current();
if (result && lexy::_detail::next(begin.position()) != end.position()
&& *begin.position() == lexy::_detail::transcode_char<char_type>('0'))
{
reader.reset(begin);
reader.bump();
end = reader.current();
forbidden_leading_zero = true;
return false;
}
return result;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader& reader)
{
if (forbidden_leading_zero)
{
auto err = lexy::error<Reader, lexy::forbidden_leading_zero>(reader.position(),
end.position());
context.on(_ev::error{}, err);
}
else
{
auto err = lexy::error<Reader, lexy::expected_char_class>(end.position(),
Base::char_class_name());
context.on(_ev::error{}, err);
}
}
};
};
template <typename Base, typename Sep>
struct _digits_s : token_base<_digits_s<Base, Sep>>
{
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
auto result = _match_digits_sep<Base, Sep>(reader);
end = reader.current();
return result;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader&)
{
auto err = lexy::error<Reader, lexy::expected_char_class>(end.position(),
Base::char_class_name());
context.on(_ev::error{}, err);
}
};
constexpr auto no_leading_zero() const
{
return _digits_st<Base, Sep>{};
}
};
template <typename Base>
struct _digits_t : token_base<_digits_t<Base>>
{
template <typename Reader>
struct tp
{
typename Reader::marker end;
bool forbidden_leading_zero;
constexpr explicit tp(const Reader& reader)
: end(reader.current()), forbidden_leading_zero(false)
{}
constexpr bool try_parse(Reader reader)
{
using char_type = typename Reader::encoding::char_type;
auto begin = reader.current();
auto result = _match_digits<Base>(reader);
end = reader.current();
if (result && lexy::_detail::next(begin.position()) != end.position()
&& *begin.position() == lexy::_detail::transcode_char<char_type>('0'))
{
reader.reset(begin);
reader.bump();
end = reader.current();
forbidden_leading_zero = true;
return false;
}
return result;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader& reader)
{
if (forbidden_leading_zero)
{
auto err = lexy::error<Reader, lexy::forbidden_leading_zero>(reader.position(),
end.position());
context.on(_ev::error{}, err);
}
else
{
auto err = lexy::error<Reader, lexy::expected_char_class>(reader.position(),
Base::char_class_name());
context.on(_ev::error{}, err);
}
}
};
template <typename Token>
constexpr auto sep(Token) const
{
static_assert(lexy::is_token_rule<Token>);
return _digits_st<Base, Token>{};
}
};
template <typename Base>
struct _digits : token_base<_digits<Base>>
{
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
auto result = _match_digits<Base>(reader);
end = reader.current();
return result;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader& reader)
{
auto err = lexy::error<Reader, lexy::expected_char_class>(reader.position(),
Base::char_class_name());
context.on(_ev::error{}, err);
}
};
template <typename Token>
constexpr auto sep(Token) const
{
static_assert(lexy::is_token_rule<Token>);
return _digits_s<Base, Token>{};
}
constexpr auto no_leading_zero() const
{
return _digits_t<Base>{};
}
};
/// Matches a non-empty list of digits.
template <typename Base = decimal>
constexpr auto digits = _digits<Base>{};
constexpr auto digit_sep_underscore = LEXY_LIT("_");
constexpr auto digit_sep_tick = LEXY_LIT("'");
} // namespace lexyd
namespace lexy
{
template <typename Base>
constexpr auto token_kind_of<lexy::dsl::_digits<Base>> = lexy::digits_token_kind;
template <typename Base>
constexpr auto token_kind_of<lexy::dsl::_digits_t<Base>> = lexy::digits_token_kind;
template <typename Base, typename Sep>
constexpr auto token_kind_of<lexy::dsl::_digits_s<Base, Sep>> = lexy::digits_token_kind;
template <typename Base, typename Sep>
constexpr auto token_kind_of<lexy::dsl::_digits_st<Base, Sep>> = lexy::digits_token_kind;
} // namespace lexy
//=== n_digits ===//
namespace lexyd
{
template <std::size_t N, typename Base, typename Sep>
struct _ndigits_s : token_base<_ndigits_s<N, Base, Sep>>
{
template <typename Reader, typename Indices = lexy::_detail::make_index_sequence<N - 1>>
struct tp;
template <typename Reader, std::size_t... Idx>
struct tp<Reader, lexy::_detail::index_sequence<Idx...>>
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
// Match the Base one time.
if (!lexy::try_match_token(digit<Base>, reader))
{
end = reader.current();
return false;
}
// Match each other digit after a separator.
auto success = (((void)Idx, lexy::try_match_token(Sep{}, reader),
lexy::try_match_token(digit<Base>, reader))
&& ...);
end = reader.current();
return success;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader&)
{
auto err = lexy::error<Reader, lexy::expected_char_class>(end.position(),
Base::char_class_name());
context.on(_ev::error{}, err);
}
};
};
template <std::size_t N, typename Base>
struct _ndigits : token_base<_ndigits<N, Base>>
{
static_assert(N > 1);
template <typename Reader, typename Indices = lexy::_detail::make_index_sequence<N>>
struct tp;
template <typename Reader, std::size_t... Idx>
struct tp<Reader, lexy::_detail::index_sequence<Idx...>>
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
// Match the Base N times.
auto success = (((void)Idx, lexy::try_match_token(digit<Base>, reader)) && ...);
end = reader.current();
return success;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader&)
{
auto err = lexy::error<Reader, lexy::expected_char_class>(end.position(),
Base::char_class_name());
context.on(_ev::error{}, err);
}
};
template <typename Token>
constexpr auto sep(Token) const
{
static_assert(lexy::is_token_rule<Token>);
return _ndigits_s<N, Base, Token>{};
}
};
/// Matches exactly N digits.
template <std::size_t N, typename Base = decimal>
constexpr auto n_digits = _ndigits<N, Base>{};
} // namespace lexyd
namespace lexy
{
template <std::size_t N, typename Base>
constexpr auto token_kind_of<lexy::dsl::_ndigits<N, Base>> = lexy::digits_token_kind;
template <std::size_t N, typename Base, typename Sep>
constexpr auto token_kind_of<lexy::dsl::_ndigits_s<N, Base, Sep>> = lexy::digits_token_kind;
} // namespace lexy
#endif // LEXY_DSL_DIGIT_HPP_INCLUDED

View File

@@ -0,0 +1,69 @@
// Copyright (C) 2022 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_EFFECT_HPP_INCLUDED
#define LEXY_DSL_EFFECT_HPP_INCLUDED
#include <lexy/_detail/detect.hpp>
#include <lexy/dsl/base.hpp>
namespace lexyd
{
template <typename EffRule, typename State>
using _detect_eff_fn = decltype(EffRule::_fn()(LEXY_DECLVAL(State&)));
template <LEXY_NTTP_PARAM Fn>
struct _eff : rule_base
{
static constexpr auto _fn()
{
return Fn;
}
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
using control_block_type = LEXY_DECAY_DECLTYPE(*context.control_block);
using state_type = typename control_block_type::state_type;
if constexpr (lexy::_detail::is_detected<_detect_eff_fn, _eff, state_type>)
{
using return_type = decltype(Fn(*context.control_block->parse_state));
if constexpr (std::is_void_v<return_type>)
{
Fn(*context.control_block->parse_state);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
else
{
return NextParser::parse(context, reader, LEXY_FWD(args)...,
Fn(*context.control_block->parse_state));
}
}
else
{
using return_type = decltype(Fn());
if constexpr (std::is_void_v<return_type>)
{
Fn();
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
else
{
return NextParser::parse(context, reader, LEXY_FWD(args)..., Fn());
}
}
}
};
};
/// Invokes Fn and produces its value as result.
template <LEXY_NTTP_PARAM Fn>
constexpr auto effect = _eff<Fn>{};
} // namespace lexyd
#endif // LEXY_DSL_EFFECT_HPP_INCLUDED

View File

@@ -0,0 +1,76 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_EOF_HPP_INCLUDED
#define LEXY_DSL_EOF_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/error.hpp>
namespace lexy
{
struct expected_eof
{
static LEXY_CONSTEVAL auto name()
{
return "expected EOF";
}
};
} // namespace lexy
namespace lexyd
{
struct _eof : branch_base
{
template <typename Reader>
struct bp
{
constexpr bool try_parse(const void*, const Reader& reader)
{
return reader.peek() == Reader::encoding::eof();
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
auto pos = reader.position();
context.on(_ev::token{}, lexy::eof_token_kind, pos, pos);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
if (reader.peek() != Reader::encoding::eof())
{
// Report that we've failed.
auto err = lexy::error<Reader, lexy::expected_eof>(reader.position());
context.on(_ev::error{}, err);
// But recover immediately, as we wouldn't have consumed anything either way.
}
else
{
auto pos = reader.position();
context.on(_ev::token{}, lexy::eof_token_kind, pos, pos);
}
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
};
/// Matches EOF.
constexpr auto eof = _eof{};
} // namespace lexyd
#endif // LEXY_DSL_EOF_HPP_INCLUDED

View File

@@ -0,0 +1,128 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_ERROR_HPP_INCLUDED
#define LEXY_DSL_ERROR_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/branch.hpp>
#include <lexy/dsl/token.hpp>
namespace lexyd
{
template <typename Rule>
struct _err_production
{
static constexpr auto name = "<error>";
static constexpr auto max_recursion_depth = 0;
static constexpr auto rule = Rule{};
};
template <typename Tag, typename Rule>
struct _err : unconditional_branch_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&...)
{
auto begin = reader.position();
auto end = reader.position();
if constexpr (!std::is_same_v<Rule, void>)
{
auto backtrack = reader.current();
// We match a dummy production that only consists of the rule.
lexy::do_action<
_err_production<Rule>,
lexy::match_action<void, Reader>::template result_type>(lexy::_mh(),
context.control_block
->parse_state,
reader);
end = reader.position();
reader.reset(LEXY_MOV(backtrack));
}
auto err = lexy::error<Reader, Tag>(begin, end);
context.on(_ev::error{}, err);
return false;
}
};
template <typename Reader>
using bp = lexy::unconditional_branch_parser<_err, Reader>;
/// Adds a rule whose match will be part of the error location.
template <typename R>
constexpr auto operator()(R) const
{
return _err<Tag, R>{};
}
};
/// Matches nothing, produces an error with the given tag.
template <typename Tag>
constexpr auto error = _err<Tag, void>{};
} // namespace lexyd
namespace lexyd
{
template <typename Branch, typename Error>
struct _must : branch_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
// Try and parse the branch.
lexy::branch_parser_for<Branch, Reader> branch{};
if (branch.try_parse(context.control_block, reader))
return branch.template finish<NextParser>(context, reader, LEXY_FWD(args)...);
branch.cancel(context);
// The branch wasn't taken, so we fail with the specific error by parsing Error.
auto result = lexy::parser_for<Error, lexy::pattern_parser<>>::parse(context, reader);
LEXY_ASSERT(!result, "error must not recover");
return false;
}
};
// As a branch we parse it exactly the same.
template <typename Reader>
using bp = lexy::branch_parser_for<Branch, Reader>;
};
template <typename Branch>
struct _must_dsl
{
template <typename Tag>
struct _err : _must<Branch, lexyd::_err<Tag, void>>
{
template <typename Rule>
constexpr auto operator()(Rule rule) const
{
auto err = lexyd::error<Tag>(rule);
return _must<Branch, decltype(err)>{};
}
};
template <typename Tag>
static constexpr _err<Tag> error = _err<Tag>{};
};
/// Tries to parse `Branch` and raises a specific error on failure.
/// It can still be used as a branch rule; then behaves exactly like `Branch.`
template <typename Branch>
constexpr auto must(Branch)
{
LEXY_REQUIRE_BRANCH_RULE(Branch, "must()");
static_assert(!lexy::is_unconditional_branch_rule<Branch>);
return _must_dsl<Branch>{};
}
} // namespace lexyd
#endif // LEXY_DSL_ERROR_HPP_INCLUDED

View File

@@ -0,0 +1,601 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_EXPRESSION_HPP_INCLUDED
#define LEXY_DSL_EXPRESSION_HPP_INCLUDED
#include <lexy/action/base.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/dsl/operator.hpp>
// Expression parsing algorithm uses an adapted version of Pratt parsing, as described here:
// https://matklad.github.io/2020/04/13/simple-but-powerful-pratt-parsing.html
// In particular:
// * precedence specified implicitly by type type hierarchy
// * support for list and single precedence
// * support for operator groups that require additional parentheses
// * generate proper parse tree events
//=== dsl ===//
namespace lexyd
{
/// Operation that just parses the atomic rule.
struct atom : _operation_base
{
static LEXY_CONSTEVAL auto name()
{
return "atom";
}
};
/// Operation that selects between multiple ones.
template <typename... Operands>
struct groups : _operation_base
{};
struct infix_op_left : _operation_base // a ~ b ~ c == (a ~ b) ~ c
{};
struct infix_op_right : _operation_base // a ~ b ~ c == a ~ (b ~ c)
{};
struct infix_op_list : _operation_base // a ~ b ~ c kept as-is
{};
struct infix_op_single : _operation_base // a ~ b ~ c is an error
{};
struct postfix_op : _operation_base
{};
struct prefix_op : _operation_base
{};
} // namespace lexyd
//=== implementation ===//
namespace lexy::_detail
{
struct binding_power
{
unsigned group;
unsigned lhs, rhs;
static constexpr auto left(unsigned group, unsigned level)
{
return binding_power{group, 2 * level, 2 * level + 1};
}
static constexpr auto right(unsigned group, unsigned level)
{
return binding_power{group, 2 * level + 1, 2 * level};
}
static constexpr auto prefix(unsigned group, unsigned level)
{
// prefix is sort of left-associative, so right side odd.
return binding_power{group, 0, 2 * level + 1};
}
static constexpr auto postfix(unsigned group, unsigned level)
{
// postfix is sort of right-associative, so left side odd.
return binding_power{group, 2 * level + 1, 0};
}
constexpr bool is_valid() const
{
return lhs > 0 || rhs > 0;
}
constexpr bool is_infix() const
{
return lhs > 0 && rhs > 0;
}
constexpr bool is_postfix() const
{
return lhs > 0 && rhs == 0;
}
constexpr bool is_prefix() const
{
return lhs == 0 && rhs > 0;
}
};
template <typename Operation>
constexpr binding_power get_binding_power(unsigned cur_group, unsigned cur_level)
{
if constexpr (std::is_base_of_v<lexyd::infix_op_left, Operation>
// We treat a list as left associative operator for simplicity here.
// It doesn't really matter, as it will only consider operators from the
// same operation anyway.
|| std::is_base_of_v<lexyd::infix_op_list, Operation>
// For the purposes of error recovery, single is left associative.
|| std::is_base_of_v<lexyd::infix_op_single, Operation>)
return binding_power::left(cur_group, cur_level);
else if constexpr (std::is_base_of_v<lexyd::infix_op_right, Operation>)
return binding_power::right(cur_group, cur_level);
else if constexpr (std::is_base_of_v<lexyd::prefix_op, Operation>)
return binding_power::prefix(cur_group, cur_level);
else if constexpr (std::is_base_of_v<lexyd::postfix_op, Operation>)
return binding_power::postfix(cur_group, cur_level);
}
template <typename DestOperation>
struct _binding_power_of
{
static constexpr auto transition(lexyd::atom, unsigned cur_group, unsigned)
{
// Not found, return an invalid operator, but return the current group.
// This is the highest group encountered.
return binding_power{cur_group, 0, 0};
}
template <typename CurOperation>
static constexpr auto transition(CurOperation op, unsigned cur_group, unsigned cur_level)
{
// Normal operation: keep group the same, but increment level.
return get(op, cur_group, cur_level + 1);
}
template <typename... Operations>
static constexpr auto transition(lexyd::groups<Operations...>, unsigned cur_group,
unsigned cur_level)
{
auto result = binding_power{cur_group, 0, 0};
// Try to find the destination in each group.
// Before we transition, we increment the group to create a new one,
// afterwards we update group to the highest group encountered so far.
// That way, we don't re-use group numbers.
// Note that we don't increment the level, as that is handled by the overload above.
(void)((result = transition(Operations{}, cur_group + 1, cur_level),
cur_group = result.group, result.is_valid())
|| ...);
return result;
}
template <typename CurOperation>
static constexpr auto get(CurOperation, unsigned cur_group, unsigned cur_level)
{
if constexpr (std::is_same_v<DestOperation, CurOperation>)
return get_binding_power<CurOperation>(cur_group, cur_level);
else
return transition(typename CurOperation::operand{}, cur_group, cur_level);
}
};
// Returns the binding power of an operator in an expression.
template <typename Expr, typename Operation>
constexpr auto binding_power_of(Operation)
{
return _binding_power_of<Operation>::transition(typename Expr::operation{}, 0, 0);
}
} // namespace lexy::_detail
namespace lexy::_detail
{
template <typename Operation>
using op_of = LEXY_DECAY_DECLTYPE(Operation::op);
template <typename... Operations>
struct operation_list
{
template <typename T>
constexpr auto operator+(T) const
{
return operation_list<Operations..., T>{};
}
template <typename... T>
constexpr auto operator+(operation_list<T...>) const
{
return operation_list<Operations..., T...>{};
}
static constexpr auto size = sizeof...(Operations);
using ops = decltype((typename op_of<Operations>::op_literals{} + ... + op_lit_list{}));
template <template <typename> typename Continuation, typename Context, typename Reader,
typename... Args>
static constexpr bool apply(Context& context, Reader& reader, parsed_operator<Reader> op,
Args&&... args)
{
auto result = false;
auto cur_idx = std::size_t(0);
(void)((cur_idx <= op.idx && op.idx < cur_idx + op_of<Operations>::op_literals::size
? (result
= Continuation<Operations>::parse(context, reader,
parsed_operator<Reader>{op.cur,
op.idx - cur_idx},
LEXY_FWD(args)...),
true)
: (cur_idx += op_of<Operations>::op_literals::size, false))
|| ...);
return result;
}
};
template <bool Pre, unsigned MinBindingPower>
struct _operation_list_of
{
template <unsigned CurLevel>
static constexpr auto get(lexyd::atom)
{
return operation_list{};
}
template <unsigned CurLevel, typename... Operations>
static constexpr auto get(lexyd::groups<Operations...>)
{
return (get<CurLevel>(Operations{}) + ...);
}
template <unsigned CurLevel, typename Operation>
static constexpr auto get(Operation)
{
constexpr auto bp = get_binding_power<Operation>(0, CurLevel);
auto tail = get<CurLevel + 1>(typename Operation::operand{});
constexpr auto is_prefix = std::is_base_of_v<lexyd::prefix_op, Operation>;
if constexpr (is_prefix == Pre
&& ((is_prefix && bp.rhs >= MinBindingPower)
|| (!is_prefix && bp.lhs >= MinBindingPower)))
return tail + Operation{};
else
return tail;
}
};
// prefix operations
template <typename Expr, unsigned MinBindingPower>
using pre_operation_list_of = decltype(_operation_list_of<true, MinBindingPower>::template get<1>(
typename Expr::operation{}));
// infix and postfix operations
template <typename Expr, unsigned MinBindingPower>
using post_operation_list_of = decltype(_operation_list_of<false, MinBindingPower>::template get<1>(
typename Expr::operation{}));
} // namespace lexy::_detail
//=== expression rule ===//
namespace lexyd
{
template <typename RootOperation>
struct _expr : rule_base
{
struct _state
{
unsigned cur_group = 0;
unsigned cur_nesting_level = 0;
};
template <typename Operation>
struct _continuation
{
struct _op_cont
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, _state& state,
Args&&... op_args)
{
using namespace lexy::_detail;
constexpr auto value_type_void = std::is_void_v<typename Context::value_type>;
constexpr auto binding_power
= binding_power_of<typename Context::production>(Operation{});
if constexpr (std::is_base_of_v<infix_op_list, Operation>)
{
// We need to handle a list infix operator specially,
// and parse an arbitrary amount of lhs.
// For that, we use a loop and a sink.
auto sink = context.value_callback().sink();
// We need to pass the initial lhs to the sink.
if constexpr (!value_type_void)
sink(*LEXY_MOV(context.value));
context.value = {};
// As well as the operator we've already got.
sink(LEXY_FWD(op_args)...);
auto result = true;
while (true)
{
// Parse (another) value.
if (!_parse<binding_power.rhs>(context, reader, state))
{
result = false;
break;
}
if constexpr (!value_type_void)
sink(*LEXY_MOV(context.value));
context.value = {};
using op_rule = op_of<Operation>;
auto op = parse_operator<typename op_rule::op_literals>(reader);
if (op.idx >= op_rule::op_literals::size)
{
// The list ends at this point.
reader.reset(op.cur);
break;
}
// Need to finish the operator properly, by passing it to the sink.
if (!op_rule::template op_finish<lexy::sink_parser>(context, reader, op,
sink))
{
result = false;
break;
}
}
// We store the final value of the sink no matter the parse result.
if constexpr (value_type_void)
{
LEXY_MOV(sink).finish();
context.value.emplace();
}
else
{
context.value.emplace(LEXY_MOV(sink).finish());
}
// If we've failed at any point, propagate failure now.
if (!result)
return false;
}
else if constexpr (binding_power.is_prefix())
{
if (!_parse<binding_power.rhs>(context, reader, state))
return false;
auto value = LEXY_MOV(context.value);
context.value = {};
if constexpr (value_type_void)
context.value.emplace_result(context.value_callback(),
LEXY_FWD(op_args)...);
else
context.value.emplace_result(context.value_callback(), LEXY_FWD(op_args)...,
*LEXY_MOV(value));
}
else if constexpr (binding_power.is_infix())
{
auto lhs = LEXY_MOV(context.value);
context.value = {};
if (!_parse<binding_power.rhs>(context, reader, state))
{
// Put it back, so we can properly recover.
context.value = LEXY_MOV(lhs);
return false;
}
auto rhs = LEXY_MOV(context.value);
context.value = {};
if constexpr (value_type_void)
context.value.emplace_result(context.value_callback(),
LEXY_FWD(op_args)...);
else
context.value.emplace_result(context.value_callback(), *LEXY_MOV(lhs),
LEXY_FWD(op_args)..., *LEXY_MOV(rhs));
if constexpr (std::is_base_of_v<infix_op_single, Operation>)
{
using op_rule = op_of<Operation>;
auto op = parse_operator<typename op_rule::op_literals>(reader);
if (op.idx < op_rule::op_literals::size)
{
using tag = typename Context::production::operator_chain_error;
auto err
= lexy::error<Reader, tag>(op.cur.position(), reader.position());
context.on(_ev::error{}, err);
}
reader.reset(op.cur);
}
}
else if constexpr (binding_power.is_postfix())
{
auto value = LEXY_MOV(context.value);
context.value = {};
if constexpr (value_type_void)
context.value.emplace_result(context.value_callback(),
LEXY_FWD(op_args)...);
else
context.value.emplace_result(context.value_callback(), *LEXY_MOV(value),
LEXY_FWD(op_args)...);
}
context.on(_ev::operation_chain_op{}, Operation{}, reader.position());
return true;
}
};
template <typename Context, typename Reader>
static constexpr bool parse(Context& context, Reader& reader,
lexy::_detail::parsed_operator<Reader> op, _state& state)
{
using namespace lexy::_detail;
using production = typename Context::production;
// Check whether we might have nested to far.
if (state.cur_nesting_level++ >= production::max_operator_nesting)
{
using tag = typename production::operator_nesting_error;
auto err = lexy::error<Reader, tag>(op.cur.position(), reader.position());
context.on(_ev::error{}, err);
// We do not recover, to prevent stack overflow.
reader.reset(op.cur);
return false;
}
// If the operator is part of a group, check whether it matches.
constexpr auto binding_power = binding_power_of<production>(Operation{});
if constexpr (binding_power.group != 0)
{
if (state.cur_group == 0)
{
// We didn't have any operator group yet, set it.
state.cur_group = binding_power.group;
}
else if (state.cur_group != binding_power.group)
{
// Operators can't be grouped.
using tag = typename production::operator_group_error;
auto err = lexy::error<Reader, tag>(op.cur.position(), reader.position());
context.on(_ev::error{}, err);
// Trivially recover, but don't update group:
// let the first one stick.
}
}
// Finish the operator and parse a RHS, if necessary.
return op_of<Operation>::template op_finish<_op_cont>(context, reader, op, state);
}
};
template <unsigned MinBindingPower, typename Context, typename Reader>
static constexpr bool _parse_lhs(Context& context, Reader& reader, _state& state)
{
using namespace lexy::_detail;
using op_list = pre_operation_list_of<typename Context::production, MinBindingPower>;
using atom_parser
= lexy::parser_for<LEXY_DECAY_DECLTYPE(Context::production::atom), final_parser>;
if constexpr (op_list::size == 0)
{
// We don't have any prefix operators, so parse an atom directly.
(void)state;
return atom_parser::parse(context, reader);
}
else
{
auto op = lexy::_detail::parse_operator<typename op_list::ops>(reader);
if (op.idx >= op_list::ops::size)
{
// We don't have a prefix operator, so it must be an atom.
reader.reset(op.cur);
return atom_parser::parse(context, reader);
}
auto start_event = context.on(_ev::operation_chain_start{}, op.cur.position());
auto result = op_list::template apply<_continuation>(context, reader, op, state);
context.on(_ev::operation_chain_finish{}, LEXY_MOV(start_event), reader.position());
return result;
}
}
template <unsigned MinBindingPower, typename Context, typename Reader>
static constexpr bool _parse(Context& context, Reader& reader, _state& state)
{
using namespace lexy::_detail;
using op_list = post_operation_list_of<typename Context::production, MinBindingPower>;
if constexpr (op_list::size == 0)
{
// We don't have any post operators, so we only parse the left-hand-side.
return _parse_lhs<MinBindingPower>(context, reader, state);
}
else
{
auto start_event = context.on(_ev::operation_chain_start{}, reader.position());
if (!_parse_lhs<MinBindingPower>(context, reader, state))
{
context.on(_ev::operation_chain_finish{}, LEXY_MOV(start_event), reader.position());
return false;
}
auto result = true;
while (true)
{
auto op = parse_operator<typename op_list::ops>(reader);
if (op.idx >= op_list::ops::size)
{
reader.reset(op.cur);
break;
}
result = op_list::template apply<_continuation>(context, reader, op, state);
if (!result)
break;
}
context.on(_ev::operation_chain_finish{}, LEXY_MOV(start_event), reader.position());
return result;
}
return false; // unreachable
}
template <typename NextParser>
struct p
{
template <typename Context, typename Reader>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader)
{
static_assert(std::is_same_v<NextParser, lexy::_detail::final_parser>);
using production = typename Context::production;
constexpr auto binding_power = lexy::_detail::binding_power_of<production>(
lexy::_detail::type_or<RootOperation, typename production::operation>{});
// The MinBindingPower is determined by the root operation.
// The initial operand is always on the left, so we use the left binding power.
// However, for a prefix operator it is zero, but then it's a right operand so we use
// that.
constexpr auto min_binding_power
= binding_power.is_prefix() ? binding_power.rhs : binding_power.lhs;
_state state;
_parse<min_binding_power>(context, reader, state);
// Regardless of parse errors, we can recover if we already had a value at some point.
return !!context.value;
}
};
};
} // namespace lexyd
//=== expression_production ===//
namespace lexy
{
struct max_operator_nesting_exceeded
{
static LEXY_CONSTEVAL auto name()
{
return "maximum operator nesting level exceeded";
}
};
struct operator_chain_error
{
static LEXY_CONSTEVAL auto name()
{
return "operator cannot be chained";
}
};
struct operator_group_error
{
static LEXY_CONSTEVAL auto name()
{
return "operator cannot be mixed with previous operators";
}
};
struct expression_production
{
using operator_nesting_error = lexy::max_operator_nesting_exceeded;
static constexpr auto max_operator_nesting = 256; // arbitrary power of two
using operator_chain_error = lexy::operator_chain_error;
using operator_group_error = lexy::operator_group_error;
static constexpr auto rule = lexyd::_expr<void>{};
};
template <typename Expr, typename RootOperation>
struct subexpression_production : Expr
{
static constexpr auto rule = lexyd::_expr<RootOperation>{};
};
} // namespace lexy
#endif // LEXY_DSL_EXPRESSION_HPP_INCLUDED

View File

@@ -0,0 +1,141 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_FLAGS_HPP_INCLUDED
#define LEXY_DSL_FLAGS_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/error.hpp>
namespace lexy
{
struct duplicate_flag
{
static LEXY_CONSTEVAL auto name()
{
return "duplicate flag";
}
};
} // namespace lexy
namespace lexyd
{
template <const auto& Table, typename Token, typename Tag>
struct _sym;
template <typename FlagRule, auto Default, typename DuplicateError = void>
struct _flags : rule_base
{
using _enum_type = LEXY_DECAY_DECLTYPE(Default);
using _int_type = std::underlying_type_t<_enum_type>;
template <typename NextParser>
struct p
{
template <typename Context, typename Reader>
static constexpr bool _parse(_int_type& result, Context& context, Reader& reader)
{
result = _int_type(Default);
while (true)
{
auto begin = reader.position();
lexy::branch_parser_for<FlagRule, Reader> bp{};
if (!bp.try_parse(context.control_block, reader))
{
bp.cancel(context);
break;
}
if (!bp.template finish<lexy::pattern_parser<_enum_type>>(context, reader))
return false;
auto flag = _int_type(bp.value());
if ((result & flag) == flag)
{
using tag = lexy::_detail::type_or<DuplicateError, lexy::duplicate_flag>;
auto err = lexy::error<Reader, tag>(begin, reader.position());
context.on(_ev::error{}, err);
// We can trivially recover.
}
result |= flag;
}
return true;
}
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
_int_type result{};
if (!_parse(result, context, reader))
return false;
return NextParser::parse(context, reader, LEXY_FWD(args)..., _enum_type(result));
}
};
template <typename Tag>
static constexpr _flags<FlagRule, Default, Tag> error = {};
};
template <auto Default, const auto& Table, typename Token, typename Tag>
constexpr auto flags(_sym<Table, Token, Tag> flag_rule)
{
using table_type = LEXY_DECAY_DECLTYPE(Table);
using enum_type = LEXY_DECAY_DECLTYPE(Default);
static_assert(std::is_same_v<enum_type, typename table_type::mapped_type>);
static_assert(std::is_enum_v<enum_type>);
return _flags<decltype(flag_rule), Default>{};
}
template <const auto& Table, typename Token, typename Tag>
constexpr auto flags(_sym<Table, Token, Tag> flag_rule)
{
using table_type = LEXY_DECAY_DECLTYPE(Table);
using enum_type = typename table_type::mapped_type;
static_assert(std::is_enum_v<enum_type>);
return _flags<decltype(flag_rule), enum_type{}>{};
}
} // namespace lexyd
namespace lexyd
{
template <typename Rule, auto If, auto Else>
struct _flag : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
lexy::branch_parser_for<Rule, Reader> branch{};
if (branch.try_parse(context.control_block, reader))
return branch.template finish<NextParser>(context, reader, LEXY_FWD(args)..., If);
else
{
branch.cancel(context);
return NextParser::parse(context, reader, LEXY_FWD(args)..., Else);
}
}
};
};
template <auto If, auto Else = LEXY_DECAY_DECLTYPE(If){}, typename Rule>
constexpr auto flag(Rule)
{
LEXY_REQUIRE_BRANCH_RULE(Rule, "flag()");
return _flag<Rule, If, Else>{};
}
template <typename Rule>
constexpr auto flag(Rule rule)
{
return flag<true, false>(rule);
}
} // namespace lexyd
#endif // LEXY_DSL_FLAGS_HPP_INCLUDED

View File

@@ -0,0 +1,122 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_FOLLOW_HPP_INCLUDED
#define LEXY_DSL_FOLLOW_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/char_class.hpp>
#include <lexy/dsl/literal.hpp>
namespace lexy
{
struct follow_restriction
{
static LEXY_CONSTEVAL auto name()
{
return "follow restriction";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Literal, typename CharClass>
struct _nf : token_base<_nf<Literal, CharClass>>, _lit_base
{
static constexpr auto lit_max_char_count = Literal::lit_max_char_count;
static constexpr auto lit_char_classes = lexy::_detail::char_class_list<CharClass>{};
using lit_case_folding = typename Literal::lit_case_folding;
template <typename Encoding>
static constexpr auto lit_first_char() -> typename Encoding::char_type
{
return Literal::template lit_first_char<Encoding>();
}
template <typename Trie>
static LEXY_CONSTEVAL std::size_t lit_insert(Trie& trie, std::size_t pos,
std::size_t char_class)
{
auto end = Literal::lit_insert(trie, pos, char_class);
trie.node_char_class[end] = char_class;
return end;
}
template <typename Reader>
struct tp
{
lexy::token_parser_for<Literal, Reader> impl;
typename Reader::marker end;
bool literal_success;
constexpr explicit tp(const Reader& reader)
: impl(reader), end(reader.current()), literal_success(false)
{}
constexpr bool try_parse(Reader reader)
{
literal_success = false;
// Need to match the literal.
if (!impl.try_parse(reader))
return false;
end = impl.end;
literal_success = true;
// To match, we must not match the char class now.
reader.reset(end);
if constexpr (std::is_void_v<lit_case_folding>)
{
return !lexy::try_match_token(CharClass{}, reader);
}
else
{
typename lit_case_folding::template reader<Reader> case_folded{reader};
return !lexy::try_match_token(CharClass{}, case_folded);
}
}
template <typename Context>
constexpr void report_error(Context& context, Reader reader)
{
if (!literal_success)
{
impl.report_error(context, reader);
}
else
{
auto err
= lexy::error<Reader, lexy::follow_restriction>(end.position(), end.position());
context.on(_ev::error{}, err);
}
}
};
};
/// Match a literal but only if not followed by the given char class.
template <typename Literal, typename CharClass>
constexpr auto not_followed_by(Literal, CharClass cc)
{
static_assert(lexy::is_literal_rule<Literal> && Literal::lit_char_classes.size == 0);
return _nf<Literal, decltype(_make_char_class(cc))>{};
}
/// Match a literal but only if followed by the given char class.
template <typename Literal, typename CharClass>
constexpr auto followed_by(Literal lit, CharClass cc)
{
return not_followed_by(lit, -cc);
}
} // namespace lexyd
namespace lexy
{
template <typename Literal, typename CharClass>
constexpr auto token_kind_of<lexy::dsl::_nf<Literal, CharClass>> = lexy::literal_token_kind;
} // namespace lexy
#endif // LEXY_DSL_FOLLOW_HPP_INCLUDED

View File

@@ -0,0 +1,414 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_IDENTIFIER_HPP_INCLUDED
#define LEXY_DSL_IDENTIFIER_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/char_class.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/dsl/token.hpp>
#include <lexy/lexeme.hpp>
//=== identifier ===//
namespace lexy
{
/// Error when we matched a reserved.
struct reserved_identifier
{
static LEXY_CONSTEVAL auto name()
{
return "reserved identifier";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Id, typename CharT, CharT... C>
struct _kw;
template <typename Literal, template <typename> typename CaseFolding>
struct _cfl;
template <typename Leading, typename Trailing>
struct _idp : token_base<_idp<Leading, Trailing>>
{
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
static_assert(lexy::is_char_encoding<typename Reader::encoding>);
// Need to match Leading character.
if (!lexy::try_match_token(Leading{}, reader))
return false;
// Match zero or more trailing characters.
while (true)
{
if constexpr (lexy::_detail::is_swar_reader<Reader>)
{
// If we have a swar reader, consume as much as possible at once.
while (Trailing{}.template char_class_match_swar<typename Reader::encoding>(
reader.peek_swar()))
reader.bump_swar();
}
if (!lexy::try_match_token(Trailing{}, reader))
break;
}
end = reader.current();
return true;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader& reader)
{
Leading::template char_class_report_error<Reader>(context, reader.position());
}
};
};
template <typename Set>
struct _idrp // reserve predicate
{
template <typename Input>
static constexpr bool is_reserved(const Input& input)
{
auto reader = input.reader();
return lexy::try_match_token(Set{}, reader)
&& reader.peek() == decltype(reader)::encoding::eof();
}
};
template <typename Set>
struct _idpp // reserve prefix predicate
{
template <typename Input>
static constexpr bool is_reserved(const Input& input)
{
auto reader = input.reader();
return lexy::try_match_token(Set{}, reader);
}
};
template <typename Set>
struct _idcp // reserve contains predicate
{
template <typename Input>
static constexpr bool is_reserved(const Input& input)
{
auto reader = input.reader();
while (true)
{
if (lexy::try_match_token(Set{}, reader))
return true;
else if (reader.peek() == decltype(reader)::encoding::eof())
return false;
else
reader.bump();
}
// unreachable
}
};
template <typename Set>
struct _idsp // reserve suffix predicate
{
template <typename Input>
static constexpr bool is_reserved(const Input& input)
{
auto reader = input.reader();
while (true)
{
if (lexy::try_match_token(Set{}, reader)
&& reader.peek() == decltype(reader)::encoding::eof())
return true;
else if (reader.peek() == decltype(reader)::encoding::eof())
return false;
else
reader.bump();
}
// unreachable
}
};
template <typename Leading, typename Trailing, typename... ReservedPredicate>
struct _id : branch_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
// Parse the pattern; this does not consume whitespace, so the range is accurate.
auto begin = reader.position();
if (!pattern().token_parse(context, reader))
return false;
auto end = reader.position();
// Check for a reserved identifier.
[[maybe_unused]] auto input = lexy::partial_input(reader, begin, end);
if ((ReservedPredicate::is_reserved(input) || ...))
{
// It is reserved, report an error but trivially recover.
auto err = lexy::error<Reader, lexy::reserved_identifier>(begin, end);
context.on(_ev::error{}, err);
}
// Skip whitespace and continue with the value.
using continuation = lexy::whitespace_parser<Context, NextParser>;
return continuation::parse(context, reader, LEXY_FWD(args)...,
lexy::lexeme<Reader>(begin, end));
}
};
template <typename Reader>
struct bp
{
typename Reader::marker end;
constexpr bool try_parse(const void*, const Reader& reader)
{
// Parse the pattern.
lexy::token_parser_for<decltype(pattern()), Reader> parser(reader);
if (!parser.try_parse(reader))
return false;
end = parser.end;
// We only succeed if it's not a reserved identifier.
[[maybe_unused]] auto input
= lexy::partial_input(reader, reader.position(), end.position());
return !(ReservedPredicate::is_reserved(input) || ...);
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC auto finish(Context& context, Reader& reader, Args&&... args)
{
auto begin = reader.position();
context.on(_ev::token{}, lexy::identifier_token_kind, begin, end.position());
reader.reset(end);
using continuation = lexy::whitespace_parser<Context, NextParser>;
return continuation::parse(context, reader, LEXY_FWD(args)...,
lexy::lexeme<Reader>(begin, end.position()));
}
};
template <typename R>
constexpr auto _make_reserve(R r) const
{
static_assert(lexy::is_literal_rule<R> || lexy::is_literal_set_rule<R>);
return r;
}
template <typename Id, typename CharT, CharT... C>
constexpr auto _make_reserve(_kw<Id, CharT, C...>) const
{
static_assert(std::is_same_v<decltype(Id{}.pattern()), decltype(pattern())>,
"must not reserve keywords from another identifier");
// No need to remember that it was a keyword originally.
return _lit<CharT, C...>{};
}
template <typename Id, typename CharT, CharT... C, template <typename> typename CaseFolding>
constexpr auto _make_reserve(_cfl<_kw<Id, CharT, C...>, CaseFolding>) const
{
static_assert(std::is_same_v<decltype(Id{}.pattern()), decltype(pattern())>,
"must not reserve keywords from another identifier");
// No need to remember that it was a keyword originally.
return _cfl<_lit<CharT, C...>, CaseFolding>{};
}
//=== dsl ===//
/// Adds a set of reserved identifiers.
template <typename... R>
constexpr auto reserve(R... r) const
{
static_assert(sizeof...(R) > 0);
auto set = (lexyd::literal_set() / ... / _make_reserve(r));
return _id<Leading, Trailing, ReservedPredicate..., _idrp<decltype(set)>>{};
}
/// Reserves everything starting with the given rule.
template <typename... R>
constexpr auto reserve_prefix(R... r) const
{
static_assert(sizeof...(R) > 0);
auto set = (lexyd::literal_set() / ... / _make_reserve(r));
return _id<Leading, Trailing, ReservedPredicate..., _idpp<decltype(set)>>{};
}
/// Reservers everything containing the given rule.
template <typename... R>
constexpr auto reserve_containing(R... r) const
{
static_assert(sizeof...(R) > 0);
auto set = (lexyd::literal_set() / ... / _make_reserve(r));
return _id<Leading, Trailing, ReservedPredicate..., _idcp<decltype(set)>>{};
}
/// Reserves everything that ends with the given rule.
template <typename... R>
constexpr auto reserve_suffix(R... r) const
{
static_assert(sizeof...(R) > 0);
auto set = (lexyd::literal_set() / ... / _make_reserve(r));
return _id<Leading, Trailing, ReservedPredicate..., _idsp<decltype(set)>>{};
}
/// Matches every identifier, ignoring reserved ones.
static constexpr auto pattern()
{
return _idp<Leading, Trailing>{};
}
/// Matches the initial char set of an identifier.
constexpr auto leading_pattern() const
{
return Leading{};
}
/// Matches the trailing char set of an identifier.
constexpr auto trailing_pattern() const
{
return Trailing{};
}
};
/// Creates an identifier that consists of one or more of the given characters.
template <typename CharClass>
constexpr auto identifier(CharClass)
{
static_assert(lexy::is_char_class_rule<CharClass>);
return _id<CharClass, CharClass>{};
}
/// Creates an identifier that consists of one leading token followed by zero or more trailing
/// tokens.
template <typename LeadingClass, typename TrailingClass>
constexpr auto identifier(LeadingClass, TrailingClass)
{
static_assert(lexy::is_char_class_rule<LeadingClass> //
&& lexy::is_char_class_rule<TrailingClass>);
return _id<LeadingClass, TrailingClass>{};
}
} // namespace lexyd
namespace lexy
{
template <typename Leading, typename Trailing>
constexpr auto token_kind_of<lexy::dsl::_idp<Leading, Trailing>> = lexy::identifier_token_kind;
} // namespace lexy
//=== keyword ===//
namespace lexyd
{
template <typename Id, typename CharT, CharT... C>
struct _kw : token_base<_kw<Id, CharT, C...>>, _lit_base
{
static constexpr auto lit_max_char_count = sizeof...(C);
// We must not end on a trailing character.
static constexpr auto lit_char_classes
= lexy::_detail::char_class_list<decltype(Id{}.trailing_pattern())>{};
using lit_case_folding = void;
template <typename Encoding>
static constexpr auto lit_first_char() -> typename Encoding::char_type
{
typename Encoding::char_type result = 0;
(void)((result = lexy::_detail::transcode_char<decltype(result)>(C), true) || ...);
return result;
}
template <typename Trie>
static LEXY_CONSTEVAL std::size_t lit_insert(Trie& trie, std::size_t pos,
std::size_t char_class)
{
auto end = ((pos = trie.insert(pos, C)), ...);
trie.node_char_class[end] = char_class;
return end;
}
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
// Need to match the literal.
if (!lexy::_detail::match_literal<0, CharT, C...>(reader))
return false;
end = reader.current();
// To qualify as a keyword, and not just the prefix of an identifier,
// we must not have a trailing identifier character.
return !lexy::try_match_token(Id{}.trailing_pattern(), reader);
}
template <typename Context>
constexpr void report_error(Context& context, Reader reader)
{
using char_type = typename Reader::encoding::char_type;
constexpr auto str = lexy::_detail::type_string<CharT, C...>::template c_str<char_type>;
// Match the entire identifier.
auto begin = reader.position();
lexy::try_match_token(Id{}.pattern(), reader);
auto end = reader.position();
auto err = lexy::error<Reader, lexy::expected_keyword>(begin, end, str, sizeof...(C));
context.on(_ev::error{}, err);
}
};
};
template <typename Id>
struct _keyword;
template <typename L, typename T, typename... R>
struct _keyword<_id<L, T, R...>>
{
template <typename CharT, CharT... C>
using get = _kw<_id<L, T>, CharT, C...>;
};
#if LEXY_HAS_NTTP
template <lexy::_detail::string_literal Str, typename L, typename T, typename... R>
constexpr auto keyword(_id<L, T, R...>)
{
return lexy::_detail::to_type_string<_keyword<_id<L, T>>::template get, Str>{};
}
#else
template <auto C, typename L, typename T, typename... R>
constexpr auto keyword(_id<L, T, R...>)
{
return _kw<_id<L, T>, LEXY_DECAY_DECLTYPE(C), C>{};
}
#endif
#define LEXY_KEYWORD(Str, Id) \
LEXY_NTTP_STRING(::lexyd::_keyword<LEXY_DECAY_DECLTYPE(Id)>::template get, Str) {}
} // namespace lexyd
namespace lexy
{
template <typename Id, typename CharT, CharT... C>
constexpr auto token_kind_of<lexy::dsl::_kw<Id, CharT, C...>> = lexy::literal_token_kind;
} // namespace lexy
#endif // LEXY_DSL_IDENTIFIER_HPP_INCLUDED

View File

@@ -0,0 +1,49 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_IF_HPP_INCLUDED
#define LEXY_DSL_IF_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/branch.hpp>
namespace lexyd
{
template <typename Branch>
struct _if : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
lexy::branch_parser_for<Branch, Reader> branch{};
if (branch.try_parse(context.control_block, reader))
// We take the branch.
return branch.template finish<NextParser>(context, reader, LEXY_FWD(args)...);
else
{
// We don't take the branch.
branch.cancel(context);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
}
};
};
/// If the branch condition matches, matches the branch then.
template <typename Branch>
constexpr auto if_(Branch)
{
LEXY_REQUIRE_BRANCH_RULE(Branch, "if()");
if constexpr (lexy::is_unconditional_branch_rule<Branch>)
// Branch is always taken, so don't wrap in if_().
return Branch{};
else
return _if<Branch>{};
}
} // namespace lexyd
#endif // LEXY_DSL_IF_HPP_INCLUDED

View File

@@ -0,0 +1,566 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_INTEGER_HPP_INCLUDED
#define LEXY_DSL_INTEGER_HPP_INCLUDED
#include <climits>
#include <lexy/_detail/assert.hpp>
#include <lexy/code_point.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/digit.hpp>
namespace lexy
{
// Number of digits to express the given value.
template <typename Integer>
constexpr std::size_t _digit_count(int radix, Integer value)
{
LEXY_PRECONDITION(value >= Integer(0));
if (value == 0)
return 1;
std::size_t result = 0;
while (value > 0)
{
value = Integer(value / Integer(radix));
++result;
}
return result;
}
template <typename T>
struct integer_traits
{
using type = T;
static constexpr auto is_bounded = true;
static constexpr auto _max = [] {
if constexpr (std::is_same_v<T, char>)
return CHAR_MAX; // NOLINT
else if constexpr (std::is_same_v<T, signed char>)
return SCHAR_MAX;
else if constexpr (std::is_same_v<T, unsigned char>)
return UCHAR_MAX; // NOLINT
else if constexpr (std::is_same_v<T, wchar_t>)
return WCHAR_MAX; // NOLINT
#if LEXY_HAS_CHAR8_T
else if constexpr (std::is_same_v<T, char8_t>)
return UCHAR_MAX; // NOLINT
#endif
else if constexpr (std::is_same_v<T, char16_t>)
return UINT_LEAST16_MAX;
else if constexpr (std::is_same_v<T, char32_t>)
return UINT_LEAST32_MAX;
else if constexpr (std::is_same_v<T, signed short>)
return SHRT_MAX;
else if constexpr (std::is_same_v<T, unsigned short>)
return USHRT_MAX;
else if constexpr (std::is_same_v<T, signed int>)
return INT_MAX;
else if constexpr (std::is_same_v<T, unsigned int>)
return UINT_MAX;
else if constexpr (std::is_same_v<T, signed long>)
return LONG_MAX;
else if constexpr (std::is_same_v<T, unsigned long>)
return ULONG_MAX;
else if constexpr (std::is_same_v<T, signed long long>)
return LLONG_MAX;
else if constexpr (std::is_same_v<T, unsigned long long>)
return ULLONG_MAX;
#ifdef __SIZEOF_INT128__
else if constexpr (std::is_same_v<T, __int128_t>)
return __int128_t(~__uint128_t{} >> 1);
else if constexpr (std::is_same_v<T, __uint128_t>)
return ~__uint128_t{};
#endif
else
static_assert(_detail::error<T>,
"specialize integer_traits for your custom integer types");
}();
template <int Radix>
static constexpr std::size_t max_digit_count = _digit_count(Radix, _max);
template <int Radix>
static constexpr void add_digit_unchecked(T& result, unsigned digit)
{
result = T(result * T(Radix) + T(digit));
}
template <int Radix>
static constexpr bool add_digit_checked(T& result, unsigned digit)
{
constexpr auto can_use_unsigned = [] {
if constexpr (sizeof(T) >= sizeof(unsigned))
// If it's bigger or of the same size as unsigned, we can't use unsigned.
return false;
else
{
// We can do it if the worst-case does not overflow unsigned.
auto worst_case = static_cast<unsigned>(_max);
return integer_traits<unsigned>::add_digit_checked<Radix>(worst_case, Radix - 1);
}
}();
// Check whether we can do an optimization for small integers,
// where we do the operation on unsigned and check later.
if constexpr (can_use_unsigned)
{
// This can't overflow, we've checked it above.
auto value = static_cast<unsigned>(result) * Radix + digit;
// Check whether the final value can fit.
if (value > static_cast<unsigned>(_max))
return false;
else
{
result = static_cast<T>(value);
return true;
}
}
else
{
// result *= Radix
constexpr auto max_per_radix = T(_max / Radix);
if (result > max_per_radix)
return false;
result = T(result * Radix);
// result += digit
if (result > T(_max - digit))
return false;
result = T(result + T(digit));
return true;
}
}
};
template <>
struct integer_traits<code_point>
{
using type = code_point;
static constexpr auto is_bounded = true;
template <int Radix>
static constexpr std::size_t max_digit_count = _digit_count(Radix, 0x10'FFFF);
template <int Radix>
static constexpr void add_digit_unchecked(type& result, unsigned digit)
{
std::uint_least32_t value = result.value();
integer_traits<std::uint_least32_t>::add_digit_unchecked<Radix>(value, digit);
result = code_point(value);
}
template <int Radix>
static constexpr bool add_digit_checked(type& result, unsigned digit)
{
std::uint_least32_t value = result.value();
if (!integer_traits<std::uint_least32_t>::add_digit_checked<Radix>(value, digit))
return false;
result = code_point(value);
return result.is_valid();
}
};
template <typename T>
struct unbounded
{};
template <typename T>
struct integer_traits<unbounded<T>>
{
using type = typename integer_traits<T>::type;
static constexpr auto is_bounded = false;
template <int Radix>
static constexpr void add_digit_unchecked(type& result, unsigned digit)
{
integer_traits<T>::template add_digit_unchecked<Radix>(result, digit);
}
};
template <typename T, T Max>
struct bounded
{};
template <typename T, T Max>
struct integer_traits<bounded<T, Max>>
{
using type = typename integer_traits<T>::type;
static constexpr auto is_bounded = true;
template <int Radix>
static constexpr std::size_t max_digit_count = _digit_count(Radix, Max);
template <int Radix>
static constexpr void add_digit_unchecked(type& result, unsigned digit)
{
integer_traits<T>::template add_digit_unchecked<Radix>(result, digit);
}
template <int Radix>
static constexpr bool add_digit_checked(type& result, unsigned digit)
{
return integer_traits<T>::template add_digit_checked<Radix>(result, digit) && result <= Max;
}
};
} // namespace lexy
namespace lexy
{
struct integer_overflow
{
static LEXY_CONSTEVAL auto name()
{
return "integer overflow";
}
};
} // namespace lexy
namespace lexyd
{
template <typename T>
constexpr bool _is_bounded = lexy::integer_traits<T>::is_bounded;
template <typename T, std::size_t N, int Radix>
constexpr bool _ndigits_can_overflow()
{
using traits = lexy::integer_traits<T>;
auto max_digit_count = traits::template max_digit_count<Radix>;
// We don't know whether the maximal value is a power of Radix,
// so we have to be conservative and don't rule out overflow on the same count.
return N >= max_digit_count;
}
// Parses T in the Base without checking for overflow.
template <typename T, typename Base>
struct _unbounded_integer_parser
{
using traits = lexy::integer_traits<T>;
using base = Base;
static constexpr auto radix = Base::digit_radix;
struct result_type
{
typename traits::type value;
std::false_type overflow;
};
template <typename Iterator>
static constexpr result_type parse(Iterator cur, Iterator end)
{
typename traits::type value(0);
// Just parse digits until we've run out of digits.
while (cur != end)
{
auto digit = Base::digit_value(*cur++);
if (digit >= Base::digit_radix)
// Skip digit separator.
continue;
traits::template add_digit_unchecked<radix>(value, digit);
}
return {value, {}};
}
};
// Parses T in the Base while checking for overflow.
template <typename T, typename Base, bool AssumeOnlyDigits>
struct _bounded_integer_parser
{
using traits = lexy::integer_traits<T>;
using base = Base;
static constexpr auto radix = Base::digit_radix;
static constexpr auto max_digit_count = traits::template max_digit_count<radix>;
static_assert(max_digit_count > 1, "integer must be able to store all possible digit values");
struct result_type
{
typename traits::type value;
bool overflow;
};
template <typename Iterator>
static constexpr result_type parse(Iterator cur, Iterator end)
{
// Find the first non-zero digit.
// Note that we always need a loop, even if leading zeros are not allowed:
// error recovery might get them anyway.
auto first_digit = 0u;
while (true)
{
if (cur == end)
return {typename traits::type(0), false};
first_digit = Base::digit_value(*cur++);
if (first_digit != 0 && first_digit < radix)
break;
}
// At this point, we've parsed exactly one non-zero digit, so we can assign.
auto value = typename traits::type(first_digit);
// Handle at most the number of remaining digits.
// Due to the fixed loop count, it is most likely unrolled.
for (std::size_t digit_count = 1; digit_count < max_digit_count; ++digit_count)
{
// Find the next digit.
auto digit = 0u;
do
{
if (cur == end)
return {value, false};
digit = Base::digit_value(*cur++);
// If we can assume it's a digit, we don't need the comparison.
} while (AssumeOnlyDigits ? false : digit >= Base::digit_radix);
// We need to handle the last loop iteration special.
// (The compiler will not generate a branch here.)
if (digit_count == max_digit_count - 1)
{
// The last digit might overflow, so check for it.
if (!traits::template add_digit_checked<radix>(value, digit))
return {value, true};
}
else
{
// Add the digit without checking as it can't overflow.
traits::template add_digit_unchecked<radix>(value, digit);
}
}
// If we've reached this point, we've parsed the maximal number of digits allowed.
// Now we can only overflow if there are still digits left.
return {value, cur != end};
}
};
template <typename T, typename Base, bool AssumeOnlyDigits>
using _integer_parser
= std::conditional_t<_is_bounded<T>, _bounded_integer_parser<T, Base, AssumeOnlyDigits>,
_unbounded_integer_parser<T, Base>>;
template <typename T, typename Digits>
struct _integer_parser_digits;
template <typename T, typename Base>
struct _integer_parser_digits<T, _digits<Base>>
{
using type = _integer_parser<T, Base, true>;
};
template <typename T, typename Base>
struct _integer_parser_digits<T, _digits_t<Base>>
{
using type = _integer_parser<T, Base, true>;
};
template <typename T, typename Base, typename Sep>
struct _integer_parser_digits<T, _digits_s<Base, Sep>>
{
using type = _integer_parser<T, Base, false>;
};
template <typename T, typename Base, typename Sep>
struct _integer_parser_digits<T, _digits_st<Base, Sep>>
{
using type = _integer_parser<T, Base, false>;
};
template <typename T, std::size_t N, typename Base>
struct _integer_parser_digits<T, _ndigits<N, Base>>
{
using value_type = std::conditional_t<_ndigits_can_overflow<T, N, Base::digit_radix>(), T,
lexy::unbounded<T>>;
using type = _integer_parser<value_type, Base, true>;
};
template <typename T, std::size_t N, typename Base, typename Sep>
struct _integer_parser_digits<T, _ndigits_s<N, Base, Sep>>
{
using value_type = std::conditional_t<_ndigits_can_overflow<T, N, Base::digit_radix>(), T,
lexy::unbounded<T>>;
using type = _integer_parser<value_type, Base, false>;
};
template <typename T, typename Digits>
using _integer_parser_for = typename _integer_parser_digits<T, Digits>::type;
template <typename Token, typename IntParser, typename Tag>
struct _int : _copy_base<Token>
{
template <typename NextParser>
struct _pc
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader,
typename Reader::iterator begin,
typename Reader::iterator end, Args&&... args)
{
auto [value, overflow] = IntParser::parse(begin, end);
if (overflow)
{
// Raise error but recover.
using tag = lexy::_detail::type_or<Tag, lexy::integer_overflow>;
context.on(_ev::error{}, lexy::error<Reader, tag>(begin, end));
}
// Need to skip whitespace now as well.
return lexy::whitespace_parser<Context, NextParser>::parse(context, reader,
LEXY_FWD(args)..., value);
}
};
template <typename Reader>
struct bp
{
typename Reader::marker end;
constexpr auto try_parse(const void*, const Reader& reader)
{
lexy::token_parser_for<Token, Reader> parser(reader);
auto result = parser.try_parse(reader);
end = parser.end;
return result;
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
auto begin = reader.position();
context.on(_ev::token{}, Token{}, begin, end.position());
reader.reset(end);
return _pc<NextParser>::parse(context, reader, begin, end.position(),
LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
auto begin = reader.position();
if (lexy::token_parser_for<Token, Reader> parser(reader); parser.try_parse(reader))
{
context.on(_ev::token{}, typename Token::token_type{}, begin,
parser.end.position());
reader.reset(parser.end);
}
else
{
parser.report_error(context, reader);
reader.reset(parser.end);
// To recover we try and skip additional digits.
while (lexy::try_match_token(digit<typename IntParser::base>, reader))
{
}
auto recovery_end = reader.position();
if (begin == recovery_end)
{
// We didn't get any digits; couldn't recover.
// We don't report error recovery events as nothing was done;
// we don't need to create an error token as nothing was consumed.
return false;
}
else
{
// We've succesfully recovered, mark everything as digits.
context.on(_ev::recovery_start{}, begin);
context.on(_ev::token{}, lexy::digits_token_kind, begin, recovery_end);
context.on(_ev::recovery_finish{}, recovery_end);
}
}
auto end = reader.position();
return _pc<NextParser>::parse(context, reader, begin, end, LEXY_FWD(args)...);
}
};
};
template <typename T, typename Base>
struct _int_dsl : _int<_digits<lexy::_detail::type_or<Base, decimal>>,
_integer_parser_for<T, _digits<lexy::_detail::type_or<Base, decimal>>>, void>
{
template <typename Digits>
constexpr auto operator()(Digits) const
{
static_assert(lexy::is_token_rule<Digits>);
if constexpr (std::is_void_v<Base>)
{
// Digits is a known rule as the user didn't specify Base.
return _int<Digits, _integer_parser_for<T, Digits>, void>{};
}
else
{
// User has specified a base, so the digits are arbitrary.
using parser = _integer_parser<T, Base, false>;
return _int<Digits, parser, void>{};
}
}
};
/// Parses the digits matched by the rule into an integer type.
template <typename T, typename Base = void>
constexpr auto integer = _int_dsl<T, Base>{};
} // namespace lexyd
//=== code_point_id ===//
namespace lexy
{
struct invalid_code_point
{
static LEXY_CONSTEVAL auto name()
{
return "invalid code point";
}
};
} // namespace lexy
namespace lexyd
{
/// Matches the integer value of a code point.
template <std::size_t N, typename Base = hex>
constexpr auto code_point_id = [] {
using type = std::conditional_t<_ndigits_can_overflow<lexy::code_point, N, Base::digit_radix>(),
lexy::code_point, lexy::unbounded<lexy::code_point>>;
using parser = _integer_parser<type, Base, true>;
return _int<_ndigits<N, Base>, parser, lexy::invalid_code_point>{};
}();
} // namespace lexyd
//=== code_unit_id ===//
namespace lexy
{
struct invalid_code_unit
{
static LEXY_CONSTEVAL auto name()
{
return "invalid code unit";
}
};
} // namespace lexy
namespace lexyd
{
/// Matches the integer value of a code unit.
template <typename Encoding, std::size_t N, typename Base = hex>
constexpr auto code_unit_id = [] {
using char_type = typename Encoding::char_type;
using type = std::conditional_t<_ndigits_can_overflow<char_type, N, Base::digit_radix>(),
char_type, lexy::unbounded<char_type>>;
using parser = _integer_parser<type, Base, true>;
return _int<_ndigits<N, Base>, parser, lexy::invalid_code_unit>{};
}();
} // namespace lexyd
#endif // LEXY_DSL_INTEGER_HPP_INCLUDED

View File

@@ -0,0 +1,448 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_LIST_HPP_INCLUDED
#define LEXY_DSL_LIST_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/choice.hpp>
#include <lexy/dsl/option.hpp>
#include <lexy/dsl/separator.hpp>
namespace lexyd
{
template <typename Item, typename Sep>
struct _lst : _copy_base<Item>
{
template <typename Context, typename Reader, typename Sink>
LEXY_PARSER_FUNC static bool _loop(Context& context, Reader& reader, Sink& sink)
{
while (true)
{
// Parse a separator if necessary.
[[maybe_unused]] auto sep_begin = reader.position();
if constexpr (!std::is_void_v<Sep>)
{
lexy::branch_parser_for<typename Sep::rule, Reader> sep{};
if (!sep.try_parse(context.control_block, reader))
{
// We didn't have a separator, list is definitely finished.
sep.cancel(context);
break;
}
if (!sep.template finish<lexy::sink_parser>(context, reader, sink))
return false;
}
[[maybe_unused]] auto sep_end = reader.position();
// Parse the next item.
if constexpr (lexy::is_branch_rule<Item>)
{
// It's a branch, so try parsing it to detect loop exit.
lexy::branch_parser_for<Item, Reader> item{};
if (!item.try_parse(context.control_block, reader))
{
// We don't have a next item, exit the loop.
// If necessary, we report a trailing separator.
item.cancel(context);
if constexpr (!std::is_void_v<Sep>)
Sep::report_trailing_error(context, reader, sep_begin, sep_end);
break;
}
// We're having an item, finish it.
if (!item.template finish<lexy::sink_parser>(context, reader, sink))
return false;
}
else
{
// Not a branch, so we need one item.
if (!lexy::parser_for<Item, lexy::sink_parser>::parse(context, reader, sink))
return false;
}
}
return true;
}
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
// Construct the sink.
auto sink = context.value_callback().sink();
// Parse the first item.
if (!lexy::parser_for<Item, lexy::sink_parser>::parse(context, reader, sink))
return false;
// Parse the remaining items.
if (!_loop(context, reader, sink))
return false;
// We're done with the list, finish the sink and continue.
return lexy::sink_finish_parser<NextParser>::parse(context, reader, sink,
LEXY_FWD(args)...);
}
};
template <typename Reader>
struct bp
{
lexy::branch_parser_for<Item, Reader> item;
template <typename ControlBlock>
constexpr bool try_parse(const ControlBlock* cb, const Reader& reader)
{
// We parse a list if we can parse its first item.
return item.try_parse(cb, reader);
}
template <typename Context>
constexpr void cancel(Context& context)
{
return item.cancel(context);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
// At this point, we have a list so construct a sink.
auto sink = context.value_callback().sink();
// Finish the first item, passing all values to the sink.
if (!item.template finish<lexy::sink_parser>(context, reader, sink))
return false;
// Parse the remaining items.
if (!_loop(context, reader, sink))
return false;
// We're done with the list, finish the sink and continue.
return lexy::sink_finish_parser<NextParser>::parse(context, reader, sink,
LEXY_FWD(args)...);
}
};
};
/// Parses a list of items without a separator.
template <typename Item>
constexpr auto list(Item)
{
LEXY_REQUIRE_BRANCH_RULE(Item, "list() without a separator");
return _lst<Item, void>{};
}
/// Parses a list of items with the specified separator.
template <typename Item, typename Sep, typename Tag>
constexpr auto list(Item, _sep<Sep, Tag>)
{
return _lst<Item, _sep<Sep, Tag>>{};
}
/// Parses a list of items with the specified separator that can be trailing.
template <typename Item, typename Sep>
constexpr auto list(Item, _tsep<Sep>)
{
LEXY_REQUIRE_BRANCH_RULE(Item, "list() with a trailing separator");
return _lst<Item, _tsep<Sep>>{};
}
template <typename Item, typename Sep>
constexpr auto list(Item, _isep<Sep>)
{
static_assert(lexy::_detail::error<Item, Sep>,
"list() does not support `dsl::ignore_trailing_sep()`");
return _lst<Item, void>{};
}
} // namespace lexyd
namespace lexyd
{
template <typename Term, typename Item, typename Sep, typename Recover>
struct _lstt : rule_base
{
// We're using an enum together with a switch to compensate a lack of goto in constexpr.
// The simple state machine goes as follows on well-formed input:
// terminator -> separator -> separator_trailing_check -> item -> terminator -> ... ->
// done
//
// The interesting case is error recovery.
// There we skip over characters until we either found the terminator, separator or
// item. We then set the enum to jump to the appropriate state of the state machine.
enum class _state
{
terminator,
separator,
separator_trailing_check,
item,
recovery,
};
template <typename TermParser, typename Context, typename Reader, typename Sink>
LEXY_PARSER_FUNC static bool _loop(_state initial_state, TermParser& term, Context& context,
Reader& reader, Sink& sink)
{
auto state = initial_state;
[[maybe_unused]] auto sep_pos = reader.position();
while (true)
{
switch (state)
{
case _state::terminator:
if (term.try_parse(context.control_block, reader))
// We had the terminator, so the list is done.
return true;
term.cancel(context);
// Parse the following list separator next.
state = _state::separator;
break;
case _state::separator:
if constexpr (!std::is_void_v<Sep>)
{
sep_pos = reader.position();
if (lexy::parser_for<typename Sep::rule, lexy::sink_parser>::parse(context,
reader,
sink))
{
// Check for a trailing separator next.
state = _state::separator_trailing_check;
break;
}
else if (sep_pos == reader.position())
{
// We don't have a separator at all.
// Assume it's missing and parse an item instead.
if constexpr (lexy::is_branch_rule<Item>)
{
lexy::branch_parser_for<Item, Reader> item{};
if (item.try_parse(context.control_block, reader)
&& item.template finish<lexy::sink_parser>(context, reader, sink))
{
// Continue after an item has been parsed.
state = _state::terminator;
break;
}
else
{
// Not an item, recover.
item.cancel(context);
state = _state::recovery;
break;
}
}
else
{
// We cannot try and parse an item.
// To avoid generating wrong errors, immediately recover.
state = _state::recovery;
break;
}
}
else
{
// We did have something that looked like a separator initially, but
// wasn't one on closer inspection. Enter generic recovery as we've
// already consumed input. (If we ignore the case where the item and
// separator share a common prefix, we know it wasn't the start of an
// item so can't just pretend that there is one).
state = _state::recovery;
break;
}
}
else
{
// List doesn't have a separator; immediately parse item next.
state = _state::item;
break;
}
case _state::separator_trailing_check:
if constexpr (!std::is_void_v<Sep>)
{
// We need to check whether we're having a trailing separator by checking
// for a terminating one.
if (term.try_parse(context.control_block, reader))
{
// We had the terminator, so the list is done.
// Report a trailing separator error if necessary.
Sep::report_trailing_error(context, reader, sep_pos, reader.position());
return true;
}
else
{
// We didn't have a separator, parse item next.
state = _state::item;
break;
}
}
break;
case _state::item:
if (lexy::parser_for<Item, lexy::sink_parser>::parse(context, reader, sink))
{
// Loop back.
state = _state::terminator;
break;
}
else
{
// Recover from missing item.
state = _state::recovery;
break;
}
case _state::recovery: {
auto recovery_begin = reader.position();
context.on(_ev::recovery_start{}, recovery_begin);
while (true)
{
// Recovery succeeds when we reach the next separator.
if constexpr (!std::is_void_v<Sep>)
{
sep_pos = reader.position();
lexy::branch_parser_for<typename Sep::rule, Reader> sep{};
if (sep.try_parse(context.control_block, reader))
{
auto recovery_end = reader.position();
context.on(_ev::token{}, lexy::error_token_kind, recovery_begin,
recovery_end);
context.on(_ev::recovery_finish{}, recovery_end);
if (sep.template finish<lexy::sink_parser>(context, reader, sink))
{
// Continue the list with the trailing separator check.
state = _state::separator_trailing_check;
break;
}
else
{
// Need to recover from this.
state = _state::recovery;
break;
}
}
else
{
sep.cancel(context);
}
}
// When we don't have a separator, but the item is a branch, we also succeed
// when we reach the next item.
//
// Note that we're doing this check only if we don't have a separator.
// If we do have one, the heuristic "end of the invalid item" is better than
// "beginning of the next one".
else if constexpr (lexy::is_branch_rule<Item>)
{
lexy::branch_parser_for<Item, Reader> item{};
if (item.try_parse(context.control_block, reader))
{
auto recovery_end = reader.position();
context.on(_ev::token{}, lexy::error_token_kind, recovery_begin,
recovery_end);
context.on(_ev::recovery_finish{}, recovery_end);
if (item.template finish<lexy::sink_parser>(context, reader, sink))
{
// Continue the list with the next terminator check.
state = _state::terminator;
break;
}
else
{
// Need to recover from this.
state = _state::recovery;
break;
}
}
else
{
item.cancel(context);
}
}
// At this point, we couldn't detect the next item.
// Recovery succeeds when we reach the terminator.
if (term.try_parse(context.control_block, reader))
{
// We're now done with the entire list.
auto recovery_end = reader.position();
context.on(_ev::token{}, lexy::error_token_kind, recovery_begin,
recovery_end);
context.on(_ev::recovery_finish{}, recovery_end);
return true;
}
else
{
term.cancel(context);
}
// At this point, we couldn't detect the next item or a terminator.
// Recovery fails when we reach the limit.
using limit_rule = decltype(Recover{}.get_limit());
if (lexy::token_parser_for<limit_rule, Reader> limit(reader);
limit.try_parse(reader) || reader.peek() == Reader::encoding::eof())
{
// Recovery has failed, propagate error.
auto recovery_end = reader.position();
context.on(_ev::token{}, lexy::error_token_kind, recovery_begin,
recovery_end);
context.on(_ev::recovery_cancel{}, recovery_end);
return false;
}
// Consume one code unit and try again.
reader.bump();
}
break;
}
}
}
return false; // unreachable
}
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
lexy::branch_parser_for<Term, Reader> term{};
auto sink = context.value_callback().sink();
// Parse initial item.
using item_parser = lexy::parser_for<Item, lexy::sink_parser>;
auto result = item_parser::parse(context, reader, sink);
// Parse the remaining items.
if (!_loop(result ? _state::terminator : _state::recovery, term, context, reader, sink))
return false;
// At this point, we just need to finish parsing the terminator.
if constexpr (std::is_same_v<typename decltype(sink)::return_type, void>)
{
LEXY_MOV(sink).finish();
return term.template finish<NextParser>(context, reader, LEXY_FWD(args)...);
}
else
{
return term.template finish<NextParser>(context, reader, LEXY_FWD(args)...,
LEXY_MOV(sink).finish());
}
}
};
};
} // namespace lexyd
#endif // LEXY_DSL_LIST_HPP_INCLUDED

View File

@@ -0,0 +1,625 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_LITERAL_HPP_INCLUDED
#define LEXY_DSL_LITERAL_HPP_INCLUDED
#include <lexy/_detail/code_point.hpp>
#include <lexy/_detail/integer_sequence.hpp>
#include <lexy/_detail/iterator.hpp>
#include <lexy/_detail/nttp_string.hpp>
#include <lexy/_detail/swar.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/token.hpp>
//=== lit_matcher ===//
namespace lexy::_detail
{
template <std::size_t CurCharIndex, typename CharT, CharT... Cs, typename Reader>
constexpr auto match_literal(Reader& reader)
{
static_assert(lexy::is_char_encoding<typename Reader::encoding>);
using char_type = typename Reader::encoding::char_type;
if constexpr (CurCharIndex >= sizeof...(Cs))
{
(void)reader;
return std::true_type{};
}
// We only use SWAR if the reader supports it and we have enough to fill at least one.
else if constexpr (is_swar_reader<Reader> && sizeof...(Cs) >= swar_length<char_type>)
{
// Try and pack as many characters into a swar as possible, starting at the current
// index.
constexpr auto pack = swar_pack<CurCharIndex>(transcode_char<char_type>(Cs)...);
// Do a single swar comparison.
if ((reader.peek_swar() & pack.mask) == pack.value)
{
reader.bump_swar(pack.count);
// Recurse with the incremented index.
return bool(match_literal<CurCharIndex + pack.count, CharT, Cs...>(reader));
}
else
{
auto partial = swar_find_difference<CharT>(reader.peek_swar() & pack.mask, pack.value);
reader.bump_swar(partial);
return false;
}
}
else
{
static_assert(CurCharIndex == 0);
// Compare each code unit, bump on success, cancel on failure.
return ((reader.peek() == transcode_int<typename Reader::encoding>(Cs)
? (reader.bump(), true)
: false)
&& ...);
}
}
} // namespace lexy::_detail
//=== lit_trie ===//
namespace lexy::_detail
{
template <typename Encoding, template <typename> typename CaseFolding, std::size_t MaxCharCount,
typename... CharClasses>
struct lit_trie
{
using encoding = Encoding;
using char_type = typename Encoding::char_type;
template <typename Reader>
using reader = CaseFolding<Reader>;
static constexpr auto max_node_count = MaxCharCount + 1; // root node
static constexpr auto max_transition_count
= max_node_count == 1 ? 1 : max_node_count - 1; // it is a tree
static constexpr auto node_no_match = std::size_t(-1);
std::size_t node_count;
std::size_t node_value[max_node_count];
// Index of a char class that must not match at the end.
// This is used for keywords.
std::size_t node_char_class[max_node_count];
char_type transition_char[max_transition_count];
std::size_t transition_from[max_transition_count];
std::size_t transition_to[max_transition_count];
LEXY_CONSTEVAL lit_trie()
: node_count(1), node_value{}, node_char_class{}, transition_char{}, transition_from{},
transition_to{}
{
node_value[0] = node_no_match;
node_char_class[0] = sizeof...(CharClasses);
}
template <typename CharT>
LEXY_CONSTEVAL std::size_t insert(std::size_t from, CharT _c)
{
auto c = transcode_char<char_type>(_c);
// We need to find a transition.
// In a tree, we're always having node_count - 1 transitions, so that's the upper bound.
// Everytime we add a transition from a node, its transition index is >= its node index.
// As such, we start looking at the node index.
for (auto i = from; i != node_count - 1; ++i)
{
if (transition_from[i] == from && transition_char[i] == c)
return transition_to[i];
}
auto to = node_count;
node_value[to] = node_no_match;
node_char_class[to] = sizeof...(CharClasses);
auto trans = node_count - 1;
transition_char[trans] = c;
transition_from[trans] = from; // trans >= from as from < node_count
transition_to[trans] = to;
++node_count;
return to;
}
template <typename CharT, CharT... C>
LEXY_CONSTEVAL std::size_t insert(std::size_t pos, type_string<CharT, C...>)
{
return ((pos = insert(pos, C)), ...);
}
LEXY_CONSTEVAL auto node_transitions(std::size_t node) const
{
struct
{
std::size_t length;
std::size_t index[max_transition_count];
} result{};
// We need to consider the same range only.
for (auto i = node; i != node_count - 1; ++i)
if (transition_from[i] == node)
{
result.index[result.length] = i;
++result.length;
}
return result;
}
};
template <typename... CharClasses>
struct char_class_list
{
template <typename Encoding, template <typename> typename CaseFolding, std::size_t N>
using trie_type = lit_trie<Encoding, CaseFolding, N, CharClasses...>;
static constexpr auto size = sizeof...(CharClasses);
template <typename... T>
constexpr auto operator+(char_class_list<T...>) const
{
return char_class_list<CharClasses..., T...>{};
}
};
template <typename CurrentCaseFolding, typename... Literals>
struct _merge_case_folding;
template <>
struct _merge_case_folding<void>
{
template <typename Reader>
using case_folding = Reader;
};
template <typename CurrentCaseFolding>
struct _merge_case_folding<CurrentCaseFolding>
{
template <typename Reader>
using case_folding = typename CurrentCaseFolding::template reader<Reader>;
};
template <typename CurrentCaseFolding, typename H, typename... T>
struct _merge_case_folding<CurrentCaseFolding, H, T...>
: _merge_case_folding<std::conditional_t<std::is_void_v<CurrentCaseFolding>,
typename H::lit_case_folding, CurrentCaseFolding>,
T...>
{
static_assert(std::is_same_v<CurrentCaseFolding,
typename H::lit_case_folding> //
|| std::is_void_v<CurrentCaseFolding>
|| std::is_void_v<typename H::lit_case_folding>,
"cannot mix literals with different case foldings in a literal_set");
};
template <typename Reader>
using lit_no_case_fold = Reader;
template <typename Encoding, typename... Literals>
LEXY_CONSTEVAL auto make_empty_trie()
{
constexpr auto max_char_count = (0 + ... + Literals::lit_max_char_count);
// Merge all mentioned character classes in a single list.
constexpr auto char_classes
= (lexy::_detail::char_class_list{} + ... + Literals::lit_char_classes);
// Need to figure out case folding as well.
return typename decltype(char_classes)::template trie_type<
Encoding, _merge_case_folding<void, Literals...>::template case_folding, max_char_count>{};
}
template <typename Encoding, typename... Literals>
using lit_trie_for = decltype(make_empty_trie<Encoding, Literals...>());
template <std::size_t CharClassIdx, bool, typename...>
struct _node_char_class_impl
{
template <typename Reader>
LEXY_FORCE_INLINE static constexpr std::false_type match(const Reader&)
{
return {};
}
};
template <typename H, typename... T>
struct _node_char_class_impl<0, true, H, T...>
{
template <typename Reader>
LEXY_FORCE_INLINE static constexpr bool match(Reader reader)
{
return lexy::token_parser_for<H, Reader>(reader).try_parse(reader);
}
};
template <std::size_t Idx, typename H, typename... T>
struct _node_char_class_impl<Idx, true, H, T...> : _node_char_class_impl<Idx - 1, true, T...>
{};
template <std::size_t CharClassIdx, typename... CharClasses>
using _node_char_class
= _node_char_class_impl<CharClassIdx, (CharClassIdx < sizeof...(CharClasses)), CharClasses...>;
template <const auto& Trie, std::size_t CurNode>
struct lit_trie_matcher;
template <typename Encoding, template <typename> typename CaseFolding, std::size_t N,
typename... CharClasses, const lit_trie<Encoding, CaseFolding, N, CharClasses...>& Trie,
std::size_t CurNode>
struct lit_trie_matcher<Trie, CurNode>
{
template <std::size_t TransIdx, typename Reader, typename IntT>
LEXY_FORCE_INLINE static constexpr bool _try_transition(std::size_t& result, Reader& reader,
IntT cur)
{
static_assert(Trie.transition_from[TransIdx] == CurNode);
using encoding = typename Reader::encoding;
constexpr auto trans_char = Trie.transition_char[TransIdx];
if (cur != encoding::to_int_type(trans_char))
return false;
reader.bump();
result = lit_trie_matcher<Trie, Trie.transition_to[TransIdx]>::try_match(reader);
return true;
}
static constexpr auto transitions = Trie.node_transitions(CurNode);
template <typename Indices = make_index_sequence<transitions.length>>
struct _impl;
template <std::size_t... Idx>
struct _impl<index_sequence<Idx...>>
{
template <typename Reader>
LEXY_FORCE_INLINE static constexpr std::size_t try_match(Reader& reader)
{
constexpr auto cur_value = Trie.node_value[CurNode];
if constexpr (sizeof...(Idx) > 0)
{
auto cur = reader.current();
auto cur_char = reader.peek();
auto next_value = Trie.node_no_match;
(void)(_try_transition<transitions.index[Idx]>(next_value, reader, cur_char)
|| ...);
if (next_value != Trie.node_no_match)
// We prefer a longer match.
return next_value;
// We haven't found a longer match, return our match.
reader.reset(cur);
}
// But first, we might need to check that we don't match that nodes char class.
constexpr auto char_class = Trie.node_char_class[CurNode];
if constexpr (cur_value == Trie.node_no_match || char_class >= sizeof...(CharClasses))
{
return cur_value;
}
else
{
if (_node_char_class<char_class, CharClasses...>::match(reader))
return Trie.node_no_match;
else
return cur_value;
}
}
};
template <typename Reader>
LEXY_FORCE_INLINE static constexpr std::size_t try_match(Reader& _reader)
{
static_assert(lexy::is_char_encoding<typename Reader::encoding>);
if constexpr (std::is_same_v<CaseFolding<Reader>, Reader>)
{
return _impl<>::try_match(_reader);
}
else
{
CaseFolding<Reader> reader{_reader};
auto result = _impl<>::try_match(reader);
_reader.reset(reader.current());
return result;
}
}
};
} // namespace lexy::_detail
//=== lit ===//
namespace lexyd
{
template <typename CharT, CharT... C>
struct _lit
: token_base<_lit<CharT, C...>,
std::conditional_t<sizeof...(C) == 0, unconditional_branch_base, branch_base>>,
_lit_base
{
static constexpr auto lit_max_char_count = sizeof...(C);
static constexpr auto lit_char_classes = lexy::_detail::char_class_list{};
using lit_case_folding = void;
template <typename Encoding>
static constexpr auto lit_first_char() -> typename Encoding::char_type
{
typename Encoding::char_type result = 0;
(void)((result = lexy::_detail::transcode_char<decltype(result)>(C), true) || ...);
return result;
}
template <typename Trie>
static LEXY_CONSTEVAL std::size_t lit_insert(Trie& trie, std::size_t pos, std::size_t)
{
return ((pos = trie.insert(pos, C)), ...);
}
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr auto try_parse(Reader reader)
{
auto result = lexy::_detail::match_literal<0, CharT, C...>(reader);
end = reader.current();
return result;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader& reader)
{
using char_type = typename Reader::encoding::char_type;
constexpr auto str = lexy::_detail::type_string<CharT, C...>::template c_str<char_type>;
auto begin = reader.position();
auto index = lexy::_detail::range_size(begin, end.position());
auto err = lexy::error<Reader, lexy::expected_literal>(begin, str, index, sizeof...(C));
context.on(_ev::error{}, err);
}
};
};
template <auto C>
constexpr auto lit_c = _lit<LEXY_DECAY_DECLTYPE(C), C>{};
template <unsigned char... C>
constexpr auto lit_b = _lit<unsigned char, C...>{};
#if LEXY_HAS_NTTP
/// Matches the literal string.
template <lexy::_detail::string_literal Str>
constexpr auto lit = lexy::_detail::to_type_string<_lit, Str>{};
#endif
#define LEXY_LIT(Str) \
LEXY_NTTP_STRING(::lexyd::_lit, Str) {}
} // namespace lexyd
namespace lexy
{
template <typename CharT, CharT... C>
inline constexpr auto token_kind_of<lexy::dsl::_lit<CharT, C...>> = lexy::literal_token_kind;
} // namespace lexy
//=== lit_cp ===//
namespace lexyd
{
template <char32_t... Cp>
struct _lcp : token_base<_lcp<Cp...>>, _lit_base
{
template <typename Encoding>
struct _string_t
{
typename Encoding::char_type data[4 * sizeof...(Cp)];
std::size_t length = 0;
constexpr _string_t() : data{}
{
((length += lexy::_detail::encode_code_point<Encoding>(Cp, data + length, 4)), ...);
}
};
template <typename Encoding>
static constexpr _string_t<Encoding> _string = _string_t<Encoding>{};
static constexpr auto lit_max_char_count = 4 * sizeof...(Cp);
static constexpr auto lit_char_classes = lexy::_detail::char_class_list{};
using lit_case_folding = void;
template <typename Encoding>
static constexpr auto lit_first_char() -> typename Encoding::char_type
{
return _string<Encoding>.data[0];
}
template <typename Trie>
static LEXY_CONSTEVAL std::size_t lit_insert(Trie& trie, std::size_t pos, std::size_t)
{
using encoding = typename Trie::encoding;
for (auto i = 0u; i != _string<encoding>.length; ++i)
pos = trie.insert(pos, _string<encoding>.data[i]);
return pos;
}
template <typename Reader,
typename Indices
= lexy::_detail::make_index_sequence<_string<typename Reader::encoding>.length>>
struct tp;
template <typename Reader, std::size_t... Idx>
struct tp<Reader, lexy::_detail::index_sequence<Idx...>>
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
using encoding = typename Reader::encoding;
auto result = lexy::_detail::match_literal<0, typename encoding::char_type,
_string<encoding>.data[Idx]...>(reader);
end = reader.current();
return result;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader& reader)
{
using encoding = typename Reader::encoding;
auto begin = reader.position();
auto index = lexy::_detail::range_size(begin, end.position());
auto err = lexy::error<Reader, lexy::expected_literal>(begin, _string<encoding>.data,
index, _string<encoding>.length);
context.on(_ev::error{}, err);
}
};
};
template <char32_t... CodePoint>
constexpr auto lit_cp = _lcp<CodePoint...>{};
} // namespace lexyd
namespace lexy
{
template <char32_t... Cp>
constexpr auto token_kind_of<lexy::dsl::_lcp<Cp...>> = lexy::literal_token_kind;
} // namespace lexy
//=== lit_set ===//
namespace lexy
{
template <typename T, template <typename> typename CaseFolding, typename... Strings>
class _symbol_table;
struct expected_literal_set
{
static LEXY_CONSTEVAL auto name()
{
return "expected literal set";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Literal, template <typename> typename CaseFolding>
struct _cfl;
template <template <typename> typename CaseFolding, typename CharT, CharT... C>
constexpr auto _make_lit_rule(lexy::_detail::type_string<CharT, C...>)
{
if constexpr (std::is_same_v<CaseFolding<lexy::_pr8>, lexy::_pr8>)
return _lit<CharT, C...>{};
else
return _cfl<_lit<CharT, C...>, CaseFolding>{};
}
template <typename... Literals>
struct _lset : token_base<_lset<Literals...>>, _lset_base
{
using as_lset = _lset;
template <typename Encoding>
static LEXY_CONSTEVAL auto _build_trie()
{
auto result = lexy::_detail::make_empty_trie<Encoding, Literals...>();
[[maybe_unused]] auto char_class = std::size_t(0);
((result.node_value[Literals::lit_insert(result, 0, char_class)] = 0,
// Keep the index correct.
char_class += Literals::lit_char_classes.size),
...);
return result;
}
template <typename Encoding>
static constexpr lexy::_detail::lit_trie_for<Encoding, Literals...> _t
= _build_trie<Encoding>();
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr bool try_parse(Reader reader)
{
using encoding = typename Reader::encoding;
using matcher = lexy::_detail::lit_trie_matcher<_t<encoding>, 0>;
auto result = matcher::try_match(reader);
end = reader.current();
return result != _t<encoding>.node_no_match;
}
template <typename Context>
constexpr void report_error(Context& context, const Reader& reader)
{
auto err = lexy::error<Reader, lexy::expected_literal_set>(reader.position());
context.on(_ev::error{}, err);
}
};
//=== dsl ===//
template <typename Lit>
constexpr auto operator/(Lit) const
{
if constexpr (lexy::is_literal_rule<Lit>)
{
return _lset<Literals..., Lit>{};
}
else if constexpr (sizeof...(Literals) == 0)
{
// We're empty, so do nothing and keep it type-erased.
static_assert(lexy::is_literal_set_rule<Lit>);
return Lit{};
}
else
{
// We're non empty, undo type erasure to append.
static_assert(lexy::is_literal_set_rule<Lit>);
return *this / typename Lit::as_lset{};
}
}
template <typename... Lit>
constexpr auto operator/(_lset<Lit...>) const
{
return _lset<Literals..., Lit...>{};
}
};
/// Matches one of the specified literals.
template <typename... Literals>
constexpr auto literal_set(Literals...)
{
static_assert((lexy::is_literal_rule<Literals> && ...));
return _lset<Literals...>{};
}
/// Matches one of the symbols in the symbol table.
template <typename T, template <typename> typename CaseFolding, typename... Strings>
constexpr auto literal_set(const lexy::_symbol_table<T, CaseFolding, Strings...>)
{
return _lset<decltype(_make_lit_rule<CaseFolding>(Strings{}))...>{};
}
} // namespace lexyd
#define LEXY_LITERAL_SET(...) \
[] { \
using impl = decltype(::lexyd::literal_set(__VA_ARGS__)); \
struct s : impl \
{ \
using impl::operator/; \
}; \
return s{}; \
}()
namespace lexy
{
template <typename... Literals>
constexpr auto token_kind_of<lexy::dsl::_lset<Literals...>> = lexy::literal_token_kind;
} // namespace lexy
#endif // LEXY_DSL_LITERAL_HPP_INCLUDED

View File

@@ -0,0 +1,140 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_LOOKAHEAD_HPP_INCLUDED
#define LEXY_DSL_LOOKAHEAD_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/error.hpp>
namespace lexy
{
/// We've failed to match a lookahead.
struct lookahead_failure
{
static LEXY_CONSTEVAL auto name()
{
return "lookahead failure";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Encoding, typename... Needle, typename... End>
LEXY_CONSTEVAL auto _build_look_trie(_lset<Needle...>, _lset<End...>)
{
auto result = lexy::_detail::make_empty_trie<Encoding, Needle..., End...>();
auto char_class = std::size_t(0);
// We insert all needles with value 0.
((result.node_value[Needle::lit_insert(result, 0, char_class)] = 0,
char_class += Needle::lit_char_classes.size),
...);
// And all ends with value 1.
((result.node_value[End::lit_insert(result, 0, char_class)] = 1,
char_class += End::lit_char_classes.size),
...);
return result;
}
template <typename Encoding, typename Needle, typename End>
static constexpr auto _look_trie
= _build_look_trie<Encoding>(typename Needle::as_lset{}, typename End::as_lset{});
template <typename Needle, typename End, typename Tag>
struct _look : branch_base
{
template <typename Reader>
struct bp
{
static_assert(lexy::is_char_encoding<typename Reader::encoding>);
typename Reader::iterator begin;
typename Reader::iterator end;
constexpr bool try_parse(const void*, Reader reader)
{
begin = reader.position();
auto result = [&] {
using matcher = lexy::_detail::lit_trie_matcher<
_look_trie<typename Reader::encoding, Needle, End>, 0>;
while (true)
{
auto result = matcher::try_match(reader);
if (result == 0)
// We've found the needle.
return true;
else if (result == 1 || reader.peek() == Reader::encoding::eof())
// We've failed.
return false;
else
// Try again.
reader.bump();
}
return false; // unreachable
}();
end = reader.position();
return result;
}
template <typename Context>
constexpr void cancel(Context& context)
{
context.on(_ev::backtracked{}, begin, end);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
context.on(_ev::backtracked{}, begin, end);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
static_assert(lexy::is_char_encoding<typename Reader::encoding>);
bp<Reader> impl{};
if (!impl.try_parse(context.control_block, reader))
{
// Report that we've failed.
using tag = lexy::_detail::type_or<Tag, lexy::lookahead_failure>;
auto err = lexy::error<Reader, tag>(impl.begin, impl.end);
context.on(_ev::error{}, err);
// But recover immediately, as we wouldn't have consumed anything either way.
}
context.on(_ev::backtracked{}, impl.begin, impl.end);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename Error>
static constexpr _look<Needle, End, Error> error = {};
};
/// Looks for the Needle before End.
/// Used as condition to implement arbitrary lookahead.
template <typename Needle, typename End>
constexpr auto lookahead(Needle _needle, End _end)
{
auto needle = literal_set() / _needle;
auto end = literal_set() / _end;
return _look<decltype(needle), decltype(end), void>{};
}
} // namespace lexyd
#endif // LEXY_DSL_LOOKAHEAD_HPP_INCLUDED

View File

@@ -0,0 +1,124 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_LOOP_HPP_INCLUDED
#define LEXY_DSL_LOOP_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/branch.hpp>
namespace lexyd
{
struct _break : unconditional_branch_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename LoopControl, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context&, Reader&, LoopControl& cntrl, Args&&...)
{
cntrl.loop_break = true;
return true;
}
};
template <typename Reader>
using bp = lexy::unconditional_branch_parser<_break, Reader>;
};
/// Exits a loop().
constexpr auto break_ = _break{};
} // namespace lexyd
namespace lexyd
{
template <typename Rule>
struct _loop : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
struct loop_control_t
{
bool loop_break = false;
} control;
while (!control.loop_break)
{
using parser = lexy::parser_for<Rule, lexy::pattern_parser<loop_control_t>>;
if (!parser::parse(context, reader, control))
return false;
}
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
};
/// Repeatedly matches the rule until a break rule matches.
template <typename Rule>
constexpr auto loop(Rule)
{
return _loop<Rule>{};
}
} // namespace lexyd
namespace lexyd
{
template <typename Branch>
struct _whl : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
lexy::branch_parser_for<Branch, Reader> branch{};
while (branch.try_parse(context.control_block, reader))
{
if (!branch.template finish<lexy::pattern_parser<>>(context, reader))
return false;
}
branch.cancel(context);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
};
/// Matches the branch rule as often as possible.
template <typename Rule>
constexpr auto while_(Rule)
{
LEXY_REQUIRE_BRANCH_RULE(Rule, "while()");
return _whl<Rule>{};
}
} // namespace lexyd
namespace lexyd
{
/// Matches the rule at least once, then as often as possible.
template <typename Rule>
constexpr auto while_one(Rule rule)
{
LEXY_REQUIRE_BRANCH_RULE(Rule, "while_one()");
return rule >> while_(rule);
}
} // namespace lexyd
namespace lexyd
{
/// Matches then once, then `while_(condition >> then)`.
template <typename Then, typename Condition>
constexpr auto do_while(Then then, Condition condition)
{
return _maybe_branch(then, while_(condition >> then));
}
} // namespace lexyd
#endif // LEXY_DSL_LOOP_HPP_INCLUDED

View File

@@ -0,0 +1,99 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_MEMBER_HPP_INCLUDED
#define LEXY_DSL_MEMBER_HPP_INCLUDED
#include <lexy/_detail/stateless_lambda.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/branch.hpp>
namespace lexy
{
template <auto Ptr>
struct _mem_ptr_fn
{
template <typename Object, typename Value>
constexpr void operator()(Object& object, Value&& value) const
{
object.*Ptr = LEXY_FWD(value);
}
};
template <typename Fn>
struct member
{};
template <auto Ptr>
using make_member_ptr = member<_mem_ptr_fn<Ptr>>;
} // namespace lexy
namespace lexyd
{
template <typename Fn, typename Rule>
struct _mem : _copy_base<Rule>
{
template <typename Reader>
struct bp
{
lexy::branch_parser_for<Rule, Reader> rule;
template <typename ControlBlock>
constexpr auto try_parse(const ControlBlock* cb, const Reader& reader)
{
return rule.try_parse(cb, reader);
}
template <typename Context>
constexpr void cancel(Context& context)
{
rule.cancel(context);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC auto finish(Context& context, Reader& reader, Args&&... args)
{
// Add member tag here.
return rule.template finish<NextParser>(context, reader, LEXY_FWD(args)...,
lexy::member<Fn>{});
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
// Forward to the rule, but add member tag.
using parser = lexy::parser_for<Rule, NextParser>;
return parser::parse(context, reader, LEXY_FWD(args)..., lexy::member<Fn>{});
}
};
};
template <typename Fn>
struct _mem_dsl
{
constexpr _mem_dsl(Fn = {}) {}
template <typename Rule>
constexpr auto operator=(Rule) const // NOLINT: it _is_ an unconventional assignment operator
{
using lambda = std::conditional_t<std::is_default_constructible_v<Fn>, Fn,
lexy::_detail::stateless_lambda<Fn>>;
return _mem<lambda, Rule>{};
}
};
/// Specifies that the output of the associated rule should be stored in the member pointer. Used
/// with `lexy::as_aggregate`.
template <auto MemPtr>
constexpr auto member = _mem_dsl<lexy::_mem_ptr_fn<MemPtr>>{};
#define LEXY_MEM(Name) \
::lexyd::_mem_dsl([](auto& obj, auto&& value) { obj.Name = LEXY_FWD(value); })
} // namespace lexyd
#endif // LEXY_DSL_MEMBER_HPP_INCLUDED

View File

@@ -0,0 +1,85 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_NEWLINE_HPP_INCLUDED
#define LEXY_DSL_NEWLINE_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/dsl/token.hpp>
namespace lexy
{
struct expected_newline
{
static LEXY_CONSTEVAL auto name()
{
return "expected newline";
}
};
} // namespace lexy
namespace lexyd
{
struct _nl : LEXY_DECAY_DECLTYPE(
literal_set(LEXY_LIT("\n"), LEXY_LIT("\r\n")).error<lexy::expected_newline>){};
/// Matches a newline character.
constexpr auto newline = _nl{};
} // namespace lexyd
namespace lexyd
{
struct _eol : branch_base
{
template <typename Reader>
struct bp
{
static_assert(lexy::is_char_encoding<typename Reader::encoding>);
constexpr bool try_parse(const void*, Reader reader)
{
return reader.peek() == Reader::encoding::eof()
|| lexy::try_match_token(newline, reader);
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
if (reader.peek() == Reader::encoding::eof())
{
auto pos = reader.position();
context.on(_ev::token{}, lexy::eof_token_kind, pos, pos);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
else
{
// Note that we're re-doing the parsing for newline,
// this looks at most at two characters, so it doesn't really matter.
return lexy::parser_for<_nl, NextParser>::parse(context, reader, LEXY_FWD(args)...);
}
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
static_assert(lexy::is_char_encoding<typename Reader::encoding>);
return bp<Reader>{}.template finish<NextParser>(context, reader, LEXY_FWD(args)...);
}
};
};
/// Matches the end of line (EOF or newline).
constexpr auto eol = _eol{};
} // namespace lexyd
#endif // LEXY_DSL_NEWLINE_HPP_INCLUDED

View File

@@ -0,0 +1,333 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_OPERATOR_HPP_INCLUDED
#define LEXY_DSL_OPERATOR_HPP_INCLUDED
#include <lexy/_detail/detect.hpp>
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/literal.hpp>
#include <lexy/dsl/token.hpp>
namespace lexyd
{
template <typename Condition, typename... R>
struct _br;
template <typename... R>
struct _seq_impl;
} // namespace lexyd
//=== tag type ===//
namespace lexy
{
template <typename Literal>
struct _op
{};
// GCC is buggy with auto parameters.
template <typename T, T Value>
struct _opv
{
constexpr operator LEXY_DECAY_DECLTYPE(Value)() const
{
return Value;
}
};
#if LEXY_HAS_NTTP
template <auto Operator>
#else
template <const auto& Operator>
#endif
using op = typename LEXY_DECAY_DECLTYPE(Operator)::op_tag_type;
} // namespace lexy
//=== op rule ===//
namespace lexy::_detail
{
template <typename... Literals>
struct op_lit_list
{
static constexpr auto size = sizeof...(Literals);
template <typename Encoding>
static LEXY_CONSTEVAL auto _build_trie()
{
auto result = make_empty_trie<Encoding, Literals...>();
auto value = std::size_t(0);
auto char_class = std::size_t(0);
((result.node_value[Literals::lit_insert(result, 0, char_class)] = value++,
char_class += Literals::lit_char_classes.size),
...);
return result;
}
template <typename Encoding>
static constexpr lit_trie_for<Encoding, Literals...> trie = _build_trie<Encoding>();
template <typename... T>
constexpr auto operator+(op_lit_list<T...>) const
{
return op_lit_list<Literals..., T...>{};
}
};
template <typename Reader>
struct parsed_operator
{
typename Reader::marker cur;
std::size_t idx;
};
template <typename OpList, typename Reader>
constexpr auto parse_operator(Reader& reader)
{
using encoding = typename Reader::encoding;
using op_matcher = lexy::_detail::lit_trie_matcher<OpList::template trie<encoding>, 0>;
auto begin = reader.current();
auto op = op_matcher::try_match(reader);
return parsed_operator<Reader>{begin, op};
}
} // namespace lexy::_detail
namespace lexyd
{
template <typename Tag, typename Reader>
using _detect_op_tag_ctor = decltype(Tag(LEXY_DECLVAL(Reader).position()));
template <typename Tag, typename Reader, typename Context>
using _detect_op_tag_ctor_with_state
= decltype(Tag(*LEXY_DECLVAL(Context).control_block->parse_state,
LEXY_DECLVAL(Reader).position()));
template <typename TagType, typename Literal, typename... R>
struct _op : branch_base
{
using op_tag_type = TagType;
using op_literals = lexy::_detail::op_lit_list<Literal>;
template <typename NextParser, typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool op_finish(Context& context, Reader& reader,
lexy::_detail::parsed_operator<Reader> op,
Args&&... args)
{
context.on(_ev::token{}, typename Literal::token_type{}, op.cur.position(),
reader.position());
using continuation
= lexy::whitespace_parser<Context, lexy::parser_for<_seq_impl<R...>, NextParser>>;
if constexpr (std::is_void_v<TagType>)
return continuation::parse(context, reader, LEXY_FWD(args)...);
else if constexpr (lexy::_detail::is_detected<_detect_op_tag_ctor_with_state, op_tag_type,
Reader, Context>)
return continuation::parse(context, reader, LEXY_FWD(args)...,
op_tag_type(*context.control_block->parse_state, op.pos));
else if constexpr (lexy::_detail::is_detected<_detect_op_tag_ctor, op_tag_type, Reader>)
return continuation::parse(context, reader, LEXY_FWD(args)...,
op_tag_type(op.cur.position()));
else
return continuation::parse(context, reader, LEXY_FWD(args)..., op_tag_type{});
}
template <typename Reader>
struct bp
{
lexy::branch_parser_for<Literal, Reader> impl;
template <typename ControlBlock>
constexpr auto try_parse(const ControlBlock* cb, const Reader& reader)
{
return impl.try_parse(cb, reader);
}
template <typename Context>
constexpr void cancel(Context& context)
{
impl.cancel(context);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
using continuation = lexy::parser_for<_seq_impl<R...>, NextParser>;
if constexpr (std::is_void_v<TagType>)
return impl.template finish<continuation>(context, reader, LEXY_FWD(args)...);
else if constexpr (lexy::_detail::is_detected<_detect_op_tag_ctor_with_state,
op_tag_type, Reader, Context>)
return impl
.template finish<continuation>(context, reader, LEXY_FWD(args)...,
op_tag_type(*context.control_block->parse_state,
reader.position()));
else if constexpr (lexy::_detail::is_detected<_detect_op_tag_ctor, op_tag_type, Reader>)
return impl.template finish<continuation>(context, reader, LEXY_FWD(args)...,
op_tag_type(reader.position()));
else
return impl.template finish<continuation>(context, reader, LEXY_FWD(args)...,
op_tag_type{});
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
[[maybe_unused]] auto pos = reader.position();
using continuation
= lexy::parser_for<Literal, lexy::parser_for<_seq_impl<R...>, NextParser>>;
if constexpr (std::is_void_v<TagType>)
return continuation::parse(context, reader, LEXY_FWD(args)...);
else if constexpr (lexy::_detail::is_detected<_detect_op_tag_ctor_with_state,
op_tag_type, Reader, Context>)
return continuation::parse(context, reader, LEXY_FWD(args)...,
op_tag_type(*context.control_block->parse_state, pos));
else if constexpr (lexy::_detail::is_detected<_detect_op_tag_ctor, op_tag_type, Reader>)
return continuation::parse(context, reader, LEXY_FWD(args)..., op_tag_type(pos));
else
return continuation::parse(context, reader, LEXY_FWD(args)..., op_tag_type{});
}
};
};
template <typename Literal>
constexpr auto op(Literal)
{
static_assert(lexy::is_literal_rule<Literal>);
return _op<lexy::_op<Literal>, Literal>{};
}
template <typename Literal, typename... R>
constexpr auto op(_br<Literal, R...>)
{
static_assert(lexy::is_literal_rule<Literal>,
"condition in the operator must be a literal rule");
return _op<lexy::_op<Literal>, Literal, R...>{};
}
template <typename Tag, typename Literal>
constexpr auto op(Literal)
{
static_assert(lexy::is_literal_rule<Literal>);
return _op<Tag, Literal>{};
}
template <typename Tag, typename Literal, typename... R>
constexpr auto op(_br<Literal, R...>)
{
static_assert(lexy::is_literal_rule<Literal>,
"condition in the operator must be a literal rule");
return _op<Tag, Literal, R...>{};
}
template <auto Tag, typename Literal>
constexpr auto op(Literal)
{
static_assert(lexy::is_literal_rule<Literal>);
return _op<lexy::_opv<LEXY_DECAY_DECLTYPE(Tag), Tag>, Literal>{};
}
template <auto Tag, typename Literal, typename... R>
constexpr auto op(_br<Literal, R...>)
{
static_assert(lexy::is_literal_rule<Literal>,
"condition in the operator must be a literal rule");
return _op<lexy::_opv<LEXY_DECAY_DECLTYPE(Tag), Tag>, Literal, R...>{};
}
} // namespace lexyd
//=== op choice ===//
namespace lexyd
{
template <typename... Ops>
struct _opc : branch_base
{
using op_literals = decltype((typename Ops::op_literals{} + ...));
template <typename NextParser, typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool op_finish(Context& context, Reader& reader,
lexy::_detail::parsed_operator<Reader> op,
Args&&... args)
{
auto result = false;
auto cur_idx = std::size_t(0);
(void)((cur_idx == op.idx
? (result = Ops::template op_finish<NextParser>(context, reader, op,
LEXY_FWD(args)...),
true)
: (++cur_idx, false))
|| ...);
return result;
}
template <typename Reader>
struct bp
{
lexy::_detail::parsed_operator<Reader> op;
typename Reader::marker end;
constexpr auto try_parse(const void*, Reader reader)
{
op = lexy::_detail::parse_operator<op_literals>(reader);
end = reader.current();
return op.idx < op_literals::size;
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
reader.reset(end);
return op_finish<NextParser>(context, reader, op, LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
bp<Reader> impl{};
if (!impl.try_parse(context.control_block, reader))
{
auto err = lexy::error<Reader, lexy::expected_literal_set>(impl.op.cur.position());
context.on(_ev::error{}, err);
return false;
}
else
{
return impl.template finish<NextParser>(context, reader, LEXY_FWD(args)...);
}
}
};
};
template <typename T1, typename L1, typename... R1, typename T2, typename L2, typename... R2>
constexpr auto operator/(_op<T1, L1, R1...> lhs, _op<T2, L2, R2...> rhs)
{
return _opc<decltype(lhs), decltype(rhs)>{};
}
template <typename... Ops, typename T2, typename L2, typename... R2>
constexpr auto operator/(_opc<Ops...>, _op<T2, L2, R2...> rhs)
{
return _opc<Ops..., decltype(rhs)>{};
}
template <typename T1, typename L1, typename... R1, typename... Ops>
constexpr auto operator/(_op<T1, L1, R1...> lhs, _opc<Ops...>)
{
return _opc<decltype(lhs), Ops...>{};
}
template <typename... O1, typename... O2>
constexpr auto operator/(_opc<O1...>, _opc<O2...>)
{
return _opc<O1..., O2...>{};
}
} // namespace lexyd
#endif // LEXY_DSL_OPERATOR_HPP_INCLUDED

View File

@@ -0,0 +1,120 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_OPTION_HPP_INCLUDED
#define LEXY_DSL_OPTION_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/branch.hpp>
#include <lexy/dsl/recover.hpp>
namespace lexy
{
// An optional type is something that has the following:
// * a default constructor: this means we can actually construct it from our `nullopt`
// * a dereference operator: this means that it actually contains something else
// * a contextual conversion to bool: this means that it might be "false" (i.e. empty)
//
// This definition should work:
// * it excludes all default constructible types that are convertible to bool (e.g. integers...)
// * it includes pointers, which is ok
// * it includes `std::optional` and all non-std implementations of it
template <typename T>
using _detect_optional_like = decltype(T(), *LEXY_DECLVAL(T&), !LEXY_DECLVAL(const T&));
struct nullopt
{
template <typename T, typename = _detect_optional_like<T>>
constexpr operator T() const
{
return T();
}
};
} // namespace lexy
namespace lexyd
{
struct _nullopt : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
return NextParser::parse(context, reader, LEXY_FWD(args)..., lexy::nullopt{});
}
};
};
constexpr auto nullopt = _nullopt{};
} // namespace lexyd
namespace lexyd
{
template <typename Branch>
struct _opt : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
lexy::branch_parser_for<Branch, Reader> branch{};
if (branch.try_parse(context.control_block, reader))
// We take the branch.
return branch.template finish<NextParser>(context, reader, LEXY_FWD(args)...);
else
{
// We don't take the branch and produce a nullopt.
branch.cancel(context);
return NextParser::parse(context, reader, LEXY_FWD(args)..., lexy::nullopt{});
}
}
};
};
/// Matches the rule or nothing.
/// In the latter case, produces a `nullopt` value.
template <typename Rule>
constexpr auto opt(Rule)
{
LEXY_REQUIRE_BRANCH_RULE(Rule, "opt()");
if constexpr (lexy::is_unconditional_branch_rule<Rule>)
// Branch is always taken, so don't wrap in opt().
return Rule{};
else
return _opt<Rule>{};
}
} // namespace lexyd
namespace lexyd
{
template <typename Term, typename Rule>
struct _optt : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
// Try to parse the terminator.
lexy::branch_parser_for<Term, Reader> term{};
if (term.try_parse(context.control_block, reader))
// We had the terminator, so produce nullopt.
return term.template finish<NextParser>(context, reader, LEXY_FWD(args)...,
lexy::nullopt{});
term.cancel(context);
// We didn't have the terminator, so we parse the rule.
using parser = lexy::parser_for<Rule, NextParser>;
return parser::parse(context, reader, LEXY_FWD(args)...);
}
};
};
} // namespace lexyd
#endif // LEXY_DSL_OPTION_HPP_INCLUDED

View File

@@ -0,0 +1,156 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_PARSE_AS_HPP_INCLUDED
#define LEXY_DSL_PARSE_AS_HPP_INCLUDED
#include <lexy/action/base.hpp>
#include <lexy/callback/object.hpp>
#include <lexy/dsl/base.hpp>
namespace lexyd
{
// Custom handler that forwards events but overrides the value callback.
template <typename Handler>
struct _pas_handler
{
Handler& _handler;
using event_handler = typename Handler::event_handler;
// We are implicitly convertible to all handler types the original handler is convertible to.
// This is because the handler is passed to event_handler::on.
template <typename H, typename = decltype(static_cast<H&>(LEXY_DECLVAL(Handler&)))>
constexpr operator H&() const
{
return static_cast<H&>(_handler);
}
// We use ::value to get a value.
// We can't use it unconditionally, as the initial production that contains the parse_as might
// not have one. So we silently fallback if that's the case - this might cause worse errors if
// the value is missing.
template <typename Production, typename State>
using value_callback
= std::conditional_t<lexy::production_has_value_callback<Production, State>,
lexy::production_value_callback<Production, State>,
lexy::_detail::void_value_callback>;
};
struct _pas_final_parser
{
template <typename Context, typename Reader, typename T, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context&, Reader&, lexy::_detail::lazy_init<T>& value,
Args&&... args)
{
value.emplace_result(lexy::construct<T>, LEXY_FWD(args)...);
return true;
}
};
template <typename Handler>
constexpr auto _make_pas_handler(Handler& handler)
{
return _pas_handler<Handler>{handler};
}
// Prevent infinite nesting when parse_as itself is recursive.
template <typename Handler>
constexpr auto _make_pas_handler(_pas_handler<Handler>& handler)
{
return handler;
}
template <typename T, typename Rule, bool Front = false>
struct _pas : _copy_base<Rule>
{
template <typename Reader>
struct bp
{
lexy::branch_parser_for<Rule, Reader> rule_parser;
template <typename ControlBlock>
constexpr bool try_parse(const ControlBlock* cb, const Reader& reader)
{
return rule_parser.try_parse(cb, reader);
}
template <typename Context>
constexpr void cancel(Context& context)
{
// No need to use the special context here; it doesn't produce any values.
rule_parser.cancel(context);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
auto handler = _make_pas_handler(context.control_block->parse_handler);
lexy::_detail::parse_context_control_block cb(LEXY_MOV(handler), context.control_block);
using context_type
= lexy::_pc<decltype(handler), typename Context::state_type,
typename Context::production, typename Context::whitespace_production>;
context_type sub_context(&cb);
sub_context.handler = LEXY_MOV(context).handler;
lexy::_detail::lazy_init<T> value;
auto result
= rule_parser.template finish<_pas_final_parser>(sub_context, reader, value);
context.control_block->copy_vars_from(&cb);
context.handler = LEXY_MOV(sub_context).handler;
if (!result)
return false;
else if constexpr (std::is_void_v<T>)
// NOLINTNEXTLINE: clang-tidy wrongly thinks the branch is repeated.
return NextParser::parse(context, reader, LEXY_FWD(args)...);
else if constexpr (Front)
return NextParser::parse(context, reader, *LEXY_MOV(value), LEXY_FWD(args)...);
else
return NextParser::parse(context, reader, LEXY_FWD(args)..., *LEXY_MOV(value));
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
auto handler = _make_pas_handler(context.control_block->parse_handler);
lexy::_detail::parse_context_control_block cb(LEXY_MOV(handler), context.control_block);
using context_type
= lexy::_pc<decltype(handler), typename Context::state_type,
typename Context::production, typename Context::whitespace_production>;
context_type sub_context(&cb);
sub_context.handler = LEXY_MOV(context).handler;
lexy::_detail::lazy_init<T> value;
auto result
= lexy::parser_for<Rule, _pas_final_parser>::parse(sub_context, reader, value);
context.control_block->copy_vars_from(&cb);
context.handler = LEXY_MOV(sub_context).handler;
if (!result)
return false;
else if constexpr (std::is_void_v<T>)
// NOLINTNEXTLINE: clang-tidy wrongly thinks the branch is repeated.
return NextParser::parse(context, reader, LEXY_FWD(args)...);
else if constexpr (Front)
return NextParser::parse(context, reader, *LEXY_MOV(value), LEXY_FWD(args)...);
else
return NextParser::parse(context, reader, LEXY_FWD(args)..., *LEXY_MOV(value));
}
};
};
template <typename T, typename Rule>
constexpr auto parse_as(Rule)
{
return _pas<T, Rule>{};
}
} // namespace lexyd
#endif // LEXY_DSL_PARSE_AS_HPP_INCLUDED

View File

@@ -0,0 +1,251 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_PARSE_TREE_NODE_HPP_INCLUDED
#define LEXY_DSL_PARSE_TREE_NODE_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/token.hpp>
#if !LEXY_EXPERIMENTAL
# error "lexy::dsl::tnode/pnode are experimental"
#endif
//=== impl ===//
namespace lexyd
{
template <typename Derived>
struct _n;
template <typename Derived, typename R>
struct _nr : branch_base
{
template <typename NextParser>
struct _cont
{
template <typename Context, typename ChildReader, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, ChildReader& child_reader,
bool& rule_succeded, Reader& reader, Args&&... args)
{
rule_succeded = true;
if (child_reader.peek() != ChildReader::encoding::eof())
{
auto begin = child_reader.position();
auto end = reader.position();
context.on(_ev::token{}, lexy::error_token_kind, begin, end);
auto err = lexy::error<Reader, typename Derived::node_end_error>(begin, end);
context.on(_ev::error{}, err);
}
return lexy::whitespace_parser<Context, NextParser>::parse(context, reader,
LEXY_FWD(args)...);
}
};
template <typename NextParser, typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool _parse_rule(Context& context, Reader& reader,
typename Reader::marker end, Args&&... args)
{
auto child_reader = Derived::node_child_reader(reader);
reader.reset(end);
using rule_parser
= lexy::whitespace_parser<Context, lexy::parser_for<R, _cont<NextParser>>>;
if (auto rule_succeded = false;
rule_parser::parse(context, child_reader, rule_succeded, reader, LEXY_FWD(args)...))
{
return true;
}
else
{
if (!rule_succeded)
// Report an error token for the child span that wasn't able to be parsed.
context.on(_ev::token{}, lexy::error_token_kind, child_reader.position(),
end.position());
return false;
}
}
template <typename Reader>
struct bp
{
typename Reader::marker end;
constexpr bool try_parse(const void*, const Reader& reader)
{
lexy::token_parser_for<_n<Derived>, Reader> parser(reader);
auto result = parser.try_parse(reader);
end = parser.end;
return result;
}
template <typename Context>
constexpr void cancel(Context&)
{}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
return _parse_rule<NextParser>(context, reader, end, LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
lexy::token_parser_for<_n<Derived>, Reader> parser(reader);
if (!parser.try_parse(reader))
{
LEXY_ASSERT(parser.end.position() == reader.position(), "impl should be LL(1)");
parser.report_error(context, reader);
return false;
}
return _parse_rule<NextParser>(context, reader, parser.end, LEXY_FWD(args)...);
}
};
};
template <typename Derived>
struct _n : token_base<Derived>
{
template <typename Reader>
struct tp
{
typename Reader::marker end;
constexpr explicit tp(const Reader& reader) : end(reader.current()) {}
constexpr auto try_parse(Reader reader)
{
if constexpr (lexy::is_node_encoding<typename Reader::encoding>)
{
if (!Reader::encoding::match(reader.peek(), Derived::node_kind()))
return false;
reader.bump();
end = reader.current();
return true;
}
else
{
// This happens when it is used as whitespace, which is inherited while parsing the
// token lexeme, we don't match anything in that case.
return std::false_type{};
}
}
template <typename Context>
constexpr void report_error(Context& context, Reader reader)
{
constexpr auto name = Derived::node_kind_name();
auto err = lexy::error<Reader, lexy::expected_char_class>(reader.position(), name);
context.on(_ev::error{}, err);
}
};
template <typename Rule>
constexpr auto operator()(Rule) const
{
return _nr<Derived, Rule>{};
}
};
} // namespace lexyd
//=== dsl::tnode ===//
namespace lexy
{
struct expected_token_end
{
static LEXY_CONSTEVAL auto name()
{
return "expected token end";
}
};
} // namespace lexy
namespace lexyd
{
template <auto Kind>
struct _tn : _n<_tn<Kind>>
{
static LEXY_CONSTEVAL auto node_kind()
{
return Kind;
}
static LEXY_CONSTEVAL auto node_kind_name()
{
using lexy::token_kind_name;
return token_kind_name(Kind);
}
using node_end_error = lexy::expected_token_end;
template <typename Reader>
static constexpr auto node_child_reader(Reader& reader)
{
return reader.lexeme_reader();
}
};
template <auto Kind>
constexpr auto tnode = _tn<Kind>{};
} // namespace lexyd
namespace lexy
{
template <auto Kind>
constexpr auto token_kind_of<lexy::dsl::_tn<Kind>> = Kind;
} // namespace lexy
//=== dsl::pnode ===//
namespace lexy
{
struct expected_production_end
{
static LEXY_CONSTEVAL auto name()
{
return "expected production end";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Production>
struct _pn : _n<_pn<Production>>
{
static_assert(lexy::is_production<Production>);
static LEXY_CONSTEVAL auto node_kind()
{
return Production{};
}
static LEXY_CONSTEVAL auto node_kind_name()
{
return lexy::production_name<Production>();
}
using node_end_error = lexy::expected_production_end;
template <typename Reader>
static constexpr auto node_child_reader(Reader& reader)
{
return reader.child_reader();
}
};
template <typename Production>
constexpr auto pnode = _pn<Production>{};
} // namespace lexyd
#endif // LEXY_DSL_PARSE_TREE_NODE_HPP_INCLUDED

View File

@@ -0,0 +1,180 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_PEEK_HPP_INCLUDED
#define LEXY_DSL_PEEK_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
#include <lexy/dsl/token.hpp>
namespace lexy
{
/// We've failed to match a peek.
struct peek_failure
{
static LEXY_CONSTEVAL auto name()
{
return "peek failure";
}
};
/// We've failed to match a peek not.
struct unexpected
{
static LEXY_CONSTEVAL auto name()
{
return "unexpected";
}
};
} // namespace lexy
namespace lexyd
{
template <typename Rule, typename Tag>
struct _peek : branch_base
{
template <typename Reader>
struct bp
{
typename Reader::iterator begin;
typename Reader::marker end;
constexpr bool try_parse(const void*, Reader reader)
{
// We need to match the entire rule.
lexy::token_parser_for<decltype(lexy::dsl::token(Rule{})), Reader> parser(reader);
begin = reader.position();
auto result = parser.try_parse(reader);
end = parser.end;
return result;
}
template <typename Context>
constexpr void cancel(Context& context)
{
context.on(_ev::backtracked{}, begin, end.position());
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
context.on(_ev::backtracked{}, begin, end.position());
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
bp<Reader> impl{};
if (!impl.try_parse(context.control_block, reader))
{
// Report that we've failed.
using tag = lexy::_detail::type_or<Tag, lexy::peek_failure>;
auto err = lexy::error<Reader, tag>(impl.begin, impl.end.position());
context.on(_ev::error{}, err);
// But recover immediately, as we wouldn't have consumed anything either way.
}
context.on(_ev::backtracked{}, impl.begin, impl.end.position());
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename Error>
static constexpr _peek<Rule, Error> error = {};
};
template <typename Rule, typename Tag>
struct _peekn : branch_base
{
template <typename Reader>
struct bp
{
typename Reader::iterator begin;
typename Reader::marker end;
constexpr bool try_parse(const void*, Reader reader)
{
// We must not match the rule.
lexy::token_parser_for<decltype(lexy::dsl::token(Rule{})), Reader> parser(reader);
begin = reader.position();
auto result = !parser.try_parse(reader);
end = parser.end;
return result;
}
template <typename Context>
constexpr void cancel(Context& context)
{
context.on(_ev::backtracked{}, begin, end.position());
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC bool finish(Context& context, Reader& reader, Args&&... args)
{
context.on(_ev::backtracked{}, begin, end.position());
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
bp<Reader> impl{};
if (!impl.try_parse(context.control_block, reader))
{
// Report that we've failed.
using tag = lexy::_detail::type_or<Tag, lexy::unexpected>;
auto err = lexy::error<Reader, tag>(impl.begin, impl.end.position());
context.on(_ev::error{}, err);
// And recover by consuming the input.
context.on(_ev::recovery_start{}, impl.begin);
context.on(_ev::token{}, lexy::error_token_kind, impl.begin, impl.end.position());
context.on(_ev::recovery_finish{}, impl.end.position());
reader.reset(impl.end);
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
else
{
context.on(_ev::backtracked{}, impl.begin, impl.end.position());
return NextParser::parse(context, reader, LEXY_FWD(args)...);
}
}
};
template <typename Error>
static constexpr _peekn<Rule, Error> error = {};
};
/// Check if at this reader position, the rule would match, but don't actually consume any
/// characters if it does.
template <typename Rule>
constexpr auto peek(Rule)
{
return _peek<Rule, void>{};
}
/// Checks if at this reader position, the rule would not match.
template <typename Rule>
constexpr auto peek_not(Rule)
{
return _peekn<Rule, void>{};
}
} // namespace lexyd
#endif // LEXY_DSL_PEEK_HPP_INCLUDED

View File

@@ -0,0 +1,83 @@
// Copyright (C) 2020-2025 Jonathan Müller and lexy contributors
// SPDX-License-Identifier: BSL-1.0
#ifndef LEXY_DSL_POSITION_HPP_INCLUDED
#define LEXY_DSL_POSITION_HPP_INCLUDED
#include <lexy/dsl/base.hpp>
namespace lexyd
{
struct _pos : rule_base
{
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
auto pos = reader.position();
context.on(_ev::token{}, lexy::position_token_kind, pos, pos);
return NextParser::parse(context, reader, LEXY_FWD(args)..., pos);
}
};
};
template <typename Rule>
struct _posr : _copy_base<Rule>
{
template <typename Reader>
struct bp
{
lexy::branch_parser_for<Rule, Reader> rule;
template <typename ControlBlock>
constexpr auto try_parse(const ControlBlock* cb, const Reader& reader)
{
return rule.try_parse(cb, reader);
}
template <typename Context>
constexpr void cancel(Context& context)
{
rule.cancel(context);
}
template <typename NextParser, typename Context, typename... Args>
LEXY_PARSER_FUNC auto finish(Context& context, Reader& reader, Args&&... args)
{
auto pos = reader.position();
context.on(_ev::token{}, lexy::position_token_kind, pos, pos);
return rule.template finish<NextParser>(context, reader, LEXY_FWD(args)..., pos);
}
};
template <typename NextParser>
struct p
{
template <typename Context, typename Reader, typename... Args>
LEXY_PARSER_FUNC static bool parse(Context& context, Reader& reader, Args&&... args)
{
auto pos = reader.position();
context.on(_ev::token{}, lexy::position_token_kind, pos, pos);
return lexy::parser_for<Rule, NextParser>::parse(context, reader, LEXY_FWD(args)...,
pos);
}
};
};
struct _pos_dsl : _pos
{
template <typename Rule>
constexpr auto operator()(Rule) const
{
return _posr<Rule>{};
}
};
/// Produces an iterator to the current reader position without parsing anything.
constexpr auto position = _pos_dsl{};
} // namespace lexyd
#endif // LEXY_DSL_POSITION_HPP_INCLUDED

Some files were not shown because too many files have changed in this diff Show More