Extend @gba-kit/debug-info for address→symbol maps - #5
Merged
Conversation
macabeus
force-pushed
the
extend-debug-info-for-symbol-maps
branch
6 times, most recently
from
July 31, 2026 21:44
a9ceec2 to
d3c7532
Compare
…aration shape Walks a variable DIE's type through typedefs/cv-qualifiers to scalar | pointer | array | struct, with element size/signedness for arrays and tag name/byte size for structs. A consumer that reconstructs a declaration needs its shape rather than a rendered C type: the shape is what decides how a name is spelled (`extern u16 tbl[]` vs a scalar vs a struct), and it is a small closed set, where a full DIE→C-type renderer is neither needed nor cheap. `null` when the name has no DWARF DIE, which doubles as the "is this name declared in the headers?" probe. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ctions Byte order is read from e_ident and threaded through every multi-byte read: Cursor carries a littleEndian flag, ElfFile exposes the container's order, and the DWARF payload always shares it. Big-endian ELF32 is what MIPS and PowerPC toolchains emit, so those images parse now. PowerPC relocatable objects are RELA-style: string and section offsets inside raw `.debug_*` bytes are zeros, and the real values live in the addends of `.rela.<sec>` (ARM/MIPS REL keeps them in the field itself). sectionData applies those addends to a cached copy, so DWARF in a raw `.o` parses identically whichever relocation style the target uses. Verified against cross-gcc sidecar objects (BE MIPS R3000: 283 vars + 209 structs from a real N64 ctx; BE PPC32 including struct layouts) and a GameCube main.elf (36,588 symbols). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…iables Layout does not describe a declaration on its own: several declarations produce the same bytes at the same offset, and what separates them lives in the type chain rather than in the offsets. `struct()` members and `variableShape` carry those facts. - members carry `signed`, from the member's base-type `DW_AT_encoding` — offset and size alone do not say whether a byte reads as -1 or as 255 - members carry `pointer`, which separates the 4-byte cases that share `signed: null` (pointer vs enum vs nested struct): indistinguishable by offset and size, and not interchangeable — pointers compare as unsigned - members carry `volatile` — the `vu16 field;` MMIO idiom, which says repeated accesses to the field are observable rather than foldable - `variableShape` carries `volatile` / `const`, collected while resolving the typedef and cv-qualifier chain. `const` is the ROM-table spelling; for an array DWARF puts the qualifier on the ELEMENT type, so the element chain feeds the same accumulator Both minimal test projects declare a volatile struct with a `signed char` member, a member-level volatile, a pointer member, a volatile scalar and a const table, so every fact is covered in both DWARF dialects (agbcc DWARF-2 and modern GCC); their ELFs and binutils oracles are rebuilt from that source. Those declarations sit above `add()` in agbcc-min/main.c, which puts its body at line 94. gba-emulator's scripting tests assert that line against the same shared ELF, so they name 94 too.
…y unit_length alone `.debug_line` is a concatenation of independent units, so finding where a unit ends is as much of the parser's job as decoding one. Two properties of the input shape how that is done. `unit_length` is not a dependable end marker. agbcc (GCC 2.95, the pret compiler) sizes a line unit by *predicting* the encoded length of every statement it is about to emit, and mispredicts, so the declared length can stop short of the program it describes: in pokeemerald 28 of 303 units undercount — 18 by 1 byte, 6 by 2, 2 by 3, 1 by 4, and event_object_movement.o by 51. It is visible in the objects themselves: build/emerald/src/text.o has a 9401-byte .debug_line whose unit_length says 9400, and the final `00 01 01` (DW_LNE_end_sequence) at 0x4c30 needs 0x4c33 while the unit is declared to end at 0x4c32. A walk that trusts the declared end desyncs — it stops mid-statement, then executes the next unit's header as line-program bytes, yielding garbage addresses, a nonsense `unit_length` and a read past the section. The line program is self-delimiting: every sequence ends with DW_LNE_end_sequence. That terminator is the authority on where a unit ends, so statements run to the declared end *and* past it while a sequence is still open. Well-formed units are unaffected — they end on the terminator exactly. Around that, one unwalkable unit costs only itself. A version this parser does not model (DWARF 5 rewrote the header: address_size/segment_selector_size, and directory/file tables described by entry formats instead of NUL-terminated lists) and 64-bit DWARF are skipped by their own length; zero-word padding is stepped over; a unit whose length runs past the section is decoded as far as it goes; every read in the program loop is bounded. A hostile section yields fewer rows, never an exception — parseDebugLine is the only thing standing between a bad `.debug_line` and DebugInfo.fromElf, which must still deliver symbols and types. pokeemerald decodes to 193,233 rows across all 303 units, consuming the section to its last byte, and agrees with readelf on every row readelf decodes. The two test-project ELFs decode identically to readelf. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ng cross-endian equivalence
Big-endian support landed with no committed big-endian artifact: no DWARF
payload was ever read MSB-first by a test, and the RELA path in `sectionData`
— the one that makes a PowerPC `.o` readable at all — had no coverage. Two
projects built by stock Ubuntu cross packages close both gaps, alongside the
two ARM ones.
`mips-min` (mips-linux-gnu-gcc 12.4, MIPS o32) and `ppc-min`
(powerpc-linux-gnu-gcc 13.3, PowerPC 32) compile ONE source — their `main.c` /
`util.c` are byte-identical — with `-g -O2
-fno-eliminate-unused-debug-types`, linked freestanding, never executed. Every
declaration is one shape the parser classifies: a scalar, a pointer, `short
g_table[4]`, `const short g_rom_table[3]`, a `volatile` scalar, `struct Probe`
(named members at 0/4/8/10/16/20/28, size 32), `struct Bits` (bitfields), and
`struct Cv` (a `signed char` next to a member-level `volatile unsigned
short`). `char` signedness is never left to the default — it is signed on MIPS
and unsigned on PowerPC — so both projects assert the same numbers. `triple`
lives in `util.c`, so each linked ELF has two CUs and its `.debug_line` two
sequences; both are read entirely MSB-first, and every function entry agrees
with `addr2line` on `{func, file, line}`.
Bitfields are the assertion class the little-endian projects structurally
cannot make. A big-endian target allocates them from the MOST significant end,
so the identical declaration that ARM pins as hearts@0>>0, stars@0>>2,
cross@0..1>>5, wide@1>>4 lands mirrored: hearts@0>>6, stars@0>>3,
cross@0..1>>4, wide@1>>0.
That mirror was NOT what the parser reported. `DW_AT_data_bit_offset` (and
DWARF 2/3's `DW_AT_bit_offset`) is measured from the end the target allocates
from, and `#memberLayout` normalized it as if that end were always the least
significant one — so every big-endian bitfield was reported at its
little-endian position, silently, with a plausible shape. `TypeIndex` now
carries the ELF's byte order and flips the intra-unit shift. The compilers'
own read-modify-write of `g_bits.cross` is the ground truth for the fixed
numbers, and both agree on a 2-byte access at offset 0, shift 4, width 7:
MIPS lhu $t2,g_bits ; ins $t2,$v0,0x4,0x7 ; sh $t2,g_bits
PPC lhz r6,0(r7) ; rlwimi r6,r9,4,21,27 ; sth r6,0(r7)
`ppc-min` also vendors `build/main.o`, a relocatable object — the only artifact
shape that exercises RELA. Its `.debug_info` has 59 relocations, and the raw
field at every one of the 59 sites is ZERO: unrelocated, every `DW_FORM_strp`
would resolve to `.debug_str` offset 0, one single name for the whole unit.
The tests assert that (all 59 raw fields zero), that each patched word equals
`symbol value + addend`, and that the five sites whose symbol is a data symbol
rather than a section symbol carry a NON-zero `st_value` (g_bits 4, g_vol 12,
g_table 16, g_ptr 24, g_counter 28 — the `.bss` offsets `nm` reports), which is
what pins the "symbol value +" half of the sum. Every struct tag and long
member name in that object resolves, and its layouts equal the linked ELF's.
Cross-endian equivalence gets its own block in `real-projects.spec.ts`,
comparing all FOUR projects against each other:
- The shared declaration set is computed, then PINNED: `Probe`, `Inner`,
`Bits`, `Cv`, `UtilPair`, and `g_counter` / `g_probe` / `g_bits` / `g_cv` /
`g_rom_table` / `g_util_pair`. Everything else is listed with the projects
that lack it (`Pair`/`g_pair`/`g_color`/`g_mode`/`g_mmio` absent from the
big-endian sources; `Shape`/`Blob`/`g_shape`/`g_wide`/`g_blob` devkitarm-only,
since GCC 2.95 rejects anonymous unions and flexible array members; `g_ptr` /
`g_table` / `g_vol` big-endian-only). A skip is a named fact, so a shape that
quietly stops parsing shrinks the comparison and fails HERE rather than
passing a smaller one.
- Every project reports the same byte layout: Probe `tag@0:1 count@4:4
flags@8:2 name@10:6 ptr@16:4 inner@20:8 tail@28:4`, size 32; Inner `x@0:4
y@4:2`, size 8; Bits `hearts@0:1 stars@0:1 cross@0:2 wide@1:1 after@4:4`,
size 8; Cv `level@0:1 gain@2:2`, size 4; UtilPair `lo@0:2 hi@2:2`, size 4.
- All four agree on every remaining declaration fact — signedness, pointer-ness,
member-level volatile, bitfield WIDTHS — with only the intra-unit shift
dropped from the comparison, since that is the one field byte order may
change.
- `variableShape` is identical in all four for each shared global.
- Bitfields carry the load-bearing assertion. Each side must match its own ABI
while offset, read size and width stay identical on both. And the two tables
must be MIRRORS, not merely different: `leShift + beShift + bitWidth ===
size * 8` holds for all four fields (0+6+2=8, 2+3+3=8, 5+4+7=16, 4+0+4=8).
The mirror survives to `resolveVariable`, which reports `g_bits.cross` as a
2-byte read at the symbol with shift 5 on the little-endian ELFs and shift 4
on the big-endian ones.
The block was checked against deliberately broken parsers, one mutation at a
time:
- `ElfFile.parse` forced to `littleEndian = true` (ELFDATA2MSB ignored): it
dies at load — `RangeError: Offset is outside the bounds of the DataView`,
reading the real MIPS ELF's section header table.
- `.debug_info` read LSB-first (`TypeIndex.fromElf` given `true` instead of
`elf.littleEndian`): 5 tests fail, including the shared-set pin — `Cv` stops
resolving in the big-endian ELFs, and the guard catches the comparison
shrinking instead of quietly comparing less.
- Bitfield normalization flipped (the little-endian rule applied to both): the
block reports mips-min at `hearts 0 / stars 2 / cross 5 / wide 4` where the
ABI says `6 / 3 / 4 / 0`, and `resolveVariable` fails with it. This is the
regression that shipped silently before `88c0bed`, and it is now caught by
the cross-endian block on its own.
CI installs gcc/binutils-{mips,powerpc}-linux-gnu — the runner is x86 Linux, so
they are native, no Docker or qemu — and `globalSetup` rebuilds all four
projects from scratch there, so the committed artifacts stay honest. Locally
the two new projects rebuild through `./build.sh` in `ubuntu:24.04`, so no
cross toolchain is needed on a contributor's box.
@gba-kit/debug-info: 186 tests, all green; `pnpm turbo build test check-types
lint check-deps` and `pnpm run format:check` clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ements, and alias naming
Facts `.debug_info` carries that the API read past.
`variableShape`'s pointer arm reported the cv-qualifiers and nothing else —
`{ kind: 'pointer', volatile, const }`. Every pointer global classified identically,
so a caller holding one could name neither the type it addresses nor its size. It
now carries `pointee`: `{ structName, size, volatile, const }` when the target
resolves — through typedefs and cv-qualifiers, the same walk the rest of the shape
uses — to a struct or union, and `null` for every other target (a scalar, another
pointer, a function, `void`).
`structName` is deliberately not "the struct tag". It is the name `struct()` looks a
layout up by, and for the `typedef struct {…} T;` idiom that is `T`: the struct there
is unnamed, and the alias is the only name it has. So the last typedef crossed on the
way to the target is reported when the tag is absent, which makes the two calls a
round trip — `struct(shape.pointee.structName)` is the layout the pointer addresses.
Null when the target has neither, since then no name retrieves it. A target that is
only forward-declared carries no `DW_AT_byte_size` of its own, so the size is read
from the definition its tag resolves to.
The `struct` arm is named by exactly the same rule. It used to read `structName`
straight off the resolved DIE's `DW_AT_name` — for `typedef struct {…} T; T g;`,
the single most common way a C header names a struct, that DIE is ANONYMOUS, so the
arm returned `structName: null` and the caller could not look the layout up with
`struct()` at all. The shared tail is `#structTarget(die, alias)`, called by both
arms, so a struct global and a pointee are named identically and either name goes
straight back into `struct()`. The size comes through it too, which also sizes a
struct global whose DIE is only a forward declaration.
The pointee's qualifiers are the ones left of the `*` and must not reach the pointer
variable's own, right of it. They cannot: the variable's cv is accumulated by the
walk that strips down TO the `DW_TAG_pointer_type` DIE, the target's by the separate
walk that starts at that pointer's `DW_AT_type`. `devkitarm-min` declares both
spellings — `volatile struct Cv *g_cv_ptr` and `struct Cv *volatile g_cv_vptr` — and
the tests assert the volatile lands on a different one of the two objects in each.
`struct()` members reported `size` — the WHOLE member, 16 for `u8 x[16]`. Nothing in
that number says where the n-th element of such a member begins, or how many there
are, so an indexed read into one was not expressible from what `struct()` returned.
Members now also carry `elemSize` / `elemSigned` / `length`, spelled exactly like
`VariableShape`'s array arm and populated by the same helpers, so one member reads
`name@10:6 elemSize 1 elemSigned false length 6`. Each key is omitted when the DWARF
does not determine it, never defaulted: a flexible array member (`char data[]`)
reports `elemSize` and NO `length`, the two being independent facts, and `elemSigned`
is absent for an array of structs/pointers/enums exactly as `signed` is null for a
member that is not a base type. The presence of `elemSize` is what identifies a
member as an array — `signed` stays null there, an array not being a base type.
`MemberLocation` is unchanged: the new keys join its `Omit` list, next to
`signed`/`pointer`/`volatile`, because they describe the declaration and not where to
read. `structMember`, `variableMember` and `resolveVariable` therefore return exactly
what they returned before, byte for byte.
Test-project declarations added to exercise the new arms on real toolchain output,
each fitted into existing lines so every file's line count is unchanged (the
committed ELFs and a cross-package `line: 94` assertion are pinned to them):
- `mips-min` / `ppc-min` (kept byte-identical): `struct Probe *g_probe_ptr =
&g_probe;` — a tag-named pointee, next to the `int *g_ptr` already there, which
is now the negative case (a scalar target reports `pointee: null`). Both are also
read out of `ppc-min`'s relocatable `main.o`, where the pointee's name lives only
in a `.rela.debug_info` addend.
- `devkitarm-min`: `Pair *g_pair_ptr = &g_pair;` — the unnamed-struct-behind-a-
typedef pointee, plus the two `Cv`-pointer cv spellings. They join the
devkitarm-only block because these are little-endian-source shapes; the
cross-endian block lists the new globals by name among the ones not shared by all
four, so a shape vanishing from a project's DWARF still fails there rather than
shrinking the comparison.
The initializers put the new globals in `.data`, leaving the `.bss` addresses the
RELA block cites unmoved. All four ELFs and their oracles were rebuilt (agbcc-min's
sources are untouched, so its artifacts are byte-identical).
Checked against deliberately broken parsers, one mutation at a time:
- the typedef-alias fallback dropped (`structName: tag ?? null`): the devkitarm
block fails — `Pair *` reports a nameless pointee, and `struct()` cannot be
handed it.
- `elemSize` read from the member's own type size instead of the element type's
(16 rather than 1 for `u8 x[16]`): 7 tests fail across all four projects,
including the flexible-array member, whose stride would become null.
- the struct/union guard removed from `#pointee` so any target reports one: the
big-endian pair fails — `int *g_ptr` gains a `{ structName: null }` pointee, the
exact "there is a layout here" claim the null is there to deny.
@gba-kit/debug-info: 186 tests → 191, all green; `pnpm turbo build test check-types
lint check-deps` and `pnpm run format:check` clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ints at
Two facts that complete `StructMember`'s declaration set.
`#memberFacts` already accumulated `{ volatile, const }` while stripping a member's
type chain and reported only the first, so a `const`-qualified member was
indistinguishable from a plain one. It now reports `const?: true` on `StructMember`
alongside `volatile?: true`, populated the same way — present is the fact, absent is
its absence.
The two are the same class of fact for the same reason: a cv-qualifier moves no
field, so nothing about a member's offset or size carries it, and a consumer
re-spelling the declaration cannot reproduce what it cannot see. They are not
decoration either — a write through a const member is a constraint violation, not
another spelling of the same access — so `MemberLocation` still omits both: a
location is where to read, not what may be done there.
A pointer member reported only `pointer: true`, so a consumer knew the cell is
four bytes but not what it addresses. That is not decoration: pointer
arithmetic scales by the pointee width, so `p - 4` through a `u16 *` and
through a `void *` reach different memory, and a consumer declaring the member
had to guess. `pointeeSize`/`pointeeSigned` are reported when the target resolves
to a base type, and omitted otherwise (`void *`, `struct S *`, function pointers)
— a present key is a fact, never a default.
`devkitarm-min`'s `struct Shape` declares its tag field `const int kind;`; const
changes no layout, so the existing offsets/sizes and the file's line count are
untouched, and `build/min.elf` and its oracle are rebuilt. All four test projects
agree that `int *ptr` is a 4-byte signed target.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The branch's whole surface so far — `variableShape` and its `pointee`, the declaration facts `struct()` members grew, big-endian ELF/DWARF and RELA-relocated `.debug_*`, and the `.debug_line` terminator walk — has been unreleased and unrecorded. One minor changeset covers it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
`functionSignature(name)` returns what a function returns and the type of each parameter, from the subprogram DIEs a compiler emits for code it compiled from source. Same width/signedness/pointer vocabulary a struct member uses, so a parameter and a field of the same declared type describe identically. Only DEFINITIONS are indexed — `low_pc` is the witness. A function that is still hand-written assembly, or merely declared in a header, has no subprogram DIE at all (gcc-2.x drops body-less declarations outright), so null means "this ELF did not compile it", never "it takes no arguments". `prototyped` reports the declaration style; the parameter list is authoritative regardless, since a definition records what it was compiled to take. Verified against pokeemerald's agbcc-emitted DWARF-2: 15,678 of 15,858 functions, with signatures matching the sources exactly (`s16 Sin2(u16)`, `u16 CalcCRC16(const void *, s32)`, `u8 *StringCopyN(u8 *, const u8 *, u8)`). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nfo) A compiler invoked with -g3 records every #define it saw. That is the only place some facts survive at all: a macro leaves no symbol, no type and no DIE, so a consumer reading an ELF has no other way to learn that a project spells a fixed address `gCounter` rather than `(*(u16 *)0x03001234)`. Parses the DWARF 2/3 form, which is a flat opcode stream carrying its strings INLINE. The DWARF 5 replacement (.debug_macro) is deliberately not read: it splits a unit's macros across COMDAT group sections joined by DW_MACRO_import and refers to .debug_str for every name, so neither survives being lifted out of one object — whereas this form is self-contained by construction. A truncated or malformed stream stops rather than throwing: a partial list is still sound (every entry in it was really read), and these sections get grafted between tools often enough that hard-failing would be the wrong default for purely additive data. Verified on two real sidecars: 1,380 definitions (41 address casts) and 1,941 (1 address cast). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The agbcc submodule moves to a0f70c9, which fixes two bugs in agbcc's own DWARF output. The relevant one here is the missing `.debug_abbrev` table terminator: every standard tool refused the section outright, so the ELF this project builds to exercise DWARF-2 parsing was one no other reader could load. `readelf --debug-dump=abbrev build/min.elf` now reports zero errors where it previously errored. The parser needed no change — it already bounds each table walk by the next unit's offset rather than trusting the terminator, which is what let it read agbcc output at all — so all 196 tests pass unchanged. The committed artifact is rebuilt with it (4 bytes larger, one terminator per unit); build/oracle.json is byte-identical, which is the evidence that the compiler fix moves no code. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
macabeus
force-pushed
the
extend-debug-info-for-symbol-maps
branch
from
August 1, 2026 23:25
e3d1983 to
be34b45
Compare
…nd fix the truncation it caught The macro parser shipped with zero in-package tests. devkitarm-min now vendors build/macinfo.o — its main.c compiled with -gdwarf-2 -g3 -gstrict-dwarf, the exact macro-sidecar recipe, relocatable on purpose because the graft source in a real project is a .o. Fixture #defines live at the END of main.c (existing pinned line numbers don't move; the spec asserts the fixtures by exact line): two address-cast RAM-cell macros, a plain constant, a function-like macro, and a body-less one. readelf --debug-dump=macro agrees on all 417 defines. min.elf and oracle.json rebuilt via build.sh, byte-identical — the appended defines are unused, and the plain -g build records no macro info, which is itself a pinned test (the -g3 requirement). The truncation test immediately caught a real bug: parseDebugMacinfo's documented contract is "a partial macro list is still sound — every entry in it was really read", but a stream cut mid-string surfaced a CORRUPTED define as a real one (Cursor.cstr returns the partial text when the NUL never arrives: "__INT8_C(c" with its body folded into the name). Grafted sections are the parser's own stated threat model. Both cstr sites now stop at an unterminated string, and the spec pins the prefix property at three cut points: every returned entry equals the full parse's entry, at any cut. 203 tests, typecheck, lint, format all green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four committed-artifact reproductions for parser bugs an adversarial review found (the spec that pins them lands with the fixes, next commit): - struct FwdPay: forward-declared in main.c's CU, DEFINED in util.c's, linked in that order — the shadowing scenario where a first-CU-wins index loses the layout to a DW_AT_declaration stub. - g_zero[0] and Flex.data[0]: agbcc encodes a zero-length array's -1 upper bound as unsigned DW_FORM_data4 0xffffffff, at variable and member level. - g_ext_table[]: an unsized extern array whose definition lives in crt0.s — the asm/ldscript-placed-table idiom. agbcc emits upper_bound 0, byte-equal to a real [1]; DW_AT_declaration on the variable is the only disambiguator. g_one_def[1] is the defined [1] that must KEEP its length. - negative controls: g_init_table[][2] and the pret-style forward-declared static g_fwd_sized_table[][2] — agbcc sizes both correctly (bounds patched at the definition), so the -1 fix must not cost them. All shapes appended after the shared core (line-stable above); min.elf and oracle.json rebuilt with the vendored agbcc, 203 existing tests green unchanged. mips-min/ppc-min already carry the abstract/concrete signature split (add/square) and devkitarm's macinfo.o the DWARF-2 prototyped flag — no rebuild needed there. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ducer-quirks.spec The spec was written first against the committed trigger artifacts and failed 8/11 (the 3 passing were its negative controls); these four fixes turn it green without moving any of the 203 existing tests. 1. DW_FORM_flag now decodes to a BOOLEAN. The raw byte satisfied no `=== true` test, so on DWARF 2/3 every declaration check in the file was inert: a forward-declared struct shadowed its own definition depending on link order (agbcc-min's FwdPay came back size null, zero members), and `prototyped` was always false on modern-gcc -gdwarf-2 output. 2. functionSignature resolves the abstract/concrete split. Modern gcc at -O1+ emits an inlined-and-emitted function as an abstract DIE (name, params) plus a concrete DIE (low_pc, abstract_origin); indexing only same-DIE name+low_pc returned null for mips-min/ppc-min's add and square — exactly the small helpers a decompiler wants callee signatures for. The concrete half is indexed under the abstract name, and each fact (params, return, prototyped) is read from the DIE that carries it, per-parameter origins included. 3. arrayLength normalizes GCC 2.95's 0xffffffff upper bound (-1 stored in unsigned data4) before the +1: a zero-length array is unknown-length, not 2^32 elements. This was live in shipped data — pokeemerald's two sWhiteoutRespawn tables claimed 4 and 16 GiB, turning lookupInterior into a wrong-name trap for every unnamed address above 0x0859F5EC. Both now report length null against the real ELF; initializer-sized arrays keep their true bounds (pinned). 4. variableShape reports length null for a DECLARATION's [1]: agbcc encodes an unsized extern array as upper_bound 0, byte-identical to a real [1], and the variable's own DW_AT_declaration is the disambiguator. klonoa's gSineTable (an asm-placed table) drops its confident wrong length: 1; a DEFINED one-element array keeps its length (pinned). The rare genuine `extern T x[1]` gives up a near-information-free fact — weakening over overstating. Validated against the real consumers: klonoa-eod.elf keeps all 222 signatures and gSineTable reads honest; pokeemerald.elf's poisoned entries are sane. 214 tests, typecheck, lint, format green. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…lease
The changeset had been written at the declaration-shape milestone and never
grew: functionSignature, the .debug_macinfo reader and the four
producer-dialect fixes — half of what 0.4.0 ships — were absent from what
will become the CHANGELOG. Rewritten as one concise bullet per capability,
contracts kept ("null means this ELF did not compile it", the truncation
prefix property, the flag/2^32/declaration-[1] fixes).
The npm-facing README gains the two missing capabilities in the feature list
and three Usage lines. Every example value is real output from the klonoa
ELF, re-verified against the built package — which caught that the example I
first reached for (gStreamPtr) no longer exists in the project's headers;
gGfxStreamBuffer is what its macro table actually says today.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fourteen self-contained commits taking
@gba-kit/debug-infofrom a GBA-shaped reader to a general ELF32/DWARF one: declaration facts alongside layout, function signatures, the preprocessor macro table, both byte orders, relocatable objects, a.debug_linewalk that survives real producer output — and four producer-dialect bug fixes found by adversarial review, TDD-pinned on committed toolchain artifacts. (The motivating consumer is asmlift, which builds an address→symbol map out of an ELF; nothing here is specific to it.) Each commit stands alone, and@gba-kit/debug-infogoes 87 → 214 tests.variableShape— classify a global's declaration shapeWalks a variable DIE's type through typedefs and cv-qualifiers to
scalar | pointer | array | struct, with element size/signedness for arrays and tag name/byte size for structs.nulldoubles as the "is this name declared in the headers?" probe. Layout was already exposed; this answers what kind of thing a name is — a small closed set, where a full DIE→C-type renderer is neither needed nor cheap.Big-endian ELF/DWARF + RELA-relocated debug sections
Byte order is read from
e_identand threaded through every multi-byte read (Cursorgains alittleEndianflag), so the MSB-first ELF32 images MIPS and PowerPC toolchains emit parse. Separately, PowerPC relocatable objects are RELA-style: string/section offsets in raw.debug_*bytes are zeros with the real values in.rela.<sec>addends (ARM/MIPS REL keeps them in the field), sosectionDataapplies those addends to a cached copy and DWARF in a raw.oparses identically across both conventions.Verified against real artifacts: a big-endian MIPS sidecar object (283 variables + 209 structs), a big-endian PPC32 sidecar including struct layouts, and a GameCube
main.elf(36,588 symbols).Declaration facts for members and variables
Layout does not describe a declaration on its own: several declarations produce the same bytes at the same offset, and what separates them lives in the type chain rather than in the offsets.
struct()members now carrysigned(the member's base-typeDW_AT_encoding— offset and size alone do not say whether a byte reads as -1 or as 255),pointer(which separates the 4-byte cases sharingsigned: null— pointer vs enum vs nested struct, indistinguishable by offset and size, and not interchangeable since pointers compare as unsigned), andvolatile(thevu16 field;MMIO idiom — accesses to the field are observable rather than foldable).variableShapecarriesvolatile/const, collected while resolving the typedef and cv-qualifier chain; for an array DWARF puts the qualifier on the element type, so the element chain feeds the same accumulator. Both ARM test projects declare a volatile struct with asigned charmember, a member-level volatile, a pointer member, a volatile scalar and a const table, so every fact is covered in both DWARF dialects; their ELFs and binutils oracles are rebuilt from that source.Walk
.debug_lineby its own terminatorsunit_lengthis not a dependable end marker. agbcc (GCC 2.95) sizes a line unit by predicting the encoded length of the statements it is about to emit, and mispredicts: in pokeemerald 28 of 303 units under-count — 18 by one byte, 6 by two, 2 by three, one by four, andevent_object_movement.oby 51. A walk that seeks to the declared end therefore resumes mid-statement and reads the next unit's header as line-program bytes. The line program is self-delimiting, soDW_LNE_end_sequenceis the authority on where a unit ends; around that, one unreadable unit does not cost the section (unsupported versions and 64-bit DWARF are skipped by their own length, zero-word padding is stepped over, every read in the program loop is bounded), and a sequence that never closes discards its own rows, so recovery cannot invent data.pokeemerald yields 193,233 rows across all 303 units, consuming the section to its last byte, and agrees with
readelf --debug-dump=decodedlineon every row readelf decodes. The regression tests build on the committed agbcc DWARF-2 bytes, shortening a unit'sunit_lengthby 1–7 and asserting the decoded rows are unchanged.Big-endian MIPS + PowerPC test projects, proving cross-endian equivalence
mips-min(mips-linux-gnu-gcc, MIPS o32) andppc-min(powerpc-linux-gnu-gcc, PowerPC 32) join the two ARM projects, compiling one byte-identical source with-g -O2 -fno-eliminate-unused-debug-types, linked freestanding and never executed. They cover what no little-endian ELF can: DWARF read MSB-first end to end, and bitfields allocated from the most significant end of the storage unit — which the parser normalized as if the low end were universal, reporting every big-endian bitfield at its little-endian position. Ground truth for the fixed numbers is each compiler's own read-modify-write ofg_bits.cross.ppc-minalso vendorsbuild/main.o— the only artifact shape that exercises the RELA path — whose.debug_infohas 59 relocations, every raw field zero.The same commit adds a cross-endian equivalence block comparing all four projects: the shared declaration set is pinned (anything absent from a project is listed by name, so a shape that quietly stops parsing fails there rather than shrinking the comparison), every project must report the same byte layout and the same declaration facts, and the bitfield shifts must be mirrors —
leShift + beShift + bitWidth === size * 8— not merely different. Checked against deliberately broken parsers one mutation at a time. CI installs the stock Ubuntu cross packages (native, no Docker or qemu) and rebuilds all four projects from scratch, so the committed artifacts stay honest.What a pointer points AT, an array member's elements, and alias naming
variableShape's pointer arm gainspointee—{ structName, size, volatile, const }when the target resolves to a struct or union,nullfor every other target.structNameis deliberately not "the struct tag": it is the namestruct()looks a layout up by, and for thetypedef struct {…} T;idiom that isT, sostruct(shape.pointee.structName)is always a round trip. Thestructarm is named by exactly the same rule. The pointee's qualifiers are the ones left of the*and provably cannot leak into the pointer variable's own, right of it — the test projects declare both spellings and assert the volatile lands on a different object in each.struct()members of array type additionally carryelemSize/elemSigned/length, so an indexed read intochar name[6]is expressible; each key is omitted when the DWARF does not determine it.A member's const, and what a pointer member points at
Completing
StructMember's declaration set:const?: truealongsidevolatile?: true(a cv-qualifier moves no field, so nothing about offset or size carries it), andpointeeSize/pointeeSignedon pointer members whose target is a base type (pointer arithmetic scales by the pointee width, sop - 4through au16 *and through avoid *reach different memory).MemberLocationstill omits all of these: a location is where to read, not what may be done there.Read a compiled function's declared signature
functionSignature(name)returns what a function returns and the type of each parameter, from subprogram DIEs — the same width/signedness/pointer vocabulary a struct member uses. Only DEFINITIONS are indexed, withlow_pcas the witness: a function that is still hand-written assembly, or merely declared in a header, has no subprogram DIE at all, sonullmeans "this ELF did not compile it", never "it takes no arguments". Verified against pokeemerald's agbcc-emitted DWARF-2: 15,678 of 15,858 functions, signatures matching the sources exactly.Read the preprocessor macro table (
.debug_macinfo)A compiler invoked with
-g3records every#defineit saw — the only place some facts survive at all: a macro leaves no symbol, no type and no DIE, so a consumer reading an ELF has no other way to learn that a project spells a fixed addressgCounterrather than(*(u16 *)0x03001234). Parses the DWARF 2/3 form, whose opcode stream carries its strings inline and is self-contained by construction — the form a macro sidecar graft produces. Verified on two real sidecars: 1,380 definitions (41 address casts) and 1,941 (1 address cast).Tested on a committed artifact:
devkitarm-minvendorsbuild/macinfo.o(itsmain.ccompiled with-gdwarf-2 -g3 -gstrict-dwarf, the exact sidecar recipe), with fixture#defines pinned by name/body/line andreadelfagreeing on all 417 defines. Writing the truncation test caught a real bug on the spot: a stream cut mid-string surfaced a corrupted define as a real entry (Cursor.cstrreturns partial text when the NUL never arrives). Both string sites now stop at an unterminated string, and the spec pins the prefix property — every returned entry equals the full parse's entry, at any cut.agbcc-min rebuilt with the fixed agbcc
The agbcc submodule moves to a commit fixing two bugs in agbcc's own DWARF output — the relevant one being the missing
.debug_abbrevtable terminator, which made every standard tool refuse the section outright. The parser needed no change; the committed artifact is rebuilt 4 bytes larger, withbuild/oracle.jsonbyte-identical — the evidence that the compiler fix moves no code.Four producer-dialect fixes, TDD-pinned (
producer-quirks.spec.ts)An adversarial review reproduced four wrong readings of real compiler output; the test projects grew the trigger shapes first (the spec failed 8/11 against them, its 3 passes being negative controls), then the fixes turned it green:
DW_FORM_flagdecodes as a boolean. DWARF 2/3's flag form is a byte; returning the raw number satisfied no=== truetest, so every declaration check was inert on those dialects — a forward-declared struct shadowed its own definition link-order-dependently (agbcc-min pins the scenario:FwdPaydeclared in the first CU, defined in the second), andprototypedwas always false on modern-gcc-gdwarf-2output.functionSignatureresolves gcc's abstract/concrete split. At-O1+a function that is both inlined and emitted becomes an abstract DIE (name, params) plus a concrete DIE (low_pc,abstract_origin); indexing only same-DIE name+low_pc returnednullfor exactly the small helpers a consumer wants callee signatures for (add/squarein mips-min and ppc-min are the committed proof). Each fact is now read from the DIE that carries it, per-parameter origins included.0xffffffffupper bound = zero-length, not 2³² elements. GCC 2.95 stores the -1 bound in unsigneddata4;upper+1on the raw u32 claimed 4-GiB shapes. This was live in shipped data — two pokeemerald tables claimed 4 and 16 GiB — and both now readlength: nullagainst the real ELF, while initializer-sized arrays keep their true bounds (pinned).[1]array reportslength: null. agbcc emitsupper_bound 0for an unsized extern array, byte-identical to a real[1]; the variable's ownDW_AT_declarationdisambiguates (agbcc-min'sg_ext_tableis defined only incrt0.s— the ldscript/asm-placed-table idiom). A defined[1]keeps its length.Real-world validation of the four: klonoa's
gSineTabledrops its confident wronglength: 1with all 222 signatures intact, and re-vendoring asmlift's benchmark maps changed exactly the predicted entries (kleod −26 phantom sizes; pokeemerald +6,701 real sizes, since prefer-definition finally works on DWARF-2) with zero rows moving across the 675-row benchmark.No API is removed or changed in shape — every addition is a new field or a new method, so existing consumers are unaffected. The changeset records it all as one minor bump; this is the
0.4.0that unblocks asmlift's vendored symbol maps.