patch/ holds the machine-readable artifacts (the runtime delta vs
upstream); docs/ keeps prose. README references updated.
Co-Authored-By: Claude Code <noreply@anthropic.com>
What Stern checks when it says GAME VALIDATION ERROR ... UPDATE SD CARD:
two independent layers. (1) the sidx per-file HMAC-SHA1+MD5 manifest over
the 1306 /games files, verified by spk/spike_menu; (2) the game's own
boot check (SD card geometry via hdparm, node firmware, SHA1) that
produces the permanent #1..#6 service-menu message — the #6 12:1:0 we
got after the 2026-10-22 card reflash. Includes the game ELF's layout
gotchas (two VA mappings, all-indirect PLT calls, unreferenced
localized catalog) that stalled the static trace, and the open
questions with the on-machine diagnostic path.
Co-Authored-By: Claude Code <noreply@anthropic.com>
When the game builds its high-score table report for Stern's agent
(/api/v1/game/high_score_events - the endpoint the Insider gate already sits
on), the logger writes an HSTD line with all four players' scores. This is
the report the game sends when a game has changed the table, so the line
marks a high-score event with no new per-game RE. The entries themselves
(initials, position) live deeper in the report body and stay uncaptured for
now - that needs locating the report builder or the in-memory table.
Tests: the endpoint trigger and the HSTD line lift into the host harness
(the endpoint pointer is a 32-bit register, so the harness maps its string
below 4 GiB and the tests run on the Linux leg like the port-gate ones).
Co-Authored-By: Claude Code <noreply@anthropic.com>
The gameplay logs sat on /dump on the machine, readable only over SSH. Now
they land in Loki for query and tail:
- k8s/loki/loki.yaml: Loki 3.4.4 monolithic on the home cluster's monitoring
namespace — filesystem storage on CephFS, 30-day retention, traefik ingress
at loki.k8s-home.lqx.net with a cert-manager cert, Grafana datasource
auto-registered via the sidecar. reject_old_samples_max_age is a year:
the machines backfill their history on first ship.
- shipper/: logshipper, a ~300-line static Go service for the machine. Tails
/dump/mode.log and /dump/switch.log, batches, pushes to Loki's push API.
Per-file offsets are saved only after a successful push (down Loki =
retry-then-catch-up, never loss); first run starts at each file's end,
SHIP_FROM_START=1 ships history; truncation is noticed. Lines keep their
own RFC 3339 stamp as the Loki timestamp; unstamped lines (the boot
banner) inherit the next stamped line's, so a stream never starts at
"now" and jumps back. After a backfill it calls /flush — old entries are
invisible in the ingester's head until they reach the store (found the
hard way: Loki answers 204 to an old entry on a new stream and then never
serves it). Labels: job=pinball, machine, source, game (from the boot
line). /etc/init.d/log_shipper runs it at boot.
Deployed: Loki is live and verified (query through the ingress). The shipper
is verified end-to-end against the cluster from this Mac; the machine was
unreachable at deploy time — the README has the install commands.
Co-Authored-By: Claude Code <noreply@anthropic.com>
The runtime tried one port file (game.port) and refused it if the game's
words did not match. Now it tries every *.port in /usr/local/padmode/ports
and /dump/ports (then the single files, the old layout), resetting the port
tables between candidates, and keeps the first whose core sites verify: a
wrong candidate hooks nothing, so trying it is safe, and the port that
verifies names the game (pm_game/pm_version).
The switch log's buttons and names come from the port's switch lines now —
the baked-in King Kong button list and switch_names.inc are gone (the port
lines carry the same ids and names, so King Kong's switch.log is unchanged).
Boot log: each refused candidate says its path and the game/version it
declares; no candidates at all says so; a game update whose port is not
deployed logs the refusal and runs stock.
tests/test_multi_game.py lifts the new code verbatim and compiles it for the
host: selection among candidates, table reset between them, the getdents64
*.port scan (it caught an inverted suffix test and an off-by-one), and the
port-driven switch log. The address-dependent tests need a Linux host (the
port's site addresses are 32 bits; darwin arm64 maps nothing below 4 GiB).
Co-Authored-By: Claude Code <noreply@anthropic.com>
enable-ssh.sh turns a factory raw SD image (King Kong LE 0.97) into an
SSH-enabled one: injects the static dropbear/dropbearkey/sftp-server
binaries, the SysV init script (/etc/init.d/sshd), the S30sshd boot
symlink and the caller-supplied public key into the ext3 rootfs
partition with debugfs (no mounting). The input image is never
modified, the output path must be explicit and distinct, and every
byte outside p2 is verified identical afterwards. The public key is an
explicit -k argument — nothing is baked in and private keys are
rejected. --dry-run supported; --force for non-standard p2 geometry.
bin/ is byte-identical to the deployed king_kong_le-0_97_0.ssh image.
validate-image.sh functionally proves a result (boots the image's own
init script under qemu, real ssh pubkey login, password auth refused,
sftp round-trip); build-binaries.sh rebuilds bin/ in Docker.
Co-Authored-By: Claude Code <noreply@anthropic.com>
All GAMELOG lines (EV/SHOT/SCORES/AWARD in mode.log, the runtime's MODE
START/END and AWARD lines) and every switch.log line now carry the wall
clock as RFC 3339 UTC with millisecond precision; the +Nms stamps relative
to runtime load are gone (dur= on MODE END stays: it is a duration). The
timestamp comes from a new gl_rfc3339() in the runtime: clock_gettime
(CLOCK_REALTIME) plus the civil-from-days algorithm, no libc time calls.
switch.log now logs only the human-activated buttons (34 lockdown, 36
start, 62/63 flippers) with quoted names. The upper-flipper button
switches (64/65) fire in parallel with the main flipper buttons on this
cabinet and are left out with every non-button switch.
Deployed and verified on the machine: the boot banner reads
2026-10-13T01:39:39.640Z against the machine's own UTC clock.
Co-Authored-By: Claude Code <noreply@anthropic.com>
The v3 runtime's edge logger could never arm: the port had no switch_edge /
switch_drain sites, no switch_records/switch_count data and no switch_*
record-layout values, and switches_arm() skips a port without switch lines.
Located in the King Kong LE 0.97 ELF with the decryptor's
portswitch.switch_source() (generation-B framework): switch_edge 0x0055fa04,
switch_drain 0x005606e4, switch_records 0x008ecc78, switch_count 0x0093437c;
record layout 384/32/24/8/26/28 as on the other generation-B builds. Added
LOCKDOWN/START/flipper button switch lines on bits the game's shot dispatch
never sends, so the logger arms and the buttons become named shots.
Deployed to the machine and verified in play: the boot log arms the drain
(switch_edge 0x0055fa04, drain 0x005606e4..0x00560864) and /dump/switch.log
logged ~6000 in-game edges, both edges of all four flipper buttons, named.
Also fixes the runtime's mangled GAMELOG comment (broke the build) and the
switch_names.inc entry count in the README.
Co-Authored-By: Claude Code <noreply@anthropic.com>
Snapshot of the stern-event-agent work (as of 2026-10-12):
- gamelog.c: passive PAD mode logging bus events (game/ball/bonus/tilt),
all named shots and all-players' scores to /dump/mode.log
- pad_mode_runtime.c: the SDK runtime + a GAMELOG section (passive
stock-mode and award watching, gl_award_seq()/gl_last_award_id()),
a per-edge switch logger to /dump/switch.log (switch_names.inc), and
a passive Insider Connected gate; docs/runtime-patch.diff shows the
exact changes vs the upstream runtime
- event_probe.c / event_read.py: event-bus measurement tooling for
identifying bus event ids on another game
- ports/king_kong_le-0.97.port: 121 verified sites, 55 named shots, 6
bus events (emulator-proven by the SDK repo authors)
Deployed and verified on the machine: events, shots, stock modes and
awards. The /dump/switch.log feature is implemented but NOT yet
deployed or verified, and the extra-ball award id (default 13) is
unconfirmed. See README.md for build/install/log usage and unfinished
work.
Co-Authored-By: Claude Code <noreply@anthropic.com>
mattlqxandClaude Code