Files
dev-updater/AGENTS.md
T
irisandClaude Opus 5 c7b3af4861 A Rebuild on every component, and the keystore AGP actually uses
Two things, both from the same stuck download.

Each component row now carries its own Rebuild, which is /build scoped to
it -- the route already took ?component=, only the button was missing.
Update builds only what the staleness rules call behind, and those rules
cannot see a command reading undeclared files, an output changed
underneath this server, or a signing key replaced since the APK was made;
in all of those Update does nothing and the only force was a project-wide
Rebuild that rebuilds every sibling. Same action at two scales, so it
takes the project Rebuild's word and colour. hasBuild is new on the
manifest component so a component with no command of its own draws none,
which the project-level needsBuild could not answer.

And debug_keystore_path() now resolves the preferences directory the way
AGP does instead of assuming ~/.android: ANDROID_USER_HOME as-is, then
ANDROID_PREFS_ROOT and ANDROID_SDK_HOME with .android appended, then
$XDG_CONFIG_HOME/.android when that directory exists, then $HOME/.android.
Measured by running real builds against AGP 32.3.2, including the
existence test on the XDG step, which the documentation does not mention.
An ordinary desktop machine with XDG_CONFIG_HOME set signs its APKs with
a keystore this server never looked at, so it reported "no debug
keystore" about a machine that had one and was using it -- and this VM
sets no XDG_CONFIG_HOME, which is why it could not happen here. The
failure now names a keystore found further down the list, since an inert
one and an absent one are otherwise identical from a phone.

Verified on the emulator (component Rebuild: press, that component's own
progress, back to resting with no sibling moving) and against a
throwaway server for both keystore branches.

Reported by the tdep-survey session, which found the two keystores on the
serving host.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-01 11:31:59 -04:00

60 KiB

dev-updater

Serves locally-built debug APKs to a phone, and an Android app that installs them. See README.md first for what this is and how to run it; this file is the working notes on top of that.

The central design point, worth not undoing by accident: an app is a project path, not a file path. Everything downstream — which APK, which package it replaces, its label, whether it's worth stripping — is derived from what's actually built under that path, at add time or per request. Nothing about any particular project is compiled in, and the app list is mutable at runtime from the phone.

Layout

  • vendor/wg-app-link/ — a submodule, and the shared half of this and ai-app: the WireGuard binding, the CA the app pins, QR enrollment, owner-only file creation, and the RON house rules. A submodule rather than a published crate because it pins an exact commit, so the two servers cannot end up on versions of it that disagree. Clone with --recurse-submodules; git::pull runs submodule update after the merge, because a merge moves the recorded pointer without touching the submodule's working tree and the build would otherwise keep compiling the old contents. What stays here is what differs: this server's routes, registry, build walk and service contract, plus auth.rs, whose middleware is generic over each project's own state -- only the token functions underneath it are shared.
  • server/ — Rust + Axum. main.rs is the bootstrap and the two listeners; routes.rs has the whole HTTP table in its module doc comment. auth.rs is the bearer token every TLS request carries, applied once around the whole router so a new route cannot forget it. registry.rs owns the live app list and every mutation of it (config writes funnel through AppState::update, so in-memory and on-disk state can't come apart; the one other write is AppState::new's startup reconciliation, before anything can read the list). discover.rs is the scanner, config.rs the persisted schema and the RON both config files are in, apkinfo.rs the aapt2 reads, strip.rs the slim-APK pipeline, sdk.rs the SDK/NDK tool lookups.
  • app/ — Kotlin + Compose, a single :androidApp module. UpdaterScreen.kt is the list, AddAppScreen.kt the add/settings screen, AppsApi.kt the management calls, UpdateManifest.kt the read side, ApkInstaller.kt / InstalledBuilds.kt the download-and-install path, DownloadServer.kt the transport and Link.kt the two values it hands the shared library. Theme.kt is Catppuccin Mocha mapped onto Material's roles, plus the four ActionTones buttons come in -- red takes something away, the scheme's mauve replaces it with the same thing, green brings it up, blue puts a new build on the phone. What a button does is said in colour rather than by which component it sits on, so the same consequence looks the same everywhere. The mapping that matters is the surface ladder: Mocha names its darks in order (Crust, Mantle, Base, Surface 0) and Material asks for the same thing under other names, so the page is Base, a component's outlined card stays Base beside it, and a project's card is Surface 0 -- one visible step, which is all the nesting has to say. NerdIcons.kt names the icon glyphs, which are drawn as text in a small Nerd Fonts subset rather than as vector assets -- an icon beside a line of text wants that line's size, colour and baseline, and a Text gets all three for free. The font is generated: add a codepoint in NerdIcons.kt and in app/build-icon-font.sh, then re-run the script, or the glyph silently isn't there.

Checking your work

  • Server, from server/: cargo fmt, cargo clippy --all-targets, and ../run-tests.sh. All three every time, not just when a change looks big enough to warrant them. The build is warning-clean and cargo fmt --check passes; keep both true. Formatting is plain rustfmt defaults with no rustfmt.toml — layout is not something to decide per line, so take what it gives rather than hand-formatting against it.

  • App: from app/, . ./android-env.sh && ./gradlew :androidApp:compileDebugKotlin (or :androidApp:assembleDebug), plus ./gradlew ktfmtFormat ktfmtFormatScripts :androidApp:ktfmtFormat :androidApp:ktfmtFormatScripts and ./gradlew :androidApp:lintDebug. Both are clean; keep them that way. ktfmt is kotlinLangStyle() with nothing else configured, so formatting is never a thing to decide per line.

  • Neither replaces running it. app/run-android.sh builds, installs and launches on an emulator; screenshot with adb shell screencap -p /sdcard/x.png && adb pull /sdcard/x.png <local>. Package com.example.devupdater, activity .MainActivity.

  • test-projects/ is what to point it at, rather than a real project. Four fake ones -- a plain APK, one building two variants, one whose build fails on demand, and a Server + Apk pair with a service and a resources: declaration. They exist because a real project only does what it happens to do, where these can be asked to fail, to be slow, or to declare nothing. That directory's own README says how they are reached (they are deliberately not scanned) and why each one builds into an app/ subdirectory; the short version of the second is that two levels is what find_apks matches, so a test APK any shallower is offered as a build of this project.

  • The server is easy to exercise directly, which is usually faster than going through the UI — but it takes three things, and leaving any of them out looks like the server being broken:

    # The CA it actually presents is the one in its config dir, not the
    # stale certs/ left in the repo. The address is wg0's -- it binds that
    # and nothing else, so 127.0.0.1 refuses the connection unless the
    # server was started with --bind. And every route on this port needs
    # the enrolled bearer token.
    curl --cacert "${XDG_CONFIG_HOME:-$HOME/.config}/dev-updater/certs/ca.pem" \
        -H "Authorization: Bearer $TOKEN" \
        "https://$(ip -4 -o addr show wg0 | awk '{print $4}' | cut -d/ -f1):8090/manifest"
    

    Only the token's SHA-256 is stored, so there is no reading it back out of config.ron: either use one printed by --rotate-token, or run a throwaway server with --config/--certs pointed somewhere temporary and --bind 127.0.0.1, which prints a fresh token at startup.

Things that have bitten

  • Don't add a general recursive search to discover.rs. It was measured and rejected; the numbers are in that module's doc comment and in the README. Extend APK_PATTERNS instead.

  • aapt2 and the strip pipeline never run on the manifest path. A process spawn or a zip walk there turns a 43 ms refresh into something a phone notices, and the manifest is fetched on every open, resume and Refresh. aapt2 is add-time work cached in config.ron; stripping belongs to the download alone, which is why the manifest reports the size of whatever is on disk (strip::serveable_now) rather than producing the slim copy to measure it.

  • Both config files are RON with two house rules, and wg_app_link::format is the only place that knows them -- ai-app's files are in the same shape, which is why they are shared rather than described twice. config.rs's own format module is now only the migration hook in front of it, and goes when that does. A file is the body of the struct -- no outer parentheses, so nothing is indented for a wrapper -- because RON has no implicit top-level struct; parse adds the paren and render strips it. And IMPLICIT_SOME is set on the deserializer rather than by a header each file would have to carry, which is why every optional field also needs skip_serializing_if so nothing is written back that nobody typed. The two halves only round-trip together: change one and every file on disk still loads while only looking wrong, which is why the config test asserts the written shape.

  • A Server component is driven through one script, run as <script> <subcommand>. service.rs knows the subcommands and the four words status may print (running, stopped, failed, not-installed); it knows nothing about systemd or OpenRC, and shouldn't. Scripts run with stdin closed, because one that prompts would hang rather than fail. On OpenRC, status reads the exit code and never the text (0 started, 3 stopped, 32 crashed, anything else "could not find out"). rc-service status prints its status line to stderr, so the obvious check -- discard stderr, grep stdout for crashed -- throws away the word it is looking for and reports a crashed service as stopped, which is the exact lie the failed state exists to prevent. The codes also distinguish "could not find out", which the text cannot: an uninitialised user softlevel makes every call fail, and that is not a state the service is in. Assign the code through || code=$?, or set -e kills the script before it can read one -- every answer but "running" is a non-zero exit. Measured on OpenRC 0.63.3, not read. There is a guest to measure in, at ~/vm/gentoo on this machine -- ./start.sh, then ./shell.sh 'cmd', ssh on 127.0.0.1:2223 as tester with guest_key. Its own README is the reference. Everything in this section, and the restart behaviour below, was run there rather than reasoned about; do the same rather than trusting either the documentation or a systemd result. Detecting OpenRC is itself a trap: rc-service --user --help fails when XDG_RUNTIME_DIR is unset, so probing with it first reports "no service manager here" on a machine that has one.

  • Which script is what service: says, and it is a sum: Script("path") for a project that carries its own, Managed("the binary") for one that wants dev-updater's built-in. Two variants rather than two optional fields, because a component picks one and "both" and "neither" would be states the file could express that mean nothing. The built-in is server/src/service-default.sh, compiled in, written to $XDG_DATA_HOME/dev-updater/service-default at every startup so an updated dev-updater cannot be driving last version's copy, and handed its identity as arguments (--name <key>-<component> --exec "...") so the subcommand the caller appends still lands last. service::driver is the single place the two variants become one command -- everything downstream takes a command and cannot tell which it was, which is what lets a project switch between them without anything else changing. It is deliberately a script rather than an in-process implementation: a built-in that ran through Rust would put systemd and OpenRC knowledge back in the one place it has been kept out of, and the contract would become a special case of itself. dev-updater declares Managed like anything else, and has no service script of its own: nothing about restarting this server lives in a script, it lives in restart.rs, so there was nothing left for one to say. Its component carries cwd: ".." because the unit's working directory has to be the checkout root -- that is where this server looks for its own app project, and started anywhere else its card silently loses the branch line, its commit count and the Pull button. start.sh knows none of this: it runs dev-updater --service <subcommand> on the binary it has just built, which derives the name and the script from the same declaration the running server reads. One place decides how this service is named and invoked, which is the only reason a bootstrap script cannot drift away from service::driver. A managed service is named <key>-<component>, because a service manager's names are one flat namespace across every project on the machine. Switching a project from its own script to Managed therefore changes the unit's name: uninstall the old one first, or its unit is orphaned with nothing left pointing at it.

  • Service state is checked off the request path, in ServiceChecks beside git::RemoteChecks and for the same reason: a process spawn per server component on /manifest is exactly what that path must not do. Which means it is subject to the same rule as a remote check: it lands after the response that started it, so checksPending counts both, and the phone waits on both (ManifestEntry.checksOutstanding for one card). And the answering starts at startup rather than on the first request (main.rs), so the window in which a component has no state is this server's own start rather than something a phone has to sit through -- restarting is how this server is updated, which makes it the moment somebody is most likely to be looking at the list. Counting only remotes was a real bug and an invisible one: the state is unknown only until the first answer of a server's life, remote checks ordinarily outlast service ones, and the symptom is a component row with no buttons -- so it showed up as "the buttons vanish after a restart if I come back to the app too quickly", and only on whichever cards lost the race. Nothing looked again until somebody hit Refresh.

  • dev-updater declares itself as an ordinary Server + Apk pair, but its server component is this process, and that changes one thing only: when it restarts. Running the script's restart inside the walk would kill it before the APK is built and before anyone is told how it went, so the restart is deferred past finish; the Restart route defers for the same reason, since a script that killed this process before it could answer would make a restart that worked read as a failed request. Once deferred, both go through the one function (restart::deferred), which asks the manager when the service is up and only otherwise execs the new binary itself. Going through the manager is what makes a refreshed unit apply -- an exec inherits the unit this process was started with, so a pull that changed the service script would otherwise land only at the next restart by hand. The button used to exec unconditionally, on the grounds that exec keeps the PID; that made it the one restart a rewritten unit never applied to, which is the kind of surprise a special case buys. Deciding it in one place is what stops the two drifting again. BuildState::is_self is the flag, and it is passed in rather than declared -- a file that could claim it is a file that can make this server exec an arbitrary binary. A build that changed nothing skips the restart altogether (restart::binary_unchanged): exec-ing into an identical binary drops the phone's connection to deliver the build it already had. It is a stat of the binary against what it was at startup rather than a hash -- cargo installs a new binary by renaming over the old one, so the inode alone answers it, and reading tens of megabytes to learn the same thing would cost more than the restart it avoids. It says "changed" whenever it cannot tell, because a needless restart costs a second while a skipped one leaves the old build serving and reads as the update having silently failed. The Restart button is unconditional -- somebody asked. Stop and Uninstall do go through the script, and do strand the phone; the app confirms them rather than hiding them.

  • A component's freshness covers its own directory plus whatever it declared with alsoWatch (Component::watched_paths, handed to git::subtree_head and subtree_dirty as pathspecs -- git takes several, so it stays one call). Scoping to cwd is right about where a component's files are and wrong about what its build reads: tdep-survey's two clients each source one shared scripts/android-sdk-env.sh, so a fix committed there changed what every build does, moved no component's subtree head, rebuilt nothing, and left every card correctly reporting "current" -- the narrower-question shape, where nothing is wrong and nothing says so. Declared rather than inferred because which files a build reads is not knowable from here, and both wrong guesses are expensive: too wide rebuilds everything on every commit, too narrow is that silence. It is part of the acceptance gate like every other declared field -- same_declaration destructures exhaustively, which is what forced the decision when the field was added.

  • The debug keystore is not ~/.android/debug.keystore -- that is the last of five places, and sdk::android_prefs_dirs resolves it the way AGP does, because the only thing that makes it the right key is being the file Gradle signed with. Measured against AGP 32.3.2 by running real builds: $ANDROID_USER_HOME is the preferences directory itself with no .android under it, $ANDROID_PREFS_ROOT and $ANDROID_SDK_HOME get .android appended, then $XDG_CONFIG_HOME/.android but only when that directory already exists (the bytecode tests it and falls through), then $HOME/.android. That fourth step is the whole bug: an ordinary desktop Linux machine with XDG_CONFIG_HOME set signs its APKs with a keystore this server never looked at, so it reported "no debug keystore" about a machine that had one and was using it -- and this VM sets no XDG_CONFIG_HOME, which is why the case could not appear here. The failure names a keystore found further down the list, because an inert one and an absent one look identical from a phone: mistaking one for the other is what turned "this machine has none" into a signature mismatch nobody could explain.

  • A stripped copy is only serveable if it is signed by the key the build was. strip.rs re-signs with ~/.android/debug.keystore, and nothing about a debug keystore says which builds it made -- it is per machine, and one recreated after an APK was built signs as an entirely unrelated certificate (measured: a fresh AGP-parameter keystore against the existing one, no relation, as a new RSA keypair should be). Where they differ Android refuses the package and the phone says "App not installed" with no cause, which reads as the download rather than the signing and is among the most expensive sentences here to be handed. So the two are compared -- apksigner verify --print-certs on the source and on what was just signed, as sets of digests so a v2 source and a v3 output still match -- and a mismatch is refused with both certificates and the keystore path named. Measured on the files rather than inferred from the keystore, so it stays true if the signing step changes. The slim copy is deleted on refusal: with no stamp beside it nothing would serve it, but serveable_now reports the size of whatever slim file is on disk, so leaving it would have the card describing a download nothing can install. Empty digests mean "not compared" rather than "no signer", and say so in the log rather than refusing -- an APK apksigner cannot read is one this comparison has no opinion about.

  • A 500 answers with its message. ApiError::Internal used to be a bare status with an empty body, on the grounds that an internal cause is not safe to hand back -- but every route here is behind the bearer token of a device somebody enrolled themselves, so there is no third party to withhold it from, and what these actually say is which tool on the build machine could not be found or would not run. Withheld, a missing NDK reached the phone as "Server returned HTTP 500", with the explanation in a log on a machine the person holding it cannot see; the app has always shown a failure body when there is one, so the whole fix was on this side. The log keeps the same {err:#} chain.

  • A remote git failure is shortened and explained in one place (git::remote_failure). Git's stderr runs to a paragraph of generic advice, so the card gets the first line and the log gets all of it; and when the failure was authentication, what this process can see of the ssh agent is appended, because "works in my terminal, fails from the service" is almost always SSH_AUTH_SOCK not being inherited by a daemon, and that is invisible from a phone. Asked rather than assumed -- the note can say the agent is fine, which is what stops it being blamed.

  • A branch with no upstream is not a broken one. can_pull requires an upstream as well as a checkout, which is what stops the card reporting the same fact twice -- once as its own note and once as the check's complaint about it -- and stops it offering a Pull that git::pull would refuse. It is also not worth saying: plenty of checkouts have no remote deliberately, and a card that mentions it every time is nagging about a choice somebody made.

  • Unrelated histories are the one pull failure the phone may override. A checkout sharing no commit with its upstream has no fast-forward and never will, so with nothing offered the card is one that can never be pulled again -- and the way out is on the build machine, which is exactly where the person holding the phone isn't. So ?force=true on the pull route resets onto the upstream instead of merging, behind a dialog that names the branch it is about to overwrite (the only place that is seen before it goes). Whether this is that failure is decided structurally -- git merge-base finding no common ancestor -- and never by matching what git printed: those messages are translated, and a button that appeared only on an English build machine is worse than no button. It travels as PullError::unrelated_histories rather than inside the message for the same reason. The dirty-tree refusal stays in front of it, so a forced pull can only ever discard something that was committed, and a merely diverged history is not offered it -- there is something better than throwing that away.

  • The branch line is how a failed remote check is visible at all. With it gone, newCommits stays false and the card reads as an app with no updates -- the same silent failure the self-entry note below describes. It was deleted once by accident in a refactor and nothing failed; only the display went quiet.

  • Forcing git onto IPv4 is done in GIT_SSH_COMMAND, not on the git subcommand. git fetch takes -4; git ls-remote does not, and answers "unknown switch 4'". ls-remoteis what the new-commits check runs, so splicing the flag onto every remote command turned the setting into a switch that broke every card's commit count -- which is how it was first written, and whatgit::ssh_commandandforcing_ipv4_does_not_break_the_checknow hold in place.fetch` keeps the flag as well, since that is what carries the preference to an https remote; ssh is covered either way.

  • This server's own project is a row in config.projects like every other, and everything in that row except gitIpv4 is re-derived at startup by registry::reconcile_self. The row is what gives a per-project setting one home -- it used to have to be keyed in a separate config.settings list, because the one card that is always in the list was the one card the setting could not be a field on. What is derived stays derived: the path is the working directory (see below), and the label and components come from the checkout's own .dev-updater.ron, so a re-clone or a moved repo fixes itself at the next start instead of leaving the row naming a directory nobody pulls. Only what somebody chose is carried across. The declaration is accepted by construction there, which is why AppEntry::pending_declaration answers nothing for it: between a pull that rewrites the declaration and the restart that follows, the row and the file can differ without that meaning anything is waiting to be accepted. A config still carrying the old keyed settings list loses it silently -- Config ignores fields it does not know -- so a machine that skips straight past the version that migrated one has its gitIpv4 to set again.

  • A component has two kinds of log, and the kind is an axis of its own. The build log is what this server captured while building it; the runtime log is what the component wrote while running, reported by its service script. They are asked for one kind at a time (?kind=), and the modal gives each a tab -- opening on runtime, since what a service is doing now is the usual question, and on build for the component a build stopped at (build_failed). They were one list indexed by generation once, and because build logs came first the runtime one sat at an index nothing ever asked for: unreachable, and silent about it. ANSI escapes are rendered rather than stripped (AnsiLog.kt) -- the colour is how a process marked its own errors -- and sequences with no meaning on a phone are consumed rather than printed, so a cursor movement cannot arrive looking like a corrupted log.

  • There are three sizes of refresh, and using the wrong one is what makes the list feel like it has a mind of its own. refresh() drops the list to a spinner and asks every remote -- it belongs to arriving, resuming, and the Refresh button, which are the moments somebody asked to be shown the current state. Pull-to-refresh is the same ask without the spinner, since the gesture brings its own indicator and taking the list away underneath it would say the same thing twice. applyOne reads GET /apps/{key} and puts that one entry back, so an action on one card cannot move or change any other; every card action uses it, including adding one -- the Add screen hands back the key and the list fetches just that app. awaitCheck is the middle one: ask a single checkout's remote and wait on that answer, which the card's own refresh control and a freshly added app both use. Waiting on the whole list to settle would make one card's refresh sit behind another card's. Arriving and resuming are one event, not two: LifecycleResumeEffect runs when the screen first reaches RESUMED, so there is no LaunchedEffect(Unit) beside it and one thing decides when the list is read. It refreshes from any state including a failed one -- guarded on "loaded", as it used to be, a card that had gone red stayed red until somebody found the Refresh button, which is the opposite of what returning to an app should do. loadingList is what stops the first composition and the first resume stacking two reads, and it is claimed before the coroutine launches, because both run in the same frame and a flag set inside the coroutine is set too late to be a guard.

  • A failure that lands while nobody is looking is not shown. Work started before the app went away keeps running -- deliberately, since a download that finishes in the background is a download that worked -- and when the device sleeps or the link drops it fails. Reported, that meant coming back an hour later to a five-second read timeout that said nothing about the server and that there was nothing left to do about. So every catch in UpdaterScreen.kt goes through failure(e), which answers null while the screen is not resumed, and null means clear, never leave at every site -- a dropped failure that left the card alone would leave a spinner up for an operation that has already stopped. setProject and setComponent both take a nullable state so the answer can be handed straight over. Iris asked for this on 2026-09-01: "I got a socket timeout by leaving the app for too long. Make sure not to show that if the app just gets unloaded." The other half of the same complaint is the read on the way back in: the link may have been asleep as long as the app was, and the first request across one still coming back times out at the five seconds every request gets. So the resume's read alone retries once (afterAGap), which costs nothing when the server really is down -- a refused connection comes back at once rather than waiting out a timeout. That is a retry, not a guess: nothing is displayed that was not measured. Dropping a failure is only safe because the resume re-reads from any state; the two changes hold each other up, and undoing either alone leaves the list stuck on a spinner or stuck on a stale error.

  • The Add screen is drawn over the list, not in place of it. Swapped out, the list is composed again from nothing on the way back -- and rebuilding it means fetching it, which is the whole-list refresh that adding an app has no business causing. describe in routes.rs is the one place a card is built, so /manifest and GET /apps/{key} cannot come to describe the same app differently.

  • A progress bar is drawn from counts the build reports, never from an estimate, and the knowledge of how to get those counts belongs to this server rather than to each project. Two ways in, both ending as @@progress done/total on the build's output: cargo reports its own once CARGO_TERM_PROGRESS_WHEN=always is set, which spawn sets for every build command, so a cargo build needs nothing and must not be wrapped. Gradle cannot -- an init script is refused by the configuration cache, and its rich console reports a percentage only as a full-screen redraw that would make the output unreadable -- so server/src/build-progress.sh does the --dry-run count and the > Task tally, shipped by this server (crate::shipped) and offered to build scripts as $DEV_UPDATER_PROGRESS. A script tests for the variable rather than depending on it, so building by hand still works. This is what stops each project carrying its own copy of the same twenty-five lines. Reading the output means splitting on carriage returns as well as newlines: a tool redrawing a counter in place puts several updates and then real output inside one \n-delimited line.

  • Discovery must stay side-effect free. It runs on every manifest request and every suggestion scan.

  • An app in the list is a project, and what it produces is its components. Ordinarily one Apk; a project that also runs a server on the build machine declares a Server beside it. They are a sum, not one struct with both halves' fields, and the common fields are repeated per variant so the file reads Apk(name: ...) with nothing nested inside a kind. Component::same_declaration destructures exhaustively, so adding a field anywhere fails to compile until someone has said whether a project declares it or this server measures it.

  • A build reinstalls every server component's unit, not just changed ones. The unit is generated from a script in the repository, so a pull can change how a service is defined rather than what it runs -- and that has to land even when the build produced a byte-identical binary, which is the ordinary case for a pull that only touched the script. Installing is safe on a running service (it writes the unit and reloads; it stops nothing), which is why start.sh has always done it every run. It is deliberately not gated on restart::binary_unchanged: that answers whether a restart is worth the interruption, and unit staleness is a different question. Wiring the two together left the unit stale exactly when the build had nothing to do.

  • A build runs every component at once, and each reports for itself. They are independent -- a Rust build and a Gradle build share nothing but the machine -- and measured here, together takes about three quarters of the time one after the other does. The saving only appears when more than one has work, which is what a pull produces. The status is per component (ComponentStatus), so a card draws each component's bar and last line inside that component's own row; the project's own area at the bottom keeps only what belongs to the whole project, which is fetching and pulling. A bar under the card could only ever say that something was happening, and with everything building at once that is exactly what the reader is trying to find out. A failure no longer stops the others -- they are already running -- and it is reported against the component whose command it was, never as the project's. The phone draws every component in a card of its own, including a project with only one: the flat layout it used to get meant two shapes to keep in step, and put that APK's size up beside the card's corner controls where it read as belonging to them.

  • There is a build slot per component, not per project, and the two halves have to agree on that. A component being built neither blocks another's build nor disables its controls: Inner has no building flag, only a ComponentRun per component whose open step is the answer, and claim writes that entry synchronously under the same lock the route answers from -- so nothing can read a component the request just claimed as idle, which the phone would take for "the build is over". BuildStatus::building stays, but it means "anything at all is happening here" and is only for the controls that act on the whole checkout; anything about one component reads that component's step. The app mirrors the split exactly: ProjectState for the pull and the project-wide Rebuild, ComponentState keyed by component name for everything a single component is asked to do. One map keyed by project alone is what the bug was -- pressing Update on one client of a two-client project disabled the other's button and drew this one's download bar under it -- and two hierarchies rather than one keyed by a pair is what stops it coming back, since a download has no project-wide meaning to be stored with. The exception, and it is worth keeping visible so it does not read as more of the same: a pull really is exclusive with everything. There is one checkout, and it rewrites the files every component builds from, so Inner::pulling blocks any component from being claimed and waits for any still building. Ending the pull and claiming what it decided to build happen under one lock for the same reason claim is synchronous -- a phone polling in the gap would see a project that is neither pulling nor building and call the run finished.

  • Text a command produced is selectable; text this app wrote is not. Theme.kt's OutputText is the whole of it, and every failure message goes through it -- a component's build or download, a service action, a checkout's remote check, and the log dialog's own failure line -- plus a SelectionContainer around the log body, which cannot use OutputText because it is an AnnotatedString the ANSI renderer coloured inside its own scrolling panel. The reason is that this is the one text on screen a person has to take somewhere else, and the machine that produced it is not the machine in their hand. A status word or a button label stays unselectable on purpose: selection handles on those are noise, and a card that starts a selection on long-press fights the gestures it already has. Iris asked for exactly that line on 2026-09-01: "not the 'failed' but the command output for build errors and stuff". OutputText also renders the ANSI escapes rather than printing them, through the same ansiAnnotated the log dialog uses -- a compiler marks its own errors in colour and the tail of a failed build is that output verbatim, so raw it arrived as [1;31merror with punctuation welded onto the one line somebody was trying to read. Selection copies AnnotatedString.text, which is the message with every escape already gone, so what lands on the clipboard is what was on screen rather than what was on the wire.

  • A finished component shows nothing, and its button goes back to normal. Iris's call, 2026-09-01: "you shouldn't see the time it took once it finishes, it should just go back to its normal enabled button state." So ComponentBuildProgress draws only while the step is open, and the elapsed times are gone from both halves of the wire -- a bar, a count and a last line all describe something happening now, and every one of them sits there looking live beside a sibling that genuinely is. What says the build landed is the control becoming pressable again and the card's own freshness. The failure is the exception, because it is an outcome rather than residue; it is drawn by the component card next to the Retry that acts on it, in the one place that reports that component's failures whether they came from a build, a download or a service action.

  • A card being worked on has no freshness, rather than the one from before the press. Nothing re-reads the manifest during a run -- the entry was fetched before the button was pressed and read again only once the run is over -- so "out of date" drawn beside the bar that is making it current is last minute's answer wearing this minute's clothes. UpdaterScreen therefore maps a busy component's freshness to unknown in the one place the component list is built for the cards, which is what makes both readers of it -- the row's own note and MismatchedPairNote -- go quiet without either having to know why. The condition is deliberately the same pair that disables the Update button (projectState.busy || componentState.busy): what cannot be acted on is exactly what cannot be measured just now. It comes back the instant the run ends, still saying "out of date" if the build failed, because by then the entry has been read again -- withheld is not the same as cleared.

  • A project's own .dev-updater.ron is a request, never an instruction. It only runs once accepted from the phone, which copies it into config.ron; AppEntry::pending_declaration is the whole gate. Evaluate it fresh rather than caching it on the entry — the file changes on a pull, and a pull writes no config, so nothing rebuilds the entry list. Caching it meant a pull could swap the command out from under a previous acceptance and nothing noticed until an unrelated config write. Enforce it wherever a command would actually run, not only where the list is built. Two things follow that both went wrong the moment resources: joined the gate. Whatever Declaration::matches_accepted compares, AppState::approve_declaration has to write -- it compared the resources declaration and wrote only the components, so accepting stored half of what was being read back and the gate could never clear. The press succeeded, so there was no error either; the card simply kept asking. Adding a field to one without the other is the shape to watch for, and the tests are what let it through: they built an "accepted" config by hand instead of calling approve_declaration, so they asserted a copy of the rule rather than the rule. And on the pull path the gate is asked after the fast-forward, never before it (build_state::pull_and_build takes a closure for exactly this reason). The declaration lives in the checkout, so the pull is the one thing that changes the answer -- a bool computed at the call site is the answer for the commit being replaced, which made a pull that changed the declaration build anyway. Pulling stays allowed while a request is unaccepted, since taking commits runs git rather than the project's command, and it is how the new request arrives to be read.

  • Which build variant to serve is the phone's choice, not the server's. It arrives as ?variant= on the download, beside the ?component= that says whose build it is, and is validated against that component's discovered builds -- an unvalidated path would let a phone name any file on disk to be served. Storing it server-side meant one enrolled device silently changing what another was offered. The stored choice is keyed by project and component on the device, so pinning one client to a release build says nothing about the other.

  • A project can produce more than one APK, and each component's builds are found under its own cwd. APK_PATTERNS is anchored at project.join(cwd) rather than at the project root, which is what cwd already meant everywhere else -- the directory the build command runs in, the subtree subtree_head scopes staleness to, a server's WorkingDirectory. A component that declares none sits at the root, which is what every single-APK project has always meant, so nothing about that case changed. tdep-survey is the project that needed it: one checkout, a backend and two independent Android clients, where the root-anchored patterns reached the first and stopped. The alternative -- naming the file on the component, apk: "path" -- was rejected because it makes a component a file path, and an app being a project path rather than a file path is the invariant at the top of this document. Everything downstream is per component in consequence: package, strip, the variant list, the size, the mtime and the rename note. Those were all apk_component()'s first match before, which was correct only while a project had one. Two clients install over different packages and differ in whether their symbols are worth carrying to a phone, so first-wins would have checked the installed state of one app and reported it as the other's -- the expensive kind of wrong, because it looks exactly like an answer. A download that names no component is refused, not guessed (ApiError::AmbiguousApk), for that reason. Naming none still answers for a project with one, which is nearly all of them and is what lets the frozen /self/apk keep working -- it cannot carry a component, and the project it describes has a single APK. The measurement carried across an acceptance is keyed by name and cwd (registry::component_id): once the directory decides which builds a component has, a reused name is not the same APK, and handing it the old one's package would be wrong until that component happened to be downloaded.

  • /prepare and /build take ?component= to build one component instead of every one with a command. Without it, pressing Update on one client of a multi-client project ran every component's build to get the one that was actually asked for -- fine when a project had one APK, expensive the moment it had two and one of them was slow. named_component in routes.rs is the one place a name from the phone is checked against the project's own components, so /prepare and /build cannot disagree about what an unknown name means, and BuildState::{trigger_if_needed,build_now,run_build,is_stale} all take the same Option<&str> -- None still means the whole project, which is what Pull & Build, the project-row Rebuild, and every project with a single component keep doing. Each component's row carries its own Rebuild, which is /build scoped to it. That used to be deliberately absent, on the grounds that Update already builds a component that needs it -- but Update runs /prepare, which builds only when the staleness rules say the output is behind, and those rules are blind to everything a commit does not describe: a command reading files nobody declared, an output changed underneath this server, a signing key replaced since the APK was made. In all of them nothing reads as stale, so Update does nothing, and the only force was a project-wide Rebuild that rebuilds every sibling too. Iris asked for the per-component button on 2026-09-01. The two are the same action at two scales, so they share a word and a colour, and each sits at the right-hand end of its own row. A component with no command of its own draws no Rebuild -- hasBuild on the manifest component says so per component, which the project's own needsBuild (the OR of every component's) cannot. Absent rather than disabled, matching the project row, because having no build step is not a state a component is in. A server's Rebuild sits in its service row but outside the condition that draws the service buttons: those wait for the script's answer, while a server whose script cannot be reached is still one this machine can compile. Where component.dir() belongs is now one definition (Component::dir in config.rs), because a second one very nearly shipped a real bug: component_is_stale's "never built at all" check still asked find_apks of the project root after per-component discovery had already moved everywhere else to component.dir(). A project with two Apk components has one's output sitting under the root-anchored patterns too -- */build/outputs/apk/*/*.apk matches any one-level subdirectory, regardless of which component put it there -- so the moment either component had ever been built, the whole project read as "something is built here," and the other component, never built, silently stopped being offered its own first build: prepare saw a component with a command and no output and declared it current. Caught by testing the actual behaviour of a two-APK project rather than trusting that scoping the build implied scoping the staleness check that decides whether to run it -- they are two different reads of "which directory is this component's," and only one of them had been moved. The same check is deliberately not asked of a Server: a service never has an APK to find under its own directory by definition, so asking would report every server "never built" forever. Guarded on matches!(component, Component::Apk { .. }) for that reason.

  • The self entry's project is the working directory itself, so this server has to be started from the root of its own checkout -- which is where everything else here is driven from, and what the service unit sets (WorkingDirectory=$REPO_ROOT, not server/). Its components then say where they live from there, exactly as any other project's do; there is nothing special about this one's layout, which is the point of it not being app/ any more. It used to be CARGO_MANIFEST_DIR, fixed when the binary was compiled, so a re-clone or a move left the card watching a directory nobody pulls. The symptom is why this is worth remembering: the entry keeps working and keeps serving an APK, but silently loses its branch line, its Pull button and its commit count, so it reads as an app that simply never has an update. It took a "why can't my server see the new commits?" to find. Startup warns when there is no checkout at the resolved path, which is the only cheap moment to notice.

  • A configured project pointing at this server's own project is a second card for it. add_app refuses it (the self entry is in the list it checks against, and self_project is canonicalized so a symlink can't make one directory look like two), so this only survives from a config written while the self entry resolved somewhere else. It is not merely redundant: it is not the self entry, so is_self is false and a build through it would restart this server partway through its own build. build_entries warns rather than dropping it -- dropping it would leave a config entry with no card and so no way to remove it from the phone. The built-in card has no Remove button, so the duplicate is the one that does.

  • A project says where its own state is; nothing infers it. The resources: declaration points at a RON file, a script that prints the same RON, or an inline struct, and it holds what a project would otherwise write down in several places -- its name, and its data and config directories when those are not $XDG_*_HOME/<name>. The file is the project's, not this server's: its own code is meant to read the same one, which is why ResourceFacts ignores unknown keys where Declaration sets deny_unknown_fields, and why it goes through wg_app_link::format like every other RON here. A project that says nothing gets nothing -- the dialog reports not knowing rather than filling in a directory. Two guesses were tried and both rejected: the config key (updater, app, which resolve to ~/.config/updater and ~/.local/share/app, neither of which exists) and the checkout's directory name (right for both projects here, and still an inference presented as a fact). Iris's call, and the reason is the failure mode: a wrong path does not error, it reads as "this component keeps nothing here". Read off the request path, through crate::checks beside the git and service checks, because Script spawns a process and /manifest is fetched on every open, resume and Refresh. It is in checksPending with the other two -- counting only some of them is the bug that once left components with no buttons after a restart. And it is part of the acceptance gate, since a Script runs: comparing the whole declaration rather than the variant means changing which file is read is a change somebody is asked about.

  • Uninstall can take three things away, and the path on screen is the only guard. The dialog offers logs, data and config as separate toggles -- only logs on by default, and ticking data forces logs on, since a service that writes its log inside its own data directory would lose it either way. purge.rs is the whole of it. Where the data and config are comes from the project's resources: declaration, above; a project that does not say gets two disabled toggles saying so, and the four reasons a toggle is disabled are distinguishable on screen because "couldn't read the resources" is a fault to fix while "doesn't say" is an ordinary project. A path is removed wherever it points, with no check that it sits under the XDG directories -- Iris's call, over the alternative of refusing anything outside them. What replaces that check is the dialog showing each resolved path before the button can be pressed, so the phone never asks for a path nobody saw. Anything that stops the path being displayed removes the only guard there is. Log paths are collected before the uninstall runs, because the script that reports where a service's log lives is the thing being removed.

  • The self entry can't be removable. The app can only be updated through this server, so an updater that can drop itself from its own list strands the installed copy (recovery is a manual reinstall over the bootstrap port).

  • /self and /self/apk are a frozen contract, and the only rescue the app has. Every other route is reachable only by an app new enough to understand it: change what /manifest says and an older app cannot read the list, which is where the button that would replace it lives. So the path that fetches a newer app depends on nothing likely to change -- two numbers, no nesting, no variants, no query parameters -- and the app asks it on every launch and again after building its own project, which is the moment the newer copy exists and the server it must keep talking to has just changed. Add fields at your peril and never rename one; anything richer belongs on a route an old app never calls. It does not survive a changed CA, port or token, since those break the connection before any route is reached: those stay one-way doors and --download stays their answer.

  • There is deliberately no general migration mechanism. Iris's call: each app decides its own, and the updater's job is only to be able to get both halves of itself to the next version. A project that renames or restructures its service is uninstalled and reinstalled from the card, which works because dev-updater is not the thing being stopped.

  • The bootstrap link (--download) serves the APK already on disk and builds nothing. Shipping a stale one there is the expensive mistake, because a fresh install is not enrolled and everything that would replace it -- Update, and the QR scan that enrols a device at all -- lives in the copy being installed, so the only way out is another trip through --download. It cost a round of "the scanner fix never landed" when the fix was in the checkout the whole time. The startup line names the variant and the build's age for that reason; run ./app/build-apk.sh first.

  • Changing the pinned CA is a one-way door for any installed copy. Same reason. wg_app_link::certs deliberately won't regenerate an existing CA; only the leaf is reissued, on every start.

  • State is per machine, not in the repo: $XDG_CONFIG_HOME/dev-updater/{config.ron,certs}, owner-only. This repo is shared with a VM at a different path, so a shared config hands each machine the other's project paths, and a CA key in it is one that VM could sign with. The APK pins the CA of whatever machine builds it (app/build-apk.sh, DEV_UPDATER_CA overrides).

  • Freshness is the APK's mtime versus PackageInfo.lastUpdateTime, not a version code — these are ad hoc rebuilds with nothing bumping a version. It follows that a build and an install landing in the same second can briefly read as "update available"; that's inherent, not a bug to chase.

Running the server for real

./start.sh is the shortest way back to a working state: it builds both halves, rewrites the service unit (so a pull that changes the service script takes effect) and starts or restarts it. Use it after a pull, or when the running server and the checkout have drifted.

To run one by hand instead, launch it from the repo root -- that is where it looks for its own app project (app/), and started anywhere else its own card has no checkout -- and fully detached from the calling shell, not via an agent's background-task tracking, which ties its lifetime to the session:

setsid nohup /path/to/dev-updater </dev/null >server/dev-updater.log 2>&1 & disown -h

Verify with ps -o pid,ppid,pgid,sid,tty,comm -p <pid> — detached means PPID 1, its own SID, TT ?. Check for a stale instance first (pgrep -af "[d]ev-updater" || true): a second instance fails to bind and silently leaves the old one answering. Stop with pkill -f "[d]ev-updater" || true.

Run pgrep -f/pkill -f as a command of its own, with nothing else on the line, and bracket the first character of the pattern ("[d]ev-updater"). Both also exit nonzero when nothing matches, which is the normal case; || true that.

Why the pattern alone isn't enough, since this keeps catching people: each command runs as bash -c '<the whole command text>', so the wrapper's own argv contains every word you wrote. An unbracketed pattern therefore matches the shell running it — pgrep reports phantom matches and pkill kills that shell outright (exit 144, truncated output, reads exactly like the thing under test having crashed). Bracketing the pattern fixes only that word: any other mention of the same string on the line re-arms it. And you usually can't fix the other mention, because it is the real command —

# Kills its own shell: the pattern is bracketed, but the path below
# still puts the plain string "target/debug/dev-updater" in the argv,
# and bracketing *that* would change which file gets executed.
pkill -f "[t]arget/debug/dev-updater" || true; setsid nohup ./target/debug/dev-updater …

so the only rule that always works is to keep them in separate commands: one to stop, one to start.

One more way this bites, which bracketing does nothing about: -f matches the whole command line, so it also catches processes that merely have the string somewhere in a path. An agent's scratchpad directory is named after the repository, so anything launched with a path through it -- the Android emulator, say -- matches [d]ev-updater and gets killed along with the server. Match the binary rather than the project when stopping one: pkill -f "[t]arget/debug/dev-updater".

Environment notes

The machine itself — where the Android SDK is, that each command runs in a fresh shell so exports have to be chained, and each repo having its own emulator — is described once in ~/.claude/MACHINE.md, which every session reads. What follows is what that means here.

  • app/android-env.sh applies the SDK override unconditionally, and .claude/settings.json puts platform-tools alone on PATH so adb works without sourcing anything. Everything else needs the env script.
  • This repo has its own AVD, named dev-updater, which is what app/run-android.sh and app/enroll-emulator.sh default to; every Android project on this machine has one named after it, and none of them share. Two sessions in two repos on one emulator each silently replace the app the other just installed, which reads as a build that never landed rather than as interference.
  • To exercise installing, add a throwaway test app, not another repo's real one. This server's whole job is putting other projects' builds on a device, so trying that out needs something to install -- and a scratch project you wrote is better for it than a real app, because you can make it do whatever the case under test needs: fail its build, produce two variants, change its label, bump nothing at all. A real app only does what it happens to do. It goes in the list like any other project, so nothing here needs to know it is a test.
  • To actually drive the app on the emulator, use app/enroll-emulator.sh rather than working enrolment out again. There is no camera to scan the QR with, so it fires the devupdater://enroll intent the app already accepts, generating a token once and adding it to config.ron beside any real phone's. Two things it exists to stop you rediscovering: adb shell am start -d loses everything after the first unescaped &, because the URI reaches the device's shell; and the server must be started with --bind 0.0.0.0 for the emulator to reach it at all, since it otherwise binds wg0 and the emulator has no route there. The script's header has the detail. It names the device rather than assuming there is only one, which matters more now that every repo has its own AVD: a second emulator running beside this one makes every bare adb call exit 1 with "more than one device/emulator", and the version that ran adb get-state read that as no emulator running -- the opposite of what had happened, which sends you off to start a third. It resolves the serial by AVD name the way run-android.sh always has, defaults to dev-updater, takes --avd NAME or AVD_NAME=, and when it cannot find that one it lists what is attached instead of guessing which of the three situations it is in.
  • The server needs aapt2 (SDK build-tools) to add an app, and, for an app that needs stripping, llvm-strip (NDK), zipalign and apksigner (build-tools) and ~/.android/debug.keystore -- the last is the least guessable of the four, since it belongs to no SDK, is created as a side effect of any Gradle Android build, and is consumed by the pipeline's final step. A machine that builds APKs and has no keystore is odd rather than new: something removed it after the build, and the APK it produced is signed by a key that is now gone. It has to find both without an environment, because the way this server usually starts is from a service manager, and one hands its process a scrubbed environment: measured in the Gentoo guest, an OpenRC user service gets 19 variables with neither $ANDROID_HOME nor $ANDROID_NDK_HOME among them, and a systemd user unit inherits an equally bare one unless somebody imported theirs. Neither branch of service-default passes any through, so this is not a Gentoo quirk and "export it in the unit" is not the fix. HOME does survive, which is what the home-relative candidates in sdk.rs are for. The trap it produces is a build that succeeds and a download that 500s -- a project's own build script sources its android-env.sh and repairs the environment inside its own process, while the strip runs in-process here and sources nothing. So sdk_roots lists candidates and every lookup takes the first that actually contains the tools it needs: an $ANDROID_HOME that exists is not one with the tools in it (this machine's system-wide /opt/android-sdk has a build-tools/36.0.0 holding nothing but package.xml), and taking it blamed zipalign for a root that should never have been chosen. Filter before taking the newest, too -- newest_child_where exists because filtering the single newest child can only reject that one directory, so Sdk sitting beside android-ndk-r27c in ~/Android hid the NDK from a message that said "install one".