A project's components were decoupled everywhere except the one place it showed: there was a single build slot per project, and a single card state in the app keyed by project alone. So pressing Update on one client of a two-client project disabled the other client's button for the length of a build it shares nothing with, drew this one's progress bar and download percentage under the other's row, and -- had the button been pressable -- would have been a silent no-op on the server, since a second request while one was running returned without starting anything. The slot is now per component. `Inner` has no `building` flag; a component's own `ComponentRun` with an open `step` is the answer, and `claim` writes that entry synchronously under the lock the route answers from, so nothing can read a just-claimed component as idle -- which the phone would take for a build that had already finished. A failure is recorded against the component whose command it was rather than in the project's one error slot, which two components building at once cannot share. A pull stays exclusive with everything, because there is one checkout and it rewrites the files every component builds from. Releasing it and claiming what it decided to build happen under one lock: a phone polling in the gap would find a project neither pulling nor building and call the run over. The app mirrors the split -- `ProjectState` for the pull and the project-wide Rebuild, `ComponentState` keyed by component for everything one component is asked to do. Two hierarchies rather than one keyed by a pair, so a download has nowhere project-wide to be stored. A component's failure is drawn in its own row beside the Retry that acts on it, which is also where a failed service action now reports. And a finished component shows nothing at all: the elapsed times are gone from both halves of the wire, and its button simply goes back to being pressable. A bar, a count and a last line all describe something happening now, and left up they sit there looking live next to a sibling that genuinely is. Verified on the emulator against test-projects/two-clients, which exists for this: while `tablet` built, its row alone carried the bar and its button alone was disabled, `phone` stayed pressable and silent, and both returned to normal with no timing left behind.
763 lines
48 KiB
Markdown
763 lines
48 KiB
Markdown
# dev-updater
|
|
|
|
Serves locally-built debug APKs to a phone, and an Android app that
|
|
installs them. See `README.md` first for what this is and how to run it;
|
|
this file is the working notes on top of that.
|
|
|
|
The central design point, worth not undoing by accident: **an app is a
|
|
project path, not a file path.** Everything downstream — which APK, which
|
|
package it replaces, its label, whether it's worth stripping — is derived
|
|
from what's actually built under that path, at add time or per request.
|
|
Nothing about any particular project is compiled in, and the app list is
|
|
mutable at runtime from the phone.
|
|
|
|
## Layout
|
|
|
|
- `vendor/wg-app-link/` — a **submodule**, and the shared half of this and
|
|
ai-app: the WireGuard binding, the CA the app pins, QR enrollment,
|
|
owner-only file creation, and the RON house rules. A submodule rather
|
|
than a published crate because it pins an exact commit, so the two
|
|
servers cannot end up on versions of it that disagree. Clone with
|
|
`--recurse-submodules`; `git::pull` runs `submodule update` after the
|
|
merge, because a merge moves the recorded pointer without touching the
|
|
submodule's working tree and the build would otherwise keep compiling
|
|
the old contents.
|
|
What stays here is what differs: this server's routes, registry, build
|
|
walk and service contract, plus `auth.rs`, whose middleware is generic
|
|
over each project's own state -- only the token functions underneath it
|
|
are shared.
|
|
- `server/` — Rust + Axum. `main.rs` is the bootstrap and the two
|
|
listeners; `routes.rs` has the whole HTTP table in its module doc
|
|
comment. `auth.rs` is the bearer token every TLS request carries,
|
|
applied once around the whole router so a new route cannot forget it. `registry.rs` owns the live app list and every mutation of it
|
|
(config writes funnel through `AppState::update`, so in-memory and
|
|
on-disk state can't come apart; the one other write is
|
|
`AppState::new`'s startup reconciliation, before anything can read the
|
|
list). `discover.rs` is the scanner,
|
|
`config.rs` the persisted schema and the RON both config files are in,
|
|
`apkinfo.rs` the `aapt2` reads,
|
|
`strip.rs` the slim-APK pipeline, `sdk.rs` the SDK/NDK tool lookups.
|
|
- `app/` — Kotlin + Compose, a single `:androidApp` module. `UpdaterScreen.kt`
|
|
is the list, `AddAppScreen.kt` the add/settings screen, `AppsApi.kt` the
|
|
management calls, `UpdateManifest.kt` the read side, `ApkInstaller.kt` /
|
|
`InstalledBuilds.kt` the download-and-install path, `DownloadServer.kt`
|
|
the transport and `Link.kt` the two values it hands the shared library. `Theme.kt` is Catppuccin Mocha
|
|
mapped onto Material's roles, plus the four `ActionTone`s buttons come in
|
|
-- red takes something away, the scheme's mauve replaces it with the same
|
|
thing, green brings it up, blue puts a new build on the phone. What a
|
|
button does is said in colour rather than by which component it sits on,
|
|
so the same consequence looks the same everywhere. The mapping that
|
|
matters is the surface ladder: Mocha names its darks in order (Crust,
|
|
Mantle, Base, Surface 0) and Material asks for the same thing under other
|
|
names, so the page is Base, a component's outlined card stays Base beside
|
|
it, and a project's card is Surface 0 -- one visible step, which is all
|
|
the nesting has to say.
|
|
`NerdIcons.kt` names the icon glyphs,
|
|
which are drawn as *text* in a small Nerd Fonts subset rather than
|
|
as vector assets -- an icon beside a line of text wants that line's size,
|
|
colour and baseline, and a `Text` gets all three for free. The font is
|
|
generated: add a codepoint in `NerdIcons.kt` **and** in
|
|
`app/build-icon-font.sh`, then re-run the script, or the glyph silently
|
|
isn't there.
|
|
|
|
## Checking your work
|
|
|
|
- Server, from `server/`: `cargo fmt`, `cargo clippy --all-targets`, and
|
|
`../run-tests.sh`. All three every time, not just when a change looks
|
|
big enough to warrant them. The build is warning-clean and
|
|
`cargo fmt --check` passes; keep both true. Formatting is plain rustfmt
|
|
defaults with no `rustfmt.toml` — layout is not something to decide per
|
|
line, so take what it gives rather than hand-formatting against it.
|
|
- App: from `app/`, `. ./android-env.sh && ./gradlew :androidApp:compileDebugKotlin`
|
|
(or `:androidApp:assembleDebug`), plus `./gradlew ktfmtFormat
|
|
ktfmtFormatScripts :androidApp:ktfmtFormat :androidApp:ktfmtFormatScripts`
|
|
and `./gradlew :androidApp:lintDebug`. Both are clean; keep them that
|
|
way. ktfmt is `kotlinLangStyle()` with nothing else configured, so
|
|
formatting is never a thing to decide per line.
|
|
- Neither replaces running it. `app/run-android.sh` builds, installs and
|
|
launches on an emulator; screenshot with `adb shell screencap -p
|
|
/sdcard/x.png && adb pull /sdcard/x.png <local>`. Package
|
|
`com.example.devupdater`, activity `.MainActivity`.
|
|
- **`test-projects/` is what to point it at**, rather than a real project.
|
|
Four fake ones -- a plain APK, one building two variants, one whose build
|
|
fails on demand, and a `Server` + `Apk` pair with a service and a
|
|
`resources:` declaration. They exist because a real project only does what
|
|
it happens to do, where these can be asked to fail, to be slow, or to
|
|
declare nothing. That directory's own README says how they are reached
|
|
(they are deliberately not scanned) and why each one builds into an
|
|
`app/` subdirectory; the short version of the second is that two levels is
|
|
what `find_apks` matches, so a test APK any shallower is offered as a
|
|
build of *this* project.
|
|
- The server is easy to exercise directly, which is usually faster than
|
|
going through the UI — but it takes three things, and leaving any of
|
|
them out looks like the server being broken:
|
|
|
|
```sh
|
|
# The CA it actually presents is the one in its config dir, not the
|
|
# stale certs/ left in the repo. The address is wg0's -- it binds that
|
|
# and nothing else, so 127.0.0.1 refuses the connection unless the
|
|
# server was started with --bind. And every route on this port needs
|
|
# the enrolled bearer token.
|
|
curl --cacert "${XDG_CONFIG_HOME:-$HOME/.config}/dev-updater/certs/ca.pem" \
|
|
-H "Authorization: Bearer $TOKEN" \
|
|
"https://$(ip -4 -o addr show wg0 | awk '{print $4}' | cut -d/ -f1):8090/manifest"
|
|
```
|
|
|
|
Only the token's SHA-256 is stored, so there is no reading it back out
|
|
of `config.ron`: either use one printed by `--rotate-token`, or run a
|
|
throwaway server with `--config`/`--certs` pointed somewhere temporary
|
|
and `--bind 127.0.0.1`, which prints a fresh token at startup.
|
|
|
|
## Things that have bitten
|
|
|
|
- **Don't add a general recursive search to `discover.rs`.** It was
|
|
measured and rejected; the numbers are in that module's doc comment and
|
|
in the README. Extend `APK_PATTERNS` instead.
|
|
- **`aapt2` and the strip pipeline never run on the manifest path.** A
|
|
process spawn or a zip walk there turns a 43 ms refresh into something a
|
|
phone notices, and the manifest is fetched on every open, resume and
|
|
Refresh. `aapt2` is add-time work cached in `config.ron`; stripping
|
|
belongs to the download alone, which is why the manifest reports the
|
|
size of whatever is on disk (`strip::serveable_now`) rather than
|
|
producing the slim copy to measure it.
|
|
- **Both config files are RON with two house rules**, and
|
|
`wg_app_link::format` is the only place that knows them -- ai-app's files
|
|
are in the same shape, which is why they are shared rather than described
|
|
twice. `config.rs`'s own `format` module is now only the migration hook
|
|
in front of it, and goes when that does. A file is the *body*
|
|
of the struct -- no outer parentheses, so nothing is indented for a
|
|
wrapper -- because RON has no implicit top-level struct; `parse` adds the
|
|
paren and `render` strips it. And `IMPLICIT_SOME` is set on the
|
|
deserializer rather than by a header each file would have to carry, which
|
|
is why every optional field also needs `skip_serializing_if` so nothing
|
|
is written back that nobody typed. The two halves only round-trip
|
|
together: change one and every file on disk still loads while only
|
|
looking wrong, which is why the config test asserts the written shape.
|
|
- **A `Server` component is driven through one script, run as
|
|
`<script> <subcommand>`.** `service.rs` knows the subcommands and the
|
|
four words `status` may print (`running`, `stopped`, `failed`,
|
|
`not-installed`); it knows nothing about systemd or OpenRC, and
|
|
shouldn't. Scripts run with stdin closed, because one that prompts would
|
|
hang rather than fail.
|
|
**On OpenRC, `status` reads the exit code and never the text**
|
|
(`0` started, `3` stopped, `32` crashed, anything else "could not find
|
|
out"). `rc-service status` prints its status line to *stderr*, so the
|
|
obvious check -- discard stderr, grep stdout for `crashed` -- throws away
|
|
the word it is looking for and reports a crashed service as `stopped`,
|
|
which is the exact lie the `failed` state exists to prevent. The codes
|
|
also distinguish "could not find out", which the text cannot: an
|
|
uninitialised user softlevel makes every call fail, and that is not a
|
|
state the service is in. Assign the code through `|| code=$?`, or `set
|
|
-e` kills the script before it can read one -- every answer but "running"
|
|
is a non-zero exit. Measured on OpenRC 0.63.3, not read.
|
|
**There is a guest to measure in**, at `~/vm/gentoo` on this machine --
|
|
`./start.sh`, then `./shell.sh 'cmd'`, ssh on 127.0.0.1:2223 as `tester`
|
|
with `guest_key`. Its own README is the reference. Everything in this
|
|
section, and the restart behaviour below, was run there rather than
|
|
reasoned about; do the same rather than trusting either the
|
|
documentation or a systemd result. Detecting OpenRC is itself a trap:
|
|
`rc-service --user --help` fails when `XDG_RUNTIME_DIR` is unset, so
|
|
probing with it first reports "no service manager here" on a machine
|
|
that has one.
|
|
- **Which script is what `service:` says**, and it is a sum:
|
|
`Script("path")` for a project that carries its own, `Managed("the
|
|
binary")` for one that wants dev-updater's built-in. Two variants rather
|
|
than two optional fields, because a component picks one and "both" and
|
|
"neither" would be states the file could express that mean nothing. The
|
|
built-in is `server/src/service-default.sh`, compiled in, written to
|
|
`$XDG_DATA_HOME/dev-updater/service-default` at every startup so an
|
|
updated dev-updater cannot be driving last version's copy, and handed
|
|
its identity as arguments (`--name <key>-<component> --exec "..."`) so
|
|
the subcommand the caller appends still lands last. `service::driver` is
|
|
the single place the two variants become one command -- everything
|
|
downstream takes a command and cannot tell which it was, which is what
|
|
lets a project switch between them without anything else changing.
|
|
It is deliberately a *script* rather than an in-process implementation:
|
|
a built-in that ran through Rust would put systemd and OpenRC knowledge
|
|
back in the one place it has been kept out of, and the contract would
|
|
become a special case of itself. **dev-updater declares `Managed` like
|
|
anything else**, and has no service script of its own: nothing about
|
|
restarting this server lives in a script, it lives in `restart.rs`, so
|
|
there was nothing left for one to say. Its component carries `cwd: ".."`
|
|
because the unit's working directory has to be the checkout root -- that
|
|
is where this server looks for its own app project, and started anywhere
|
|
else its card silently loses the branch line, its commit count and the
|
|
Pull button. `start.sh` knows none of this: it runs
|
|
`dev-updater --service <subcommand>` on the binary it has just built,
|
|
which derives the name and the script from the same declaration the
|
|
running server reads. One place decides how this service is named and
|
|
invoked, which is the only reason a bootstrap script cannot drift away
|
|
from `service::driver`.
|
|
A managed service is named `<key>-<component>`, because a service
|
|
manager's names are one flat namespace across every project on the
|
|
machine. Switching a project from its own script to `Managed` therefore
|
|
changes the unit's name: **uninstall the old one first**, or its unit is
|
|
orphaned with nothing left pointing at it.
|
|
- **Service state is checked off the request path**, in `ServiceChecks`
|
|
beside `git::RemoteChecks` and for the same reason: a process spawn per
|
|
server component on `/manifest` is exactly what that path must not do.
|
|
Which means it is subject to the same rule as a remote check: it lands
|
|
*after* the response that started it, so `checksPending` counts both, and
|
|
the phone waits on both (`ManifestEntry.checksOutstanding` for one card).
|
|
And the answering starts at startup rather than on the first request
|
|
(`main.rs`), so the window in which a component has no state is this
|
|
server's own start rather than something a phone has to sit through --
|
|
restarting is how this server is updated, which makes it the moment
|
|
somebody is most likely to be looking at the list.
|
|
Counting only remotes was a real bug and an invisible one: the state is
|
|
unknown only until the first answer of a server's life, remote checks
|
|
ordinarily outlast service ones, and the symptom is a component row with
|
|
no buttons -- so it showed up as "the buttons vanish after a restart if I
|
|
come back to the app too quickly", and only on whichever cards lost the
|
|
race. Nothing looked again until somebody hit Refresh.
|
|
- **dev-updater declares itself as an ordinary `Server` + `Apk` pair**, but
|
|
its server component *is this process*, and that changes one thing only:
|
|
*when* it restarts. Running the script's `restart` inside the walk would
|
|
kill it before the APK is built and before anyone is told how it went, so
|
|
the restart is deferred past `finish`; the Restart route defers for the
|
|
same reason, since a script that killed this process before it could
|
|
answer would make a restart that worked read as a failed request. Once
|
|
deferred, both go through the one function (`restart::deferred`), which
|
|
asks the manager when the service is up and only otherwise execs the new
|
|
binary itself. Going through the manager is what makes a refreshed unit
|
|
apply -- an exec inherits the unit this process was started with, so a
|
|
pull that changed the service script would otherwise land only at the
|
|
next restart by hand. The button used to exec unconditionally, on the grounds
|
|
that exec keeps the PID; that made it the one restart a rewritten unit
|
|
never applied to, which is the kind of surprise a special case buys.
|
|
Deciding it in one place is what stops the two drifting again. `BuildState::is_self` is the flag, and it is passed in
|
|
rather than declared -- a file that could claim it is a file that can
|
|
make this server exec an arbitrary binary. A build that changed nothing
|
|
skips the restart altogether (`restart::binary_unchanged`): exec-ing into
|
|
an identical binary drops the phone's connection to deliver the build it
|
|
already had. It is a `stat` of the binary against what it was at startup
|
|
rather than a hash -- cargo installs a new binary by renaming over the
|
|
old one, so the inode alone answers it, and reading tens of megabytes to
|
|
learn the same thing would cost more than the restart it avoids. It says
|
|
"changed" whenever it cannot tell, because a needless restart costs a
|
|
second while a skipped one leaves the old build serving and reads as the
|
|
update having silently failed. The Restart *button* is unconditional --
|
|
somebody asked. Stop and Uninstall do go
|
|
through the script, and do strand the phone; the app confirms them
|
|
rather than hiding them.
|
|
- **A remote git failure is shortened and explained in one place**
|
|
(`git::remote_failure`). Git's stderr runs to a paragraph of generic
|
|
advice, so the card gets the first line and the log gets all of it; and
|
|
when the failure was authentication, what this process can see of the
|
|
ssh agent is appended, because "works in my terminal, fails from the
|
|
service" is almost always `SSH_AUTH_SOCK` not being inherited by a
|
|
daemon, and that is invisible from a phone. Asked rather than assumed --
|
|
the note can say the agent is fine, which is what stops it being blamed.
|
|
- **A branch with no upstream is not a broken one.** `can_pull` requires an
|
|
upstream as well as a checkout, which is what stops the card reporting
|
|
the same fact twice -- once as its own note and once as the check's
|
|
complaint about it -- and stops it offering a Pull that `git::pull`
|
|
would refuse. It is also not worth *saying*: plenty of checkouts have no
|
|
remote deliberately, and a card that mentions it every time is nagging
|
|
about a choice somebody made.
|
|
|
|
- **Unrelated histories are the one pull failure the phone may override.**
|
|
A checkout sharing no commit with its upstream has no fast-forward and
|
|
never will, so with nothing offered the card is one that can never be
|
|
pulled again -- and the way out is on the build machine, which is
|
|
exactly where the person holding the phone isn't. So `?force=true` on
|
|
the pull route resets onto the upstream instead of merging, behind a
|
|
dialog that names the branch it is about to overwrite (the only place
|
|
that is seen before it goes). Whether this *is* that failure is decided
|
|
structurally -- `git merge-base` finding no common ancestor -- and never
|
|
by matching what git printed: those messages are translated, and a
|
|
button that appeared only on an English build machine is worse than no
|
|
button. It travels as `PullError::unrelated_histories` rather than
|
|
inside the message for the same reason. The dirty-tree refusal stays in
|
|
front of it, so a forced pull can only ever discard something that was
|
|
committed, and a merely *diverged* history is not offered it -- there is
|
|
something better than throwing that away.
|
|
|
|
- **The branch line is how a failed remote check is visible at all.** With
|
|
it gone, `newCommits` stays false and the card reads as an app with no
|
|
updates -- the same silent failure the self-entry note below describes.
|
|
It was deleted once by accident in a refactor and nothing failed; only
|
|
the display went quiet.
|
|
|
|
- **Forcing git onto IPv4 is done in `GIT_SSH_COMMAND`, not on the git
|
|
subcommand.** `git fetch` takes `-4`; `git ls-remote` does not, and
|
|
answers "unknown switch `4'". `ls-remote` is what the new-commits check
|
|
runs, so splicing the flag onto every remote command turned the setting
|
|
into a switch that broke every card's commit count -- which is how it
|
|
was first written, and what `git::ssh_command` and
|
|
`forcing_ipv4_does_not_break_the_check` now hold in place. `fetch` keeps
|
|
the flag as well, since that is what carries the preference to an https
|
|
remote; ssh is covered either way.
|
|
|
|
- **This server's own project is a row in `config.projects` like every
|
|
other, and everything in that row except `gitIpv4` is re-derived at
|
|
startup** by `registry::reconcile_self`. The row is what gives a
|
|
per-project setting one home -- it used to have to be keyed in a
|
|
separate `config.settings` list, because the one card that is always in
|
|
the list was the one card the setting could not be a field on. What is
|
|
derived stays derived: the path is the working directory (see below),
|
|
and the label and components come from the checkout's own
|
|
`.dev-updater.ron`, so a re-clone or a moved repo fixes itself at the
|
|
next start instead of leaving the row naming a directory nobody pulls.
|
|
Only what somebody chose is carried across. The declaration is accepted
|
|
by construction there, which is why `AppEntry::pending_declaration`
|
|
answers nothing for it: between a pull that rewrites the declaration and
|
|
the restart that follows, the row and the file can differ without that
|
|
meaning anything is waiting to be accepted. A config still carrying the
|
|
old keyed `settings` list loses it silently -- `Config` ignores fields it
|
|
does not know -- so a machine that skips straight past the version that
|
|
migrated one has its `gitIpv4` to set again.
|
|
|
|
- **A component has two kinds of log, and the kind is an axis of its
|
|
own.** The build log is what this server captured while building it; the
|
|
runtime log is what the component wrote while running, reported by its
|
|
service script. They are asked for one kind at a time (`?kind=`), and the
|
|
modal gives each a tab -- opening on runtime, since what a service is
|
|
doing now is the usual question, and on build for the component a build
|
|
stopped at (`build_failed`). They were one list indexed by generation
|
|
once, and because build logs came first the runtime one sat at an index
|
|
nothing ever asked for: unreachable, and silent about it. ANSI escapes
|
|
are rendered rather than stripped (`AnsiLog.kt`) -- the colour is how a
|
|
process marked its own errors -- and sequences with no meaning on a
|
|
phone are consumed rather than printed, so a cursor movement cannot
|
|
arrive looking like a corrupted log.
|
|
- **There are three sizes of refresh, and using the wrong one is what
|
|
makes the list feel like it has a mind of its own.** `refresh()` drops
|
|
the list to a spinner and asks every remote -- it belongs to arriving,
|
|
resuming, and the Refresh button, which are the moments somebody asked
|
|
to be shown the current state. Pull-to-refresh is the same ask without
|
|
the spinner, since the gesture brings its own indicator and taking the
|
|
list away underneath it would say the same thing twice. `applyOne` reads `GET /apps/{key}`
|
|
and puts that one entry back, so an action on one card cannot move or
|
|
change any other; every card action uses it, including adding one --
|
|
the Add screen hands back the key and the list fetches just that app.
|
|
`awaitCheck` is the middle one: ask a single checkout's remote and wait
|
|
on *that* answer, which the card's own refresh control and a freshly
|
|
added app both use. Waiting on the whole list to settle would make one
|
|
card's refresh sit behind another card's.
|
|
|
|
- **The Add screen is drawn over the list, not in place of it.** Swapped
|
|
out, the list is composed again from nothing on the way back -- and
|
|
rebuilding it means fetching it, which is the whole-list refresh that
|
|
adding an app has no business causing. `describe` in `routes.rs` is the
|
|
one place a card is built, so `/manifest` and `GET /apps/{key}` cannot
|
|
come to describe the same app differently.
|
|
|
|
- **A progress bar is drawn from counts the build reports, never from an
|
|
estimate**, and the knowledge of how to get those counts belongs to this
|
|
server rather than to each project. Two ways in, both ending as
|
|
`@@progress done/total` on the build's output:
|
|
cargo reports its own once `CARGO_TERM_PROGRESS_WHEN=always` is set,
|
|
which `spawn` sets for every build command, so a cargo build needs
|
|
nothing and must not be wrapped. Gradle cannot -- an init script is
|
|
refused by the configuration cache, and its rich console reports a
|
|
percentage only as a full-screen redraw that would make the output
|
|
unreadable -- so `server/src/build-progress.sh` does the `--dry-run`
|
|
count and the `> Task` tally, shipped by this server (`crate::shipped`)
|
|
and offered to build scripts as `$DEV_UPDATER_PROGRESS`. A script tests
|
|
for the variable rather than depending on it, so building by hand still
|
|
works. This is what stops each project carrying its own copy of the
|
|
same twenty-five lines.
|
|
Reading the output means splitting on carriage returns as well as
|
|
newlines: a tool redrawing a counter in place puts several updates and
|
|
then real output inside one `\n`-delimited line.
|
|
- **Discovery must stay side-effect free.** It runs on every manifest
|
|
request and every suggestion scan.
|
|
- **An app in the list is a *project*, and what it produces is its
|
|
components.** Ordinarily one `Apk`; a project that also runs a server on
|
|
the build machine declares a `Server` beside it. They are a sum, not one
|
|
struct with both halves' fields, and the common fields are repeated per
|
|
variant so the file reads `Apk(name: ...)` with nothing nested inside a
|
|
`kind`. `Component::same_declaration` destructures exhaustively, so
|
|
adding a field anywhere fails to compile until someone has said whether a
|
|
project declares it or this server measures it.
|
|
- **A build reinstalls every server component's unit, not just changed
|
|
ones.** The unit is generated from a script in the repository, so a pull
|
|
can change how a service is *defined* rather than what it runs -- and
|
|
that has to land even when the build produced a byte-identical binary,
|
|
which is the ordinary case for a pull that only touched the script.
|
|
Installing is safe on a running service (it writes the unit and reloads;
|
|
it stops nothing), which is why `start.sh` has always done it every run.
|
|
It is deliberately not gated on `restart::binary_unchanged`: that answers
|
|
whether a *restart* is worth the interruption, and unit staleness is a
|
|
different question. Wiring the two together left the unit stale exactly
|
|
when the build had nothing to do.
|
|
- **A build runs every component at once, and each reports for itself.**
|
|
They are independent -- a Rust build and a Gradle build share nothing but
|
|
the machine -- and measured here, together takes about three quarters of
|
|
the time one after the other does. The saving only appears when more than
|
|
one has work, which is what a pull produces.
|
|
The status is per component (`ComponentStatus`), so a card draws each
|
|
component's bar and last line inside that component's own row;
|
|
the project's own area at the bottom keeps only what belongs to the whole
|
|
project, which is fetching and pulling. A bar under the card could only
|
|
ever say that *something* was happening, and with everything building at
|
|
once that is exactly what the reader is trying to find out.
|
|
A failure no longer stops the others -- they are already running -- and
|
|
it is reported against the component whose command it was, never as the
|
|
project's. The phone draws every component in
|
|
a card of its own, including a project with only one: the flat layout it
|
|
used to get meant two shapes to keep in step, and put that APK's size up
|
|
beside the card's corner controls where it read as belonging to them.
|
|
- **There is a build slot per component, not per project, and the two
|
|
halves have to agree on that.** A component being built neither blocks
|
|
another's build nor disables its controls: `Inner` has no `building`
|
|
flag, only a `ComponentRun` per component whose open `step` *is* the
|
|
answer, and `claim` writes that entry synchronously under the same lock
|
|
the route answers from -- so nothing can read a component the request
|
|
just claimed as idle, which the phone would take for "the build is
|
|
over". `RunningBuild::building` stays, but it means "anything at all is
|
|
happening here" and is only for the controls that act on the whole
|
|
checkout; anything about one component reads that component's `step`.
|
|
The app mirrors the split exactly: `ProjectState` for the pull and the
|
|
project-wide Rebuild, `ComponentState` keyed by component name for
|
|
everything a single component is asked to do. One map keyed by project
|
|
alone is what the bug was -- pressing Update on one client of a
|
|
two-client project disabled the other's button and drew this one's
|
|
download bar under it -- and two hierarchies rather than one keyed by a
|
|
pair is what stops it coming back, since a download has no
|
|
project-wide meaning to be stored with.
|
|
The exception, and it is worth keeping visible so it does not read as
|
|
more of the same: **a pull really is exclusive with everything.** There
|
|
is one checkout, and it rewrites the files every component builds from,
|
|
so `Inner::pulling` blocks any component from being claimed and waits
|
|
for any still building. Ending the pull and claiming what it decided to
|
|
build happen under one lock for the same reason `claim` is
|
|
synchronous -- a phone polling in the gap would see a project that is
|
|
neither pulling nor building and call the run finished.
|
|
- **A finished component shows nothing, and its button goes back to
|
|
normal.** Iris's call, 2026-09-01: "you shouldn't see the time it took
|
|
once it finishes, it should just go back to its normal enabled button
|
|
state." So `ComponentBuildProgress` draws only while the step is open,
|
|
and the elapsed times are gone from both halves of the wire -- a bar, a
|
|
count and a last line all describe something happening *now*, and every
|
|
one of them sits there looking live beside a sibling that genuinely is.
|
|
What says the build landed is the control becoming pressable again and
|
|
the card's own freshness. The failure is the exception, because it is an
|
|
outcome rather than residue; it is drawn by the component card next to
|
|
the Retry that acts on it, in the one place that reports that
|
|
component's failures whether they came from a build, a download or a
|
|
service action.
|
|
- **A project's own `.dev-updater.ron` is a request, never an
|
|
instruction.** It only runs once accepted from the phone, which copies
|
|
it into `config.ron`; `AppEntry::pending_declaration` is the whole gate.
|
|
Evaluate it *fresh* rather than caching it on the entry — the file
|
|
changes on a pull, and a pull writes no config, so nothing rebuilds the
|
|
entry list. Caching it meant a pull could swap the command out from
|
|
under a previous acceptance and nothing noticed until an unrelated
|
|
config write. Enforce it wherever a command would actually run, not
|
|
only where the list is built.
|
|
Two things follow that both went wrong the moment `resources:` joined
|
|
the gate. **Whatever `Declaration::matches_accepted` compares,
|
|
`AppState::approve_declaration` has to write** -- it compared the
|
|
resources declaration and wrote only the components, so accepting stored
|
|
half of what was being read back and the gate could never clear. The
|
|
press succeeded, so there was no error either; the card simply kept
|
|
asking. Adding a field to one without the other is the shape to watch
|
|
for, and the tests are what let it through: they built an "accepted"
|
|
config by hand instead of calling `approve_declaration`, so they
|
|
asserted a copy of the rule rather than the rule.
|
|
And **on the pull path the gate is asked after the fast-forward, never
|
|
before it** (`build_state::pull_and_build` takes a closure for exactly
|
|
this reason). The declaration lives in the checkout, so the pull is the
|
|
one thing that changes the answer -- a bool computed at the call site is
|
|
the answer for the commit being replaced, which made a pull that changed
|
|
the declaration build anyway. Pulling stays allowed while a request is
|
|
unaccepted, since taking commits runs git rather than the project's
|
|
command, and it is how the new request arrives to be read.
|
|
- **Which build variant to serve is the phone's choice, not the server's.**
|
|
It arrives as `?variant=` on the download, beside the `?component=` that
|
|
says whose build it is, and is validated against *that component's*
|
|
discovered builds -- an unvalidated path would let a phone name any file
|
|
on disk to be served. Storing it server-side meant one enrolled device
|
|
silently changing what another was offered. The stored choice is keyed
|
|
by project *and* component on the device, so pinning one client to a
|
|
release build says nothing about the other.
|
|
|
|
- **A project can produce more than one APK, and each component's builds
|
|
are found under its own `cwd`.** `APK_PATTERNS` is anchored at
|
|
`project.join(cwd)` rather than at the project root, which is what
|
|
`cwd` already meant everywhere else -- the directory the build command
|
|
runs in, the subtree `subtree_head` scopes staleness to, a server's
|
|
`WorkingDirectory`. A component that declares none sits at the root,
|
|
which is what every single-APK project has always meant, so nothing
|
|
about that case changed. tdep-survey is the project that needed it: one
|
|
checkout, a backend and *two* independent Android clients, where the
|
|
root-anchored patterns reached the first and stopped.
|
|
The alternative -- naming the file on the component, `apk: "path"` --
|
|
was rejected because it makes a component a file path, and **an app
|
|
being a project path rather than a file path** is the invariant at the
|
|
top of this document.
|
|
Everything downstream is per component in consequence: `package`,
|
|
`strip`, the variant list, the size, the mtime and the rename note.
|
|
Those were all `apk_component()`'s *first* match before, which was
|
|
correct only while a project had one. Two clients install over
|
|
different packages and differ in whether their symbols are worth
|
|
carrying to a phone, so first-wins would have checked the installed
|
|
state of one app and reported it as the other's -- the expensive kind
|
|
of wrong, because it looks exactly like an answer.
|
|
**A download that names no component is refused, not guessed**
|
|
(`ApiError::AmbiguousApk`), for that reason. Naming none still answers
|
|
for a project with one, which is nearly all of them and is what lets
|
|
the frozen `/self/apk` keep working -- it cannot carry a component, and
|
|
the project it describes has a single APK.
|
|
The measurement carried across an acceptance is keyed by name *and*
|
|
`cwd` (`registry::component_id`): once the directory decides which
|
|
builds a component has, a reused name is not the same APK, and handing
|
|
it the old one's package would be wrong until that component happened
|
|
to be downloaded.
|
|
- **`/prepare` and `/build` take `?component=` to build one component
|
|
instead of every one with a command.** Without it, pressing Update on
|
|
one client of a multi-client project ran every component's build to
|
|
get the one that was actually asked for -- fine when a project had one
|
|
APK, expensive the moment it had two and one of them was slow.
|
|
`named_component` in `routes.rs` is the one place a name from the phone
|
|
is checked against the project's own components, so `/prepare` and
|
|
`/build` cannot disagree about what an unknown name means, and
|
|
`BuildState::{trigger_if_needed,build_now,run_build,is_stale}` all take
|
|
the same `Option<&str>` -- `None` still means the whole project, which
|
|
is what `Pull & Build`, the project-row `Rebuild`, and every project
|
|
with a single component keep doing. There is deliberately no
|
|
component-scoped Rebuild: forcing one component's build without
|
|
touching the rest happens by pressing Update on it, which runs
|
|
`/prepare` scoped to that component.
|
|
**Where `component.dir()` belongs is now one definition**
|
|
(`Component::dir` in `config.rs`), because a second one very nearly
|
|
shipped a real bug: `component_is_stale`'s "never built at all" check
|
|
still asked `find_apks` of the *project root* after per-component
|
|
discovery had already moved everywhere else to `component.dir()`. A
|
|
project with two `Apk` components has one's output sitting under the
|
|
root-anchored patterns too -- `*/build/outputs/apk/*/*.apk` matches any
|
|
one-level subdirectory, regardless of which component put it there --
|
|
so the moment *either* component had ever been built, the whole
|
|
project read as "something is built here," and the *other* component,
|
|
never built, silently stopped being offered its own first build:
|
|
`prepare` saw a component with a command and no output and declared it
|
|
current. Caught by testing the actual behaviour of a two-APK project
|
|
rather than trusting that scoping the build implied scoping the
|
|
staleness check that decides whether to run it -- they are two
|
|
different reads of "which directory is this component's," and only one
|
|
of them had been moved.
|
|
The same check is deliberately *not* asked of a `Server`: a service
|
|
never has an APK to find under its own directory by definition, so
|
|
asking would report every server "never built" forever. Guarded on
|
|
`matches!(component, Component::Apk { .. })` for that reason.
|
|
- **The self entry's project is the working directory itself**, so this
|
|
server has to be started from the root of its own checkout -- which is
|
|
where everything else here is driven from, and what the service unit
|
|
sets (`WorkingDirectory=$REPO_ROOT`, not `server/`). Its components then
|
|
say where they live from there, exactly as any other project's do; there
|
|
is nothing special about this one's layout, which is the point of it not
|
|
being `app/` any more. It used to be
|
|
`CARGO_MANIFEST_DIR`, fixed when the binary was *compiled*, so a
|
|
re-clone or a move left the card watching a directory nobody pulls. The
|
|
symptom is why this is worth remembering: the entry keeps working and
|
|
keeps serving an APK, but silently loses its branch line, its Pull
|
|
button and its commit count, so it reads as an app that simply never has
|
|
an update. It took a "why can't my server see the new commits?" to find.
|
|
Startup warns when there is no checkout at the resolved path, which is
|
|
the only cheap moment to notice.
|
|
- **A configured project pointing at this server's own project is a second
|
|
card for it.** `add_app` refuses it (the self entry is in the list it
|
|
checks against, and `self_project` is canonicalized so a symlink can't
|
|
make one directory look like two), so this only survives from a config
|
|
written while the self entry resolved somewhere else. It is not merely
|
|
redundant: it is not the self entry, so `is_self` is false and a build
|
|
through it would restart this server partway through its own build. `build_entries` warns rather than dropping
|
|
it -- dropping it would leave a config entry with no card and so no way
|
|
to remove it from the phone. The built-in card has no Remove button, so
|
|
the duplicate is the one that does.
|
|
- **A project says where its own state is; nothing infers it.** The
|
|
`resources:` declaration points at a RON file, a script that prints the
|
|
same RON, or an inline struct, and it holds what a project would
|
|
otherwise write down in several places -- its `name`, and its `data` and
|
|
`config` directories when those are not `$XDG_*_HOME/<name>`. **The file
|
|
is the project's, not this server's**: its own code is meant to read the
|
|
same one, which is why `ResourceFacts` ignores unknown keys where
|
|
`Declaration` sets `deny_unknown_fields`, and why it goes through
|
|
`wg_app_link::format` like every other RON here.
|
|
A project that says nothing gets nothing -- the dialog reports not
|
|
knowing rather than filling in a directory. Two guesses were tried and
|
|
both rejected: the config key (`updater`, `app`, which resolve to
|
|
`~/.config/updater` and `~/.local/share/app`, neither of which exists)
|
|
and the checkout's directory name (right for both projects here, and
|
|
still an inference presented as a fact). Iris's call, and the reason is
|
|
the failure mode: a wrong path does not error, it reads as "this
|
|
component keeps nothing here".
|
|
Read **off the request path**, through `crate::checks` beside the git
|
|
and service checks, because `Script` spawns a process and `/manifest` is
|
|
fetched on every open, resume and Refresh. It is in `checksPending` with
|
|
the other two -- counting only some of them is the bug that once left
|
|
components with no buttons after a restart. And it is part of the
|
|
acceptance gate, since a `Script` runs: comparing the whole declaration
|
|
rather than the variant means changing which file is read is a change
|
|
somebody is asked about.
|
|
|
|
- **Uninstall can take three things away, and the path on screen is the
|
|
only guard.** The dialog offers logs, data and config as separate
|
|
toggles -- only logs on by default, and ticking data forces logs on,
|
|
since a service that writes its log inside its own data directory would
|
|
lose it either way. `purge.rs` is the whole of it. Where the data and
|
|
config are comes from the project's `resources:` declaration, above; a
|
|
project that does not say gets two disabled toggles saying so, and the
|
|
four reasons a toggle is disabled are distinguishable on screen because
|
|
"couldn't read the resources" is a fault to fix while "doesn't say" is
|
|
an ordinary project.
|
|
A path is removed **wherever it points**, with no check that it
|
|
sits under the XDG directories -- Iris's call, over the alternative of
|
|
refusing anything outside them. What replaces that check is the dialog
|
|
showing each resolved path before the button can be pressed, so the
|
|
phone never asks for a path nobody saw. Anything that stops the path
|
|
being displayed removes the only guard there is.
|
|
Log paths are collected *before* the uninstall runs, because the script
|
|
that reports where a service's log lives is the thing being removed.
|
|
|
|
- **The self entry can't be removable.** The app can only be updated
|
|
through this server, so an updater that can drop itself from its own list
|
|
strands the installed copy (recovery is a manual reinstall over the
|
|
bootstrap port).
|
|
- **`/self` and `/self/apk` are a frozen contract, and the only rescue the
|
|
app has.** Every other route is reachable only by an app new enough to
|
|
understand it: change what `/manifest` says and an older app cannot read
|
|
the list, which is where the button that would replace it lives. So the
|
|
path that fetches a newer app depends on nothing likely to change --
|
|
two numbers, no nesting, no variants, no query parameters -- and the app
|
|
asks it on every launch and again after building its own project, which
|
|
is the moment the newer copy exists and the server it must keep talking
|
|
to has just changed. Add fields at your peril and never rename one;
|
|
anything richer belongs on a route an old app never calls. It does not
|
|
survive a changed CA, port or token, since those break the connection
|
|
before any route is reached: those stay one-way doors and `--download`
|
|
stays their answer.
|
|
- **There is deliberately no general migration mechanism.** Iris's call:
|
|
each app decides its own, and the updater's job is only to be able to
|
|
get both halves of *itself* to the next version. A project that renames
|
|
or restructures its service is uninstalled and reinstalled from the card,
|
|
which works because dev-updater is not the thing being stopped.
|
|
- **The bootstrap link (`--download`) serves the APK already on disk and
|
|
builds nothing.** Shipping a stale one there is the expensive mistake,
|
|
because a fresh install is not enrolled and everything that would replace
|
|
it -- Update, and the QR scan that enrols a device at all -- lives in the
|
|
copy being installed, so the only way out is another trip through
|
|
`--download`. It cost a round of "the scanner fix never landed" when the
|
|
fix was in the checkout the whole time. The startup line names the
|
|
variant and the build's age for that reason; run `./app/build-apk.sh`
|
|
first.
|
|
- **Changing the pinned CA is a one-way door for any installed copy.** Same
|
|
reason. `wg_app_link::certs` deliberately won't regenerate an existing
|
|
CA; only the leaf is reissued, on every start.
|
|
- **State is per machine, not in the repo**:
|
|
`$XDG_CONFIG_HOME/dev-updater/{config.ron,certs}`, owner-only. This
|
|
repo is shared with a VM at a different path, so a shared config hands
|
|
each machine the other's project paths, and a CA key in it is one that VM
|
|
could sign with. The APK pins the CA of whatever machine builds it
|
|
(`app/build-apk.sh`, `DEV_UPDATER_CA` overrides).
|
|
- Freshness is the APK's **mtime** versus `PackageInfo.lastUpdateTime`, not
|
|
a version code — these are ad hoc rebuilds with nothing bumping a
|
|
version. It follows that a build and an install landing in the same
|
|
second can briefly read as "update available"; that's inherent, not a
|
|
bug to chase.
|
|
|
|
## Running the server for real
|
|
|
|
`./start.sh` is the shortest way back to a working state: it builds both
|
|
halves, rewrites the service unit (so a pull that changes the service script
|
|
takes effect) and starts or restarts it. Use it after a pull, or when the
|
|
running server and the checkout have drifted.
|
|
|
|
To run one by hand instead, launch it **from the repo root** -- that is
|
|
where it looks for its own app project (`app/`), and started anywhere else
|
|
its own card has no checkout -- and **fully detached from the calling
|
|
shell**, not via an agent's background-task tracking, which ties its
|
|
lifetime to the session:
|
|
|
|
```sh
|
|
setsid nohup /path/to/dev-updater </dev/null >server/dev-updater.log 2>&1 & disown -h
|
|
```
|
|
|
|
Verify with `ps -o pid,ppid,pgid,sid,tty,comm -p <pid>` — detached means
|
|
`PPID 1`, its own `SID`, `TT ?`. Check for a stale instance first
|
|
(`pgrep -af "[d]ev-updater" || true`): a second instance fails to bind and
|
|
silently leaves the old one answering. Stop with
|
|
`pkill -f "[d]ev-updater" || true`.
|
|
|
|
**Run `pgrep -f`/`pkill -f` as a command of its own, with nothing else on
|
|
the line, and bracket the first character of the pattern**
|
|
(`"[d]ev-updater"`). Both also exit nonzero when nothing matches, which is
|
|
the normal case; `|| true` that.
|
|
|
|
Why the pattern alone isn't enough, since this keeps catching people: each
|
|
command runs as `bash -c '<the whole command text>'`, so the wrapper's own
|
|
argv contains every word you wrote. An unbracketed pattern therefore
|
|
matches the shell running it — `pgrep` reports phantom matches and `pkill`
|
|
kills that shell outright (exit 144, truncated output, reads exactly like
|
|
the thing under test having crashed). Bracketing the pattern fixes only
|
|
*that* word: any **other** mention of the same string on the line re-arms
|
|
it. And you usually can't fix the other mention, because it is the real
|
|
command —
|
|
|
|
```sh
|
|
# Kills its own shell: the pattern is bracketed, but the path below
|
|
# still puts the plain string "target/debug/dev-updater" in the argv,
|
|
# and bracketing *that* would change which file gets executed.
|
|
pkill -f "[t]arget/debug/dev-updater" || true; setsid nohup ./target/debug/dev-updater …
|
|
```
|
|
|
|
so the only rule that always works is to keep them in separate commands:
|
|
one to stop, one to start.
|
|
|
|
One more way this bites, which bracketing does nothing about: `-f` matches
|
|
the **whole command line**, so it also catches processes that merely have
|
|
the string somewhere in a path. An agent's scratchpad directory is named
|
|
after the repository, so anything launched with a path through it -- the
|
|
Android emulator, say -- matches `[d]ev-updater` and gets killed along
|
|
with the server. Match the binary rather than the project when stopping
|
|
one: `pkill -f "[t]arget/debug/dev-updater"`.
|
|
|
|
## Environment notes
|
|
|
|
The machine itself — where the Android SDK is, that each command runs in a
|
|
fresh shell so exports have to be chained, and each repo having its own
|
|
emulator — is described once in `~/.claude/MACHINE.md`, which every
|
|
session reads. What follows is what that means here.
|
|
|
|
- `app/android-env.sh` applies the SDK override unconditionally, and
|
|
`.claude/settings.json` puts `platform-tools` alone on `PATH` so `adb`
|
|
works without sourcing anything. Everything else needs the env script.
|
|
- **This repo has its own AVD, named `dev-updater`**, which is what
|
|
`app/run-android.sh` and `app/enroll-emulator.sh` default to; every
|
|
Android project on this machine has one named after it, and none of them
|
|
share. Two sessions in two repos on one emulator each silently replace
|
|
the app the other just installed, which reads as a build that never
|
|
landed rather than as interference.
|
|
- **To exercise installing, add a throwaway test app**, not another repo's
|
|
real one. This server's whole job is putting *other* projects' builds on a
|
|
device, so trying that out needs something to install -- and a scratch
|
|
project you wrote is better for it than a real app, because you can make
|
|
it do whatever the case under test needs: fail its build, produce two
|
|
variants, change its label, bump nothing at all. A real app only does what
|
|
it happens to do. It goes in the list like any other project, so nothing
|
|
here needs to know it is a test.
|
|
- **To actually drive the app on the emulator, use
|
|
`app/enroll-emulator.sh`** rather than working enrolment out again. There
|
|
is no camera to scan the QR with, so it fires the `devupdater://enroll`
|
|
intent the app already accepts, generating a token once and adding it to
|
|
`config.ron` beside any real phone's. Two things it exists to stop you
|
|
rediscovering: `adb shell am start -d` loses everything after the first
|
|
unescaped `&`, because the URI reaches the *device's* shell; and the
|
|
server must be started with `--bind 0.0.0.0` for the emulator to reach
|
|
it at all, since it otherwise binds wg0 and the emulator has no route
|
|
there. The script's header has the detail.
|
|
**It names the device rather than assuming there is only one**, which
|
|
matters more now that every repo has its own AVD: a second emulator
|
|
running beside this one makes every bare `adb` call exit 1 with "more
|
|
than one device/emulator", and the version that ran `adb get-state` read
|
|
that as *no emulator running* -- the opposite of what had happened, which
|
|
sends you off to start a third. It resolves the serial by AVD name the
|
|
way `run-android.sh` always has, defaults to `dev-updater`, takes
|
|
`--avd NAME` or `AVD_NAME=`, and when it cannot find that one it lists
|
|
what *is* attached instead of guessing which of the three situations it
|
|
is in.
|
|
- The server needs `aapt2` (SDK build-tools) to add an app, and
|
|
`llvm-strip` (NDK) only for apps that need stripping.
|