emulator-tools/README.md

# emulator-tools

Everything this machine uses to drive an Android emulator, in one place:
which emulator a command means, whether there is room to start it, reading a
screen as text, and taking a screenshot that does not cost a thousand tokens.

    ./install.sh

symlinks `bin/*` into `~/.local/bin` and `share/ui-trace` into
`~/.local/share`, so editing this repo is editing what runs. It keeps
anything it replaces as `<name>.bak`.

## What is here

| | |
|---|---|
| `emu` | this checkout's emulator: `name`, `serial`, `list`, `up`, `down`, `check` |
| `adb` | adb, aimed at this checkout's emulator, with screenshots scaled |
| `ui-trace` | records a screen as text at 60Hz, and drives it |
| `lib/project-avd.sh` | the rules all three share |
| `share/ui-trace/` | the device recorder (`UiTrace.java`) and its build |
| `share/gradle-init/` | the init script that stops Gradle installing on every device |

## One emulator per checkout, named after it

`emu` and `adb` both work out which AVD you mean from the enclosing git
checkout: in `~/repos/ai-app-2` that is `ai-app-2`. Nothing has to be
configured, and `AVD_NAME` overrides it where the guess is wrong.

This is the rule that lets several agent sessions work here at once. The
emulator used to be the one resource that could not be parallelised: taking
it meant messaging the other sessions, waiting, and handing it back — and
installing an app onto somebody else's running emulator steals the foreground
from whatever they were looking at.

It is also why `adb` fills in `-s`. With two emulators attached, a bare
`adb shell pm list packages` comes back **empty** rather than failing, which
reads as the app having been uninstalled rather than as the question being
ambiguous. An explicit `-s`, `ANDROID_SERIAL`, or a subcommand that is not
about one device (`adb devices`) all turn the defaulting off, and nothing is
guessed when this checkout's emulator is not running: adb's own error is
better than a wrapper picking a stranger's device.

## Gradle is the hole the wrapper cannot cover

`adb` can only aim a call that goes through it, and Gradle's Android tasks
do not: `installDebug`, `uninstallDebug` and `connectedAndroidTest` ask the
adb server for every attached device and act on all of them. On 2026-08-31
one session ran `./gradlew :androidApp:installDebug` with two emulators up
and replaced the app on both. Gradle reported success; from the other
session it looked like its own build had never landed.

So `install.sh` also links `share/gradle-init/emulator-tools.gradle` into
`~/.gradle/init.d`, where Gradle applies it to **every** build on this
machine — including checkouts nobody has adopted it in, which are exactly
the ones this is protecting. Before any of those tasks runs it calls

    emu check

in the project's own directory. That is the same rule the `adb` wrapper
follows, in one place: what must not be touched is *another checkout's
emulator*. A physical phone is nobody's checkout, one emulator on its own is
nobody else's business, and a caller who named a device has already narrowed
the fan-out to it — so `check` passes all of those, and refuses only when a
stranger's emulator is in range. `ANDROID_SERIAL` pointing at one is refused
too, because that is what a serial exported into a long-lived shell decays
into once an emulator restarts and another checkout's takes the port.

Say which device you mean at the moment you use it rather than exporting it:

    ANDROID_SERIAL=$(emu serial) ./gradlew :androidApp:installDebug

`EMU_ANY_DEVICE=1` means every attached device anyway, for the case that is
genuinely what you want.

## Rendering on the GPU with no display

This machine has no screen, and for a long time that meant the emulator
rasterised in software. It does not have to: the VM sees the host's GPU
through virtio-gpu, and `/dev/dri/renderD128` is readable without being in
the `video` group.

What was in the way is that the emulator's `-gpu host` renderer speaks GLX,
so with no `DISPLAY` it cannot open a context at all -- it says
`GlxEnginegetDefaultDisplay: Failed to open display 0` and falls back. So
`emu up` starts a **headless sway** whose only job is to provide an
Xwayland, and Xwayland's GLX runs on the render node. The guest's GLES then
reports `virgl (AMD Radeon RX 7900 XT)` instead of a software rasteriser.
One compositor is started for the machine and reused, on a named socket
under `XDG_RUNTIME_DIR`; it needs no seat, no DRM master and no card node.

Measured 2026-08-31, scrolling the same list in the stock **Settings** app
on the same AVD with the same gesture -- so this is the platform's own cost,
with no app of ours in it:

| | janky frames | legacy janky | 90th percentile | slow UI-thread frames |
|---|---|---|---|---|
| software | 57.4% | 100% | 73ms | 18 |
| on the GPU | 2.3% | 5.7% | 26ms | 0 |

Read **both** jank columns. The legacy figure counts a frame against the
16.7ms deadline and the modern one against what the compositor actually
needed, so on a fast enough path the modern number can barely move while the
legacy one halves. When comparing two builds of an app in here, the legacy
column is usually the one carrying the signal.

An emulator also costs about **620 MB less** resident this way: measured
twenty seconds after boot on one AVD, 3537 MB on software against 2918 MB on
the GPU. (An earlier draft of this file said 1.1 GB. That number came from
`emu list` before the prefix bug in it was fixed, and was one emulator's size
reported against another's.)

This matters for more than speed. Before it, frame timings measured in here
said nothing at all about an app: every app was bounded by the rasteriser
rather than by anything it was doing, and the stock Settings app missed
*every single frame's* deadline while scrolling its own list. Now the
platform floor is low enough that an app's own jank is visible above it.

`GPU_HOST_FEATURES` carries the one workaround: host Vulkan is switched off,
because gfxstream asks Venus for a memory type for its ColorBuffers that
Venus does not offer, and the emulator dies before adb sees it. GLES is
unaffected. It is worth retrying whenever mesa moves -- the failure is loud
and costs one boot to find.

If sway is missing, or the compositor will not start, `emu up` says so and
starts on software rendering as it always did. None of this is essential.

## Refusing to start one

`emu up` checks `MemAvailable` first and refuses if starting an emulator
would leave the machine short, printing what is attached, what is large, and
what usually frees enough. `EMU_FORCE=1` overrides it.

The emulator is launched in its own session with stdin closed, so it outlives
both an interactive terminal and an isolated command runner. A bare background
job appeared to work from long-lived agent shells but was shut down as soon as
a one-command execution session ended; the emulator's own log showed a clean
shutdown after boot rather than a crash.

The reason is a real incident rather than tidiness. On 2026-08-30 an emulator
was started on this box with 2.8 GB available; the OOM killer ran, and what
it took was not the emulator that had just started — it walked the user slice
and killed pipewire, dbus-broker and *another session's* emulator first. The
cost of one too many lands on somebody else's work, minutes later, looking
like an unrelated crash.

Measured numbers behind the defaults: a headless x86_64 AVD is about 3.8 GB
resident, this VM has 23 GB with a swap that is routinely full, two emulators
are comfortable, and three were not.

## Reading a screen without screenshots

`ui-trace` drives a small resident recorder on the device and samples the
accessibility tree at 60Hz, so an animation or a settling layout is visible
rather than happening between two samples.

    ui-trace record -d 3000 --do "tap 'Session settings'" -o /tmp/t.txt
    ui-trace elements /tmp/t.txt              # what is on screen, to pick from
    ui-trace show /tmp/t.txt                  # what moved
    ui-trace show /tmp/t.txt -m 'Send|Stop' --field box

Bounds come out in **device** pixels, so they feed straight back into a
`--do tap`. Prefer this to a screenshot for anything positional: two buttons
that measured identically at 171x105 looked different in a screenshot,
because the filled one read as larger than the disabled grey one. Screenshots
are for appearance — colour, weight, whether it looks right.

### Press things by name, never by coordinate

`tap 'Save'` finds whatever currently carries that label — the text, or the
padded control around it, preferring the one that is clickable — resolves its
box from the tree **at the moment of the gesture**, and presses its centre. It
is the name the control already has for assistive technology, so nothing has
to be kept in step with it.

`tap X Y` still exists and is the exception. A coordinate is a position
measured once by hand, and anything that moves the control — a button added to
the row, a font size, a density, another device — makes the tap land on
whatever now sits there. The script then reports a number that was never
measured, which reads exactly like a result rather than like a failure. Asked
for by Iris on 2026-09-03, after ai-app's two benchmark scripts pressed a
header button at `tap 723 205` and that button moved.

`hold 'helper 3'` is the same resolution with a press held past the
platform's 500ms long-press threshold, for a row that is held to select it or
anything else that distinguishes the two. Neither other action reaches it:
`tap`'s press is 60ms, and `holddrag` takes coordinates and always drags
afterwards — so the one gesture "hold a list row" needs was the one that could
only be written as a pair of numbers.

A label that is not on screen ends the recording with `# error action failed`
and a non-zero exit, so a run that could not press what it meant to press
produces no numbers at all.

The check that none is left in a project's scripts is one grep:

    grep -n "tap [0-9]" path/to/*.sh

`adb exec-out screencap -p` still works and comes back scaled to 800px on its
long edge, which is ~380 tokens to read instead of ~1460 and still legible
for layout, contrast and small labels. `ADB_SCREENCAP_MAX_EDGE` raises it for
one call.

## Using it from a project

A project's own `run-android.sh` should build and install; the emulator half
belongs here:

    SERIAL=$(emu up)
    ./gradlew :androidApp:assembleDebug
    adb install -r <apk>          # already aimed at this checkout's emulator