Author SHA1 Message Date
iris 62199aa3a7 Merge remote-tracking branch 'origin/rustify' into worktree-agent-afe80868604fef704
# Conflicts:
#	IRIS.md
#	RUST.md
2026-09-05 12:59:39 -04:00
irisandClaude Fable 5.1 b133d85943 RUST.md, IRIS.md, CLIENT_CORE.md: record E4 done
RUST.md: E4 ticked with the screenshot path, the exact commands against
app/ui-sandbox.sh, and the streaming-duplication bug the screenshot found;
"Where things stand" moved E4 out of "in flight" into its own done bullet.
IRIS.md: transcript_ui::build_tree, the public API change transcript-ui
gained for this. CLIENT_CORE.md: client_core::config's table row and its
correspondence note.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 12:55:59 -04:00
irisandClaude Fable 5.1 ba6817fee5 iris: run-headless.sh --bin, for screenshotting a real binary not just an example
desktop-app (RUST.md's E4) is a real crate binary a person runs, not a
demo under examples/, and it needs its own argv (--ca, --link) to start
at all -- neither of which the script had a way to express. --bin swaps
`cargo build --example`/`target/debug/examples/NAME` for the `--bin`
equivalents; $RUN_HEADLESS_ARGS is word-split into the launched binary's
own argv, since no example ever needed one before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 12:55:52 -04:00
irisandClaude Fable 5.1 73ee63bc1b iris: desktop-app, a winit window for the transcript screen (RUST.md's E4)
The pass condition was the same screen, from the same crate, running in a
window with only the layout differing. desktop-app is a new workspace
member: a session list (plain iris::widget::Span, rebuilt on selection)
beside transcript_ui::build_tree's screen, talking to a real ai-server
through client-core's ApiClient/UreqTransport/follow_session_events, with
background network I/O on plain std::threads reporting back through
winit's EventLoopProxy rather than iris's Tasks (which only redraws once
per async closure, not once per SSE event).

Both pass-condition proofs held against app/ui-sandbox.sh's real server:
the list showed a spawned session, selecting it loaded its transcript, and
a message sent from the composer streamed its reply back live. Along the
way, a real bug: resuming the SSE stream from a folded item's seq (which
for a still-open assistant message is its *first* delta's seq by design)
replayed already-folded deltas and duplicated the tail of the reply --
found by a run-headless.sh screenshot, fixed by resuming from the raw wire
seq instead, and covered by a regression test.

Deliberately simple and said so in app.rs's module doc: every SSE event
refolds the whole transcript and rebuilds the right-hand tree from
scratch rather than reaching for TranscriptScreen::push_row's incremental
append, since a streaming reply is a row whose text keeps changing after
it appears and push_row can only add a new one. Fine at a desktop
session's scale; the real fix needs transcript-ui to expose updating a
row in place. Android is untouched by this step.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 12:55:41 -04:00
irisandClaude Fable 5.1 8f0aec449a transcript-ui: build_tree, the screen without claiming the window root (RUST.md's E4)
build() always finished by calling ui_state.set_root(), which is right for
a window that *is* the transcript screen and wrong for a caller embedding
it beside something else (the desktop app's session list). build_tree()
is build() minus that last step, returning the widget tree instead of
planting it; build() is now one line on top of it, so nothing else
changes for existing callers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 12:55:29 -04:00
irisandClaude Fable 5.1 6d5fd64bb0 client-core: EnrolledServer, the aiapp:// enrol-link parser (RUST.md's E4)
A Rust client needs the same host/port/token an Android phone gets from
scanning an aiapp://enroll?... QR, so a desktop build can enrol from the
identical text pasted rather than a second format invented for it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 12:55:22 -04:00
irisandClaude Fable 5.1 e5880c33f4 iris: DragArbiter closes I5's touch-drag pan-vs-select gap
A row's own click_or_drag() selection handler always won the same
gesture a list-level pan wanted, since run_sensors gives the inner
layer first refusal every frame it's pressed. DragArbiter
(iris/src/sense.rs) decides pan vs. select the way Android does:
vertical drag pans immediately, a held stationary press starts a
selection after LONG_PRESS, and a horizontal drag on already-selected
text extends immediately. transcript-ui's Selection::drag routes
every row's drag through one arbiter per list, driving List::scroll
for a pan instead of a second scroll mechanism.

8 new unit tests (iris::sense::drag_arbiter_tests); cargo
fmt/clippy/test --workspace and cargo ndk (iris, transcript-ui) all
clean; run-headless.sh screenshot byte-identical to before the change.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 12:26:54 -04:00
irisandClaude Fable 5.1 a853eb5a4d DECISIONS.md: the summary file for choices made without Iris; RUST.md: note the two in-flight pieces
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
2026-09-05 12:18:26 -04:00
iris 22d5c6585a Merge branch 'worktree-agent-a23aa965694b9eaf5' into rustify (I5: transcript-ui, SpanStyle) 2026-09-05 07:56:09 -04:00
irisandClaude Sonnet b063fbd7f9 RUST.md, IRIS.md, IRIS_TODO.md: record I5 -- transcript-ui built and
partial, the recommendation's numbers still missing

I5's own box: the seven "hard to get back" behaviours each shown or
given a sourced reason, the exact verification commands and results,
and what remains (Android integration, touch-drag-vs-selection
arbitration, row accessibility names, a tappable link, code-span chip,
selection's anchor-row shortcut, code-fence syntax highlighting) --
each also a dated IRIS_TODO.md item so it is not silently dropped.
Ticked [~] rather than [x]: the widget-tree half is built and tested,
the emulator half is not.

"Where things stand" and the Recommendation's item 3 updated in place
to say plainly that neither Masonry (E2) nor iris (I5) has produced a
render number yet, and why -- not a bad measurement, no measurement
obtainable yet on either side -- with the structural findings that do
exist (iris now does cross-row selection and per-span inline rich text,
neither of which exists in masonry/masonry_core/xilem today) recorded
as what currently favours iris absent a number.

IRIS.md gets SpanStyle's own entry: what changed, why, and the one
thing a future TextBuilderOutput impl must remember (both TextOutput
and TextEditOutput apply .spans() -- this box shipped the bug of
missing one of the pair once already).

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 07:55:04 -04:00
irisandClaude Sonnet 3f25e7ebca iris: transcript-ui, the transcript screen (RUST.md's I5)
A new workspace member, iris/transcript-ui/, built the same way
tabs-ui is: generic over Rsc: HasEvents + Rsc::State: FocusHost, on
client-core/event-model by path (real code, matching E2's precedent).
Four modules:

- markdown.rs: CommonMark (pulldown-cmark) -> one plain string plus a
  Vec<SpanStyle>, so a row's headings/bold/italic/inline-code/links
  render inline inside one wrapped TextEdit rather than one widget per
  block -- the actual proof that iris can do what E2 found Masonry
  structurally unable to (masonry/src/widgets/text_area.rs's
  "TODO: RichTextInput").
- row.rs: one iris::widget::List row per folded TranscriptRow. A
  TranscriptRow::Tools group collapses to a summary and expands to
  every call's own tool/input/output on tap, using List::extent +
  note_tap for hold-the-edge exactly as list.rs's module doc describes.
- selection.rs: cross-row selection -- a drag that starts in one row's
  TextEdit and crosses into another's, coordinating each visible row's
  own select/select_all/deselect from one pointer gesture. The one
  Masonry's own text_area.rs cites as impossible (no
  SelectionContainer-shaped type anywhere in masonry/masonry_core/
  xilem).
- composer.rs: a growing multi-line composer with no fixed height,
  wired beside the list with .height(rest(1)) -- the real screen for
  IRIS_TODO.md's "input box" benchmark case.

9 new tests (5 pure markdown, 4 selection), all passing. Screenshotted
via run-headless.sh: real inline rich text visible (bold, italic,
inline code, a bigger bold heading, a coloured link, a monospaced
fenced block, a collapsed tool-call row).

What this box does not close, each recorded at its own point (RUST.md's
I5 box, IRIS_TODO.md's dated entries): no Android integration exists
yet for this screen (no cdylib/Gradle shell the way iris-android-app
wraps tabs-ui), so the emulator-side render-number pass condition was
not attempted; a touch-drag pan over a row's own text currently loses
gesture arbitration to that row's own drag-select (diagnosed and named,
not silently broken); row-level accessibility names, a tappable link, a
code-span background chip, and Selection's anchor-row shortcut are
scoped shortcuts recorded in place.

cargo fmt/build/clippy/test --workspace clean; cargo ndk -t x86_64
-P 26 build/clippy clean for both transcript-ui and iris.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 07:54:51 -04:00
irisandClaude Sonnet 0af4c88d08 iris: SpanStyle, per-range text styling (RUST.md's I5)
A TextBuffer used to have exactly one style for its whole string,
applied via parley's push_default. SpanStyle adds a second, optional
layer -- a byte range plus whichever of colour/family/font size/
bold/italic/underline it overrides, pushed with parley's own
push(property, range) -- so a heading, bold, inline code and a link can
each carry their own look inside one wrapped, selectable TextEdit. This
is the actual answer to RUST.md's E2 finding against Masonry
(TextArea::edit_styles() is one StyleSet for the whole editor).

PlacedGlyph gains a color field, read from parley's own per-run
Style::brush, and Painter::glyphs draws each glyph in its own colour
instead of one colour for the whole RenderedText.

Real bug found while wiring this into a live screen (not caught by any
test, since markdown's own tests only check string/range logic): spans
were threaded through TextOutput::run but not the sibling
TextEditOutput::run, so every editable field silently dropped them.
Fixed in build.rs; see IRIS.md's entry for why both call sites are a
pair to keep in sync.

cargo fmt/build/clippy/test --workspace and cargo ndk (iris,
iris-android excluded per its own workspace exclusion) all clean; 28
existing iris tests unaffected.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 07:54:28 -04:00
irisandClaude Sonnet ceabd00805 E5: package app/shellApp without Gradle (cargo xtask apk)
New xtask/ crate (no deps) runs cargo ndk -> javac -> d8 -> aapt2 ->
zipalign -> apksigner directly, signed with the same key build-apk.sh
uses. Both pass conditions proved on the ai-app-2 emulator: the xtask
APK installs over the Gradle-built shellApp, and the notification
service starts and posts a real notification while backgrounded.

Adds one printRuntimeClasspathJars task to shellApp/build.gradle.kts
(and a matching signingConfig) -- the one disclosed Gradle call the
xtask still makes, to resolve the AndroidX/​:link dependency graph.
That call's Kotlin compilation of :link as a side effect also answers
E3's open kotlinc question, so no Java port of ServerStore was needed.
Wires a second Apk component into .dev-updater.ron beside the existing
one. Full writeup in RUST.md's E5 box.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 07:11:44 -04:00
iris 32a5256a0d Merge branch 'worktree-agent-a466c08a4014dbcbe' into rustify (I4: AccessKit names) 2026-09-05 07:06:29 -04:00
irisandClaude Sonnet 4cfe0ef6e6 iris: I4 -- accessibility names via AccessKit
Builds one flat AccessKit tree (iris_core::ui::access::AccessTree) from
iris's own widget tree: a synthetic Role::Window root with every named
widget as a direct child, names from the existing `.label()`, roles from
a new Widget::access_role() (default Unknown, TextEdit overrides to
TextInput/MultilineTextInput), bounds from UiRenderState::window_region
so a moved subtree reports where it actually is. Modular the way input's
sense registry is: Widgets gained one HashSet<WidgetId> ("named"),
populated only by .label()/set_label and drained by free_next (the
existing removal path), and AccessTree walks only that set -- a widget
nobody named costs it nothing. Updates only when the named set's name,
role or bounds actually changed, with a rebuild counter mirroring
take_counters (confirmed 1/0/1 across first-draw/unchanged/moved in
access_tests.rs).

Pushed through accesskit_winit on the desktop (DefaultApp::new now
creates the window hidden, builds the adapter, then shows it, per that
constructor's requirement) and accesskit_android on Android
(IrisViewPeer now implements AccessibilityNodeProvider). Both action
handlers are inert on purpose: AGENTS.md's tap-by-name is a real touch
at the node's bounds, not an AccessKit action request, so the ordinary
pointer path already answers it once bounds are right. E1's
detach-abort mitigation is carried into android/access.rs's
raise_if_enabled, which gates every QueuedEvents::raise on
AccessibilityManager.isEnabled().

tabs-ui's five switch buttons now carry .label()s matching their
on-screen text, giving both the desktop run and the emulator step real
names to find.

Verified on host: cargo fmt/build/clippy/test all clean (28 tests, 3
new), cargo ndk build+clippy clean for iris and iris-android-app,
run-headless.sh tabs --shot byte-identical to I2's prior screenshot
(27266 bytes). Not run: the emulator step (ui-trace tap-by-name against
iris-android-app), held by another session this pass -- exact commands
recorded in RUST.md's I4 box.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 07:05:41 -04:00
irisandClaude Sonnet c9b273ff16 E3: the Kotlin/Java shell over a JNI bridge into Rust (RUST.md)
Two Java classes (MainActivity, NotificationService) hand their lifecycle
to a new android-shell crate built on client-core; client-core gains
notifications.rs (the /notifications SSE parse and attention_line, ported
from Notifications.kt). Packaged as a new app/shellApp Gradle module
rather than a rewrite of app/androidApp in place, so that module's working
Compose UI is untouched.

Both pass conditions held on the emulator: a notification arrived in
Android's drawer with the app closed, and a shared text share landed as a
real message in a sandbox session's transcript. Found and fixed three
real bugs along the way (a silently-wrong JNI signature from a generic
JObject parameter, a class-by-name lookup failing on this crate's own
background thread for lack of an app ClassLoader, and onStartCommand
opening two /notifications connections per enrollment -- the last a
latent bug in Notifications.kt itself). Full account, exact commands and
what was deliberately cut are in RUST.md's E3 box.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 06:47:32 -04:00
iris 8adda94a7a Merge branch 'worktree-agent-a33c31aef1fd6d868' into rustify (I3: iris::widget::List) 2026-09-05 06:29:08 -04:00
irisandClaude Sonnet 3a9208f38b RUST.md, IRIS.md: record I3 -- List built and benchmarked, emulator step named
Ticks I3's box with the numbers (all flat across N as required),
updates "Where things stand", and adds IRIS.md's public-API entry for
List plus the fill-shaped-background lesson. The remaining emulator
comparison against transcript-bench.sh needs List wired into an actual
transcript/session screen (closer to I5's scope than I3's), so it's
recorded as the next step with the exact command rather than left
silently undone.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 06:27:58 -04:00
irisandClaude Sonnet e898370bf4 iris: fix List placing a fill-shaped background at its oversized measurement size
Building the I3 example (800 rows, some with images, styled with
.background(rect(tint))) surfaced a real bug: place()'s Bottom-known
branch measured a row at an oversized, fixed-size region and moved it
into its final box with reposition -- a pure translation. That is
correct for wrapped text, whose reported height doesn't depend on the
height it was offered, but Rect (used for every row's background) is
is_size_independent because it fills *whatever region it is given*,
so it painted at the oversized size and reposition never shrank it
back down. The screenshot showed one oversized tinted rectangle
covering the whole visible window instead of per-row backgrounds.

Fixed by caching each row's height once measured and placing an
already-measured row directly at its exact box (one widget_within/
reposition pass, same as any known-size placement) instead of
re-measuring every frame. A first-ever appearance still pays a
two-draw measurement (draw_twice), and a row whose real height
changed since it was cached is corrected the same frame it redraws
(not a one-frame lag) via an explicit reposition when the two
disagree. Steady-state scroll cost is unaffected: an unchanged row's
single placement call still hits draw_inner's existing skip-or-move
fast path.

Also fixes repair_anchor unconditionally re-snapping a bottom-anchored
list's offset to the viewport's edge on every frame snap_end was true
-- which discarded a live scroll() call the moment it ran, since
snap_end is only recomputed at the end of a layout pass and so still
read true from before the scroll. Now only re-snaps when the viewport
itself actually resized (tracked via last_viewport_len).

Added a_fill_shaped_background_is_not_left_oversized, a direct
regression test for the background bug (checks the background rect's
own painted pixel size, not just the row's reported extent, which was
already correct). cargo test -p iris (26 passed), clippy --all-targets
and --benches --release, fmt --all -- --check all clean. Rebenched:
all five scenarios still flat across N = 100/1,000/10,000 (numbers in
RUST.md's I3 box). Visually verified via
run-headless.sh message_list --shot, cropped with a throwaway PNG
decoder since no image tooling is installed here.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 06:26:02 -04:00
irisandClaude Sonnet 6e0bd06e4d RUST.md: E2 -- a transcript in Masonry on android-view, and the touch-scroll gap it found
Built a real transcript screen (e2-transcript, beside E1's demo in
~/src/android-view) against a live app/ui-sandbox.sh session through
client-core: real fold, real ApiClient, VirtualScroll<dyn Widget> over 854
events, block-level markdown via pulldown-cmark into Prose, and a tool row
that holds its top edge on expand via overwrite_anchor.

The headline result is negative and load-bearing: neither VirtualScroll nor
Portal reacts to a touch drag, only to wheel-style PointerEvent::Scroll
(virtual_scroll.rs:504-523, portal.rs:259-267), confirmed both by reading
and empirically (a real swipe and a synthetic Android scroll event both
moved nothing). That blocks transcript-bench.sh's own gesture, so the
render-number half of E2's pass condition has no comparison to make yet.
Selection across rows and per-span rich text are also confirmed impossible
on the pinned xilem commit, each cited to its source. Full writeup, repro
commands and screenshots list in E2's own box.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 06:03:42 -04:00
irisandClaude Sonnet 03da47e550 iris: benches/message_list.rs measures the real List, adds insert-above and expand-hold
The (a)/(b)/(c) scenarios built their own Span+Scroll pair, so they
never exercised the virtualised widget the transcript screen actually
needs. Rewritten on top of iris::widget::List, plus two new scenarios
from RUST.md's I3: (d) insert-above-anchor (paging older history onto
an already-scrolled list) and (e) expand-a-row-holding-its-edge
(list.rs's note_tap mechanism). Both come out flat across N =
100/1,000/10,000, as required.

Also fixes a real inefficiency this rewrite surfaced: List::place's
"generous" measurement bound was derived from viewport_len, so a
sibling resizing the list itself (the (c) scenario) changed that
bound every tick and defeated draw_inner's same-size fast path,
forcing a full redraw of every visible row instead of a move. It is
now a fixed module constant (GENEROUS_PADDING), independent of the
list's own size -- draws for (c) dropped from 3059 to 684 over 40
ticks.

cargo test -p iris (5 List tests still pass), cargo clippy
--all-targets and --benches --release, cargo fmt --all -- --check all
clean. Numbers recorded in RUST.md's I3 box.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 05:42:54 -04:00
irisandClaude Sonnet a2cd119985 iris: add List, a virtualised bottom-anchored list (RUST.md I3, part 1)
Variable-height rows, keyed by a u64, composed only while visible via
the existing draw_inner old-children diff (LAYOUT.md), moved not
re-laid-out on scroll (Painter::widget_within/reposition, an O(1)
offset write), a scroll anchor named by slot index so a row inserted
above costs one index increment rather than a content-offset
recompute, "more" sentinels as two ordinary optional widgets, and
"hold the edge nearest the tap" resolved in the layout pass before any
primitive is written for the frame.

cargo test -p iris (24 passed, 5 new), cargo clippy --all-targets and
cargo fmt --all -- --check clean.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 05:35:12 -04:00
irisandClaude Sonnet 19c36e37f2 iris: give masks/move_offsets their own bind group, fixing O(N) image append
GpuTextures folded the masks and move_offsets storage buffers into every
standalone image's own bind group (group 2), alongside that image's
texture view. Since ArrBuf::update hands back a new Buffer identity
whenever either buffer's length changes -- which a widget getting its
first move-offset slot can trigger, unrelated to any image -- every
live image's bind group had to be rebuilt whenever either buffer grew.
Appending a 1,001st image to 1,000 already-settled ones cost 1,001
bind-group creates, not 1 (IRIS_TODO.md, run-bench.sh images).

Moved both buffers into their own bind group (group 3 in shader.wgsl
and UiRenderNode), bound once per frame in draw() rather than once per
per-image bind group. GpuTextures's image bind groups now only
reference the atlas array view, the image's own view and the sampler --
none of which change when masks/move_offsets resize -- so a resize
touches exactly one bind group regardless of how many images are live.
This also closes the "two frames to reach steady state" item, which was
the same bug measured a second way.

Verified: cargo build/clippy/test clean (19 tests), cargo ndk build/clippy
clean, run-headless.sh tabs --shot byte-identical (27266 bytes). New
run-bench.sh images numbers: cold load unchanged at 1000/0/0/0, append
now 1 instead of 1001. Both Fix items in IRIS_TODO.md ticked with the
before/after numbers.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 05:28:38 -04:00
irisandClaude Sonnet e2873df92e iris: fix I2's render gap -- window uniform never left (0, 0) on android-view
UiRenderNode::new seeded the GPU window uniform from
WindowUniform::default() rather than the surface's real size, so
shader.wgsl's vertex stage divided every primitive's position by
(0, 0) and produced NaN/Inf clip coordinates on both Vulkan and GLES.
winit's backend never hit this because winit fires an initial
WindowEvent::Resized that corrects the uniform before the first frame;
android-view has no equivalent event, so the node it built never got
corrected. Seed the uniform from the SurfaceConfiguration passed to
UiRenderNode::new instead, which is already right on both backends at
construction time.

Verified on the ai-app-2 emulator (Vulkan/SwiftShader and, temporarily
forced, GLES/virgl): the tabs example now draws its widgets instead of
just the clear colour. Ticks I2 in RUST.md.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 05:19:55 -04:00
iris ea13889a21 Merge branch 'worktree-agent-a23e63cec86723942' into rustify 2026-09-05 05:09:20 -04:00
irisandClaude Sonnet 6317685d1a iris: android-app's Gradle shell, and the emulator run for I2
The Gradle side of RUST.md's I2: MainActivity, IrisView (extending
android-view's RustView with the two native methods it has no hook
for -- window insets, and unregistering this view's entry in
iris::android::insets's side table), and RustView.java/
RustInputConnection.java vendored from android-view (no published AAR
to depend on) with one deliberate diff noted in a comment: mViewPeer
is protected rather than package-private, so a subclass in a different
package can reach it.

Measured on the emulator (x86_64, API 26, SwiftShader Vulkan):
dumpsys input_method shows the served InputConnection is ours, and
Gboard's suggestion strip reads real buffer content back through
text_before_cursor ("hi | Hi | HI" after typing "hi") -- the same bar
E1 set, met. Not met: nothing draws. The clear colour reaches the
screen (confirmed by swapping it to magenta) and the layout engine
reports the correct widget count and pixel regions (log::debug! calls
left in view.rs's render() for exactly this), but no primitive shows
up, on both Vulkan/SwiftShader and GLES/virgl. Root cause not found;
one unconfirmed lead (a GLES-only D2/D2Array warning that could point
at the glyph atlas) is written up in RUST.md's I2 rather than chased
into core/src/render/, which is mid-flight in a separate benchmark
branch this session.

I2 is therefore built and wired but not tickable -- RUST.md has the
full writeup, what was ruled out, and where to pick this up.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 05:08:25 -04:00
irisandClaude Sonnet f79bd7ca71 iris: add the android-app cdylib crate (I2, part 3)
iris/android-app is the concrete app RUST.md's I2 is judged against:
JNI_OnLoad, a Client implementing AndroidAppState, and new_view_peer
wrapping iris::android::new_peer's generic function in the plain
function pointer register_view_class needs. Its UI is tabs-ui::build,
unchanged from the winit example.

Deliberately excluded from the iris workspace (iris/Cargo.toml's new
`exclude`): android-view needs the NDK sysroot to link, so folding this
crate in would break `cargo build --workspace --all-targets` on the
host. It resolves as its own single-crate workspace instead, built
with `cd iris/android-app && cargo ndk -t x86_64 -P 26 build`.

Verified: cross-compiles and clippys clean for x86_64-linux-android
API 26; the host iris workspace (build/clippy/fmt/19 tests) is
unaffected. Not yet built: the Gradle shell (IrisView.java,
MainActivity, AndroidManifest) to actually install and run this on the
emulator -- next in RUST.md's I2.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 04:46:10 -04:00
irisandClaude Sonnet 9c935f8ce8 iris: factor the tabs example's widget tree into tabs-ui (I2, part 2)
The pass condition for android-view backend (RUST.md's I2) is that the
tabs example itself, text field included, runs there -- not a second
demo with the same shape. tabs-ui/src/lib.rs is that widget tree moved
out of examples/tabs/main.rs into a small crate generic over `Rsc:
HasEvents` and `Rsc::State: FocusHost`, so the winit example and the
upcoming android-app cdylib both call the same `build()` rather than
carrying two copies. Nothing in it names either backend.

Verified: the winit tabs example still renders pixel-identically via
run-headless.sh (27266 bytes, unchanged), tabs-ui cross-compiles clean
for x86_64-linux-android alongside iris, and host build/clippy/fmt/
tests are unaffected.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 04:42:13 -04:00
irisandClaude Sonnet 982449293d iris: add an android-view backend beside winit (I2, part 1)
Relocates the platform-neutral halves of the winit `default` backend
(WidgetState, CursorState/sense, Tasks, Selector/Selectable's focus
handling) into shared crate-root modules so both backends can use them
without duplication, generalizes Tasks' redraw nudge behind a
RequestRedraw trait instead of a concrete winit::window::Window, and
adds iris/src/android/: a second backend on android-view's ViewPeer --
wgpu on the view's surface, touch as a mouse-like cursor, an
InputConnection bridge onto TextEdit (I1's parley editor), and a
window-insets side channel since android-view has no hook for it.
winit's own Android support pulls in android-activity without a
selected backend feature, so `default`/`android` are now target-gated
rather than both compiled in; confirmed by cross-compiling before this
split (cargo ndk failed inside android-activity) and after (clean).

Host build/clippy/fmt/tests and the android (x86_64, API 26)
cross-compile of the iris crate are all clean; the winit tabs example
still renders via run-headless.sh. Not yet exercised: an actual
android-app crate and Gradle shell to run this on the emulator -- next
in RUST.md's I2.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 04:39:06 -04:00
irisandClaude Sonnet 288853c094 iris: on-demand message-list/image benchmarks, and two O(N) findings
IRIS_TODO.md's "Benchmarks" item: a message list of N wrapped-text rows
(first-frame cost), scrolling it, and growing an input box above which
the list must move rather than re-layout -- all as a plain, harness=false
`cargo bench` binary (iris/benches/message_list.rs) since UiRenderState
touches no GPU or window, chosen over criterion because every scenario
here reduces to a count take_counters already answers exactly, and a
new dependency wasn't worth it. Scroll (200 ticks) and the input-grow
case (40 lines) are flat across N=100/1,000/10,000: LAYOUT.md's O(1)
move chain holds.

The many-images case (d) needs a real wgpu device, so it's a headless
example (iris/examples/bench_images.rs) plus a new
GpuTextures/UiRenderNode counter, take_image_bind_group_creates,
mirroring take_counters. It found two real non-O(1) costs, recorded as
new Fix items rather than redesigned: bind-group creation takes two
frames to settle after a cold load instead of one, and appending a
single image to an already-loaded 1,000-image list rebuilds all 1,000
existing bind groups (masks/move_offsets buffer growth triggers
rebuild_image_bind_groups unconditionally).

run-bench.sh wraps both. Numbers and commands are in IRIS_TODO.md.

cargo fmt --all -- --check, cargo clippy --all-targets, and
cargo test --workspace (19 passed) all clean; benches are not run by
cargo test.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
2026-09-05 00:13:58 -04:00
101 changed files with 19193 additions and 545 deletions

No files matched your search

+6
View File
@@ -0,0 +1,6 @@
# xtask convention (https://github.com/matklad/cargo-xtask), without folding
# every crate in this repo into one workspace -- they are deliberately
# independent (see run-tests.sh, which cds into each). `cargo xtask apk`
# from the repo root runs xtask/src/main.rs directly.
[alias]
xtask = "run --quiet --manifest-path xtask/Cargo.toml --"
+17
View File
@@ -55,4 +55,21 @@ components: [
// the terminal the QR would be printed on.
enroll: "server/enroll-link.sh",
),
// E5 (RUST.md): app/shellApp packaged by the xtask instead of Gradle
// (cargo ndk -> javac -> d8 -> aapt2 -> zipalign -> apksigner), signed
// with the same release key as "app" above so the two can install
// over each other -- a separate component, not a mode of "app" above,
// because it is a different applicationId (com.example.aiapp.shell)
// built by a different tool from different sources. No `cwd`: it
// defaults to this checkout's root, which both the `cargo xtask`
// alias (`.cargo/config.toml`, resolved relative to the working
// directory cargo is run from) and `cargo xtask apk`'s own publishing
// step (`xtask/build/outputs/apk/<mode>/*.apk`, matching discover.rs's
// `*/build/outputs/apk/*/*.apk` pattern -- see apk.rs's module doc)
// both need.
Apk(
name: "shell",
modes: ["release", "debug"],
build: "cargo xtask apk",
),
],
+14
View File
@@ -1,6 +1,7 @@
.gradle/
build/
app/androidApp/build/
app/shellApp/build/
local.properties
.kotlin/
*.iml
@@ -9,6 +10,11 @@ local.properties
server/target/
event-model/target/
client-core/target/
android-shell/target/
# E3's native library, built by cargo-ndk straight into the Gradle module
# (RUST.md) -- an artifact, like server/target/ above, not source.
app/shellApp/src/main/jniLibs/
# Server logs from a development run (ai-server.log by convention,
# wg-test.log from ./test-wg-tunnel.sh).
@@ -26,3 +32,11 @@ sessions/
# iris, the in-house UI library, is vendored at iris/ and built by cargo.
iris/target/
iris/android-app/target/
# E5's packaging xtask (RUST.md). `build/` above already covers
# xtask/build/outputs/apk (the published APK, see apk.rs's module doc).
# The repo root has no Cargo workspace, so this is xtask's own
# intermediate working files (target/xtask/apk/...), not a shared one.
xtask/target/
/target/
+16
View File
@@ -26,6 +26,7 @@ next (a Masonry or iris transcript screen, most likely).
| `api.rs` | `Api.kt` | Partial -- see below |
| `event_stream.rs` | `EventStream.kt` | Done |
| `transcript_fold.rs` | `TranscriptItems.kt`, `ToolRows.kt` | Partial -- see below |
| `config.rs` | `ServerConfig.kt`'s `handleEnrollment` | New, desktop-only so far -- see below |
| *(not started)* | `TranscriptSource.kt` | Not started |
| *(not ported, and may never be)* | `TranscriptUnits.kt` | Out of scope -- see below |
@@ -103,6 +104,21 @@ deciding how `event_model` itself represents "a shape I don't recognise"
-- a shared-model decision affecting `server/` too, not a `client-core`-only
fix, so it is recorded here rather than silently worked around.
## `config.rs`: `EnrolledServer`
`EnrolledServer` (host, port, bearer token) plus `parse_link`, which reads
the exact `aiapp://enroll?host=H&port=P&token=T` deep link
`wg-app-link`'s `enroll` mints and `ServerConfig.kt`'s `handleEnrollment`
parses on the phone -- so any Rust client enrols from the same text a
phone would scan as a QR, with no second format invented for it (RUST.md's
E4, DECISIONS.md 2026-09-05). Deliberately does not decide where it is
persisted or under what file permissions -- a phone seals its token in the
Android Keystore, `iris/desktop-app/src/config.rs` writes it to
`$XDG_CONFIG_HOME/ai-app-desktop/enrollment.json` at 0600 -- since that is
caller-specific (the code rules' "ask for the least you need"). Its only
caller today is `desktop-app`; a future Android build of this crate would
be a second one, not a reason to move the type.
## What is not started at all
- **`TranscriptSource.kt`** -- the layer that decides whether a page comes
+30
View File
@@ -0,0 +1,30 @@
# Decisions taken for Iris to review
Short list of design choices made by the design agent without asking, so
they can be judged and reversed later. Detail lives in RUST.md (and IRIS.md
for iris API changes); this file is only the summary. Newest first. Items
marked **DEFERRED** are ones the agent chose not to decide alone.
## 2026-09-05
- **Touch drag on a transcript row follows Android's own rule**: a vertical
drag pans the list immediately; a stationary press held 500 ms starts a
text selection which further dragging extends; a horizontal drag while
something is already selected extends that selection without the wait.
One `DragArbiter` per list decides it (`iris/src/sense.rs`). Chosen over a
"text layer always wins" or "list always wins" rule because either loses
one of the two gestures a reader expects.
- **E4's desktop shape is a new `iris/desktop-app` crate**: a winit window
holding `transcript-ui`'s screen beside a session list, talking to a real
`ai-server` through `client-core`. It enrols by pasting the same
`aiapp://enroll?…` link a phone scans (`client-core::config::EnrolledServer`)
and keeps it owner-only under `$XDG_CONFIG_HOME/ai-app-desktop/`. The
pinned CA is a path given on the command line, not baked in. Chosen so
the phone and desktop share one enrolment format and no second one is
invented.
- **Order of remaining work**: finish the two in-flight pieces above, then
the transcript screen's Android integration and the `transcript-bench.sh`
comparison against Compose — the numbers the recommendation still lacks.
- **DEFERRED — whether to commit to iris over Masonry for `ai-app`.** Waits
on the bench numbers above; RUST.md's recommendation says what the
measurements must show.
+198
View File
@@ -8,6 +8,204 @@ capability that moved. Small and trivial changes do not go here.
An entry gives the date, what changed, why, and a short before/after where
it helps judge the change without the session that made it. Newest first.
## 2026-09-05: `transcript_ui::build_tree` (RUST.md's E4)
`transcript_ui::build` claimed the whole window (`ui_state.set_root(tree)`)
as its last step, which is right for a window that *is* the transcript
screen (the winit example, an eventual Android cdylib) and wrong for the
desktop app, which puts a session list beside it. `build_tree` is `build`
minus that last step: it returns `(TranscriptScreen, StrongWidget)` instead
of just `TranscriptScreen`, and the caller decides where the tree goes —
into `ui_state.set_root`, or into a `WidgetPtr` alongside something else
(`iris/desktop-app`'s `rebuild_transcript`). `build` is now one line calling
`build_tree` and doing the `set_root` itself, so existing callers are
unaffected.
```rust
// before, and still available, for a caller that wants to *be* the window:
let screen = transcript_ui::build(rsc, &mut ui_state, rows);
// new, for a caller embedding the screen beside something else:
let (screen, tree) = transcript_ui::build_tree(rsc, rows);
some_widget_ptr(rsc).set(tree);
```
## 2026-09-05: `DragArbiter`, pan-vs-select for one shared touch gesture (RUST.md's I5)
New public type, `iris::sense::DragArbiter`. Why: a widget author who
registers both a list-level pan and a row-level drag-to-select on the same
touch gesture has no way to arbitrate between them — `core/src/sense.rs`'s
`run_sensors` always gives the innermost layer first refusal, so the inner
one wins every frame it is pressed, not just the frame the press started
(this is exactly what left transcript-ui's touch-drag panning unreachable
until now). `DragArbiter` is one small state machine, one instance per
gesture surface (a whole list, not per row), that a caller drives with its
own `press_start`/`update`/`release` calls and a caller-supplied `Instant`
(so it is unit-testable without a real clock or a render harness). It
decides the way Android itself does: an ordinary vertical drag pans
immediately; a stationary press held `LONG_PRESS` (500ms) starts a
selection, which any further drag then extends; a horizontal drag while
something is already selected extends it immediately, skipping the wait.
```rust
// One per list, held alongside whatever state coordinates the rows:
let mut arbiter = DragArbiter::new();
// On press-down:
arbiter.press_start(pos, Instant::now(), already_selected);
// Every frame the button/finger stays down:
match arbiter.update(pos, Instant::now()) {
DragOutcome::Pan(dy) => list.scroll(-dy),
DragOutcome::SelectStart => selection.begin(...),
DragOutcome::SelectExtend => selection.extend(...),
DragOutcome::Undecided => {}
}
// On release:
arbiter.release();
```
`transcript-ui`'s `Selection::drag` (`transcript-ui/src/selection.rs`) is
the reference caller: every row's `CursorSense::click_or_drag() |
CursorSense::unclick()` handler routes through one `Selection`-owned
arbiter instead of calling `begin`/`extend` directly, so a drag that starts
on a row's own rendered text now pans the list correctly instead of
always starting a selection. 8 new unit tests in `iris/src/sense.rs`'s
`drag_arbiter_tests` module.
## 2026-09-05: `SpanStyle`, per-range text styling (RUST.md's I5)
A `TextBuffer` used to have exactly one style (`TextAttrs`: colour, size,
family, ...) for its whole string, applied via `push_default` into parley's
ranged builder. `SpanStyle` is a second, optional layer: a byte range plus
whichever of colour/family/font size/bold/italic/underline it overrides,
pushed with parley's own `push(property, range)` instead. Why: a transcript
row's markdown (a heading, **bold**, `inline code`, a link) all inside one
wrapped paragraph needs each to carry its own look while the paragraph
still wraps and selects as a single buffer — the thing `masonry`'s
`TextArea` cannot do (`StyleSet` is one style for the whole editor,
`text_area.rs:43-44`'s `// TODO: RichTextInput`), and the reason this
existed at all.
```rust
let (text, spans) = transcript_ui::markdown::render_markdown(src, 16.0);
wtext(text)
.spans(spans) // new: TextBuilder::spans, on both Text and TextEdit
.editable(EditMode::MultiLine)
.add(rsc);
```
Two things a widget author should know before reaching for it:
- **Call `.spans()` before or after `.editable()`, both work** — the field
lives on `TextBuilder` itself, not either output type, and both
`TextOutput::run` and `TextEditOutput::run` apply it to the buffer via
`TextBuffer::set_spans`. **These two call sites are a pair**: adding a
third `TextBuilderOutput` impl without also calling `set_spans` there
reproduces the exact bug this box shipped once already (spans silently
dropped for `TextEdit`, found only by screenshotting, not by any test —
`markdown.rs`'s own unit tests check string/range logic, which is
correct in isolation and proves nothing about whether the render path
ever sees it).
- **Colour is now per-glyph, not per-buffer.** `PlacedGlyph` gained a
`color: UiColor` field (from parley's own per-run `Style::brush`), and
`Painter::glyphs` draws each glyph in its own colour instead of
`RenderedText::color` uniformly. `RenderedText::color` still exists (the
buffer's *base* colour, for a caller that wants it as a whole, e.g. to
tint a cursor) but no longer drives what a glyph actually renders as.
## 2026-09-05: accessibility names via AccessKit (RUST.md's I4)
`.label()` (already in `trait_fns.rs`, previously unused anywhere in-tree)
is now load-bearing: it's the one thing that puts a widget in the AccessKit
tree `iris_core::ui::access::AccessTree` builds and both backends push
out. A widget author who wants a control to be findable by name (and
tappable by name, through `ui-trace`/a real screen reader) calls `.label()`
on it; nothing else is required, and a widget nobody labels is invisible
to this system at zero cost, not just zero UI.
```rust
let button = rect(Color::LIME)
.on(CursorSense::click(), move |_, rsc| { ... })
.label("Add task"); // now findable by uiautomator/AccessKit as "Add task"
```
Two new things a widget author might touch directly:
- **`Widget::access_role(&self) -> accesskit::Role`**, default `Unknown`.
Override it if your widget has a real platform equivalent —
`TextEdit` now returns `TextInput`/`MultilineTextInput` by `EditMode`.
Only consulted for a widget that also has a `.label()`; an unlabelled
widget's `access_role` is never called.
- **`Widgets::named() -> impl Iterator<Item = WidgetId>`** — every widget
with an explicit label, for anything else that wants to walk the same
set `AccessTree` does.
Nothing about `Painter`, `draw`, or the layout/move machinery changed —
this sits entirely beside them, reading `resolved_region`'s output rather
than participating in producing it.
## 2026-09-05: `List`, a virtualised bottom-anchored list (RUST.md's I3)
A new widget, `iris::widget::List` (`iris/src/widget/list.rs` -- read its
module doc first), for the transcript's kind of screen: variable-height
rows, keyed by a `u64`, composed only while visible, moved rather than
re-laid-out on scroll, a scroll anchor that survives a row inserted above
it, "more" sentinels at each end, and "hold the edge nearest the tap" when
a row's height changes (`note_tap`, resolved in the layout pass).
```rust
let mut list = List::new(Axis::Y);
list.push_back(ListRow::new(key, row_widget)); // O(1)
list.push_front(ListRow::new(older_key, row)); // O(1), anchor unaffected
list.set_more_before(Some(spinner_widget)); // sentinel, drawn at the edge
list.note_tap(viewport_y); // before mutating a row's height
let (top, bottom) = list.extent(key).unwrap(); // last frame's on-screen box, if visible
```
Built entirely out of existing primitives (`Painter::widget`/`widget_within`/
`reposition`/`draw_twice`, and `draw_inner`'s own old-children diffing) --
no new mechanism was added to the render core for it. One correctness
lesson worth reading even for other widgets: a row that fills whatever
region it is offered (`Rect`, `is_size_independent`) cannot be measured at
a throwaway oversized region and then merely `reposition`ed into place --
`reposition` only ever writes an offset, never a size, so the oversized
primitive stays oversized. `List` fixes this by caching each row's real
height once measured and placing an already-known row directly at its
exact box; see `list.rs`'s `place` for the full reasoning and
`a_fill_shaped_background_is_not_left_oversized` for the regression test.
## 2026-09-05: a second backend (android-view), and what moved to make room for it
RUST.md's I2. Three changes a widget or app author would notice, all in
service of the same thing: `default` (winit) and the new `android`
(android-view) backends sharing what does not depend on windowing.
- **`Selector`/`Selectable`'s bound changed from `Rsc::State:
HasDefaultUiState` to `Rsc::State: FocusHost`** (new trait, `attr.rs`).
`HasDefaultUiState` still exists and still works — `default/attr.rs` now
implements `FocusHost` for anything that has it — so a winit app's
existing code is unaffected. An Android app implements `FocusHost` via
`HasAndroidUiState` instead. Affects only an app that referenced
`HasDefaultUiState` directly at a `Selectable`/`Selector` call site
rather than through `.attr::<Selectable>(())`, which nothing in-tree
does.
- **`Tasks::init` takes `Arc<dyn RequestRedraw>` instead of
`Arc<winit::window::Window>`.** `RequestRedraw` (`task.rs`) is one method,
`fn request_redraw(&self)`; `winit::window::Window` implements it
(`default/render.rs`), so `Tasks::init(window)` at a call site is
unchanged by inference. Only matters if something constructed a `Tasks`
directly rather than through `DefaultRsc`/`AndroidRsc`.
- **`TextEdit::apply_event`/`TextInputResult` are `#[cfg(not(target_os =
"android"))]`** — they take a `winit::event::KeyEvent`, which does not
exist on Android; `android/input.rs` drives the same primitives
(`backspace`/`delete`/`motion`/`insert`, all still unconditional) from
`ndk::event::Keycode` directly instead. New unconditional getters on the
way: `TextEdit::text()`/`selection_range()`/`caret()`, and
`TextEditCtx::delete_byte_range`/`set_cursor_byte` — the primitives
`android/ime.rs`'s `InputConnection` bridge needed and that were not
previously exposed publicly.
## 2026-09-04: `Widget::draw` reports the size it used; `desired_width`/`desired_height` are gone
A widget used to implement three methods (`draw`, `desired_width`,
+198 -10
View File
@@ -26,18 +26,206 @@ order and what "done" looks like. Tick and date them in place.
still reaches the button — confirmed to fail on the pre-fix code and
pass after.
- [x] **Appending one image to an already-loaded list rebuilds every other
image's bind group (2026-09-05, fixed 2026-09-05).** Found by the
benchmark below: `GpuTextures::update` (`core/src/render/texture.rs`)
triggered `rebuild_image_bind_groups` — a loop over *every live
standalone image*, rebuilding its `BindGroup` — whenever the shared
`masks` or `move_offsets` GPU buffer was resized (`masks_resized ||
moves_resized` in `UiRenderNode::update`, `core/src/render/mod.rs`), and
a widget getting its *first* move-offset slot (LAYOUT.md section 2 —
every widget gets one on first draw) could be exactly what grows that
buffer. So one new message with one new image, appended to a transcript
that already has N images loaded, did not cost O(1): it cost one
`create_image` for the new image plus one `make_image_bind_group` per
*existing* image, because the new widget's own move slot pushed the
arena past its capacity. Measured directly in
`iris/examples/bench_images.rs`: appending a 1,001st image to 1,000
already-settled ones reported **1,001** bind-group creates for that one
frame, not 1 (`./run-bench.sh images`, frame 5 in the transcript below).
**Fix**: `masks`/`move_offsets` never belonged in a standalone image's own
bind group (group 2) in the first place — the group also holds that
image's own texture view, which is the only thing that is genuinely
per-image, so a buffer shared by *everything* forced a rebuild of
*every* group the moment it moved. Gave masks/move_offsets their own
bind group (group 3 in `shader.wgsl` and `UiRenderNode`: `masks_layout`/
`masks_group`), bound once per frame in `UiRenderNode::draw` rather than
once per draw call, instead of duplicating them into every per-image
group. `GpuTextures` and its image bind groups now know nothing about
either buffer — `rebuild_image_bind_groups` is called only from
`grow_array` (the atlas array texture growing, which genuinely does
change what every image's own bind group must reference) — so a
masks/move_offsets resize now touches exactly one bind group, ever,
regardless of how many images are live. Numbers after the fix, same
benchmark and command:
./run-bench.sh images
frame=1 bind_group_creates=1000 (cold load, unchanged)
frame=2 bind_group_creates=0 (was 1000 -- see the item below)
frame=3 bind_group_creates=0
frame=4 bind_group_creates=0
(append one image here)
frame=5 bind_group_creates=1 (was 1001)
frame=6 bind_group_creates=0
`run-headless.sh tabs --shot` still 27266 bytes, byte-for-byte unchanged,
confirming the bind-group restructuring changed nothing about what is
drawn.
- [x] **Bind-group creation takes two frames to reach the steady state, not
one (2026-09-05, closed by the fix above, 2026-09-05).** Same benchmark:
loading 1,000 images cold used to report 1,000 creates on frame 1
(expected — `create_image`, one per new image) *and again* 1,000 on
frame 2, before settling to 0 from frame 3. This was `rebuild_image_bind_groups`
firing a second time for the same masks/move-offsets buffer-growth
reason as the item above, confirming the guess recorded here — the two
were exactly the same root cause measured two different ways. Frame 2
now reports 0 (see the numbers above); not a separate fix.
## Build
- [ ] **Benchmarks**, not unit tests, run on demand (a `benches/` or a
script under `iris/`, never in `cargo test`). The scenario that matters
most is a **message list** — chat apps and this app's transcript alike —
stressed with many messages and many images. One case in particular:
**resizing an input box** (typing enough text to grow it) that pushes a
long list of messages above it must stay very fast and recalculate
almost nothing — a move of everything above, not a re-layout. That is
exactly the O(1) move chain in LAYOUT.md; the benchmark is what proves
it. Done when the numbers are in this file with the command, and the
input-box case reports draws re-run, not just frame time.
- [x] **Benchmarks**, not unit tests, run on demand (2026-09-05; a
`benches/` or a script under `iris/`, never in `cargo test`). The
scenario that matters most is a **message list** — chat apps and this
app's transcript alike — stressed with many messages and many images.
One case in particular: **resizing an input box** (typing enough text to
grow it) that pushes a long list of messages above it must stay very
fast and recalculate almost nothing — a move of everything above, not a
re-layout. That is exactly the O(1) move chain in LAYOUT.md; the
benchmark is what proves it. Done when the numbers are in this file with
the command, and the input-box case reports draws re-run, not just frame
time.
**Built as two rigs**, chosen per scenario by whether a real `wgpu`
device is needed (`UiRenderState`/`Widgets` touch no GPU or window, so
most of this runs as an ordinary binary — the same property
`layout_tests.rs` relies on):
- `iris/benches/message_list.rs` — a plain `Instant`-timed binary
(`[[bench]] harness = false` in `iris/Cargo.toml`), not criterion: see
the file's own header for why (short version — every scenario here
reduces to a *count* `UiRenderState::take_counters` already produces,
which criterion's statistical machinery adds nothing to and which a
new dependency is not worth pulling in for). Covers (a) first-frame
cost of a message list of N wrapped-text rows (one in 20 also carrying
a small in-memory image) for N = 100/1,000/10,000; (b) per-frame cost
of scrolling that list, 200 ticks; (c) the input-box case — a
fixed-height field at the bottom of the screen growing by a line 40
times, with the message list above it filling the rest of the screen.
Run: `cd iris && cargo bench --bench message_list` (always release —
`cargo bench` builds the `bench` profile, which is optimized).
- `iris/examples/bench_images.rs` — needs a real device, so it runs
through `iris/run-headless.sh bench_images`, printing
`UiRenderNode::take_image_bind_group_creates()` (a new counter, added
in `core/src/render/texture.rs` and `core/src/render/mod.rs`,
mirroring `UiRenderState::take_counters`) each frame. Covers (d): 1,000
image rows, checked both cold (does bind-group creation reach zero
once loaded) and after appending one more image once settled (does
*that* stay cheap) — the second question is what actually matters for
a live transcript and is what turned up the two Fix items above.
- `iris/run-bench.sh [list|images]` runs either or both and is what to
run before/after touching `Scroll`, `Span`, `Sized`, the move-offset
chain, or `GpuTextures`.
**Numbers (2026-09-05, release, `cargo bench`/`run-headless.sh`, this
VM: AMD Ryzen 7 3800X, 8 cores, rustc 1.98.0 nightly-2026-09-03):**
cd iris && cargo bench --bench message_list
(a) first frame, N=100: 30.30ms draws=227 rewrites=15 moves=0
(a) first frame, N=1000: 186.04ms draws=2252 rewrites=150 moves=0
(a) first frame, N=10000:1770.36ms draws=22502 rewrites=1500 moves=0
(b) scroll, N=100/1000/10000, 200 ticks each:
draws=200 rewrites=0 moves=200 (identical at every N)
per-tick average: 0.0002ms (identical at every N)
(c) input grows 40 lines, N=100/1000/10000 rows above it:
draws=320 rewrites=40 moves=160 (identical at every N)
per-line average: 0.0012-0.0013ms (identical at every N)
cd iris && ./run-bench.sh images (2026-09-05, before the fix)
frame=1 bind_group_creates=1000 (cold load)
frame=2 bind_group_creates=1000 (see Fix item above)
frame=3 bind_group_creates=0
frame=4 bind_group_creates=0
(append one image here)
frame=5 bind_group_creates=1001 (see Fix item above)
frame=6 bind_group_creates=0
cd iris && ./run-bench.sh images (2026-09-05, after the fix)
frame=1 bind_group_creates=1000 (cold load, unchanged -- genuine work)
frame=2 bind_group_creates=0
frame=3 bind_group_creates=0
frame=4 bind_group_creates=0
(append one image here)
frame=5 bind_group_creates=1 (one image's own create_image, O(1))
frame=6 bind_group_creates=0
**Reading it**: (a) is real, necessary work — shaping and laying out N
never-before-seen text rows — and scales with N as it must, ~10x cost
per 10x N. (b) and (c) are the pass conditions that matter: both are
**exactly flat across N = 100 to 10,000**, confirming LAYOUT.md's O(1)
move chain holds for both scrolling and for a growing input box pushing
the message list — draws/moves per tick or per line do not grow with
list size, and the per-operation cost (a fraction of a microsecond) is
nowhere near a frame budget. (d)'s cold-load and steady-state halves
behave as designed; its *append* half did not, until the fix above moved
masks/move_offsets out of the per-image bind group — now flat at O(1)
the same way (b) and (c) are.
- **I5's transcript screen (`iris/transcript-ui/`, 2026-09-05) — what it
left, each recorded at the point in the code it would go rather than
silently dropped. See RUST.md's I5 box for the full account of what
*was* built (the screen, `SpanStyle`, cross-row selection, the growing
composer).**
- [ ] **Android integration for this screen does not exist yet.** No
cdylib/Gradle shell the way `iris-android-app` wraps `tabs-ui` (I2),
so `transcript-bench.sh`'s render-number pass condition against the
Compose baseline cannot be run. Needs: real `client-core::ApiClient`/
`event_stream::follow_session_events` wiring against
`app/ui-sandbox.sh --delay` (this crate deliberately fetches nothing
itself, `transcript-ui/src/lib.rs`'s doc), a new cdylib + Gradle
module, then the bench script pointed at it.
- [x] **Touch-drag panning over a row's own rendered text — done,
2026-09-05.** `row.rs` used to register `CursorSense::click_or_drag()`
on each row's `TextEdit` for cross-row selection; `TextEdit::draw`'s
`painter.child_layer()` (`iris/src/widget/text/edit.rs:87`) meant that
registration won `core/src/sense.rs::run_sensors`'s per-layer
arbitration on every frame it was pressed, not just the frame the
press started, so a list pan gesture registered on `List` itself never
got a turn while a row was under the finger. Fixed with
`iris::sense::DragArbiter` (recorded in `IRIS.md`), one small state
machine per list deciding pan vs. select the way Android does (a
vertical drag pans immediately; a stationary press held `LONG_PRESS`
(500ms) starts a selection which further drag extends; a horizontal
drag while something is already selected extends immediately) —
`transcript-ui/src/selection.rs`'s `Selection::drag` is the one place
every row's drag now routes through. 8 new unit tests
(`iris/src/sense.rs`'s `drag_arbiter_tests`); `cargo fmt/clippy/test
--workspace` and `cargo ndk` (both `iris` and `transcript-ui`) all
clean; `run-headless.sh` screenshot byte-identical to before the
change (38578 bytes). See RUST.md's I5 box, "Gap closed, 2026-09-05".
- [ ] **Row-level accessibility names.** The composer carries
`.label("Message")`; transcript rows do not carry a `.label()` of
their own yet, so `Widgets::named()` (I4) does not include them —
`row.rs`'s `build_text_row` is where one would go, keyed to something
stable per row (its sender + a short excerpt, matching what a screen
reader announcing a chat message would say).
- [ ] **A tappable link and a background chip behind inline code.**
Both need per-range glyph geometry that `TextEditCtx` does not expose
outside `iris::widget::text` (`edit.rs`'s `layout()` helper is
private) — see `markdown.rs`'s module doc for the exact shape the fix
would take (the same primitive `TextEdit::draw`'s own selection
highlight already uses internally,
`iris/src/widget/text/edit.rs:99`).
- [ ] **`Selection`'s anchor-row shortcut.** The row a drag started in
is selected in full (`select_all`) the moment the drag leaves it,
rather than "from the click point to whichever edge points away from
the drag" — needs the same private `layout()` access as the item
above. `selection.rs`'s module doc has the exact reasoning.
- [ ] **No syntax highlighting inside a fenced code block.**
`client_core::highlight` exists (built for the file explorer) and
could feed per-token `SpanStyle`s into a code block's span; wiring it
in was not attempted this pass.
- [ ] **Masks defined relative to each other.** Wanted: mask A multiplies
by something *and also* applies mask B — a mask can reference a parent
mask, the way the move chain references a parent offset. Today masks
+1536 -43
View File
File diff suppressed because it is too large. Load diff
+1081
View File
File diff suppressed because it is too large. Load diff
+36
View File
@@ -0,0 +1,36 @@
[package]
name = "android-shell"
version = "0.1.0"
edition = "2024"
# The JNI bridge behind E3's two Java stub classes (`MainActivity`,
# `NotificationService` -- see RUST.md's "How much Java is unavoidable" for
# why those two classes cannot be anything but Java/Kotlin, registered from
# the manifest by name). Everything they would otherwise have done in
# Kotlin -- the SSE follow loop, deciding where a notification is shown,
# picking a session for a share -- is here instead, built on `client-core`
# so the networking and parsing are not duplicated a third time next to the
# server and the Kotlin app.
#
# `cdylib` for `System.loadLibrary`; `lib` too so `cargo test`/`clippy` run
# on a normal host target without an Android NDK toolchain, the same
# posture `client-core` and `server` already have.
[lib]
name = "android_shell"
crate-type = ["cdylib", "lib"]
[dependencies]
client-core = { path = "../client-core" }
jni = "0.22"
log = "0.4"
# `LogErrorAndDefault` (the `native_method!` error policy this crate uses
# throughout, see lib.rs) logs through the `log` facade, which is a no-op
# without a backend installed -- so without this, every recoverable error
# at a native entry point would be silently dropped rather than reaching
# logcat. Android-only: nothing else here needs it, and it does not build
# off-device (see `notify::ensure_logger`'s call site, the only place this
# is used).
[target.'cfg(target_os = "android")'.dependencies]
android_logger = "0.15"
+152
View File
@@ -0,0 +1,152 @@
//! Thin wrappers around the five `Env` calls this crate makes constantly
//! (a class name, a method name and a signature, all as plain `&str`).
//!
//! `jni` 0.22 wants a class or method *name* as `AsRef<JNIStr>` (its own
//! modified-UTF-8 type; `JNIString::new` is the runtime conversion, used
//! here uniformly rather than switching to the compile-time `jni_str!`
//! literal macro call by call -- these are a handful of short, one-off
//! lookups, not a hot loop, so the difference is not worth two code paths
//! for the same thing) and a *signature* as a parsed `MethodSignature`/
//! `FieldSignature`, which is why those go through
//! `RuntimeMethodSignature`/`RuntimeFieldSignature::from_str` instead: the
//! parsed form is what lets these calls skip re-validating the signature
//! against the arguments on every call, which is the whole reason `jni`
//! moved to it.
//!
//! **The classloader gotcha, found by testing (2026-09-05).** A class
//! lookup by name (`find_class`, `new_object`, `call_static_method`,
//! `get_static_field` -- anything that resolves a *class*, as opposed to
//! `call_method` on an object it already has, which needs no such lookup)
//! defaults to `FindClass`'s ordinary search when it cannot find the
//! calling thread a classloader through `Thread.getContextClassLoader()`.
//! That default is fine on a thread the JVM itself started -- an
//! `onCreate`/`onStartCommand` callback -- but every one of these calls
//! from `android-shell`'s own background thread (the notification
//! follow-loop, the share upload) is running on a thread *Rust* spawned
//! and attached with `JavaVM::attach_current_thread`, which the platform
//! never gave an app classloader. Framework classes
//! (`android.app.Notification$Builder`, ...) still resolve, because they
//! are reachable from the bootstrap loader `FindClass` falls back to --
//! `androidx.core.app.NotificationManagerCompat` is not, since it is
//! packaged inside this app's own APK. The failure was
//! `Error::NoClassDefFound`, logged by `notify::show`'s `LogErrorAndDefault`
//! as "failed to resolve Java class ... (class not found or linkage
//! error)" -- on a real device this reads as "the notification silently
//! never arrives," since the whole call is inside the follow loop and the
//! ongoing foreground notification (built on the main thread, in
//! `try_start`, before the background thread exists) posts fine either
//! way. `remember_class_loader` caches the app's own `ClassLoader` the
//! first time any entry point has a `Context` to ask, and every class
//! lookup below goes through it explicitly via `LoaderContext::Loader`
//! rather than the thread-dependent default -- so it is correct on the
//! main thread and on this crate's own background threads alike.
use jni::Env;
use jni::errors::Result;
use jni::objects::{JClass, JClassLoader, JObject, JValue, JValueOwned};
use jni::refs::{Global, LoaderContext};
use jni::signature::{RuntimeFieldSignature, RuntimeMethodSignature};
use jni::strings::JNIString;
use std::sync::OnceLock;
static CLASS_LOADER: OnceLock<Global<JClassLoader<'static>>> = OnceLock::new();
/// Caches `context`'s own `ClassLoader`, the first time this is called.
/// Cheap to call from every entry point that has a `Context` on hand
/// (`MainActivity`'s and `NotificationService`'s all do): later calls are
/// a `OnceLock::get` and nothing else.
pub fn remember_class_loader(env: &mut Env, context: &JObject) -> Result<()> {
if CLASS_LOADER.get().is_some() {
return Ok(());
}
// context.getClass().getClassLoader() -- resolved via `call_method` on
// real objects throughout, so this needs no class-name lookup of its
// own and has nothing to bootstrap.
let class_obj = call_method(env, context, "getClass", "()Ljava/lang/Class;", &[])?.l()?;
let loader_obj = call_method(
env,
&class_obj,
"getClassLoader",
"()Ljava/lang/ClassLoader;",
&[],
)?
.l()?;
let loader = env.cast_local::<JClassLoader>(loader_obj)?;
let global = env.new_global_ref(&loader)?;
// Lost the race with another entry point calling this concurrently --
// both loaders name the same app, so either one is fine and there is
// nothing to reconcile.
let _ = CLASS_LOADER.set(global);
Ok(())
}
/// Resolves `name` (slash-separated, e.g. `androidx/core/app/NotificationCompat`)
/// through the cached app classloader when one has been remembered, and
/// through the ordinary default otherwise -- which is every call made
/// before any entry point has run, and is also correct for a main-thread
/// caller, so there is no case this makes worse.
fn resolve_class<'local>(env: &mut Env<'local>, name: &str) -> Result<JClass<'local>> {
match CLASS_LOADER.get() {
Some(loader) => {
let binary_name = name.replace('/', ".");
LoaderContext::Loader(loader).load_class(env, JNIString::new(&binary_name), true)
}
None => env.find_class(JNIString::new(name)),
}
}
pub fn find_class<'local>(env: &mut Env<'local>, name: &str) -> Result<JClass<'local>> {
resolve_class(env, name)
}
/// A new Java string as a plain `JObject` -- what every call site here
/// wants it as (`JValue::Object` takes `&JObject`, not `&JString`, and
/// `JString: Into<JObject>` is the documented way across).
pub fn jstr_obj<'local>(env: &mut Env<'local>, text: impl AsRef<str>) -> Result<JObject<'local>> {
Ok(env.new_string(text)?.into())
}
pub fn new_object<'local>(
env: &mut Env<'local>,
class: &str,
sig: &str,
args: &[JValue],
) -> Result<JObject<'local>> {
let sig = RuntimeMethodSignature::from_str(sig)?;
let class = resolve_class(env, class)?;
env.new_object(class, sig.method_signature(), args)
}
pub fn call_method<'local>(
env: &mut Env<'local>,
obj: &JObject,
method: &str,
sig: &str,
args: &[JValue],
) -> Result<JValueOwned<'local>> {
let sig = RuntimeMethodSignature::from_str(sig)?;
env.call_method(obj, JNIString::new(method), sig.method_signature(), args)
}
pub fn call_static_method<'local>(
env: &mut Env<'local>,
class: &str,
method: &str,
sig: &str,
args: &[JValue],
) -> Result<JValueOwned<'local>> {
let sig = RuntimeMethodSignature::from_str(sig)?;
let class = resolve_class(env, class)?;
env.call_static_method(class, JNIString::new(method), sig.method_signature(), args)
}
pub fn get_static_field<'local>(
env: &mut Env<'local>,
class: &str,
field: &str,
sig: &str,
) -> Result<JValueOwned<'local>> {
let sig = RuntimeFieldSignature::from_str(sig)?;
let class = resolve_class(env, class)?;
env.get_static_field(class, JNIString::new(field), sig.field_signature())
}
+131
View File
@@ -0,0 +1,131 @@
//! The JNI bridge behind E3's two Java stub classes. See `Cargo.toml`'s
//! package comment for what this crate is and RUST.md's E3 entry for the
//! design decisions.
//!
//! Each native method is declared with `jni`'s [`native_method!`] macro
//! rather than a hand-written `#[no_mangle] extern "system" fn Java_...`:
//! the macro derives the mangled export name and the JNI signature from the
//! Rust function itself, so the two cannot drift apart the way a
//! hand-typed name string and a hand-typed `"(Landroid/...;)V"` signature
//! routinely do. `error_policy = LogErrorAndDefault` matches
//! `Notifications.kt`'s own posture: a failure here (a lost connection, a
//! JNI call that threw) is reported to logcat, not thrown back into Java
//! as an exception that would crash the app over something recoverable.
//!
//! Each `const _: NativeMethod = native_method! { ... };` binding is
//! otherwise unused by name -- `_` is the idiomatic way to keep a
//! side-effecting const (here, generating the `#[export_name]`d function
//! the JVM resolves by the JNI naming convention) without a `dead_code`
//! warning for a binding nothing reads.
mod jcall;
mod notify;
mod settings;
mod share;
use jni::errors::LogErrorAndDefault;
use jni::objects::{JClass, JObject};
use jni::sys::jint;
use jni::{Env, NativeMethod, native_method};
/// Installs the `log` backend that routes to logcat, once per process.
/// Without it, `LogErrorAndDefault` (every native method below) and any
/// `log::error!` inside `jni` itself (e.g. `JString`'s `Display` fallback)
/// call into the `log` facade's default no-op logger, and a real failure
/// vanishes with nothing on logcat to say so -- silently *more* wrong than
/// crashing, since nothing on screen or in the log says a notification was
/// dropped. Called from every entry point below rather than a Java-side
/// `Application.onCreate`, since this crate deliberately has no such class
/// to hook (see RUST.md's E3 entry on the two-Java-classes floor).
fn ensure_logger() {
static ONCE: std::sync::Once = std::sync::Once::new();
ONCE.call_once(|| {
#[cfg(target_os = "android")]
android_logger::init_once(
android_logger::Config::default()
.with_max_level(log::LevelFilter::Debug)
.with_tag("android-shell"),
);
});
}
// The parameters are spelled as their Java types, not as `JObject`: the
// macro encodes each argument into the exported symbol's JNI signature
// (and JNI resolves `Java_...` names *by* that signature), so a generic
// `JObject` here would export `(Ljava/lang/Object;...)` against a Java
// method actually declared `(Landroid/app/Activity;...)` -- two different
// symbols that never resolve to each other, silently, with no compiler
// error on either side. `android.app.Activity` etc. have no dedicated
// Rust wrapper in this crate, so they fall back to plain `JObject` in the
// implementation functions below (the "Built-in Types" note in
// `native_method!`'s docs).
const _: NativeMethod = native_method! {
java_type = "com.example.aiapp.shell.MainActivity",
static extern fn native_handle_intent(activity: android.app.Activity, intent: android.content.Intent) -> (),
error_policy = LogErrorAndDefault,
};
/// `MainActivity.nativeHandleIntent` -- called from `onCreate` and
/// `onNewIntent`. See `share::handle_intent` for what an intent can mean.
fn native_handle_intent<'local>(
env: &mut Env<'local>,
_class: JClass<'local>,
activity: JObject<'local>,
intent: JObject<'local>,
) -> Result<(), jni::errors::Error> {
ensure_logger();
jcall::remember_class_loader(env, &activity)?;
share::handle_intent(env, &activity, &intent)
}
const _: NativeMethod = native_method! {
java_type = "com.example.aiapp.shell.NotificationService",
static extern fn native_sync(context: android.content.Context) -> (),
error_policy = LogErrorAndDefault,
};
/// `NotificationService.nativeSync` -- called both from `MainActivity` (an
/// enrollment may have just landed) and from `NotificationService.sync`
/// itself. See `notify::sync`.
fn native_sync<'local>(
env: &mut Env<'local>,
_class: JClass<'local>,
context: JObject<'local>,
) -> Result<(), jni::errors::Error> {
ensure_logger();
jcall::remember_class_loader(env, &context)?;
notify::sync(env, &context)
}
const _: NativeMethod = native_method! {
java_type = "com.example.aiapp.shell.NotificationService",
static extern fn native_on_start_command(service: android.app.Service) -> jint,
error_policy = LogErrorAndDefault,
};
/// `NotificationService.nativeOnStartCommand`. See `notify::on_start_command`.
fn native_on_start_command<'local>(
env: &mut Env<'local>,
_class: JClass<'local>,
service: JObject<'local>,
) -> Result<jint, jni::errors::Error> {
ensure_logger();
jcall::remember_class_loader(env, &service)?;
Ok(notify::on_start_command(env, service))
}
const _: NativeMethod = native_method! {
java_type = "com.example.aiapp.shell.NotificationService",
static extern fn native_on_destroy() -> (),
error_policy = LogErrorAndDefault,
};
/// `NotificationService.nativeOnDestroy`. See `notify::on_destroy`.
fn native_on_destroy<'local>(
_env: &mut Env<'local>,
_class: JClass<'local>,
) -> Result<(), jni::errors::Error> {
ensure_logger();
notify::on_destroy();
Ok(())
}
+556
View File
@@ -0,0 +1,556 @@
//! Where a notification is said, and the foreground service that keeps
//! the connection open while the app is closed. Ported from
//! `Notifications.kt`'s `NotificationService`, minus the "session on
//! screen" / "hand to the app as a banner" branches: those read
//! process-wide state that only exists because a screen is drawn to
//! register against, and this experiment draws no screen yet (that is
//! E4's job, on iris). So every notification here takes the third branch
//! Kotlin's `show` already had -- the platform's own drawer -- which is
//! also exactly the case E3's pass condition asks for: **a notification
//! arrives with the app closed.**
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::Duration;
use client_core::api::UreqTransport;
use client_core::notifications::{SessionNotification, follow_notifications};
use jni::Env;
use jni::errors::Result;
use jni::objects::{JObject, JValue};
use jni::sys::{JNI_TRUE, jint};
use crate::settings::{self, ServerSettings};
const ALERT_CHANNEL: &str = "sessions";
const ONGOING_CHANNEL: &str = "connection";
const ONGOING_ID: i32 = 1;
const ALERT_ID: i32 = 2;
/// Same backoff as `Notifications.kt`'s `RECONNECT_DELAY_MS`.
const RECONNECT_DELAY: Duration = Duration::from_millis(5_000);
/// Whether the follow-loop thread is already running. **A deviation from
/// `Notifications.kt`, found by testing rather than planned**: the Kotlin
/// `onStartCommand` spawns a fresh `thread(isDaemon = true) { follow(...) }`
/// on *every* call, with nothing to notice a previous one is still going --
/// and `sync()` calling `startForegroundService` when the service is
/// already running is an ordinary Android start, not a restart, so
/// `onStartCommand` runs again. Enrolling from `MainActivity` (which calls
/// `sync` once itself, then again inside `handle_enrollment` after saving
/// the token) hits exactly this path and was observed opening **two**
/// concurrent connections to `/notifications` from one process -- caught
/// on this build via `adb logcat` showing two `jni::vm::java_vm: Attached
/// thread ai-app-notifications` lines for one enrollment. Guarded here
/// rather than left to match Kotlin's behaviour exactly, since duplicating
/// a live connection is a resource leak with no upside; worth carrying the
/// same guard back to `Notifications.kt` separately.
static RUNNING: AtomicBool = AtomicBool::new(false);
/// Set by `nativeOnDestroy`, checked by the follow loop between
/// reconnects. **Known gap, recorded rather than hidden**: unlike
/// `HttpURLConnection.disconnect()` in the Kotlin original, nothing here
/// can interrupt a `ureq` read already blocked inside one connection --
/// `Transport::stream` hands back a plain `Read` with no cancellation
/// handle. So a stop lands at the next reconnect, not mid-read. `/notifications`
/// is idle between events (a keep-alive, per `server/src/routes.rs`), so in
/// practice this is a bounded wait rather than a hang; closing that gap
/// for real means adding a cancellation point to `client_core::Transport`,
/// which is a decision affecting every caller of that trait, not just this
/// one -- left for whoever next depends on prompt shutdown.
static STOPPING: AtomicBool = AtomicBool::new(false);
fn static_int(env: &mut Env, class: &str, field: &str) -> Result<i32> {
crate::jcall::get_static_field(env, class, field, "I")?.i()
}
fn notification_manager<'l>(env: &mut Env<'l>, context: &JObject) -> Result<JObject<'l>> {
crate::jcall::call_static_method(
env,
"androidx/core/app/NotificationManagerCompat",
"from",
"(Landroid/content/Context;)Landroidx/core/app/NotificationManagerCompat;",
&[JValue::Object(context)],
)?
.l()
}
fn create_channel(
env: &mut Env,
manager: &JObject,
id: &str,
name: &str,
importance: i32,
) -> Result<()> {
let id_j = crate::jcall::jstr_obj(env, id)?;
let builder = crate::jcall::new_object(
env,
"androidx/core/app/NotificationChannelCompat$Builder",
"(Ljava/lang/String;I)V",
&[JValue::Object(&id_j), JValue::Int(importance)],
)?;
let name_j = crate::jcall::jstr_obj(env, name)?;
crate::jcall::call_method(
env,
&builder,
"setName",
"(Ljava/lang/CharSequence;)Landroidx/core/app/NotificationChannelCompat$Builder;",
&[JValue::Object(&name_j)],
)?;
let channel = crate::jcall::call_method(
env,
&builder,
"build",
"()Landroidx/core/app/NotificationChannelCompat;",
&[],
)?
.l()?;
crate::jcall::call_method(
env,
manager,
"createNotificationChannel",
"(Landroidx/core/app/NotificationChannelCompat;)V",
&[JValue::Object(&channel)],
)?;
Ok(())
}
/// Two channels, because they are two different things to be told -- see
/// `Notifications.kt`'s `createChannels` for the reasoning; the names and
/// importances here are copied from it exactly, since a phone that has
/// seen both apps should not learn two different vocabularies for the
/// same fact.
fn create_channels(env: &mut Env, context: &JObject) -> Result<()> {
let manager = notification_manager(env, context)?;
let default = static_int(
env,
"androidx/core/app/NotificationManagerCompat",
"IMPORTANCE_DEFAULT",
)?;
let min = static_int(
env,
"androidx/core/app/NotificationManagerCompat",
"IMPORTANCE_MIN",
)?;
create_channel(
env,
&manager,
ALERT_CHANNEL,
"Sessions needing attention",
default,
)?;
create_channel(env, &manager, ONGOING_CHANNEL, "Staying connected", min)?;
Ok(())
}
fn new_intent_for<'l>(
env: &mut Env<'l>,
context: &JObject,
class_name: &str,
) -> Result<JObject<'l>> {
let target_class = crate::jcall::find_class(env, class_name)?;
crate::jcall::new_object(
env,
"android/content/Intent",
"(Landroid/content/Context;Ljava/lang/Class;)V",
&[JValue::Object(context), JValue::Object(&target_class)],
)
}
/// The intent a tap on an alert opens -- mirrors `Notifications.kt`'s
/// `sessionIntent`, including building the URI through `Uri.Builder`
/// rather than string concatenation, for the same reason: an id needing
/// escaping must survive the round trip.
fn session_intent<'l>(
env: &mut Env<'l>,
context: &JObject,
session_id: &str,
) -> Result<JObject<'l>> {
let intent = new_intent_for(env, context, "com/example/aiapp/shell/MainActivity")?;
let action_view = crate::jcall::jstr_obj(env, "android.intent.action.VIEW")?;
crate::jcall::call_method(
env,
&intent,
"setAction",
"(Ljava/lang/String;)Landroid/content/Intent;",
&[JValue::Object(&action_view)],
)?;
let builder = crate::jcall::new_object(env, "android/net/Uri$Builder", "()V", &[])?;
let scheme = crate::jcall::jstr_obj(env, settings::SCHEME)?;
crate::jcall::call_method(
env,
&builder,
"scheme",
"(Ljava/lang/String;)Landroid/net/Uri$Builder;",
&[JValue::Object(&scheme)],
)?;
let authority = crate::jcall::jstr_obj(env, "session")?;
crate::jcall::call_method(
env,
&builder,
"authority",
"(Ljava/lang/String;)Landroid/net/Uri$Builder;",
&[JValue::Object(&authority)],
)?;
let path = crate::jcall::jstr_obj(env, session_id)?;
crate::jcall::call_method(
env,
&builder,
"appendPath",
"(Ljava/lang/String;)Landroid/net/Uri$Builder;",
&[JValue::Object(&path)],
)?;
let uri = crate::jcall::call_method(env, &builder, "build", "()Landroid/net/Uri;", &[])?.l()?;
crate::jcall::call_method(
env,
&intent,
"setData",
"(Landroid/net/Uri;)Landroid/content/Intent;",
&[JValue::Object(&uri)],
)?;
Ok(intent)
}
fn pending_activity<'l>(
env: &mut Env<'l>,
context: &JObject,
intent: &JObject,
) -> Result<JObject<'l>> {
let update_current = static_int(env, "android/app/PendingIntent", "FLAG_UPDATE_CURRENT")?;
let immutable = static_int(env, "android/app/PendingIntent", "FLAG_IMMUTABLE")?;
crate::jcall::call_static_method(
env,
"android/app/PendingIntent",
"getActivity",
"(Landroid/content/Context;ILandroid/content/Intent;I)Landroid/app/PendingIntent;",
&[
JValue::Object(context),
JValue::Int(0),
JValue::Object(intent),
JValue::Int(update_current | immutable),
],
)?
.l()
}
fn builder_call<'l>(
env: &mut Env<'l>,
builder: &JObject<'l>,
method: &str,
sig: &str,
args: &[JValue],
) -> Result<()> {
crate::jcall::call_method(env, builder, method, sig, args)?;
Ok(())
}
/// The type Android 14+ requires a foreground service to declare, and
/// nothing before it -- mirrors `Notifications.kt`'s `foregroundType`.
fn foreground_type(env: &mut Env) -> Result<i32> {
let sdk = static_int(env, "android/os/Build$VERSION", "SDK_INT")?;
let upside_down_cake = static_int(env, "android/os/Build$VERSION_CODES", "UPSIDE_DOWN_CAKE")?;
if sdk >= upside_down_cake {
static_int(
env,
"android/content/pm/ServiceInfo",
"FOREGROUND_SERVICE_TYPE_SPECIAL_USE",
)
} else {
Ok(0)
}
}
fn ongoing_notification<'l>(env: &mut Env<'l>, context: &JObject) -> Result<JObject<'l>> {
let channel = crate::jcall::jstr_obj(env, ONGOING_CHANNEL)?;
let builder = crate::jcall::new_object(
env,
"androidx/core/app/NotificationCompat$Builder",
"(Landroid/content/Context;Ljava/lang/String;)V",
&[JValue::Object(context), JValue::Object(&channel)],
)?;
let title = crate::jcall::jstr_obj(env, "Watching for sessions that need you")?;
builder_call(
env,
&builder,
"setContentTitle",
"(Ljava/lang/CharSequence;)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Object(&title)],
)?;
let icon = static_int(env, "android/R$drawable", "stat_notify_sync")?;
builder_call(
env,
&builder,
"setSmallIcon",
"(I)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Int(icon)],
)?;
builder_call(
env,
&builder,
"setOngoing",
"(Z)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Bool(JNI_TRUE)],
)?;
let priority_min = static_int(env, "androidx/core/app/NotificationCompat", "PRIORITY_MIN")?;
builder_call(
env,
&builder,
"setPriority",
"(I)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Int(priority_min)],
)?;
crate::jcall::call_method(env, &builder, "build", "()Landroid/app/Notification;", &[])?.l()
}
/// Starts the service if there is a server to connect to, and stops it
/// otherwise -- mirrors `Notifications.kt`'s `NotificationService.sync`.
pub fn sync(env: &mut Env, context: &JObject) -> Result<()> {
let service_intent =
new_intent_for(env, context, "com/example/aiapp/shell/NotificationService")?;
if settings::load(env, context)?.is_none() {
crate::jcall::call_method(
env,
context,
"stopService",
"(Landroid/content/Intent;)Z",
&[JValue::Object(&service_intent)],
)?;
return Ok(());
}
create_channels(env, context)?;
crate::jcall::call_static_method(
env,
"androidx/core/content/ContextCompat",
"startForegroundService",
"(Landroid/content/Context;Landroid/content/Intent;)V",
&[JValue::Object(context), JValue::Object(&service_intent)],
)?;
Ok(())
}
/// The `Service.onStartCommand` body -- loads settings, starts the
/// foreground notification, and spawns the follow-loop thread. Answers the
/// platform's `START_STICKY`/`START_NOT_STICKY` constant, read from the
/// framework rather than hardcoded so a wrong guess at their values cannot
/// silently pick the other behaviour.
pub fn on_start_command(env: &mut Env, service: JObject) -> jint {
match try_start(env, &service) {
Ok(true) => static_int(env, "android/app/Service", "START_STICKY").unwrap_or(1),
Ok(false) => {
let _ = crate::jcall::call_method(env, &service, "stopSelf", "()V", &[]);
static_int(env, "android/app/Service", "START_NOT_STICKY").unwrap_or(2)
}
Err(e) => {
log_error(env, "onStartCommand", &e);
static_int(env, "android/app/Service", "START_NOT_STICKY").unwrap_or(2)
}
}
}
fn try_start(env: &mut Env, service: &JObject) -> Result<bool> {
let Some(settings) = settings::load(env, service)? else {
return Ok(false);
};
let ca = settings::load_pinned_ca(env)?;
let notification = ongoing_notification(env, service)?;
let fg_type = foreground_type(env)?;
crate::jcall::call_static_method(
env,
"androidx/core/app/ServiceCompat",
"startForeground",
"(Landroid/app/Service;ILandroid/app/Notification;I)V",
&[
JValue::Object(service),
JValue::Int(ONGOING_ID),
JValue::Object(&notification),
JValue::Int(fg_type),
],
)?;
// See `RUNNING`'s doc: a second `onStartCommand` while the loop from
// the first is still going -- the ordinary case for this service,
// since `sync()` is called from more than one place -- must not open
// a second connection.
if RUNNING.swap(true, Ordering::SeqCst) {
return Ok(true);
}
let vm = env.get_java_vm()?;
let context = env.new_global_ref(service)?;
STOPPING.store(false, Ordering::SeqCst);
std::thread::Builder::new()
.name("ai-app-notifications".to_string())
.spawn(move || {
// Requests a *permanent* attachment (detached only when this thread
// exits), matching the Kotlin original's `thread(isDaemon = true)`:
// this is the long-lived follow loop, not a one-shot callback.
let _: jni::errors::Result<()> = vm.attach_current_thread(|env| {
follow_loop(env, &context, settings, &ca);
Ok(())
});
})
.ok();
Ok(true)
}
/// Follows the backend's notification stream, reconnecting until stopped
/// -- mirrors `Notifications.kt`'s `follow`. A dropped connection is the
/// ordinary case, so it retries quietly and forever; nothing is shown when
/// it cannot connect, for the same reason as the Kotlin original: a
/// notification saying "I could not tell you whether anything happened" is
/// noise about a condition nobody can act on.
fn follow_loop(env: &mut Env, context: &JObject, settings: ServerSettings, ca: &[u8]) {
while !STOPPING.load(Ordering::SeqCst) {
if let Ok(transport) = UreqTransport::new(settings.base_url(), settings.token.clone(), ca) {
let _ = follow_notifications(&transport, |notification| {
if let Err(e) = show(env, context, &notification) {
log_error(env, "show", &e);
}
!STOPPING.load(Ordering::SeqCst)
});
}
if STOPPING.load(Ordering::SeqCst) {
return;
}
std::thread::sleep(RECONNECT_DELAY);
}
}
/// One notification per session, replacing that session's previous one --
/// mirrors `Notifications.kt`'s `show`, minus the on-screen/banner
/// branches this module's doc comment explains.
fn show(env: &mut Env, context: &JObject, notification: &SessionNotification) -> Result<()> {
let manager = notification_manager(env, context)?;
let sdk = static_int(env, "android/os/Build$VERSION", "SDK_INT")?;
let tiramisu = static_int(env, "android/os/Build$VERSION_CODES", "TIRAMISU")?;
let allowed = if sdk < tiramisu {
true
} else {
let permission = crate::jcall::jstr_obj(env, "android.permission.POST_NOTIFICATIONS")?;
let granted = static_int(
env,
"android/content/pm/PackageManager",
"PERMISSION_GRANTED",
)?;
let result = crate::jcall::call_static_method(
env,
"androidx/core/content/ContextCompat",
"checkSelfPermission",
"(Landroid/content/Context;Ljava/lang/String;)I",
&[JValue::Object(context), JValue::Object(&permission)],
)?
.i()?;
result == granted
};
let enabled =
crate::jcall::call_method(env, &manager, "areNotificationsEnabled", "()Z", &[])?.z()?;
if !allowed || !enabled {
return Ok(());
}
let intent = session_intent(env, context, &notification.session_id)?;
let pending = pending_activity(env, context, &intent)?;
let channel = crate::jcall::jstr_obj(env, ALERT_CHANNEL)?;
let builder = crate::jcall::new_object(
env,
"androidx/core/app/NotificationCompat$Builder",
"(Landroid/content/Context;Ljava/lang/String;)V",
&[JValue::Object(context), JValue::Object(&channel)],
)?;
let title = crate::jcall::jstr_obj(env, &notification.title)?;
builder_call(
env,
&builder,
"setContentTitle",
"(Ljava/lang/CharSequence;)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Object(&title)],
)?;
let text = crate::jcall::jstr_obj(env, notification.kind.attention_line())?;
builder_call(
env,
&builder,
"setContentText",
"(Ljava/lang/CharSequence;)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Object(&text)],
)?;
let icon = static_int(env, "android/R$drawable", "stat_notify_chat")?;
builder_call(
env,
&builder,
"setSmallIcon",
"(I)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Int(icon)],
)?;
builder_call(
env,
&builder,
"setContentIntent",
"(Landroid/app/PendingIntent;)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Object(&pending)],
)?;
builder_call(
env,
&builder,
"setAutoCancel",
"(Z)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Bool(JNI_TRUE)],
)?;
let when = (notification.at * 1000.0) as i64;
builder_call(
env,
&builder,
"setWhen",
"(J)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Long(when)],
)?;
builder_call(
env,
&builder,
"setShowWhen",
"(Z)Landroidx/core/app/NotificationCompat$Builder;",
&[JValue::Bool(JNI_TRUE)],
)?;
let built =
crate::jcall::call_method(env, &builder, "build", "()Landroid/app/Notification;", &[])?
.l()?;
let tag = crate::jcall::jstr_obj(env, &notification.session_id)?;
crate::jcall::call_method(
env,
&manager,
"notify",
"(Ljava/lang/String;ILandroid/app/Notification;)V",
&[
JValue::Object(&tag),
JValue::Int(ALERT_ID),
JValue::Object(&built),
],
)?;
Ok(())
}
/// Ends the follow loop -- mirrors `Notifications.kt`'s `onDestroy`, with
/// the gap this module's `STOPPING` doc explains.
pub fn on_destroy() {
STOPPING.store(true, Ordering::SeqCst);
// `RUNNING`'s path out. Same race as `STOPPING` itself (this doc's own
// comment): the old thread may still be inside a blocked read when a
// new `onStartCommand` follows immediately, which would spawn a
// second one before the first has actually stopped. Narrower than not
// resetting at all -- a service destroyed and never restarted would
// otherwise wedge `RUNNING` true forever -- and no worse than the
// known gap already accepted above.
RUNNING.store(false, Ordering::SeqCst);
}
pub fn log_error(env: &mut Env, where_: &str, error: &jni::errors::Error) {
let message = format!("android-shell: {where_}: {error}");
let _ = (|| -> Result<()> {
let tag = crate::jcall::jstr_obj(env, "android-shell")?;
let msg = crate::jcall::jstr_obj(env, &message)?;
crate::jcall::call_static_method(
env,
"android/util/Log",
"e",
"(Ljava/lang/String;Ljava/lang/String;)I",
&[JValue::Object(&tag), JValue::Object(&msg)],
)?;
Ok(())
})();
}
+141
View File
@@ -0,0 +1,141 @@
//! Enrollment: where the backend is, and the Keystore-sealed token to
//! reach it. This crate does not reimplement the Android Keystore AES-GCM
//! sealing in Rust -- it calls the same `wg-app-link` `ServerStore` Kotlin
//! class the production app already uses (see `ServerConfig.kt`), through
//! JNI, for two reasons: that code is shared with Dev Updater and already
//! tested, and the sealed value on a real phone is keyed to the exact
//! Keystore alias that class already uses -- reimplementing the crypto
//! here would either duplicate it or invalidate an existing enrollment.
use jni::Env;
use jni::errors::Result;
use jni::objects::{JObject, JString, JValue};
/// Where the backend is and how to authenticate to it -- the Rust twin of
/// `wg-app-link`'s `ServerSettings` data class, read back field by field
/// rather than kept as a live JNI reference, so it can cross a thread
/// boundary (a `JObject` is tied to one `Env`/thread).
#[derive(Debug, Clone)]
pub struct ServerSettings {
pub host: String,
pub port: i32,
pub token: String,
}
impl ServerSettings {
pub fn base_url(&self) -> String {
format!("https://{}:{}", self.host, self.port)
}
}
/// This experiment's own scheme and Keystore alias -- distinct from the
/// production app's (`aiapp` / `aiapp-token-key`) so the two can be
/// installed side by side on the same development device without
/// colliding over which one a scanned QR or a deep link resolves to. See
/// RUST.md's E3 entry for why they are not the same value.
pub(crate) const SCHEME: &str = "aiappshell";
const KEY_ALIAS: &str = "aiapp-shell-token-key";
const STORE_CLASS: &str = "com/example/wgapplink/ServerStore";
const SETTINGS_CLASS: &str = "com/example/wgapplink/ServerSettings";
fn new_store<'l>(env: &mut Env<'l>) -> Result<JObject<'l>> {
let scheme = crate::jcall::jstr_obj(env, SCHEME)?;
let alias = crate::jcall::jstr_obj(env, KEY_ALIAS)?;
crate::jcall::new_object(
env,
STORE_CLASS,
"(Ljava/lang/String;Ljava/lang/String;)V",
&[JValue::Object(&scheme), JValue::Object(&alias)],
)
}
fn read_settings(env: &mut Env, settings_obj: &JObject) -> Result<ServerSettings> {
let host = get_string(env, settings_obj, "getHost")?;
let port = crate::jcall::call_method(env, settings_obj, "getPort", "()I", &[])?.i()?;
let token = get_string(env, settings_obj, "getToken")?;
Ok(ServerSettings { host, port, token })
}
fn get_string(env: &mut Env, obj: &JObject, getter: &str) -> Result<String> {
let value = crate::jcall::call_method(env, obj, getter, "()Ljava/lang/String;", &[])?.l()?;
let jstr: JString = env.cast_local::<JString>(value)?;
jstr.try_to_string(env)
}
/// The stored enrollment, or `None` when there is not one -- mirrors
/// `ServerConfig.kt`'s `loadServerSettings`.
pub fn load(env: &mut Env, context: &JObject) -> Result<Option<ServerSettings>> {
let store = new_store(env)?;
let settings_obj = crate::jcall::call_method(
env,
&store,
"load",
"(Landroid/content/Context;)Lcom/example/wgapplink/ServerSettings;",
&[JValue::Object(context)],
)?
.l()?;
if settings_obj.is_null() {
return Ok(None);
}
Ok(Some(read_settings(env, &settings_obj)?))
}
/// Seals and stores `settings` -- mirrors `ServerConfig.kt`'s `saveServerSettings`.
pub fn save(env: &mut Env, context: &JObject, settings: &ServerSettings) -> Result<()> {
let store = new_store(env)?;
let host = crate::jcall::jstr_obj(env, &settings.host)?;
let token = crate::jcall::jstr_obj(env, &settings.token)?;
let settings_obj = crate::jcall::new_object(
env,
SETTINGS_CLASS,
"(Ljava/lang/String;ILjava/lang/String;)V",
&[
JValue::Object(&host),
JValue::Int(settings.port),
JValue::Object(&token),
],
)?;
crate::jcall::call_method(
env,
&store,
"save",
"(Landroid/content/Context;Lcom/example/wgapplink/ServerSettings;)V",
&[JValue::Object(context), JValue::Object(&settings_obj)],
)?;
Ok(())
}
/// Parses an `aiappshell://enroll?...` URI -- mirrors `ServerConfig.kt`'s
/// `parseEnrollmentUri`, asking the same Kotlin code that already owns the
/// query-parameter rules rather than re-deriving them here.
pub fn parse_enrollment_uri(env: &mut Env, uri: &JObject) -> Result<Option<ServerSettings>> {
let store = new_store(env)?;
let settings_obj = crate::jcall::call_method(
env,
&store,
"parseEnrollmentUri",
"(Landroid/net/Uri;)Lcom/example/wgapplink/ServerSettings;",
&[JValue::Object(uri)],
)?
.l()?;
if settings_obj.is_null() {
return Ok(None);
}
Ok(Some(read_settings(env, &settings_obj)?))
}
/// The CA this build pins, generated at build time the same way
/// `androidApp`'s `generatePinnedCert` task does (see `build.gradle.kts`)
/// but into a plain Java constant, since this module has no Kotlin of its
/// own to generate into.
pub fn load_pinned_ca(env: &mut Env) -> Result<Vec<u8>> {
let value = crate::jcall::get_static_field(
env,
"com/example/aiapp/shell/PinnedCa",
"PINNED_CA_PEM",
"Ljava/lang/String;",
)?
.l()?;
let jstr: JString = env.cast_local::<JString>(value)?;
Ok(jstr.try_to_string(env)?.into_bytes())
}
+183
View File
@@ -0,0 +1,183 @@
//! Deep links and the share sheet -- ported from `MainActivity.kt`'s
//! `handleIntent`/`onNewIntent` and `Share.kt`'s `sharedContent`.
//!
//! **Scope cut, recorded rather than silent**: only shared *text*
//! (`Intent.EXTRA_TEXT`) is attached to a session. `Attachments.kt`'s
//! upload path -- `ContentResolver` reads of a shared file/photo URI,
//! bitmap downscaling, EXIF rotation -- is real work of its own and is not
//! ported here, because `client-core`'s `ApiClient` does not have the
//! `/sessions/{id}/attachments` route yet either (see `CLIENT_CORE.md`'s
//! "not covered" list). So `ACTION_SEND`/`ACTION_SEND_MULTIPLE` with a
//! `content://` stream and no text falls through to a toast saying so,
//! rather than silently doing nothing. Closing this gap is the same
//! `client-core` work whichever caller needs it next.
//!
//! **Which session a share lands in** is also a placeholder: with no
//! screen drawn yet (E4's job), there is no picker to ask, so this attaches
//! to whichever session has the latest `last_activity` -- the one most
//! likely to be what somebody meant. Worth revisiting once a real screen
//! exists to ask instead of guessing.
use client_core::api::{ApiClient, UreqTransport};
use jni::Env;
use jni::errors::Result;
use jni::objects::{JObject, JString, JValue};
use crate::notify;
use crate::settings;
const ACTION_SEND: &str = "android.intent.action.SEND";
const ACTION_SEND_MULTIPLE: &str = "android.intent.action.SEND_MULTIPLE";
const ACTION_VIEW: &str = "android.intent.action.VIEW";
const EXTRA_TEXT: &str = "android.intent.extra.TEXT";
fn get_string_method(env: &mut Env, obj: &JObject, method: &str) -> Result<Option<String>> {
let value = crate::jcall::call_method(env, obj, method, "()Ljava/lang/String;", &[])?.l()?;
if value.is_null() {
return Ok(None);
}
let jstr: JString = env.cast_local::<JString>(value)?;
Ok(Some(jstr.try_to_string(env)?))
}
fn toast(env: &mut Env, context: &JObject, message: &str) -> Result<()> {
let message = crate::jcall::jstr_obj(env, message)?;
crate::jcall::call_static_method(
env,
"com/example/aiapp/shell/MainActivity",
"toast",
"(Landroid/content/Context;Ljava/lang/String;)V",
&[JValue::Object(context), JValue::Object(&message)],
)?;
Ok(())
}
/// The one place an incoming intent is sorted into what it means -- mirrors
/// `MainActivity.kt`'s `handleIntent`.
pub fn handle_intent(env: &mut Env, activity: &JObject, intent: &JObject) -> Result<()> {
let action = get_string_method(env, intent, "getAction")?;
if matches!(
action.as_deref(),
Some(ACTION_SEND) | Some(ACTION_SEND_MULTIPLE)
) {
return handle_share(env, activity, intent);
}
if action.as_deref() != Some(ACTION_VIEW) {
return Ok(());
}
let uri = crate::jcall::call_method(env, intent, "getData", "()Landroid/net/Uri;", &[])?.l()?;
if uri.is_null() {
return Ok(());
}
let scheme = get_string_method(env, &uri, "getScheme")?;
if scheme.as_deref() != Some(settings::SCHEME) {
return Ok(());
}
match get_string_method(env, &uri, "getHost")?.as_deref() {
Some("session") => handle_session_open(env, activity, &uri),
Some("enroll") => handle_enrollment(env, activity, &uri),
_ => Ok(()),
}
}
fn handle_session_open(env: &mut Env, activity: &JObject, uri: &JObject) -> Result<()> {
let Some(session_id) = get_string_method(env, uri, "getLastPathSegment")? else {
return Ok(());
};
// There is no session screen yet (E4's job); the toast is this
// experiment's stand-in proof that the tap was routed to the right
// session id.
toast(env, activity, &format!("Opened session {session_id}"))
}
fn handle_enrollment(env: &mut Env, activity: &JObject, uri: &JObject) -> Result<()> {
match settings::parse_enrollment_uri(env, uri)? {
Some(parsed) => {
settings::save(env, activity, &parsed)?;
notify::sync(env, activity)?;
toast(
env,
activity,
&format!("Enrolled with {}", parsed.base_url()),
)
}
None => toast(env, activity, "Not a valid enrollment code"),
}
}
/// The share sheet -- mirrors `Share.kt`'s `sharedContent` for what counts
/// as a share, and `AttachmentButton`'s upload-then-message pattern for
/// what happens to it, minus attachments per this module's doc comment.
fn handle_share(env: &mut Env, activity: &JObject, intent: &JObject) -> Result<()> {
let extra_text = crate::jcall::jstr_obj(env, EXTRA_TEXT)?;
let text = crate::jcall::call_method(
env,
intent,
"getStringExtra",
"(Ljava/lang/String;)Ljava/lang/String;",
&[JValue::Object(&extra_text)],
)?
.l()?;
let text = if text.is_null() {
None
} else {
let jstr: JString = env.cast_local::<JString>(text)?;
Some(jstr.try_to_string(env)?)
};
let Some(text) = text.filter(|t| !t.trim().is_empty()) else {
return toast(
env,
activity,
"Nothing to share -- only shared text is supported so far",
);
};
// Network I/O must not run on the calling thread: `handle_intent` is
// called from `onCreate`/`onNewIntent`, both on the main thread, and a
// blocking socket read there is a `NetworkOnMainThreadException`. So
// the actual send happens on a JNI-attached background thread, the
// same shape `notify::try_start`'s follow loop uses; `toast` from that
// thread is safe because `MainActivity.toast` itself hops back to the
// main looper (see that method).
let vm = env.get_java_vm()?;
let activity_ref = env.new_global_ref(activity)?;
std::thread::spawn(move || {
let _: jni::errors::Result<()> = vm.attach_current_thread(|env| {
share_in_background(env, &activity_ref, text);
Ok(())
});
});
Ok(())
}
fn share_in_background(env: &mut Env, activity: &JObject, text: String) {
let outcome = attach_to_a_session(env, activity, &text);
let message = match outcome {
Ok(title) => format!("Shared into \"{title}\""),
Err(message) => message,
};
let _ = toast(env, activity, &message);
}
fn attach_to_a_session(
env: &mut Env,
activity: &JObject,
text: &str,
) -> std::result::Result<String, String> {
let settings = settings::load(env, activity)
.map_err(|e| e.to_string())?
.ok_or_else(|| "Not enrolled yet".to_string())?;
let ca = settings::load_pinned_ca(env).map_err(|e| e.to_string())?;
let transport = UreqTransport::new(settings.base_url(), settings.token.clone(), &ca)
.map_err(|e| e.to_string())?;
let client = ApiClient::new(transport);
let sessions = client.fetch_sessions().map_err(|e| e.to_string())?;
let target = sessions
.into_iter()
.max_by(|a, b| a.last_activity.total_cmp(&b.last_activity))
.ok_or_else(|| "No session to share into".to_string())?;
client
.send_message(&target.id, text, &[])
.map_err(|e| e.to_string())?;
Ok(target.title)
}
+5
View File
@@ -17,6 +17,11 @@ dependencyResolutionManagement {
include(":androidApp")
// E3 (RUST.md): the Kotlin/Java shell over android-shell's JNI bridge, a
// separate module from :androidApp so the ~13,000 lines of working Compose
// UI there are untouched. See shellApp/build.gradle.kts's module comment.
include(":shellApp")
// The app half of wg-app-link, resolved by path through the submodule so
// this checkout and the crate it consumes move together -- the same
// arrangement `server/` uses for the Rust half. See that repo's README.
+163
View File
@@ -0,0 +1,163 @@
plugins { alias(libs.plugins.androidApplication) }
// E3 (RUST.md): the Kotlin/Java shell being replaced by a thin JNI bridge
// into Rust (`../../android-shell`). Deliberately its own module rather
// than a rewrite of `:androidApp` in place -- that module is ~13,000 lines
// of working Compose UI this experiment does not touch, and the two can be
// installed side by side on the same development device (see
// `settings.SCHEME`'s doc in `android-shell` for why the deep-link scheme
// and Keystore alias are not the production app's). No Compose plugin, no
// Kotlin source of its own: `MainActivity`/`NotificationService` are plain
// Java, and the CA constant below is generated as Java too.
//
// The CA this build pins is baked in the same way `androidApp`'s does --
// see that module's `build.gradle.kts` comment for the reasoning (the
// trust boundary follows the machine that builds, never a pasted copy).
// `PinnedCa.java`'s package must match `android-shell`'s
// `settings::load_pinned_ca` lookup (`com/example/aiapp/shell/PinnedCa`).
val pinnedCaPath: String =
System.getenv("AI_APP_CA")
?: "${System.getenv("XDG_CONFIG_HOME") ?: "${System.getProperty("user.home")}/.config"}" +
"/ai-app/certs/ca.pem"
abstract class GeneratePinnedCa : DefaultTask() {
@get:Input abstract val caPath: Property<String>
@get:InputFile
@get:Optional
@get:PathSensitive(PathSensitivity.NONE)
abstract val caCertificate: RegularFileProperty
@get:OutputDirectory abstract val outputDir: DirectoryProperty
@TaskAction
fun generate() {
val path = caPath.get()
val ca = File(path)
if (!ca.isFile) {
throw GradleException(
"No CA certificate at $path.\n" +
"Start ai-server (or app/ui-sandbox.sh) once on this machine first -- it " +
"generates the CA this build pins.\n" +
"Set AI_APP_CA=/path/to/ca.pem to build against a different one."
)
}
val pem = ca.readText().trim()
if (!pem.startsWith("-----BEGIN CERTIFICATE-----")) {
throw GradleException("$path is not a PEM certificate.")
}
val dir = outputDir.get().dir("com/example/aiapp/shell").asFile
dir.mkdirs()
// Same reasoning as androidApp's generatePinnedCert: the text block
// must start immediately after the opening `"""`, or
// CertificateFactory stops recognising the "-----BEGIN" preamble.
File(dir, "PinnedCa.java")
.writeText(
"""
|// Generated from $path by the generatePinnedCa task. Do not edit.
|package com.example.aiapp.shell;
|
|public final class PinnedCa {
| private PinnedCa() {}
| public static final String PINNED_CA_PEM = ""${'"'}
|$pem""${'"'};
|}
|"""
.trimMargin()
)
}
}
val generatePinnedCa =
tasks.register<GeneratePinnedCa>("generatePinnedCa") {
val ca = file(pinnedCaPath)
caPath.set(pinnedCaPath)
if (ca.isFile) {
caCertificate.set(ca)
}
}
android {
namespace = "com.example.aiapp.shell"
compileSdk = 37
defaultConfig {
applicationId = "com.example.aiapp.shell"
minSdk = 24
targetSdk = 37
versionCode = 1
versionName = "1.0"
}
// Same reasoning and same key as androidApp's (see that module's comment): E5 (RUST.md)
// signs its own, Gradle-free build with this same keystore, and the two can only
// `adb install -r` over each other if they carry the same certificate.
val keystore = System.getenv("AI_APP_KEYSTORE")
signingConfigs {
if (keystore != null) {
create("release") {
storeFile = file(keystore)
storePassword = System.getenv("AI_APP_KEYSTORE_PASSWORD")
keyAlias = "ai-app"
keyPassword = storePassword
}
}
}
buildTypes {
getByName("release") {
isMinifyEnabled = false
if (keystore != null) signingConfig = signingConfigs.getByName("release")
}
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_21
targetCompatibility = JavaVersion.VERSION_21
}
}
// E5 (RUST.md): the xtask dexes and packages this module's Java sources itself, but it does
// not resolve Maven dependencies -- reimplementing a dependency resolver was out of scope for a
// packaging step, so this one task is the single place Gradle still runs in that pipeline. It
// asks the dependency graph for the *post-transform* jars (AARs already unpacked to a classes
// jar, the same artifact type AGP's own dexing task consumes) rather than the raw configuration,
// which would hand back .aar files d8 cannot read directly.
val artifactType = Attribute.of("artifactType", String::class.java)
tasks.register("printRuntimeClasspathJars") {
description = "Writes the resolved release runtime classpath jars, one per line, for xtask."
val outputFile = layout.buildDirectory.file("xtask/runtime-classpath.txt")
outputs.file(outputFile)
val jars =
configurations
.getByName("releaseRuntimeClasspath")
.incoming
.artifactView { attributes.attribute(artifactType, "android-classes-jar") }
.files
// Captured as a plain FileCollection (not the ArtifactView itself, which the
// configuration cache cannot serialize) so this task is still cacheable.
inputs.files(jars)
doLast {
val file = outputFile.get().asFile
file.parentFile.mkdirs()
file.writeText(jars.joinToString("\n") { it.absolutePath })
}
}
androidComponents {
onVariants { variant ->
variant.sources.java?.addGeneratedSourceDirectory(generatePinnedCa, GeneratePinnedCa::outputDir)
}
}
dependencies {
// The Keystore-sealed enrollment (ServerStore/ServerSettings) --
// android-shell's settings.rs calls into this Kotlin class directly
// over JNI rather than re-sealing the token in Rust; see that file's
// module doc.
implementation(project(":link"))
// NotificationCompat/NotificationManagerCompat/NotificationChannelCompat/
// ServiceCompat -- android-shell's notify.rs calls these classes over
// JNI so the pre-26 fallback behaviour (no channels) lives once, in
// the library that already has it, rather than being re-derived as a
// set of Build.VERSION.SDK_INT branches in Rust.
implementation(libs.androidx.core.ktx)
}
+62
View File
@@ -0,0 +1,62 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android"
xmlns:tools="http://schemas.android.com/tools">
<!-- Mirrors androidApp's manifest (AGENTS.md: reuse it rather than
re-deriving it) for the permissions and declarations E3 actually
exercises. Not carried over: the QR scanner activity (this
experiment enrolls via the aiappshell://enroll deep link directly,
per AGENTS.md's ui-sandbox.sh banner) and the app icon warning
suppression below, for the same reason androidApp's is there. -->
<uses-permission android:name="android.permission.INTERNET" />
<uses-permission android:name="android.permission.ACCESS_LOCAL_NETWORK" />
<uses-permission android:name="android.permission.POST_NOTIFICATIONS" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE" />
<uses-permission android:name="android.permission.FOREGROUND_SERVICE_SPECIAL_USE" />
<application
android:label="AI Sessions (shell)"
android:allowBackup="true"
android:theme="@android:style/Theme.Material.Light.NoActionBar"
tools:ignore="MissingApplicationIcon">
<activity
android:name=".MainActivity"
android:exported="true"
android:launchMode="singleTop">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<!-- Enrollment: aiappshell://enroll?host=...&port=...&token=...,
per AGENTS.md's ui-sandbox.sh banner (fed to this app with
`adb shell am start -a android.intent.action.VIEW -d
'aiappshell://enroll?...'`, or -n'd at this component
directly if a second app also claims the aiapp scheme). -->
<intent-filter>
<action android:name="android.intent.action.VIEW" />
<category android:name="android.intent.category.DEFAULT" />
<category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="aiappshell" android:host="enroll" />
</intent-filter>
<!-- The share sheet - see android-shell's share.rs. -->
<intent-filter>
<action android:name="android.intent.action.SEND" />
<action android:name="android.intent.action.SEND_MULTIPLE" />
<category android:name="android.intent.category.DEFAULT" />
<data android:mimeType="*/*" />
</intent-filter>
</activity>
<!-- specialUse, not dataSync, for the reason androidApp's manifest
gives: a connection that has to keep listening overnight
cannot accept dataSync's six-hour cap. -->
<service
android:name=".NotificationService"
android:exported="false"
android:foregroundServiceType="specialUse">
<property
android:name="android.app.PROPERTY_SPECIAL_USE_FGS_SUBTYPE"
android:value="E3 experiment: holds one connection to the sandbox server so a
session that needs an answer can be reported while the app is closed." />
</service>
</application>
</manifest>
@@ -0,0 +1,50 @@
package com.example.aiapp.shell;
import android.app.Activity;
import android.content.Context;
import android.content.Intent;
import android.os.Bundle;
import android.os.Handler;
import android.os.Looper;
import android.widget.Toast;
/**
* E3's floor, per RUST.md's "How much Java is unavoidable": a class the framework
* constructs by name from the manifest, with its lifecycle methods handing straight to Rust
* (android-shell's {@code share::handle_intent}). No Compose, no layout -- there is no screen to
* draw yet (that is E4's job, on iris); {@link #toast} is this experiment's stand-in for showing
* something happened.
*/
public class MainActivity extends Activity {
static {
System.loadLibrary("android_shell");
}
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
NotificationService.sync(this);
nativeHandleIntent(this, getIntent());
}
// launchMode="singleTop": a notification tap or a share while this activity is already on
// top lands here rather than in a second instance -- same reasoning as MainActivity.kt's.
@Override
protected void onNewIntent(Intent intent) {
super.onNewIntent(intent);
setIntent(intent);
nativeHandleIntent(this, intent);
}
/**
* Called from android-shell, sometimes from a background thread (a share's network call is
* never made on the calling thread -- see share.rs). {@code Toast} itself is main-thread-only,
* so this hops there with a {@link Handler} rather than assuming the caller already has.
*/
static void toast(Context context, String message) {
new Handler(Looper.getMainLooper())
.post(() -> Toast.makeText(context, message, Toast.LENGTH_LONG).show());
}
private static native void nativeHandleIntent(Activity activity, Intent intent);
}
@@ -0,0 +1,45 @@
package com.example.aiapp.shell;
import android.app.Service;
import android.content.Context;
import android.content.Intent;
import android.os.IBinder;
/**
* E3's second unavoidable Java class (RUST.md): a foreground service constructed by the framework
* from the manifest, existing only to hand its lifecycle to android-shell's {@code notify} module
* -- the SSE follow loop, deciding what a notification says, and posting it are all Rust reached
* through these three native calls. See {@code Notifications.kt}'s {@code NotificationService} for
* the Kotlin original this mirrors.
*/
public class NotificationService extends Service {
static {
System.loadLibrary("android_shell");
}
@Override
public IBinder onBind(Intent intent) {
return null;
}
@Override
public int onStartCommand(Intent intent, int flags, int startId) {
return nativeOnStartCommand(this);
}
@Override
public void onDestroy() {
nativeOnDestroy();
}
/** Starts this service if there is a server to connect to, and stops it otherwise. */
static void sync(Context context) {
nativeSync(context);
}
private static native void nativeSync(Context context);
private static native int nativeOnStartCommand(Service service);
private static native void nativeOnDestroy();
}
+152
View File
@@ -0,0 +1,152 @@
//! What a Rust client needs to reach one enrolled server: host, port and
//! bearer token. Mirrors the shape `ServerConfig.kt`/`Api.kt`'s
//! `handleEnrollment` parses out of an `aiapp://enroll?host=H&port=P&token=T`
//! deep link -- the exact link `wg-app-link`'s `enroll` module mints and
//! `app/ui-sandbox.sh`'s banner prints, so any Rust client can enrol from
//! the same text a phone would scan as a QR, with no second format
//! invented for it (RUST.md's E4).
//!
//! What this type deliberately does not decide: where it is persisted, and
//! under what file permissions. A phone seals its token in the Android
//! Keystore; a desktop client has its own `$XDG_CONFIG_HOME/<app>/`
//! directory and its own file-mode conventions (MACHINE.md: owner-only,
//! never in the repo). Both are caller-specific, so they stay out of this
//! crate per the code rules' "ask for the least you need" -- see
//! `iris/desktop-app/src/config.rs` for the desktop instance.
use serde::{Deserialize, Serialize};
/// One enrolled server: reachable at `https://{host}:{port}`, authenticated
/// with `token` as a bearer header. Does not carry the pinned CA -- that is
/// a public certificate rather than a secret, and where to find it differs
/// by caller (a phone pins the one its APK was built against; a desktop
/// client is told a path).
#[derive(Debug, Clone, PartialEq, Serialize, Deserialize)]
pub struct EnrolledServer {
pub host: String,
pub port: u16,
pub token: String,
}
impl EnrolledServer {
/// Parses `aiapp://enroll?host=H&port=P&token=T` (query order does not
/// matter; unrecognised keys are ignored). `token` is percent-decoded,
/// since `ui-sandbox.sh` encodes it precisely because a raw token can
/// contain `+`, which turns into a space if left to a naive splitter.
pub fn parse_link(link: &str) -> Result<Self, String> {
let query = link.split_once('?').map(|(_, q)| q).ok_or_else(|| {
format!(
"'{link}' has no query string (expected \
aiapp://enroll?host=...&port=...&token=...)"
)
})?;
let mut host = None;
let mut port = None;
let mut token = None;
for pair in query.split('&') {
let Some((key, value)) = pair.split_once('=') else {
continue;
};
let value = percent_decode(value);
match key {
"host" => host = Some(value),
"port" => port = Some(value),
"token" => token = Some(value),
_ => {}
}
}
let host = host.ok_or_else(|| format!("'{link}' is missing 'host'"))?;
let port_str = port.ok_or_else(|| format!("'{link}' is missing 'port'"))?;
let port: u16 = port_str
.parse()
.map_err(|e| format!("'{link}''s port ('{port_str}') is not a number: {e}"))?;
let token = token.ok_or_else(|| format!("'{link}' is missing 'token'"))?;
Ok(Self { host, port, token })
}
/// Where a `client_core::api::UreqTransport` reaches this server.
pub fn base_url(&self) -> String {
format!("https://{}:{}", self.host, self.port)
}
}
fn percent_decode(s: &str) -> String {
let bytes = s.as_bytes();
let mut out = Vec::with_capacity(bytes.len());
let mut i = 0;
while i < bytes.len() {
if bytes[i] == b'%' && i + 2 < bytes.len() {
if let Ok(byte) =
u8::from_str_radix(std::str::from_utf8(&bytes[i + 1..i + 3]).unwrap_or(""), 16)
{
out.push(byte);
i += 3;
continue;
}
}
out.push(bytes[i]);
i += 1;
}
String::from_utf8_lossy(&out).into_owned()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_host_port_and_token() {
let server =
EnrolledServer::parse_link("aiapp://enroll?host=127.0.0.1&port=8547&token=abcDEF123")
.unwrap();
assert_eq!(
server,
EnrolledServer {
host: "127.0.0.1".to_string(),
port: 8547,
token: "abcDEF123".to_string(),
}
);
assert_eq!(server.base_url(), "https://127.0.0.1:8547");
}
#[test]
fn field_order_does_not_matter() {
let server =
EnrolledServer::parse_link("aiapp://enroll?token=tok&port=443&host=example.com")
.unwrap();
assert_eq!(server.host, "example.com");
assert_eq!(server.port, 443);
assert_eq!(server.token, "tok");
}
#[test]
fn a_percent_encoded_token_is_decoded() {
// ui-sandbox.sh's own reason for encoding: a raw '+' would
// otherwise arrive as a space.
let server =
EnrolledServer::parse_link("aiapp://enroll?host=h&port=1&token=a%2Bb%2Fc").unwrap();
assert_eq!(server.token, "a+b/c");
}
#[test]
fn a_missing_field_is_named_in_the_error() {
let err = EnrolledServer::parse_link("aiapp://enroll?host=h&port=1").unwrap_err();
assert!(
err.contains("token"),
"error should name the missing field: {err}"
);
}
#[test]
fn a_non_numeric_port_is_named_in_the_error() {
let err = EnrolledServer::parse_link("aiapp://enroll?host=h&port=x&token=t").unwrap_err();
assert!(
err.contains("port"),
"error should name the offending field: {err}"
);
}
}
+2
View File
@@ -4,8 +4,10 @@
pub mod ansi;
pub mod api;
pub mod config;
pub mod event_stream;
pub mod highlight;
pub mod notifications;
pub mod sse;
pub mod transcript_cache;
pub mod transcript_fold;
+162
View File
@@ -0,0 +1,162 @@
//! `GET /notifications`, the attention stream PLAN.md's "Notifications: two
//! places, never both" describes. Ported from the parsing half of
//! `app/.../Notifications.kt`'s `NotificationService` -- the framing
//! ([`crate::sse`]) and the wire shape ([`SessionNotification`],
//! [`NotificationKind`], mirroring `server/src/session/mod.rs`'s
//! `Notification`/`NotificationKind`).
//!
//! What is deliberately **not** here, because it is a decision rather than
//! logic: whether a given notification is shown at all (the session on
//! screen gets nothing), handed to the app as a banner, or posted to the
//! platform's own notification drawer. That three-way choice reads
//! process-wide state (what screen is open, whether the app is in front)
//! that has no meaning to a pure crate with no UI and no Android in it --
//! see `android-shell` for where it lives for this port.
use std::io::{BufRead, BufReader};
use serde::Deserialize;
use crate::api::{ApiError, Transport};
use crate::sse::SseReader;
/// One frame of `GET /notifications`, matching `server/src/session/mod.rs`'s
/// `Notification` field for field.
#[derive(Debug, Clone, PartialEq, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct SessionNotification {
pub session_id: String,
pub title: String,
pub kind: NotificationKind,
/// Epoch seconds, so a phone that was asleep can say how long ago.
pub at: f64,
}
/// Mirrors `server/src/session/mod.rs`'s `NotificationKind` -- serialized
/// the same way, so this deserializes the wire's `"awaitingInput"` /
/// `"finished"` directly rather than through a string match.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Deserialize)]
#[serde(rename_all = "camelCase")]
pub enum NotificationKind {
AwaitingInput,
Finished,
}
impl NotificationKind {
/// What a notification asks of the reader, in the words they see --
/// ported verbatim from `Notifications.kt`'s `attentionLine`. One
/// function because the same fact is shown in two places (the
/// platform's drawer and the app's own banner) and two mappings of one
/// word drift.
pub fn attention_line(self) -> &'static str {
match self {
NotificationKind::AwaitingInput => "Waiting for you",
NotificationKind::Finished => "Finished",
}
}
}
/// Follows `/notifications`, calling `on_notification` for each frame until
/// the connection drops or the callback asks to stop (by returning
/// `false`). Reconnecting is the caller's job -- mirroring
/// `NotificationService.follow`'s retry loop, which is a platform policy
/// (how long to wait, whether to give up) rather than parsing logic.
pub fn follow_notifications(
transport: &dyn Transport,
mut on_notification: impl FnMut(SessionNotification) -> bool,
) -> Result<(), ApiError> {
let body = transport.stream("/notifications")?;
let mut lines = BufReader::new(body).lines();
let mut reader = SseReader::new();
while let Some(line) = lines.next().transpose().map_err(|e| ApiError {
message: format!("Can't reach the server -- retrying. ({e})"),
status: None,
})? {
let Some(frame) = reader.feed_line(&line) else {
continue;
};
if frame.data.is_empty() {
continue;
}
let notification: SessionNotification =
serde_json::from_str(&frame.data).map_err(|e| ApiError {
message: format!("The server sent a notification this build couldn't parse: {e}"),
status: None,
})?;
if !on_notification(notification) {
return Ok(());
}
}
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
use crate::api::{Body, RawResponse};
use std::io::Cursor;
struct FixtureTransport {
body: &'static str,
}
impl Transport for FixtureTransport {
fn request(
&self,
_method: &str,
_path: &str,
_body: Option<Body>,
) -> Result<RawResponse, ApiError> {
unimplemented!("this fixture only serves a stream")
}
fn stream(&self, _path: &str) -> Result<Box<dyn std::io::Read + Send>, ApiError> {
Ok(Box::new(Cursor::new(self.body.as_bytes().to_vec())))
}
}
#[test]
fn a_notification_frame_parses_both_kinds() {
let transport = FixtureTransport {
body: "data:{\"sessionId\":\"s1\",\"title\":\"fix the bug\",\"kind\":\"awaitingInput\",\"at\":1.0}\n\n\
data:{\"sessionId\":\"s2\",\"title\":\"add tests\",\"kind\":\"finished\",\"at\":2.0}\n\n",
};
let mut seen = Vec::new();
follow_notifications(&transport, |n| {
seen.push((n.session_id, n.kind));
true
})
.unwrap();
assert_eq!(
seen,
vec![
("s1".to_string(), NotificationKind::AwaitingInput),
("s2".to_string(), NotificationKind::Finished),
]
);
}
#[test]
fn the_caller_can_stop_early() {
let transport = FixtureTransport {
body: "data:{\"sessionId\":\"s1\",\"title\":\"a\",\"kind\":\"finished\",\"at\":1.0}\n\n\
data:{\"sessionId\":\"s2\",\"title\":\"b\",\"kind\":\"finished\",\"at\":2.0}\n\n",
};
let mut count = 0;
follow_notifications(&transport, |_| {
count += 1;
count < 1
})
.unwrap();
assert_eq!(count, 1);
}
#[test]
fn attention_line_matches_the_kotlin_original() {
assert_eq!(
NotificationKind::AwaitingInput.attention_line(),
"Waiting for you"
);
assert_eq!(NotificationKind::Finished.attention_line(), "Finished");
}
}
+1162 -37
View File
File diff suppressed because it is too large. Load diff
+68 -3
View File
@@ -10,18 +10,77 @@ iris-core = { workspace = true }
iris-macro = { workspace = true }
parley = { workspace = true }
swash = { workspace = true }
winit = { workspace = true }
arboard = { workspace = true, features = ["wayland-data-control"] }
pollster = { workspace = true }
wgpu = { workspace = true }
image = { workspace = true }
accesskit = { workspace = true }
tokio = { workspace = true, features = ["sync", "rt", "rt-multi-thread"] }
# winit everywhere except Android; android-view (below) is what stands in
# for it there. Both backends live in this crate (see `src/android/mod.rs`'s
# doc comment) but are never compiled together: winit's own Android support
# pulls in `android-activity`, which panics at compile time unless one of
# its own backend features is picked, and picking one is exactly what
# `iris-core` was kept free of (RUST.md's I0b). Confirmed by trying it
# 2026-09-05: `cargo ndk -t x86_64 -P 26 build -p iris` failed inside
# `android-activity` itself with "Either game-activity or native-activity
# must be enabled" before this split existed.
[target.'cfg(not(target_os = "android"))'.dependencies]
winit = { workspace = true }
arboard = { workspace = true, features = ["wayland-data-control"] }
# I4 (RUST.md): the desktop half of the AccessKit push, `winit`'s own
# adapter over `accesskit`. No pin needed the way android-view's rev is
# pinned -- this is an ordinary crates.io release with no local abort to
# track (that finding is Android-only, see below).
accesskit_winit = "0.34.0"
# Pinned to the exact commit RUST.md's E1 (2026-09-04) measured on this
# emulator -- real Vulkan rendering, a working `InputConnection`, and the
# accesskit-detach abort, all against this rev specifically. Advancing it
# wants re-running E1's checks, the same reason the nightly toolchain pin
# is dated rather than floating.
[target.'cfg(target_os = "android")'.dependencies]
android-view = { git = "https://github.com/rust-mobile/android-view.git", rev = "bec6c62a96cef8239b0fd7fedeef9b184d02e3a1" }
# I4 (RUST.md): the Android half of the AccessKit push, over android-view's
# `AccessibilityNodeProvider`. **0.8.0 carries the same detach-abort E1
# found on 0.4.0** (the `State` enum still never returns to `Inactive`,
# and `send_completed_event` still unwraps a Java exception) -- advancing
# the version is not the fix, so pinning to a specific rev buys nothing
# here the way it does for android-view itself. `android/view.rs`'s
# `raise_if_enabled` is the mitigation, carried from E1.
accesskit_android = "0.8.0"
# Not re-exported by android-view (only `jni` and `ndk` are), and needed
# for `android/insets.rs`'s own id -> state map -- the same reason
# android-view's own `PEER_MAP` carries one.
send_wrapper = "0.6.0"
# For diagnostics visible through android_logger, wherever the app crate
# installs it -- this crate never installs a logger itself.
log = "0.4.28"
[dev-dependencies]
tokio = { workspace = true, features = ["sync", "rt", "rt-multi-thread", "time"] }
# The tabs example's widget tree. A dev-dependency cycle back to this
# package is fine -- cargo excludes dev-dependencies from the graph used
# to build the library itself, so this only matters for `--examples`.
tabs-ui = { path = "tabs-ui" }
# Plain Instant-timed binaries, not criterion -- see benches/message_list.rs's
# header for why. `harness = false` opts out of the unstable `#[bench]`
# test-crate harness cargo would otherwise want, in favour of an ordinary
# `fn main()`.
[[bench]]
name = "message_list"
harness = false
[workspace]
members = ["core", "macro"]
members = ["core", "macro", "tabs-ui", "transcript-ui", "desktop-app"]
# android-app pulls in android-view, which needs the NDK sysroot to link
# -- excluded so `cargo build --workspace --all-targets` on the host stays
# buildable. Cross-compile it from its own directory (its own single-crate
# workspace, since it has no `[workspace]` table of its own and this
# exclusion stops it inheriting this one): `cd android-app && cargo ndk
# -t x86_64 -P 26 build`.
exclude = ["android-app"]
[workspace.package]
version = "0.1.0"
@@ -37,6 +96,12 @@ parley = "0.11.1"
swash = "0.2.10"
fxhash = "0.2.1"
arboard = "3.6.1"
accesskit = "0.25.0"
iris-core = { path = "core" }
iris-macro = { path = "macro" }
tokio = "1.49.0"
# Current stable as of 2026-09-05 (`cargo search`) -- I5's markdown block
# model, the same crate E2's uncommitted `e2-transcript` experiment used for
# the identical job (RUST.md), rather than reimplementing a CommonMark
# parser.
pulldown-cmark = "0.13.4"
+6
View File
@@ -0,0 +1,6 @@
.gradle/
build/
app/build/
# Rebuilt by `cargo ndk -o app/src/main/jniLibs/ build` before every
# Gradle build -- see RUST.md's I2 for the exact command.
app/src/main/jniLibs/
+4855
View File
File diff suppressed because it is too large. Load diff
+29
View File
@@ -0,0 +1,29 @@
[package]
name = "iris-android-app"
version = "0.1.0"
edition = "2024"
# Deliberately outside the `iris` workspace (see that Cargo.toml's
# `[workspace] exclude`): this crate exists only to be cross-compiled with
# `cargo ndk` for the emulator/a phone, and pulls in android-view, which
# needs the NDK sysroot to link. Folding it into the main workspace would
# make `cargo build --workspace --all-targets` -- the host command RUST.md
# and AGENTS.md both require to stay clean -- try to link a cdylib against
# libraries that do not exist on this machine. See RUST.md's I2.
[lib]
name = "main"
crate-type = ["cdylib"]
[dependencies]
iris = { path = "../" }
tabs-ui = { path = "../tabs-ui" }
android-view = { git = "https://github.com/rust-mobile/android-view.git", rev = "bec6c62a96cef8239b0fd7fedeef9b184d02e3a1" }
android_logger = "0.15.0"
log = "0.4.28"
[profile.release]
panic = "abort"
[profile.dev]
panic = "abort"
+31
View File
@@ -0,0 +1,31 @@
plugins {
id("com.android.application")
}
// The Rust side (this directory's Cargo.toml) is built separately with
// `cargo ndk`, straight into src/main/jniLibs/ -- see the repo-root
// AGENTS.md-style comment at the top of Cargo.toml for why this crate
// stays outside the main Rust workspace, and RUST.md's I2 for the exact
// build command.
android {
namespace = "dev.iris.android.demo"
compileSdk = 37
defaultConfig {
applicationId = "dev.iris.android.demo"
minSdk = 26
targetSdk = 34
versionCode = 1
versionName = "1.0"
}
buildTypes {
debug {
}
}
compileOptions {
sourceCompatibility = JavaVersion.VERSION_17
targetCompatibility = JavaVersion.VERSION_17
}
}
@@ -0,0 +1,22 @@
<?xml version="1.0" encoding="utf-8"?>
<manifest xmlns:android="http://schemas.android.com/apk/res/android">
<application
android:allowBackup="true"
android:label="iris android-view demo"
android:theme="@android:style/Theme.Material.Light.NoActionBar">
<activity
android:name=".MainActivity"
android:configChanges="orientation|screenSize|screenLayout|keyboardHidden"
android:exported="true"
android:windowSoftInputMode="adjustResize">
<intent-filter>
<action android:name="android.intent.action.MAIN" />
<category android:name="android.intent.category.LAUNCHER" />
</intent-filter>
<meta-data android:name="android.app.lib_name" android:value="main" />
</activity>
</application>
</manifest>
@@ -0,0 +1,36 @@
package dev.iris.android.demo;
import android.content.Context;
import org.linebender.android.rustview.RustView;
/**
* android-view's abstract base plus the two native methods it has no hook
* for: window insets and unregistering this view's entry in
* iris::android::insets's side table. See iris/src/android/insets.rs's doc
* comment for why those could not ride along on an existing android-view
* callback the way the back gesture does.
*/
public final class IrisView extends RustView {
@Override
protected native long newViewPeer(Context context);
native void applyWindowInsetsNative(
long peer, int left, int top, int right, int bottom, int imeBottom);
native void unregisterInsetsNative(long peer);
public IrisView(Context context) {
super(context);
}
void applyWindowInsets(int left, int top, int right, int bottom, int imeBottom) {
applyWindowInsetsNative(mViewPeer, left, top, right, bottom, imeBottom);
}
@Override
protected void onDetachedFromWindow() {
unregisterInsetsNative(mViewPeer);
super.onDetachedFromWindow();
}
}
@@ -0,0 +1,47 @@
package dev.iris.android.demo;
import android.app.Activity;
import android.os.Build;
import android.os.Bundle;
import android.view.WindowInsets;
import android.widget.FrameLayout;
/**
* The android-view backend's demo activity (RUST.md's I2): one IrisView
* filling the window, running iris's tabs example through
* iris-android-app's Rust side. Mirrors android-view's own
* DemoActivity, plus the window-insets wiring that has no android-view
* counterpart.
*/
public final class MainActivity extends Activity {
static {
System.loadLibrary("main");
}
@Override
public void onCreate(Bundle state) {
super.onCreate(state);
IrisView view = new IrisView(this);
view.setLayoutParams(new FrameLayout.LayoutParams(
FrameLayout.LayoutParams.MATCH_PARENT, FrameLayout.LayoutParams.MATCH_PARENT));
view.setFocusable(true);
view.setFocusableInTouchMode(true);
FrameLayout layout = new FrameLayout(this);
layout.addView(view);
setContentView(layout);
view.requestFocus();
view.setOnApplyWindowInsetsListener((v, insets) -> {
int left = insets.getSystemWindowInsetLeft();
int top = insets.getSystemWindowInsetTop();
int right = insets.getSystemWindowInsetRight();
int bottom = insets.getSystemWindowInsetBottom();
int imeBottom = 0;
if (Build.VERSION.SDK_INT >= Build.VERSION_CODES.R) {
imeBottom = insets.getInsets(WindowInsets.Type.ime()).bottom;
}
((IrisView) v).applyWindowInsets(left, top, right, bottom, imeBottom);
return insets;
});
}
}
@@ -0,0 +1,153 @@
package org.linebender.android.rustview;
import android.os.Bundle;
import android.os.Handler;
import android.view.KeyEvent;
import android.view.inputmethod.CompletionInfo;
import android.view.inputmethod.CorrectionInfo;
import android.view.inputmethod.ExtractedText;
import android.view.inputmethod.ExtractedTextRequest;
import android.view.inputmethod.InputConnection;
import android.view.inputmethod.InputContentInfo;
class RustInputConnection implements InputConnection {
private final RustView mView;
RustInputConnection(RustView view) {
mView = view;
}
private long getViewPeer() {
return mView.mViewPeer;
}
@Override
public CharSequence getTextBeforeCursor(int n, int flags) {
return mView.getTextBeforeCursorNative(getViewPeer(), n);
}
@Override
public CharSequence getTextAfterCursor(int n, int flags) {
return mView.getTextAfterCursorNative(getViewPeer(), n);
}
@Override
public CharSequence getSelectedText(int flags) {
return mView.getSelectedTextNative(getViewPeer());
}
@Override
public int getCursorCapsMode(int reqModes) {
return mView.getCursorCapsModeNative(getViewPeer(), reqModes);
}
@Override
public ExtractedText getExtractedText(ExtractedTextRequest request, int flags) {
return null;
}
@Override
public boolean deleteSurroundingText(int beforeLength, int afterLength) {
return mView.deleteSurroundingTextNative(getViewPeer(), beforeLength, afterLength);
}
@Override
public boolean deleteSurroundingTextInCodePoints(int beforeLength, int afterLength) {
return mView.deleteSurroundingTextInCodePointsNative(getViewPeer(), beforeLength, afterLength);
}
@Override
public boolean setComposingText(CharSequence text, int newCursorPosition) {
return mView.setComposingTextNative(getViewPeer(), text.toString(), newCursorPosition);
}
@Override
public boolean setComposingRegion(int start, int end) {
return mView.setComposingRegionNative(getViewPeer(), start, end);
}
@Override
public boolean finishComposingText() {
return mView.finishComposingTextNative(getViewPeer());
}
@Override
public boolean commitText(CharSequence text, int newCursorPosition) {
return mView.commitTextNative(getViewPeer(), text.toString(), newCursorPosition);
}
@Override
public boolean commitCompletion(CompletionInfo text) {
return false;
}
@Override
public boolean commitCorrection(CorrectionInfo correctionInfo) {
return false;
}
@Override
public boolean setSelection(int start, int end) {
return mView.setSelectionNative(getViewPeer(), start, end);
}
@Override
public boolean performEditorAction(int editorAction) {
return mView.performEditorActionNative(getViewPeer(), editorAction);
}
@Override
public boolean performContextMenuAction(int id) {
return mView.performContextMenuActionNative(getViewPeer(), id);
}
@Override
public boolean beginBatchEdit() {
return mView.beginBatchEditNative(getViewPeer());
}
@Override
public boolean endBatchEdit() {
return mView.endBatchEditNative(getViewPeer());
}
@Override
public boolean sendKeyEvent(KeyEvent event) {
return mView.inputConnectionSendKeyEventNative(getViewPeer(), event);
}
@Override
public boolean clearMetaKeyStates(int states) {
return mView.inputConnectionClearMetaKeyStatesNative(getViewPeer(), states);
}
@Override
public boolean reportFullscreenMode(boolean enabled) {
return mView.inputConnectionReportFullscreenModeNative(getViewPeer(), enabled);
}
@Override
public boolean performPrivateCommand(String action, Bundle data) {
return false;
}
@Override
public boolean requestCursorUpdates(int cursorUpdateMode) {
return mView.requestCursorUpdatesNative(getViewPeer(), cursorUpdateMode);
}
@Override
public Handler getHandler() {
return null;
}
@Override
public void closeConnection() {
mView.closeInputConnectionNative(getViewPeer());
}
@Override
public boolean commitContent(InputContentInfo inputContentInfo, int flags, Bundle opts) {
return false;
}
}
@@ -0,0 +1,291 @@
package org.linebender.android.rustview;
import android.content.Context;
import android.graphics.Rect;
import android.os.Bundle;
import android.view.Choreographer;
import android.view.KeyEvent;
import android.view.MotionEvent;
import android.view.SurfaceHolder;
import android.view.SurfaceView;
import android.view.accessibility.AccessibilityNodeInfo;
import android.view.accessibility.AccessibilityNodeProvider;
import android.view.inputmethod.EditorInfo;
import android.view.inputmethod.InputConnection;
import android.view.inputmethod.InputMethodManager;
public abstract class RustView extends SurfaceView
implements SurfaceHolder.Callback, Choreographer.FrameCallback {
// Vendored from android-view (bec6c62, https://github.com/rust-mobile/android-view)
// with one deliberate change: `protected` rather than package-private, so a
// subclass in a different package (dev.iris.android.demo.IrisView) can pass
// it to the window-insets native call android-view itself has no hook for --
// see iris/src/android/insets.rs's doc comment for why that call exists at
// all. No other line differs from upstream.
protected final long mViewPeer;
final InputMethodManager mInputMethodManager;
protected abstract long newViewPeer(Context context);
public RustView(Context context) {
super(context);
mViewPeer = newViewPeer(context);
getHolder().addCallback(this);
mInputMethodManager =
(InputMethodManager) context.getSystemService(Context.INPUT_METHOD_SERVICE);
}
private native int[] onMeasureNative(long peer, int widthSpec, int heightSpec);
@Override
protected void onMeasure(int widthSpec, int heightSpec) {
int[] result = onMeasureNative(mViewPeer, widthSpec, heightSpec);
if (result != null) {
setMeasuredDimension(result[0], result[1]);
} else {
super.onMeasure(widthSpec, heightSpec);
}
}
private native void onLayoutNative(
long peer, boolean changed, int left, int top, int right, int bottom);
@Override
protected void onLayout(boolean changed, int left, int top, int right, int bottom) {
onLayoutNative(mViewPeer, changed, left, top, right, bottom);
super.onLayout(changed, left, top, right, bottom);
}
private native void onSizeChangedNative(long peer, int w, int h, int oldw, int oldh);
@Override
protected void onSizeChanged(int w, int h, int oldw, int oldh) {
onSizeChangedNative(mViewPeer, w, h, oldw, oldh);
super.onSizeChanged(w, h, oldw, oldh);
}
private native boolean onKeyDownNative(long peer, int keyCode, KeyEvent event);
@Override
public boolean onKeyDown(int keyCode, KeyEvent event) {
return onKeyDownNative(mViewPeer, keyCode, event) || super.onKeyDown(keyCode, event);
}
private native boolean onKeyUpNative(long peer, int keyCode, KeyEvent event);
@Override
public boolean onKeyUp(int keyCode, KeyEvent event) {
return onKeyUpNative(mViewPeer, keyCode, event) || super.onKeyUp(keyCode, event);
}
private native boolean onTrackballEventNative(long peer, MotionEvent event);
@Override
public boolean onTrackballEvent(MotionEvent event) {
return onTrackballEventNative(mViewPeer, event) || super.onTrackballEvent(event);
}
private native boolean onTouchEventNative(long peer, MotionEvent event);
@Override
public boolean onTouchEvent(MotionEvent event) {
return onTouchEventNative(mViewPeer, event) || super.onTouchEvent(event);
}
private native boolean onGenericMotionEventNative(long peer, MotionEvent event);
@Override
public boolean onGenericMotionEvent(MotionEvent event) {
return onGenericMotionEventNative(mViewPeer, event) || super.onGenericMotionEvent(event);
}
private native boolean onHoverEventNative(long peer, MotionEvent event);
@Override
public boolean onHoverEvent(MotionEvent event) {
return onHoverEventNative(mViewPeer, event) || super.onHoverEvent(event);
}
private native void onFocusChangedNative(
long peer, boolean gainFocus, int direction, Rect previouslyFocusedRect);
@Override
protected void onFocusChanged(boolean gainFocus, int direction, Rect previouslyFocusedRect) {
super.onFocusChanged(gainFocus, direction, previouslyFocusedRect);
onFocusChangedNative(mViewPeer, gainFocus, direction, previouslyFocusedRect);
}
private native void onWindowFocusChangedNative(long peer, boolean hasWindowFocus);
@Override
public void onWindowFocusChanged(boolean hasWindowFocus) {
super.onWindowFocusChanged(hasWindowFocus);
onWindowFocusChangedNative(mViewPeer, hasWindowFocus);
}
private native void onAttachedToWindowNative(long peer);
@Override
protected void onAttachedToWindow() {
super.onAttachedToWindow();
onAttachedToWindowNative(mViewPeer);
}
private native void onDetachedFromWindowNative(long peer);
@Override
protected void onDetachedFromWindow() {
super.onDetachedFromWindow();
onDetachedFromWindowNative(mViewPeer);
}
private native void onWindowVisibilityChangedNative(long peer, int visibility);
@Override
protected void onWindowVisibilityChanged(int visibility) {
super.onWindowVisibilityChanged(visibility);
onWindowVisibilityChangedNative(mViewPeer, visibility);
}
private native void surfaceCreatedNative(long peer, SurfaceHolder holder);
@Override
public void surfaceCreated(SurfaceHolder holder) {
surfaceCreatedNative(mViewPeer, holder);
}
private native void surfaceChangedNative(
long peer, SurfaceHolder holder, int format, int width, int height);
@Override
public void surfaceChanged(SurfaceHolder holder, int format, int width, int height) {
surfaceChangedNative(mViewPeer, holder, format, width, height);
}
private native void surfaceDestroyedNative(long peer, SurfaceHolder holder);
@Override
public void surfaceDestroyed(SurfaceHolder holder) {
surfaceDestroyedNative(mViewPeer, holder);
}
void postFrameCallback() {
Choreographer c = Choreographer.getInstance();
c.removeFrameCallback(this);
c.postFrameCallback(this);
}
void removeFrameCallback() {
Choreographer.getInstance().removeFrameCallback(this);
}
private native void doFrameNative(long peer, long frameTimeNanos);
@Override
public void doFrame(long frameTimeNanos) {
doFrameNative(mViewPeer, frameTimeNanos);
}
private native void delayedCallbackNative(long peer);
private final Runnable mDelayedCallback =
new Runnable() {
@Override
public void run() {
delayedCallbackNative(mViewPeer);
}
};
boolean postDelayed(long delayMillis) {
return postDelayed(mDelayedCallback, delayMillis);
}
boolean removeDelayedCallbacks() {
return removeCallbacks(mDelayedCallback);
}
private native boolean hasAccessibilityNodeProviderNative(long peer);
private native AccessibilityNodeInfo createAccessibilityNodeInfoNative(
long peer, int virtualViewId);
private native AccessibilityNodeInfo accessibilityFindFocusNative(long peer, int virtualViewId);
private native boolean performAccessibilityActionNative(
long peer, int virtualViewId, int action, Bundle arguments);
@Override
public AccessibilityNodeProvider getAccessibilityNodeProvider() {
if (!hasAccessibilityNodeProviderNative(mViewPeer)) {
return super.getAccessibilityNodeProvider();
}
return new AccessibilityNodeProvider() {
@Override
public AccessibilityNodeInfo createAccessibilityNodeInfo(int virtualViewId) {
return createAccessibilityNodeInfoNative(mViewPeer, virtualViewId);
}
@Override
public AccessibilityNodeInfo findFocus(int focusType) {
return accessibilityFindFocusNative(mViewPeer, focusType);
}
@Override
public boolean performAction(int virtualViewId, int action, Bundle arguments) {
return performAccessibilityActionNative(
mViewPeer, virtualViewId, action, arguments);
}
};
}
private native boolean onCreateInputConnectionNative(long peer, EditorInfo outAttrs);
@Override
public InputConnection onCreateInputConnection(EditorInfo outAttrs) {
if (!onCreateInputConnectionNative(mViewPeer, outAttrs)) {
return null;
}
return new RustInputConnection(this);
}
native String getTextBeforeCursorNative(long peer, int n);
native String getTextAfterCursorNative(long peer, int n);
native String getSelectedTextNative(long peer);
native int getCursorCapsModeNative(long peer, int reqModes);
native boolean deleteSurroundingTextNative(long peer, int beforeLength, int afterLength);
native boolean deleteSurroundingTextInCodePointsNative(
long peer, int beforeLength, int afterLength);
native boolean setComposingTextNative(long peer, String text, int newCursorPosition);
native boolean setComposingRegionNative(long peer, int start, int end);
native boolean finishComposingTextNative(long peer);
native boolean commitTextNative(long peer, String text, int newCursorPosition);
native boolean setSelectionNative(long peer, int start, int end);
native boolean performEditorActionNative(long peer, int editorAction);
native boolean performContextMenuActionNative(long peer, int id);
native boolean beginBatchEditNative(long peer);
native boolean endBatchEditNative(long peer);
native boolean inputConnectionSendKeyEventNative(long peer, KeyEvent event);
native boolean inputConnectionClearMetaKeyStatesNative(long peer, int states);
native boolean inputConnectionReportFullscreenModeNative(long peer, boolean enabled);
native boolean requestCursorUpdatesNative(long peer, int cursorUpdateMode);
native void closeInputConnectionNative(long peer);
}
+3
View File
@@ -0,0 +1,3 @@
plugins {
id("com.android.application") version "9.4.0" apply false
}
+15
View File
@@ -0,0 +1,15 @@
pluginManagement {
repositories {
google()
mavenCentral()
gradlePluginPortal()
}
}
dependencyResolutionManagement {
repositories {
google()
mavenCentral()
}
}
rootProject.name = "iris-android-demo"
include(":app")
+87
View File
@@ -0,0 +1,87 @@
//! The android-view demo app: iris's `tabs` widget tree (`tabs_ui::build`,
//! shared with the winit example) running through
//! `iris::android`'s `ViewPeer`. This is RUST.md's I2 pass condition made
//! concrete -- there is no UI here beyond what `tabs-ui` already draws.
//!
//! `JNI_OnLoad` and `new_view_peer` mirror android-view's own demo
//! (`~/src/android-view/demo/src/lib.rs`): the only android-view-specific
//! plumbing a real app needs is registering its `View` subclass and
//! wrapping `iris::android::new_peer`'s generic function in a concrete
//! `extern "system" fn`, since `register_view_class` wants a plain
//! function pointer.
use android_view::{
Context, View,
jni::{
JNIEnv, JavaVM,
sys::{JNI_VERSION_1_6, JavaVM as RawJavaVM, jint, jlong},
},
register_view_class,
};
use iris::android::{AndroidAppState, AndroidRsc, AndroidUiState, HasAndroidUiState};
use iris::prelude::*;
use log::LevelFilter;
use std::ffi::c_void;
/// The app's `View` subclass, matching the Java side's package --
/// `app/src/main/java/dev/iris/android/demo/IrisView.java`.
const VIEW_CLASS: &str = "dev/iris/android/demo/IrisView";
pub struct Client {
ui_state: AndroidUiState,
}
impl HasAndroidUiState for Client {
fn android_state(&self) -> &AndroidUiState {
&self.ui_state
}
fn android_state_mut(&mut self) -> &mut AndroidUiState {
&mut self.ui_state
}
}
impl AndroidAppState for Client {
fn new(mut ui_state: AndroidUiState, rsc: &mut AndroidRsc<Self>) -> Self {
// `widgets.info` is the winit example's frame-debug readout, kept
// current from `DefaultAppState::window_event` -- android-view has
// no per-frame hook to drive the equivalent from here yet, so it
// is left at its built "" text rather than wired to nothing.
let _ = tabs_ui::build(rsc, &mut ui_state);
Self { ui_state }
}
fn back_pressed(&mut self, _rsc: &mut AndroidRsc<Self>, _render: &mut UiRenderState) -> bool {
// Nothing in the tabs example has a back stack of its own to pop --
// declining lets the activity finish, which is the same "no
// handler" behaviour the default impl gives. Present as an
// explicit override (rather than relying on the default) so a
// reader checking "does the back gesture reach this app" finds an
// answer here rather than nothing.
false
}
}
extern "system" fn new_view_peer<'local>(
env: JNIEnv<'local>,
view: View<'local>,
context: Context<'local>,
) -> jlong {
iris::android::new_peer::<Client>(env, view, context)
}
/// # Safety
/// Interacting with JNI at load time is always unsafe at some level --
/// mirrors android-view's own demo, which carries the same comment.
#[unsafe(no_mangle)]
pub unsafe extern "system" fn JNI_OnLoad(vm: *mut RawJavaVM, _: *mut c_void) -> jint {
android_logger::init_once(
android_logger::Config::default()
.with_max_level(LevelFilter::Debug)
.with_tag("iris-android-app"),
);
let vm = unsafe { JavaVM::from_raw(vm) }.unwrap();
let mut env = vm.get_env().unwrap();
register_view_class(&mut env, VIEW_CLASS, new_view_peer);
iris::android::register_native_methods(&mut env, VIEW_CLASS);
JNI_VERSION_1_6
}
+425
View File
@@ -0,0 +1,425 @@
//! On-demand benchmarks for iris's message-list scenario -- IRIS_TODO.md's
//! "Benchmarks" item, and RUST.md's I3. Never run by `cargo test`; run
//! explicitly with `cargo bench --bench message_list --release` or
//! `./run-bench.sh`.
//!
//! **Why a plain `Instant`-timed binary, not criterion.** Every scenario
//! here is really "how many `Widget::draw` calls and primitive rewrites did
//! this frame cost," which `UiRenderState::take_counters` already answers
//! exactly (see `iris/src/layout_tests.rs`, which this file's harness
//! mirrors). A short loop that times itself and prints the counters
//! alongside the wall time says everything criterion's warm-up/sampling/
//! outlier-removal machinery would add on top, for scenarios that are
//! fundamentally about a *count*, not a noisy microbenchmark distribution
//! -- and it avoids a new dependency this crate does not otherwise need.
//! Per the code rules, the plain option is also the one shorter to explain.
//!
//! **The list under test is `iris::widget::List` (RUST.md's I3), not a
//! `Scroll` over a `Span` of pre-built rows.** Earlier versions of this
//! file built their own giant `Span` and wrapped it in `Scroll`, which
//! meant (a)/(b)/(c) below were measuring "move one big child," never the
//! virtualised widget the app's transcript screen actually needs. `List`
//! still needs every row's *widget* built up front by the caller (its
//! module doc explains why: it only ever sees `&dyn Widget` through
//! `Painter`, so it cannot construct a row lazily on its own) -- what
//! virtualisation buys is that only the rows currently on screen are ever
//! *drawn*, which is what the draw/rewrite/move counters below are
//! measuring, not construction time.
//!
//! Scenarios (LAYOUT.md's O(1) move chain, list.rs's module doc, and
//! IRIS_TODO.md's "Benchmarks" wording):
//!
//! - (a) first-frame cost of a message list of N wrapped-text rows, some
//! with an image, for N = 100 / 1,000 / 10,000. With a virtualised list
//! this is expected to stop scaling with N once N exceeds a screenful --
//! the draw/rewrite counters below are the number that used to grow 10x
//! per 10x N and should not any more.
//! - (b) per-frame cost of scrolling that list -- must be O(1) moves, not
//! re-layout.
//! - (c) the input-box case: growing a fixed-height field at the bottom of
//! the screen must move the message list above it, not re-lay its rows.
//! Reports frame time *and* the draw/rewrite/move counters LAYOUT.md
//! section 8 defines.
//! - (d) insert-above-anchor: paging older history onto the front of an
//! already-scrolled list. `List::push_front` is an O(1) index update
//! (list.rs's module doc); this measures that none of the rows already
//! on screen are touched by it.
//! - (e) expand-a-row-holding-its-edge: growing one row's height with a
//! tap recorded near one of its edges (list.rs's `note_tap`) must move
//! only the rows on the far side of it, never redraw the ones already
//! correctly placed.
//!
//! (f), many images with zero steady-state bind-group creation, needs a
//! real `wgpu` device and lives in `iris/examples/bench_images.rs` instead,
//! driven through `run-headless.sh` -- see that file's header.
//!
//! `UiRenderState`/`Widgets` touch no GPU or window (as `layout_tests.rs`
//! notes), so everything here runs as an ordinary `--release` binary with
//! no compositor. Numbers are recorded in RUST.md's I3 box, not here --
//! this file is the rig, not the result.
use iris::prelude::*;
use std::time::Instant;
/// The minimal `UiRsc` a benchmark needs -- identical in shape to
/// `layout_tests.rs`'s `TestRsc`.
struct BenchRsc {
ui: UiData,
}
impl UiRsc for BenchRsc {
fn ui(&self) -> &UiData {
&self.ui
}
fn ui_mut(&mut self) -> &mut UiData {
&mut self.ui
}
}
/// Long enough to force real wrapping at a phone-plausible column width, and
/// varied enough (no two rows byte-identical) that nothing can special-case
/// on repeated content.
const BODY: &str = "The quick brown fox jumps over the lazy dog. Iris lays \
out wrapped text by shaping once per width and caching the result, so a \
row that is offered the same width twice does not reshape. This sentence \
exists only to give a row enough text to wrap across several lines at a \
typical phone column width.";
/// One message row: a wrapped `Text`, and every `image_every`th row also an
/// `Image` beneath it -- a small in-memory RGBA square rather than a file,
/// so N=10,000 rows costs no disk I/O.
fn build_row(rsc: &mut BenchRsc, i: usize, image_every: usize) -> StrongWidget {
let mut text = Text::new(format!("Message {i}: {BODY}"));
text.wrap = true;
let text = rsc.ui.widgets.add_strong(text).any();
if image_every > 0 && i.is_multiple_of(image_every) {
let img = image::DynamicImage::new_rgba8(64, 64);
let image_widget = image::<BenchRsc>(img)(rsc);
let image_widget = rsc.ui.widgets.add_strong(image_widget).any();
let mut row = Span::empty(Dir::DOWN);
row.push(text);
row.push(image_widget);
rsc.ui.widgets.add_strong(row).any()
} else {
text
}
}
/// A virtualised `List` of `n` message rows, one in `image_every` of them
/// carrying an image (0 disables images entirely). Returns the list widget
/// (weak, so the caller can drive it) and the erased root to render.
fn build_message_list(
rsc: &mut BenchRsc,
n: usize,
image_every: usize,
) -> (WeakWidget<List>, StrongWidget) {
let mut list = List::new(Axis::Y);
for i in 0..n {
let row = build_row(rsc, i, image_every);
list.push_back(ListRow::new(i as u64, row));
}
let list = rsc.ui.widgets.add_strong(list);
(list.weak(), list.any())
}
fn report(label: &str, elapsed: std::time::Duration, draws: u64, rewrites: u64, moves: u64) {
println!(
"{label}: {:.2}ms draws={draws} rewrites={rewrites} moves={moves}",
elapsed.as_secs_f64() * 1000.0
);
}
/// (a) First-frame cost of a message list of N rows.
fn bench_first_frame(n: usize) {
let mut rsc = BenchRsc {
ui: UiData::default(),
};
let (_list, root) = build_message_list(&mut rsc, n, 20);
let mut render = UiRenderState::new();
render.resize((1080.0, 2000.0));
let start = Instant::now();
render.update(&root, &mut rsc);
let elapsed = start.elapsed();
let (draws, rewrites, moves) = render.take_counters();
report(
&format!("(a) first frame, N={n}"),
elapsed,
draws,
rewrites,
moves,
);
}
/// (b) Per-frame cost of scrolling an already-laid-out list of N rows.
/// Warms up (one no-op tick, matching `Scroll`'s own need for it before an
/// ordinary Rust `layout_tests.rs` scrolling test becomes a same-size move
/// rather than a resize), then times a run of individual scroll ticks.
fn bench_scroll(n: usize, ticks: usize) {
let mut rsc = BenchRsc {
ui: UiData::default(),
};
let (list, root) = build_message_list(&mut rsc, n, 20);
let mut render = UiRenderState::new();
render.resize((1080.0, 2000.0));
render.update(&root, &mut rsc);
rsc.ui.widgets.get_mut(&list).unwrap().scroll(0.0);
render.update(&root, &mut rsc);
render.take_counters();
let mut total = std::time::Duration::ZERO;
let mut total_draws = 0u64;
let mut total_rewrites = 0u64;
let mut total_moves = 0u64;
for _ in 0..ticks {
rsc.ui.widgets.get_mut(&list).unwrap().scroll(-8.0);
let start = Instant::now();
render.update(&root, &mut rsc);
total += start.elapsed();
let (draws, rewrites, moves) = render.take_counters();
total_draws += draws;
total_rewrites += rewrites;
total_moves += moves;
}
report(
&format!("(b) scroll, N={n}, {ticks} ticks (totals; expect draws/moves independent of N)"),
total,
total_draws,
total_rewrites,
total_moves,
);
println!(
" per-tick average: {:.4}ms",
total.as_secs_f64() * 1000.0 / ticks as f64
);
}
/// (c) The input-box case: a fixed-height field at the bottom of the screen
/// growing by a line at a time, with a message list of N rows filling the
/// rest of the screen above it. Growing the input shrinks the *offered*
/// height of the list container (a single widget, from the outer `Span`'s
/// point of view) without changing the width it offers its content -- so
/// the rows underneath, which only care about width, must not redraw; the
/// list's own re-registration of where its content sits is the one O(1)
/// move this is checking for.
fn bench_input_grows(n: usize, lines: usize) {
let mut rsc = BenchRsc {
ui: UiData::default(),
};
let (list, list_root) = build_message_list(&mut rsc, n, 20);
let list_area = rsc.ui.widgets.add_strong(Sized {
inner: list_root,
x: None,
y: Some(rest(1.0)),
});
let line_height = 24.0;
let input_rect = rsc.ui.widgets.add_strong(Rect::new(UiColor::WHITE));
let input_area = rsc.ui.widgets.add_strong(Sized {
inner: input_rect.any(),
x: None,
y: Some(abs(line_height)),
});
let input_area_weak = input_area.weak();
let mut root_span = Span::empty(Dir::DOWN);
root_span.push(list_area.any());
root_span.push(input_area.any());
let root = rsc.ui.widgets.add_strong(root_span).any();
let mut render = UiRenderState::new();
render.resize((1080.0, 2000.0));
render.update(&root, &mut rsc);
rsc.ui.widgets.get_mut(&list).unwrap().scroll(0.0);
render.update(&root, &mut rsc);
render.take_counters();
let mut total = std::time::Duration::ZERO;
let mut total_draws = 0u64;
let mut total_rewrites = 0u64;
let mut total_moves = 0u64;
for line in 1..=lines {
rsc.ui.widgets.get_mut(&input_area_weak).unwrap().y =
Some(abs(line_height * (line + 1) as f32));
let start = Instant::now();
render.update(&root, &mut rsc);
total += start.elapsed();
let (draws, rewrites, moves) = render.take_counters();
total_draws += draws;
total_rewrites += rewrites;
total_moves += moves;
}
report(
&format!(
"(c) input grows by {lines} lines above N={n} rows (totals; \
draws/rewrites must not scale with N)"
),
total,
total_draws,
total_rewrites,
total_moves,
);
println!(
" per-line average: {:.4}ms",
total.as_secs_f64() * 1000.0 / lines as f64
);
}
/// (d) Insert-above-anchor: the list is scrolled to its very first loaded
/// row (`jump_to_start`, an O(1) re-anchor) rather than left at the default
/// bottom, so a row prepended above it is genuinely "inserted above the
/// anchor" rather than merely far off-screen at the far end. Each
/// `push_front` is O(1) (list.rs's module doc: the anchor's slot is an
/// index, bumped by one) and, since the prepended rows never enter the
/// viewport, none of them should cost a draw either.
fn bench_insert_above_anchor(n: usize, inserts: usize) {
let mut rsc = BenchRsc {
ui: UiData::default(),
};
let (list, root) = build_message_list(&mut rsc, n, 20);
let mut render = UiRenderState::new();
render.resize((1080.0, 2000.0));
render.update(&root, &mut rsc);
rsc.ui.widgets.get_mut(&list).unwrap().jump_to_start();
render.update(&root, &mut rsc);
render.take_counters();
let mut total = std::time::Duration::ZERO;
let mut total_draws = 0u64;
let mut total_rewrites = 0u64;
let mut total_moves = 0u64;
for i in 0..inserts {
// Older-history rows: distinct keys below every existing one, so a
// real caller's paging code (prepending an older page) is exactly
// what this loop does.
let row = build_row(&mut rsc, usize::MAX - i, 20);
rsc.ui
.widgets
.get_mut(&list)
.unwrap()
.push_front(ListRow::new(i as u64, row));
let start = Instant::now();
render.update(&root, &mut rsc);
total += start.elapsed();
let (draws, rewrites, moves) = render.take_counters();
total_draws += draws;
total_rewrites += rewrites;
total_moves += moves;
}
report(
&format!(
"(d) insert-above-anchor, N={n}, {inserts} pushes (totals; \
must not scale with N)"
),
total,
total_draws,
total_rewrites,
total_moves,
);
println!(
" per-push average: {:.4}ms",
total.as_secs_f64() * 1000.0 / inserts as f64
);
}
/// (e) Expand-a-row-holding-its-edge: one row (fixed-height, so its size is
/// directly controllable) is grown a little at a time, each time preceded
/// by `note_tap` aimed at its own top edge -- the exact mechanism list.rs's
/// module doc describes and its unit tests check for correctness. This
/// measures its *cost*: only the rows on the far side of the grown one
/// (below it, since the top edge is held) should ever move, and nothing
/// should be redrawn purely because the list overall got taller.
fn bench_expand_holds_edge(n: usize, growths: usize) {
let mut rsc = BenchRsc {
ui: UiData::default(),
};
let mut list = List::new(Axis::Y);
// Near the end (not the very last row) so it is already on screen
// under the list's default bottom-anchored placement, for every N --
// no scrolling needed to bring it into view before measuring.
let growable_index = n.saturating_sub(3);
let mut growable = None;
for i in 0..n {
if i == growable_index {
let rect = rsc.ui.widgets.add_strong(Rect::new(UiColor::WHITE));
let sized = rsc.ui.widgets.add_strong(Sized {
inner: rect.any(),
x: None,
y: Some(abs(40.0)),
});
growable = Some(sized.weak());
list.push_back(ListRow::new(i as u64, sized.any()));
} else {
let row = build_row(&mut rsc, i, 20);
list.push_back(ListRow::new(i as u64, row));
}
}
let list = rsc.ui.widgets.add_strong(list);
let list_weak = list.weak();
let root = list.any();
let growable = growable.unwrap();
let mut render = UiRenderState::new();
render.resize((1080.0, 2000.0));
render.update(&root, &mut rsc);
render.take_counters();
let mut total = std::time::Duration::ZERO;
let mut total_draws = 0u64;
let mut total_rewrites = 0u64;
let mut total_moves = 0u64;
let mut height = 40.0f32;
let key = growable_index as u64;
for _ in 0..growths {
height += 10.0;
if let Some((top, _bottom)) = rsc.ui.widgets.get(&list_weak).unwrap().extent(key) {
rsc.ui
.widgets
.get_mut(&list_weak)
.unwrap()
.note_tap(top + 1.0);
}
rsc.ui.widgets.get_mut(&growable).unwrap().y = Some(abs(height));
let start = Instant::now();
render.update(&root, &mut rsc);
total += start.elapsed();
let (draws, rewrites, moves) = render.take_counters();
total_draws += draws;
total_rewrites += rewrites;
total_moves += moves;
}
report(
&format!(
"(e) expand-hold, N={n}, {growths} growths (totals; \
must not scale with N)"
),
total,
total_draws,
total_rewrites,
total_moves,
);
println!(
" per-growth average: {:.4}ms",
total.as_secs_f64() * 1000.0 / growths as f64
);
}
fn main() {
println!("iris message-list benchmark -- release build, this machine's CPU");
for &n in &[100usize, 1_000, 10_000] {
bench_first_frame(n);
}
for &n in &[100usize, 1_000, 10_000] {
bench_scroll(n, 200);
}
for &n in &[100usize, 1_000, 10_000] {
bench_input_grows(n, 40);
}
for &n in &[100usize, 1_000, 10_000] {
bench_insert_above_anchor(n, 200);
}
for &n in &[100usize, 1_000, 10_000] {
bench_expand_holds_edge(n, 40);
}
}
+1
View File
@@ -10,3 +10,4 @@ image = { workspace = true }
parley = { workspace = true }
swash = { workspace = true }
fxhash = { workspace = true }
accesskit = { workspace = true }
+1 -1
View File
@@ -421,7 +421,7 @@ impl Display for UiRegion {
}
}
#[derive(Debug)]
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct PixelRegion {
pub top_left: Vec2,
pub bot_right: Vec2,
+113 -6
View File
@@ -1,8 +1,9 @@
use crate::{Align, GlyphAtlas, GlyphKey, PlacedGlyph, RegionAlign, Textures, UiColor, util::Vec2};
use parley::{
Alignment, AlignmentOptions, FontContext, FontFamily, FontFamilyName, GenericFamily, Layout,
LayoutContext, LineHeight, PositionedLayoutItem, StyleProperty,
Alignment, AlignmentOptions, FontContext, FontFamily, FontFamilyName, FontStyle, FontWeight,
GenericFamily, Layout, LayoutContext, LineHeight, PositionedLayoutItem, StyleProperty,
};
use std::ops::Range;
use swash::{
FontRef,
scale::{Render, ScaleContext, Source, StrikeWith},
@@ -51,6 +52,72 @@ impl Family {
}
}
/// One styled run inside a `TextBuffer`, overriding `TextAttrs`' base style
/// over `range` (a byte range into the buffer's text). Every field is
/// optional so a span only says what it changes -- e.g. a link span sets
/// `color` and `underline` and leaves weight/family at the paragraph's own
/// default. This is I5's answer to RUST.md's inline-rich-text ceiling
/// (`masonry/src/widgets/text_area.rs`'s `StyleSet` is one style for the
/// whole editor, with `// TODO: RichTextInput` beside it): parley's own
/// `RangedBuilder::push` already takes a style and a range, so per-span
/// bold/italic/monospace/colour/underline only needed plumbing this struct
/// through to it and giving each glyph its own colour at draw time (see
/// `PlacedGlyph::color` and `TextData::place` below) instead of the one
/// `RenderedText::color` every glyph used to share.
#[derive(Clone, PartialEq)]
pub struct SpanStyle {
pub range: Range<usize>,
pub color: Option<UiColor>,
pub family: Option<Family>,
/// Overrides `TextAttrs::font_size` for just this range -- what lets a
/// heading inside a transcript row's single `TextEdit` be bigger than
/// the paragraph text around it, so a whole markdown-folded row (block
/// and inline styling both) can stay one selectable text buffer instead
/// of one widget per block.
pub font_size: Option<f32>,
pub bold: bool,
pub italic: bool,
pub underline: bool,
}
impl SpanStyle {
pub fn new(range: Range<usize>) -> Self {
Self {
range,
color: None,
family: None,
font_size: None,
bold: false,
italic: false,
underline: false,
}
}
pub fn color(mut self, color: UiColor) -> Self {
self.color = Some(color);
self
}
pub fn family(mut self, family: Family) -> Self {
self.family = Some(family);
self
}
pub fn font_size(mut self, size: f32) -> Self {
self.font_size = Some(size);
self
}
pub fn bold(mut self) -> Self {
self.bold = true;
self
}
pub fn italic(mut self) -> Self {
self.italic = true;
self
}
pub fn underline(mut self) -> Self {
self.underline = true;
self
}
}
#[derive(Clone, PartialEq)]
pub struct TextAttrs {
pub color: UiColor,
@@ -86,8 +153,12 @@ impl Default for TextAttrs {
pub struct TextBuffer {
text: String,
layout: Layout<UiColor>,
spans: Vec<SpanStyle>,
/// What the current layout was built for, so `shape` can decline to redo
/// work that would come out the same.
/// work that would come out the same. Spans are not part of this key --
/// `set_spans` forces `shaped` to `None` directly, the same way `edit`
/// does, since spans change far less often than a naive equality check
/// on the whole `Vec` would cost to compute every frame.
shaped: Option<(TextAttrs, Option<f32>)>,
}
@@ -96,10 +167,19 @@ impl TextBuffer {
Self {
text: text.into(),
layout: Layout::new(),
spans: Vec::new(),
shaped: None,
}
}
/// Replace this buffer's per-range style overrides (I5's rich text --
/// see `SpanStyle`). Invalidates the layout unconditionally, mirroring
/// `set_text`.
pub fn set_spans(&mut self, spans: Vec<SpanStyle>) {
self.spans = spans;
self.shaped = None;
}
pub fn new_empty() -> Self {
Self::new("")
}
@@ -150,6 +230,27 @@ impl TextBuffer {
attrs.line_height,
)));
builder.push_default(StyleProperty::Brush(attrs.color));
for span in &self.spans {
let range = span.range.clone();
if let Some(color) = span.color {
builder.push(StyleProperty::Brush(color), range.clone());
}
if let Some(family) = &span.family {
builder.push(StyleProperty::FontFamily(family.family()), range.clone());
}
if let Some(size) = span.font_size {
builder.push(StyleProperty::FontSize(size), range.clone());
}
if span.bold {
builder.push(StyleProperty::FontWeight(FontWeight::BOLD), range.clone());
}
if span.italic {
builder.push(StyleProperty::FontStyle(FontStyle::Italic), range.clone());
}
if span.underline {
builder.push(StyleProperty::Underline(true), range.clone());
}
}
builder.build_into(&mut self.layout, &self.text);
self.layout.break_all_lines(width);
self.layout
@@ -175,6 +276,7 @@ impl TextData {
let font = run.run().font();
let font_size = run.run().font_size();
let coords = run.run().normalized_coords();
let run_color = run.style().brush;
let Some(font_ref) = FontRef::from_index(font.data.as_ref(), font.index as usize)
else {
continue;
@@ -227,6 +329,7 @@ impl TextData {
glyph.x.floor() + entry.left as f32,
glyph.y.floor() - entry.top as f32,
),
color: run_color,
});
}
}
@@ -245,11 +348,15 @@ fn hash_coords(coords: &[i16]) -> u64 {
h
}
/// A laid-out string, ready to draw: where each glyph goes, how big the whole
/// thing is, and what colour to tint the atlas with.
/// A laid-out string, ready to draw: where each glyph goes and how big the
/// whole thing is.
///
/// Cheap to clone and to keep, which is the point -- a widget holds one across
/// frames and re-emits its quads without going near the rasteriser.
/// frames and re-emits its quads without going near the rasteriser. `color`
/// is the buffer's *base* colour (`TextAttrs::color`) for a caller that wants
/// it as a whole (e.g. tinting a cursor to match); the colour each glyph is
/// actually drawn in is `PlacedGlyph::color`, which a `SpanStyle` can
/// override per range.
#[derive(Clone)]
pub struct RenderedText {
pub glyphs: std::sync::Arc<Vec<PlacedGlyph>>,
+9 -1
View File
@@ -10,7 +10,7 @@
//! it, and a resize re-emits quads without touching the GPU's copy at all.
use crate::{
PatchRect, TextureHandle, Textures,
PatchRect, TextureHandle, Textures, UiColor,
util::{HashMap, Vec2},
};
use image::RgbaImage;
@@ -228,8 +228,16 @@ fn write_glyph(page: &mut RgbaImage, image: &Image, x: u32, y: u32) {
}
/// Where a glyph goes on screen, in pixels relative to the text's origin.
///
/// `color` is per-glyph (read from the parley run's own `Brush`, since
/// `UiColor` is parley's brush type here) rather than a single colour for
/// the whole `RenderedText`, so that a span pushed with its own
/// `StyleProperty::Brush` (I5's inline rich text: a link, a diff of colour
/// inside one wrapped paragraph) actually renders in that colour instead of
/// the buffer's base one.
#[derive(Clone, Copy)]
pub struct PlacedGlyph {
pub entry: GlyphEntry,
pub offset: Vec2,
pub color: UiColor,
}
+116 -47
View File
@@ -35,6 +35,19 @@ pub struct UiRenderNode {
textures: GpuTextures,
masks: ArrBuf<Mask>,
move_offsets: ArrBuf<MoveOffset>,
/// Group 3: the masks and move-offsets storage buffers, on their own --
/// see IRIS_TODO.md's "Appending one image ... rebuilds every other
/// image's bind group". These used to live in group 2 alongside each
/// standalone image's own texture view, so an image's bind group named
/// the masks/move_offsets buffer directly; the moment either buffer
/// resized (which a widget getting its *first* move slot can trigger,
/// unrelated to any image), `ArrBuf::update` handed back a new `Buffer`
/// identity and every image's bind group -- one per live image -- had
/// to be rebuilt to reference it. Pulling both buffers into their own
/// group, bound once per frame rather than once per draw call, means a
/// buffer resize now rebuilds exactly this one group instead of N.
masks_layout: BindGroupLayout,
masks_group: BindGroup,
}
struct RenderLayer {
@@ -54,6 +67,13 @@ impl UiRenderNode {
pub fn draw<'a>(&'a self, pass: &mut RenderPass<'a>) {
pass.set_pipeline(&self.pipeline);
pass.set_bind_group(0, &self.uniform_group, &[]);
// Set once, not per layer or per image: masks/move_offsets are read
// by every primitive and every standalone image alike, and living
// in their own group (rather than folded into group 2 alongside the
// per-image texture view) is what keeps an image's own bind group
// from naming a buffer that changes size on an unrelated widget's
// first draw -- see the comment on `masks_group` below.
pass.set_bind_group(3, &self.masks_group, &[]);
for i in &self.active {
let layer = &self.layers[i];
if layer.instance.len() == 0 && layer.image_instance.len() == 0 {
@@ -164,21 +184,13 @@ impl UiRenderNode {
} else {
false
};
let rebuild_main = self.textures.update(
&mut ui.textures,
&self.rsc_layout,
&self.masks,
&self.move_offsets,
masks_resized || moves_resized,
);
if masks_resized || moves_resized {
self.masks_group =
Self::masks_group(device, &self.masks_layout, &self.masks, &self.move_offsets);
}
let rebuild_main = self.textures.update(&mut ui.textures, &self.rsc_layout);
if rebuild_main {
self.rsc_group = Self::rsc_group(
device,
&self.rsc_layout,
&self.textures,
&self.masks,
&self.move_offsets,
);
self.rsc_group = Self::rsc_group(device, &self.rsc_layout, &self.textures);
}
}
@@ -201,7 +213,21 @@ impl UiRenderNode {
source: ShaderSource::Wgsl(SHAPE_SHADER.into()),
});
let window_uniform = WindowUniform::default();
// Seeded from the surface's own size, not `WindowUniform::default()`
// (0, 0): the vertex shader divides by `window.dim` to reach clip
// space, so a window this buffer disagrees with means every
// primitive's position is NaN/Inf and is dropped before
// rasterization -- the clear colour still reaches the screen (the
// pass runs regardless) while nothing drawn on top of it ever does.
// winit's backend gets away with the old default because winit
// fires an initial `WindowEvent::Resized` that calls `resize()`
// before the first frame; android-view has no such automatic
// event, so `AndroidRenderer::new` built a node whose window buffer
// was never corrected -- this is I2's "nothing draws" bug (RUST.md).
let window_uniform = WindowUniform {
width: config.width as f32,
height: config.height as f32,
};
let window_buffer = device.create_buffer_init(&BufferInitDescriptor {
label: Some("window"),
contents: bytemuck::cast_slice(&[window_uniform]),
@@ -251,11 +277,18 @@ impl UiRenderNode {
);
let rsc_layout = Self::rsc_layout(device);
let rsc_group = Self::rsc_group(device, &rsc_layout, &tex_manager, &masks, &move_offsets);
let rsc_group = Self::rsc_group(device, &rsc_layout, &tex_manager);
let masks_layout = Self::masks_layout(device);
let masks_group = Self::masks_group(device, &masks_layout, &masks, &move_offsets);
let pipeline_layout = device.create_pipeline_layout(&PipelineLayoutDescriptor {
label: Some("UI Shape Pipeline Layout"),
bind_group_layouts: &[&uniform_layout, &primitive_layout, &rsc_layout],
bind_group_layouts: &[
&uniform_layout,
&primitive_layout,
&rsc_layout,
&masks_layout,
],
immediate_size: 0,
});
let pipeline = device.create_render_pipeline(&RenderPipelineDescriptor {
@@ -308,6 +341,8 @@ impl UiRenderNode {
textures: tex_manager,
masks,
move_offsets,
masks_layout,
masks_group,
}
}
@@ -341,12 +376,13 @@ impl UiRenderNode {
})
}
/// Group 2: the shared atlas array, one standalone-image slot (a null
/// Group 2: the shared atlas array and one standalone-image slot (a null
/// view for the main draw, a real one for each image's own bind group --
/// see `GpuTextures`), one sampler and the masks buffer. No `count` on
/// any entry: this needs nothing beyond plain Vulkan 1.0 / GLES
/// sampling, unlike the `binding_array` layout it replaced (see
/// TEXTURES.md's "Recommended shape").
/// see `GpuTextures`), plus one sampler. No `count` on any entry: this
/// needs nothing beyond plain Vulkan 1.0 / GLES sampling, unlike the
/// `binding_array` layout it replaced (see TEXTURES.md's "Recommended
/// shape"). Masks and move_offsets are deliberately *not* here -- see
/// `masks_layout` below for why they get their own group.
fn rsc_layout(device: &Device) -> BindGroupLayout {
device.create_bind_group_layout(&BindGroupLayoutDescriptor {
entries: &[
@@ -376,26 +412,6 @@ impl UiRenderNode {
ty: BindingType::Sampler(SamplerBindingType::NonFiltering),
count: None,
},
BindGroupLayoutEntry {
binding: 3,
visibility: ShaderStages::FRAGMENT,
ty: BindingType::Buffer {
ty: BufferBindingType::Storage { read_only: true },
has_dynamic_offset: false,
min_binding_size: None,
},
count: None,
},
BindGroupLayoutEntry {
binding: 4,
visibility: ShaderStages::VERTEX | ShaderStages::FRAGMENT,
ty: BindingType::Buffer {
ty: BufferBindingType::Storage { read_only: true },
has_dynamic_offset: false,
min_binding_size: None,
},
count: None,
},
],
label: Some("ui rsc"),
})
@@ -407,8 +423,6 @@ impl UiRenderNode {
device: &Device,
layout: &BindGroupLayout,
tex_manager: &GpuTextures,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
) -> BindGroup {
device.create_bind_group(&BindGroupDescriptor {
layout,
@@ -425,20 +439,75 @@ impl UiRenderNode {
binding: 2,
resource: BindingResource::Sampler(tex_manager.sampler()),
},
],
label: Some("ui rsc"),
})
}
/// Group 3: the masks and move_offsets storage buffers, shared by the
/// main draw and every standalone image alike (see the field comment on
/// `masks_group`). Bound once per frame in `draw()` rather than folded
/// into group 2, so a resize of either buffer -- which an unrelated
/// widget's first move slot can trigger -- rebuilds this one group
/// instead of every image's.
fn masks_layout(device: &Device) -> BindGroupLayout {
device.create_bind_group_layout(&BindGroupLayoutDescriptor {
entries: &[
BindGroupLayoutEntry {
binding: 0,
visibility: ShaderStages::FRAGMENT,
ty: BindingType::Buffer {
ty: BufferBindingType::Storage { read_only: true },
has_dynamic_offset: false,
min_binding_size: None,
},
count: None,
},
BindGroupLayoutEntry {
binding: 1,
visibility: ShaderStages::VERTEX | ShaderStages::FRAGMENT,
ty: BindingType::Buffer {
ty: BufferBindingType::Storage { read_only: true },
has_dynamic_offset: false,
min_binding_size: None,
},
count: None,
},
],
label: Some("ui masks"),
})
}
fn masks_group(
device: &Device,
layout: &BindGroupLayout,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
) -> BindGroup {
device.create_bind_group(&BindGroupDescriptor {
layout,
entries: &[
BindGroupEntry {
binding: 3,
binding: 0,
resource: masks.buffer.as_entire_binding(),
},
BindGroupEntry {
binding: 4,
binding: 1,
resource: move_offsets.buffer.as_entire_binding(),
},
],
label: Some("ui rsc"),
label: Some("ui masks"),
})
}
pub fn view_count(&self) -> usize {
self.textures.view_count()
}
/// Standalone-image bind groups built since the last call -- see
/// `GpuTextures::take_bind_group_creates`. Call once per frame before
/// `update()` to measure exactly that frame.
pub fn take_image_bind_group_creates(&mut self) -> u64 {
self.textures.take_bind_group_creates()
}
}
+5 -2
View File
@@ -72,9 +72,12 @@ var atlas: texture_2d_array<f32>;
var image_texture: texture_2d<f32>;
@group(2) @binding(2)
var samp: sampler;
@group(2) @binding(3)
// Their own group, bound once per frame rather than folded into group 2: see
// UiRenderNode::masks_layout for why an image's own bind group must not name
// either buffer.
@group(3) @binding(0)
var<storage> masks: array<Mask>;
@group(2) @binding(4)
@group(3) @binding(1)
var<storage> move_offsets: array<MoveOffset>;
// A move chain more than this deep means something else is wrong (an
+45 -70
View File
@@ -1,9 +1,7 @@
use image::{DynamicImage, EncodableLayout, GenericImageView};
use wgpu::{util::DeviceExt, *};
use crate::{
Mask, MoveOffset, PatchRect, TextureKind, TextureUpdate, Textures, render::util::ArrBuf,
};
use crate::{PatchRect, TextureKind, TextureUpdate, Textures};
use super::atlas::PAGE;
@@ -61,37 +59,34 @@ pub struct GpuTextures {
/// nothing of its own to put there: rects and glyphs never sample it,
/// but the layout requires something bound regardless.
null_view: TextureView,
/// Standalone-image bind groups actually built (`create_image`'s own
/// build, or one per slot touched by `rebuild_image_bind_groups`) since
/// the last `take_bind_group_creates`. IRIS_TODO.md's "many images"
/// benchmark reads this to prove the steady-state cost of an
/// unchanging image list is zero, the same way `UiRenderState`'s
/// `draw_count`/`region_mut_count` prove the layout side.
bind_group_creates: u64,
}
impl GpuTextures {
/// Applies queued `Textures` updates, then reports whether the *main*
/// bind group (the one rects and glyphs draw with) needs rebuilding --
/// true when the atlas array was recreated (its view identity changed)
/// or the masks buffer was, since both are bound there. Pushing or
/// freeing a standalone image never touches that group: it built or drops
/// its own.
pub fn update(
&mut self,
textures: &mut Textures,
rsc_layout: &BindGroupLayout,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
masks_resized: bool,
) -> bool {
let mut rebuild_main = masks_resized;
if masks_resized {
// The masks or move-offsets buffer just moved, so every bind
// group holding a reference to either -- one per live
// standalone image -- is stale.
self.rebuild_image_bind_groups(rsc_layout, masks, move_offsets);
}
/// true exactly when the atlas array was recreated (its view identity
/// changed). Pushing or freeing a standalone image never touches that
/// group: it built or drops its own. Masks/move_offsets resizing is
/// `UiRenderNode`'s own concern now (its `masks_group`, group 3) --
/// see that struct's field comment for why standalone images no longer
/// hear about either buffer at all.
pub fn update(&mut self, textures: &mut Textures, rsc_layout: &BindGroupLayout) -> bool {
let mut rebuild_main = false;
for update in textures.updates() {
match update {
TextureUpdate::Push(kind, image) => {
rebuild_main |= self.push(kind, image, rsc_layout, masks, move_offsets);
rebuild_main |= self.push(kind, image, rsc_layout);
}
TextureUpdate::Set(kind, i, image) => {
rebuild_main |= self.set(kind, i, image, rsc_layout, masks, move_offsets);
rebuild_main |= self.set(kind, i, image, rsc_layout);
}
// A patch changes texture contents, not which layer or bind
// group exists, so it never asks for a rebuild -- rebuilding
@@ -110,10 +105,8 @@ impl GpuTextures {
kind: TextureKind,
image: &DynamicImage,
rsc_layout: &BindGroupLayout,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
) -> bool {
let (slot, rebuilt) = self.make_slot(kind, image, rsc_layout, masks, move_offsets);
let (slot, rebuilt) = self.make_slot(kind, image, rsc_layout);
self.slots.push(slot);
rebuilt
}
@@ -124,10 +117,8 @@ impl GpuTextures {
i: u32,
image: &DynamicImage,
rsc_layout: &BindGroupLayout,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
) -> bool {
let (slot, rebuilt) = self.make_slot(kind, image, rsc_layout, masks, move_offsets);
let (slot, rebuilt) = self.make_slot(kind, image, rsc_layout);
self.slots[i as usize] = slot;
rebuilt
}
@@ -137,18 +128,16 @@ impl GpuTextures {
kind: TextureKind,
image: &DynamicImage,
rsc_layout: &BindGroupLayout,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
) -> (Slot, bool) {
match kind {
TextureKind::Image => {
let gpu = self.create_image(image, rsc_layout, masks, move_offsets);
let gpu = self.create_image(image, rsc_layout);
(Slot::Image(gpu), false)
}
TextureKind::Page { layer } => {
let mut rebuilt = false;
if layer >= self.array_capacity {
self.grow_array(rsc_layout, masks, move_offsets);
self.grow_array(rsc_layout);
rebuilt = true;
}
self.write_full_layer(layer, image);
@@ -236,12 +225,7 @@ impl GpuTextures {
/// copies the old layers across GPU-side -- no readback. Recreates the
/// array's view, which invalidates every bind group that referenced it,
/// so this also rebuilds all of them before returning.
fn grow_array(
&mut self,
rsc_layout: &BindGroupLayout,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
) {
fn grow_array(&mut self, rsc_layout: &BindGroupLayout) {
let new_capacity = self.array_capacity * 2;
let new_texture = Self::create_array_texture(&self.device, new_capacity);
if self.page_count > 0 {
@@ -277,15 +261,14 @@ impl GpuTextures {
..Default::default()
});
self.array_capacity = new_capacity;
self.rebuild_image_bind_groups(rsc_layout, masks, move_offsets);
self.rebuild_image_bind_groups(rsc_layout);
}
fn rebuild_image_bind_groups(
&mut self,
rsc_layout: &BindGroupLayout,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
) {
/// Called only from `grow_array`: the atlas array's view identity is the
/// one thing an image's bind group (group 2) still names that can
/// change out from under it. Masks/move_offsets resizing no longer
/// reaches here at all -- see `UiRenderNode::masks_group`.
fn rebuild_image_bind_groups(&mut self, rsc_layout: &BindGroupLayout) {
for slot in &mut self.slots {
if let Slot::Image(gpu) = slot {
gpu.bind_group = Self::make_image_bind_group(
@@ -294,20 +277,13 @@ impl GpuTextures {
&self.array_view,
&gpu.view,
&self.sampler,
masks,
move_offsets,
);
self.bind_group_creates += 1;
}
}
}
fn create_image(
&self,
image: &DynamicImage,
rsc_layout: &BindGroupLayout,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
) -> ImageGpu {
fn create_image(&mut self, image: &DynamicImage, rsc_layout: &BindGroupLayout) -> ImageGpu {
let rgba = image.to_rgba8();
let (width, height) = rgba.dimensions();
let texture = self.device.create_texture_with_data(
@@ -336,9 +312,8 @@ impl GpuTextures {
&self.array_view,
&view,
&self.sampler,
masks,
move_offsets,
);
self.bind_group_creates += 1;
ImageGpu {
texture,
view,
@@ -347,16 +322,16 @@ impl GpuTextures {
}
/// Builds group 2 for one standalone image: the shared atlas array, this
/// image's own view, the shared sampler, and the shared masks buffer --
/// the same layout the main draw uses with a null view in the image slot.
/// image's own view and the shared sampler -- the same layout the main
/// draw uses with a null view in the image slot. Deliberately does not
/// touch masks/move_offsets (group 3, `UiRenderNode::masks_group`): see
/// that field's comment for why folding them in here was the bug.
fn make_image_bind_group(
device: &Device,
rsc_layout: &BindGroupLayout,
array_view: &TextureView,
image_view: &TextureView,
sampler: &Sampler,
masks: &ArrBuf<Mask>,
move_offsets: &ArrBuf<MoveOffset>,
) -> BindGroup {
device.create_bind_group(&BindGroupDescriptor {
layout: rsc_layout,
@@ -373,14 +348,6 @@ impl GpuTextures {
binding: 2,
resource: BindingResource::Sampler(sampler),
},
BindGroupEntry {
binding: 3,
resource: masks.buffer.as_entire_binding(),
},
BindGroupEntry {
binding: 4,
resource: move_offsets.buffer.as_entire_binding(),
},
],
label: Some("ui rsc image"),
})
@@ -424,9 +391,17 @@ impl GpuTextures {
page_count: 0,
sampler,
null_view,
bind_group_creates: 0,
}
}
/// Reads and zeroes the standalone-image bind-group creation counter --
/// call once per frame before `update()`, mirroring
/// `UiRenderState::take_counters`.
pub fn take_bind_group_creates(&mut self) -> u64 {
std::mem::take(&mut self.bind_group_creates)
}
pub fn array_view(&self) -> &TextureView {
&self.array_view
}
+152
View File
@@ -0,0 +1,152 @@
//! I4 (RUST.md): an AccessKit tree built from iris's own widget tree,
//! shared by both backends -- `android/view.rs` pushes its `TreeUpdate`s
//! through `accesskit_android::Adapter`, `default/mod.rs` through
//! `accesskit_winit::Adapter`. Kept modular the way input's sense registry
//! is: `Widgets::named()` is a side set populated only by `.label()`, so a
//! widget nobody named is never visited here at all, not even to decide it
//! has no name.
//!
//! The tree itself is deliberately flat -- one synthetic `Role::Window`
//! root with every named widget as a direct child, in no particular order.
//! iris's actual widget nesting (a label three `Span`s deep inside a
//! `Scroll`) carries no accessibility meaning of its own here: nothing
//! upstream of a named leaf needs a node, since a screen reader's own
//! traversal (and uiautomator's tap-by-name, the pass condition this was
//! built for) works from each node's on-screen bounds rather than from
//! tree structure. Mirroring the real widget tree exactly would also mean
//! rebuilding intermediate nodes whenever *any* container above a named
//! widget resizes, which is most frames -- the flat shape is what keeps
//! rebuilds tied to "a name, a role or a position actually changed".
use crate::{PixelRegion, UiRenderState, UiRsc, WidgetId, Widgets, util::HashMap};
use accesskit::{Node, NodeId, Rect, Role, TreeId, TreeInfo, TreeUpdate};
/// Reserved for the synthetic root; every real widget's `SlotId::as_u64`
/// starts at 1, so this can never collide with one (see that method's
/// doc comment).
const WINDOW_NODE: NodeId = NodeId(0);
fn node_id(id: WidgetId) -> NodeId {
NodeId(id.as_u64())
}
#[derive(Clone, PartialEq)]
struct Entry {
name: String,
role: Role,
bounds: PixelRegion,
}
fn entry_node(entry: &Entry) -> Node {
let mut node = Node::new(entry.role);
node.set_label(entry.name.clone());
node.set_bounds(Rect {
x0: entry.bounds.top_left.x as f64,
y0: entry.bounds.top_left.y as f64,
x1: entry.bounds.bot_right.x as f64,
y1: entry.bounds.bot_right.y as f64,
});
node
}
/// Owns the last tree pushed out, so `update` can tell "nothing
/// accessibility-relevant changed" from "something did" without asking
/// the platform adapter to diff two `Node`s itself. One of these per
/// window/view -- `default::DefaultUiState` and `android::AndroidUiState`
/// each keep one.
#[derive(Default)]
pub struct AccessTree {
known: HashMap<WidgetId, Entry>,
/// `TreeUpdate`s actually produced since the last `take_rebuilds` --
/// the AccessKit-tree twin of `UiRenderState::take_counters`. Should
/// stay at 0 across an unchanged frame and move by exactly 1 when a
/// named widget's position, name or role changes, however many other
/// widgets are on screen; see `iris/src/access_tests.rs`.
rebuilds: u64,
}
impl AccessTree {
pub fn new() -> Self {
Self::default()
}
fn collect(
widgets: &Widgets,
render: &UiRenderState,
rsc: &dyn UiRsc,
) -> HashMap<WidgetId, Entry> {
let mut current = HashMap::default();
for id in widgets.named() {
let Some(bounds) = render.window_region(&id, rsc) else {
continue;
};
let Some(widget) = widgets.get_dyn(id) else {
continue;
};
current.insert(
id,
Entry {
name: widgets.label(id).clone(),
role: widget.access_role(),
bounds,
},
);
}
current
}
/// Walks `widgets.named()`, looks up each one's current screen bounds
/// via `render.window_region` (which resolves the same move-chain
/// `resolved_region` does, so a moved subtree reports where it
/// actually is), and returns a full `TreeUpdate` if and only if that
/// set differs from the last call -- added, removed, renamed, or
/// moved/resized. A widget that is named but not currently active
/// (not drawn this frame) is left out, the same as one never named at
/// all.
pub fn update(
&mut self,
widgets: &Widgets,
render: &UiRenderState,
rsc: &dyn UiRsc,
) -> Option<TreeUpdate> {
let current = Self::collect(widgets, render, rsc);
if current == self.known {
return None;
}
self.known = current.clone();
self.rebuilds += 1;
Some(build_update(&current))
}
/// The unconditional twin of `update`, for a platform adapter's
/// activation handler (`android/access.rs`'s `AndroidAccessSource`) --
/// AccessKit asks for a full tree the first time a client attaches,
/// which is exactly the case `update`'s diff-against-`known` is not
/// meant to answer (it may have already sent this same snapshot to a
/// client that has since detached and reattached).
pub fn build_full(widgets: &Widgets, render: &UiRenderState, rsc: &dyn UiRsc) -> TreeUpdate {
build_update(&Self::collect(widgets, render, rsc))
}
/// Reads and zeroes the rebuild counter, the same call shape as
/// `UiRenderState::take_counters`.
pub fn take_rebuilds(&mut self) -> u64 {
std::mem::take(&mut self.rebuilds)
}
}
fn build_update(current: &HashMap<WidgetId, Entry>) -> TreeUpdate {
let mut window = Node::new(Role::Window);
let mut nodes = Vec::with_capacity(current.len() + 1);
for (&id, entry) in current {
window.push_child(node_id(id));
nodes.push((node_id(id), entry_node(entry)));
}
nodes.push((WINDOW_NODE, window));
TreeUpdate {
nodes,
tree: Some(TreeInfo::new(WINDOW_NODE)),
tree_id: TreeId::ROOT,
focus: WINDOW_NODE,
}
}
+2
View File
@@ -2,10 +2,12 @@ use crate::{
Mask, MoveOffset, TextData, Textures, WeakWidget, WidgetId, Widgets, util::TrackedArena,
};
mod access;
mod active;
mod painter;
mod render_state;
pub use access::*;
pub use active::*;
pub use painter::Painter;
pub use render_state::*;
+1 -1
View File
@@ -194,7 +194,7 @@ impl<'a> Painter<'a> {
glyph.entry.uv_min,
glyph.entry.uv_max,
glyph.entry.layer,
text.color,
glyph.color,
flags_for(glyph.entry.is_color),
),
region,
+11
View File
@@ -4,6 +4,17 @@ pub struct SlotId {
genr: u32,
}
impl SlotId {
/// A stable, collision-free `u64` encoding of this id -- for a caller
/// (accesskit's `NodeId`, today) that wants a flat integer key rather
/// than the two `u32`s. `idx` is offset by one so no real id ever
/// encodes to 0, which callers can then reserve for their own
/// out-of-band root/window node.
pub fn as_u64(&self) -> u64 {
((self.idx as u64) + 1) << 32 | self.genr as u64
}
}
pub struct SlotVec<T> {
data: Vec<(u32, Option<T>)>,
free: Vec<u32>,
+12
View File
@@ -29,6 +29,18 @@ pub trait Widget: Any {
fn is_size_independent(&self) -> bool {
false
}
/// What kind of control this is, for the AccessKit tree `ui::access`
/// builds (RUST.md's I4). Only consulted for a widget that also has an
/// explicit `.label()` -- an unnamed widget is never visited by that
/// tree at all, named or not, so the default here costs nothing except
/// at the handful of call sites that opt in. Default `Unknown` (a
/// generic control with no more specific semantics); a widget with a
/// real platform equivalent -- `TextEdit`'s `MultilineTextInput` --
/// overrides it.
fn access_role(&self) -> accesskit::Role {
accesskit::Role::Unknown
}
}
impl Widget for () {
+20 -2
View File
@@ -11,6 +11,11 @@ pub struct Widgets {
send: Sender<WidgetId>,
recv: Receiver<WidgetId>,
pub(crate) waiting: HashSet<WidgetId>,
/// Every widget that has ever been given an explicit `.label()` --
/// `ui::access::AccessTree` walks exactly this set, not the whole
/// arena, so a widget nobody named costs it nothing. Symmetric with
/// `free_next` below, which is this set's one removal path.
named: HashSet<WidgetId>,
}
impl Widgets {
@@ -20,6 +25,7 @@ impl Widgets {
needs_redraw: Default::default(),
vec: Default::default(),
waiting: Default::default(),
named: Default::default(),
send,
recv,
}
@@ -95,9 +101,20 @@ impl Widgets {
&self.data(id.id()).unwrap().label
}
/// useful for debugging
/// Also the one place a widget opts into `ui::access`'s AccessKit tree
/// (RUST.md's I4) -- see `named`'s doc comment.
pub fn set_label(&mut self, id: impl IdLike, label: String) {
self.data_mut(id.id()).unwrap().label = label;
let id = id.id();
self.data_mut(id).unwrap().label = label;
self.named.insert(id);
}
/// Every widget with an explicit name, for `ui::access::AccessTree` to
/// walk. Order is unspecified; `AccessTree` doesn't need one; a screen
/// reader's own traversal is worked out by uiautomator from each
/// node's on-screen bounds instead.
pub fn named(&self) -> impl Iterator<Item = WidgetId> + '_ {
self.named.iter().copied()
}
pub fn data_mut(&mut self, id: impl IdLike) -> Option<&mut WidgetData> {
@@ -107,6 +124,7 @@ impl Widgets {
pub fn free_next(&mut self) -> Option<WidgetId> {
let next = self.recv.try_recv().ok()?;
self.vec.free(next);
self.named.remove(&next);
Some(next)
}
+27
View File
@@ -0,0 +1,27 @@
[package]
name = "desktop-app"
version.workspace = true
edition.workspace = true
# RUST.md's E4: the same transcript-ui screen (I5) in a winit window on the
# desktop, beside a session list, talking to a real `ai-server` through
# `client-core`'s REST + SSE clients. Enrolment reuses the phone's own
# `aiapp://enroll?...` link (`client-core::config`) rather than inventing a
# second format -- see DECISIONS.md's 2026-09-05 entry. An ordinary
# workspace member (unlike `android-app`): nothing here needs the NDK, so
# `cargo build --workspace --all-targets` at the host stays clean with it
# included.
[dependencies]
iris = { path = ".." }
transcript-ui = { path = "../transcript-ui" }
client-core = { path = "../../client-core" }
event-model = { path = "../../event-model" }
# Already pulled in transitively through client-core; used directly here
# only to persist `EnrolledServer` as the app's own tiny config file (see
# `config.rs`) -- no new dependency.
serde_json = { version = "1", features = ["float_roundtrip"] }
winit = { workspace = true }
[dev-dependencies]
tempfile = "3"
+538
View File
@@ -0,0 +1,538 @@
//! RUST.md's E4: a session list on the left, `transcript-ui`'s screen (I5)
//! filling the rest, both against a real `ai-server` reached through
//! `client-core`. The layout is the simplest thing that shows both at
//! once -- a fixed-width column and `rest(1)` for everything else, using
//! `iris::widget::{Span, WidgetPtr}` the way `tabs-ui` already switches
//! panes, rather than anything desktop-specific:
//!
//! ```text
//! +-----------+--------------------------------------+
//! | session | transcript_ui::TranscriptScreen |
//! | list | (List of folded rows + composer) |
//! | (WidgetPtr| |
//! | swapped | (WidgetPtr swapped whole on session |
//! | on data) | switch or a new transcript event) |
//! +-----------+--------------------------------------+
//! ```
//!
//! **Deliberately left simple, and why**: every incoming SSE event refolds
//! the *entire* transcript (`client_core::transcript_fold::fold_event` is
//! already `O(items)` and a desktop session's conversation is small) and
//! rebuilds the whole right-hand widget tree from scratch, rather than
//! reaching for `TranscriptScreen::push_row`'s incremental append.
//! `push_row` cannot update a row already on screen -- only append a new
//! one -- and a streaming assistant reply is exactly a row whose *text*
//! keeps changing after it first appears (see `transcript-ui`'s own doc on
//! `fold_event` folding deltas into one growing item). A full rebuild
//! shows that growth correctly at the cost of redrawing everything each
//! time; fine for this proof, wrong for a long, fast-streaming transcript
//! -- the incremental path that fixes it needs `transcript-ui` to expose
//! updating a row in place, which it does not yet. The composer's
//! in-progress text survives a rebuild (`rebuild_transcript`'s
//! `in_progress` local) since the user typing a followup while a reply
//! streams in is the one case a naive rebuild would otherwise lose data
//! on.
//!
//! Background network I/O (`client_core::api`/`event_stream`, both
//! blocking by design -- see `client-core`'s `Cargo.toml`) runs on plain
//! `std::thread`s that report back through `winit`'s `EventLoopProxy`
//! (`Proxy<AppEvent>`), rather than through iris's own `Tasks`/`task_on`:
//! `Tasks` only requests a redraw once, after its whole async closure
//! finishes, which fits a single request-then-update but not a live SSE
//! loop that needs to be seen redrawing after *each* event it relays.
//! `Proxy::send_event` wakes the window's event loop immediately, once per
//! event, which is what a stream wants.
use client_core::api::{ApiClient, SessionSummary, UreqTransport};
use client_core::event_stream::{StreamItem, follow_session_events};
use client_core::transcript_fold::{TranscriptItem, fold_event, group_tool_runs};
use event_model::SeqEvent;
use iris::prelude::*;
use std::sync::Arc;
use std::sync::atomic::{AtomicU64, Ordering};
/// The session list column's width -- a fixed size for the simplest
/// layout that shows both panels at once (UI_RULES's text-truncation and
/// no-shrink rules apply to what's drawn inside it, not to this choice of
/// column width itself).
const LIST_WIDTH: f32 = 260.0;
/// Everything a background thread hands back to the window's event loop.
/// `generation` on the session-scoped variants is the generation
/// `select_session` was on when the thread started (`Client::generation`)
/// -- compared back against the current one before being applied, so a
/// slow response from a session the reader has since clicked away from
/// can't overwrite what replaced it.
enum AppEvent {
Sessions(Result<Vec<SessionSummary>, String>),
TranscriptLoaded {
session_id: String,
generation: u64,
result: Result<Vec<TranscriptItem>, String>,
},
StreamEvent {
session_id: String,
generation: u64,
event: SeqEvent,
},
StreamEnded {
session_id: String,
generation: u64,
message: Option<String>,
},
SendFailed(String),
}
pub fn run() {
DefaultApp::<Client>::run();
}
#[derive(DefaultUiState)]
struct Client {
ui_state: DefaultUiState,
api: Arc<ApiClient<UreqTransport>>,
/// A second, independent `UreqTransport` to the same server, used only
/// by `select_session`'s live-follow loop. `ApiClient` keeps its
/// transport private (rightly -- nothing outside it should reach past
/// the typed calls), so a caller that also needs the raw
/// `Transport::stream` for SSE, as this one does, builds its own
/// rather than the crate growing a getter whose only purpose would be
/// letting one caller reach around its own abstraction.
stream_transport: Arc<UreqTransport>,
proxy: Proxy<AppEvent>,
sessions: Vec<SessionSummary>,
selected: Option<String>,
items: Vec<TranscriptItem>,
list_ptr: WeakWidget<WidgetPtr>,
transcript_ptr: WeakWidget<WidgetPtr>,
screen: Option<transcript_ui::TranscriptScreen>,
/// Bumped every time the selected session changes; see `AppEvent`'s
/// doc for what it guards against.
generation: Arc<AtomicU64>,
}
impl DefaultAppState for Client {
type Event = AppEvent;
fn new(
mut ui_state: DefaultUiState,
rsc: &mut DefaultRsc<Self>,
proxy: Proxy<AppEvent>,
) -> Self {
// Re-validated here rather than threaded through from `main` --
// `DefaultApp::run()` takes no payload, so there is no other way
// to get `main`'s parsed CLI/config into this constructor. `main`
// already called this once to fail fast before a window opens;
// this call only fails if the filesystem changed underneath the
// process in between, which is not a case worth a nicer message.
let (server, ca_pem) = crate::load_startup_config().unwrap_or_else(|e| {
eprintln!("desktop-app: {e}");
std::process::exit(2);
});
let build_transport =
|| UreqTransport::new(server.base_url(), server.token.clone(), &ca_pem);
let (rest_transport, stream_transport) = build_transport()
.and_then(|rest| build_transport().map(|stream| (rest, stream)))
.unwrap_or_else(|e| {
eprintln!(
"desktop-app: couldn't set up TLS to {}: {e}",
server.base_url()
);
std::process::exit(1);
});
let api = Arc::new(ApiClient::new(rest_transport));
let stream_transport = Arc::new(stream_transport);
let list_ptr = WidgetPtr::new().add(rsc);
let transcript_ptr = WidgetPtr::new().add(rsc);
let loading = placeholder(rsc, "Loading sessions...");
transcript_ptr(rsc).set(loading);
(list_ptr.width(LIST_WIDTH), transcript_ptr.width(rest(1)))
.span(Dir::RIGHT)
.set_root(rsc, &mut ui_state);
let client = Self {
ui_state,
api,
stream_transport,
proxy,
sessions: Vec::new(),
selected: None,
items: Vec::new(),
list_ptr,
transcript_ptr,
screen: None,
generation: Arc::new(AtomicU64::new(0)),
};
client.spawn_fetch_sessions();
client
}
fn event(&mut self, event: AppEvent, rsc: &mut DefaultRsc<Self>, _render: &mut UiRenderState) {
match event {
AppEvent::Sessions(Ok(sessions)) => {
self.sessions = sessions;
self.rebuild_list(rsc);
if self.selected.is_none() {
self.show_message(rsc, "Select a session.");
}
}
AppEvent::Sessions(Err(message)) => {
self.show_message(rsc, &format!("Couldn't list sessions: {message}"));
}
AppEvent::TranscriptLoaded {
session_id,
generation,
result,
} => {
if self.current(&session_id, generation) {
match result {
Ok(items) => {
self.items = items;
self.rebuild_transcript(rsc);
}
Err(message) => {
self.show_message(
rsc,
&format!("Couldn't load {session_id}: {message}"),
);
}
}
}
}
AppEvent::StreamEvent {
session_id,
generation,
event,
} => {
if self.current(&session_id, generation) {
self.items = fold_event(&self.items, &event);
self.rebuild_transcript(rsc);
}
}
AppEvent::StreamEnded {
session_id,
generation,
message: Some(message),
} => {
if self.current(&session_id, generation) {
eprintln!("desktop-app: {session_id}'s live connection ended: {message}");
}
}
AppEvent::StreamEnded { .. } => {}
AppEvent::SendFailed(message) => {
eprintln!("desktop-app: couldn't send: {message}");
}
}
self.ui_state.window.request_redraw();
}
}
impl Client {
fn current(&self, session_id: &str, generation: u64) -> bool {
self.selected.as_deref() == Some(session_id)
&& self.generation.load(Ordering::SeqCst) == generation
}
/// Replaces the right-hand panel with a line of text -- built before
/// `transcript_ptr` is reached for, since building the message and
/// swapping it in both need `rsc` and can't overlap as one borrow.
fn show_message(&mut self, rsc: &mut DefaultRsc<Self>, message: &str) {
let widget = placeholder(rsc, message);
(self.transcript_ptr)(rsc).set(widget);
}
fn spawn_fetch_sessions(&self) {
let api = self.api.clone();
let proxy = self.proxy.clone();
std::thread::spawn(move || {
let result = api.fetch_sessions().map_err(|e| e.to_string());
let _ = proxy.send_event(AppEvent::Sessions(result));
});
}
fn rebuild_list(&mut self, rsc: &mut DefaultRsc<Self>) {
let list = Span::empty(Dir::DOWN).gap(2).add(rsc);
for session in &self.sessions {
let selected = self.selected.as_deref() == Some(session.id.as_str());
let row = session_row(rsc, session, selected);
list(rsc).push(row);
}
let tree = list
.background(rect(Color::rgb(24, 24, 28)))
.add_strong(rsc)
.any();
(self.list_ptr)(rsc).set(tree);
}
/// Selecting a session starts a fresh generation: any thread still
/// working for the previous one checks `Client::current` before
/// touching state, so a slow response for a session the reader has
/// clicked away from is silently dropped rather than overwriting what
/// replaced it.
fn select_session(&mut self, rsc: &mut DefaultRsc<Self>, session_id: String) {
let generation = self.generation.fetch_add(1, Ordering::SeqCst) + 1;
self.selected = Some(session_id.clone());
self.items.clear();
self.screen = None;
self.rebuild_list(rsc);
self.show_message(rsc, "Loading transcript...");
let api = self.api.clone();
let stream_transport = self.stream_transport.clone();
let proxy = self.proxy.clone();
let live_generation = self.generation.clone();
std::thread::spawn(move || {
// The most recent 200 events, coalesced -- plenty for a
// desktop proof; RUST.md's I3/history-paging work is what a
// real scrollback would reuse, out of scope here (E4 is only
// "the same screen runs in a window").
let page: Result<Vec<serde_json::Value>, String> = api
.fetch_transcript_page(&session_id, None, 200, true)
.map_err(|e| e.to_string());
// The raw wire `seq` of the last line fetched -- not the seq of
// the last *folded item*. A `TranscriptItem::AssistantMsg` keeps
// the seq of the first delta it accumulated (`fold_event`'s own
// doc: "a row whose identity changed with every delta would be
// a new row every frame"), so resuming the live stream from
// that seq re-delivers every delta already folded into it,
// duplicating the tail of whatever reply was mid-stream when
// the page was fetched. Found by screenshotting a real reply
// through `run-headless.sh`: the assistant's line read "You
// said: ... testsaid: ... test", the back half being deltas 2
// through N replayed onto an already-complete message.
let after = page
.as_ref()
.ok()
.and_then(|values| raw_seq(values.last()?))
.unwrap_or(0);
let result = page.and_then(|values| fold_page(&values));
let _ = proxy.send_event(AppEvent::TranscriptLoaded {
session_id: session_id.clone(),
generation,
result,
});
// Follows live from here in the same thread -- sequential
// rather than a second thread, since there is nothing to do
// with the stream until the page above has been sent anyway.
let stop = || live_generation.load(Ordering::SeqCst) != generation;
if stop() {
return;
}
let outcome =
follow_session_events(&*stream_transport, &session_id, after, |item| match item {
StreamItem::Open | StreamItem::Reset => !stop(),
StreamItem::Event { event, .. } => {
if stop() {
return false;
}
let _ = proxy.send_event(AppEvent::StreamEvent {
session_id: session_id.clone(),
generation,
event,
});
true
}
});
let _ = proxy.send_event(AppEvent::StreamEnded {
session_id,
generation,
message: outcome.err().map(|e| e.to_string()),
});
});
}
fn send_message(&mut self, session_id: String, text: String) {
let api = self.api.clone();
let proxy = self.proxy.clone();
std::thread::spawn(move || {
if let Err(e) = api.send_message(&session_id, &text, &[]) {
let _ = proxy.send_event(AppEvent::SendFailed(e.to_string()));
}
});
}
fn rebuild_transcript(&mut self, rsc: &mut DefaultRsc<Self>) {
let in_progress = self
.screen
.as_ref()
.map(|screen| screen.composer.field.edit(rsc).text.text().to_string())
.filter(|t| !t.is_empty());
let rows = group_tool_runs(&self.items);
let (screen, tree) = transcript_ui::build_tree(rsc, rows);
if let Some(text) = in_progress {
screen.composer.field.edit(rsc).set(&text);
}
if let Some(session_id) = self.selected.clone() {
let field = screen.composer.field;
rsc.register_event(field, Submit, move |ctx, rsc| {
let text = field.edit(rsc).take();
let text = text.trim().to_string();
if !text.is_empty() {
ctx.state.send_message(session_id.clone(), text);
}
});
}
(self.transcript_ptr)(rsc).set(tree);
self.screen = Some(screen);
}
}
/// One row in the session list: title on top, status below, highlighted
/// when it's the one currently shown.
fn session_row(
rsc: &mut DefaultRsc<Client>,
session: &SessionSummary,
selected: bool,
) -> StrongWidget {
let bg = if selected {
Color::rgb(58, 90, 138)
} else {
Color::rgb(38, 38, 44)
};
let id = session.id.clone();
let label = format!("{}\n{}", session.title, session.status);
wtext(label)
.color(Color::WHITE)
.wrap(true)
.pad(10)
.width(rest(1))
.background(rect(bg))
.on(
CursorSense::click(),
move |ctx, rsc: &mut DefaultRsc<Client>| {
ctx.state.select_session(rsc, id.clone());
},
)
.add_strong(rsc)
.any()
}
fn placeholder(rsc: &mut DefaultRsc<Client>, message: &str) -> StrongWidget {
wtext(message.to_string())
.color(Color::WHITE)
.wrap(true)
.pad(16)
.add_strong(rsc)
.any()
}
/// Folds a page of raw transcript lines (`ApiClient::fetch_transcript_page`'s
/// `Vec<Value>`) into the flat item list `client_core::transcript_fold`
/// works over. A line this build can't parse fails the whole page rather
/// than being skipped -- CODE_RULES's "an enumeration must be able to say
/// 'it broke'" -- since silently dropping one event could hide, say, the
/// user message the composer is about to look like it never sent.
fn fold_page(values: &[serde_json::Value]) -> Result<Vec<TranscriptItem>, String> {
let mut items = Vec::new();
for value in values {
let event: SeqEvent = serde_json::from_value(value.clone()).map_err(|e| {
format!("the server sent a transcript line this build couldn't parse: {e}")
})?;
items = fold_event(&items, &event);
}
Ok(items)
}
/// The wire `seq` a raw transcript line carries -- see `select_session`'s
/// comment on why the live-stream cursor has to be this, not a folded
/// item's `seq()`.
fn raw_seq(value: &serde_json::Value) -> Option<u64> {
value.get("seq")?.as_u64()
}
#[cfg(test)]
mod tests {
use super::*;
fn line(seq: u64, json: serde_json::Value) -> serde_json::Value {
let mut obj = json;
obj["seq"] = serde_json::json!(seq);
obj["ts"] = serde_json::json!(1.0);
obj
}
/// The regression for the bug a real `run-headless.sh` screenshot
/// found (see `select_session`'s comment): resuming the live stream
/// from the last *item's* seq re-delivers the deltas already folded
/// into a still-open assistant message, doubling its tail. `raw_seq`
/// of the last wire line must be the true high-water mark instead,
/// which for a run of deltas is higher than every item's own `seq()`.
#[test]
fn the_resume_cursor_is_the_last_wire_seq_not_the_last_items_seq() {
let values = vec![
line(1, serde_json::json!({"type": "userMessage", "text": "hi"})),
line(
2,
serde_json::json!({"type": "assistantText", "delta": "a"}),
),
line(
3,
serde_json::json!({"type": "assistantText", "delta": "b"}),
),
line(
4,
serde_json::json!({"type": "assistantText", "delta": "c"}),
),
];
let after = raw_seq(values.last().unwrap()).unwrap();
assert_eq!(after, 4);
let items = fold_page(&values).unwrap();
// The folded item keeps the *first* delta's seq (2), which is
// exactly the value that must not be used as the resume cursor.
let assistant_seq = items
.iter()
.find(|i| matches!(i, TranscriptItem::AssistantMsg { .. }))
.unwrap()
.seq();
assert_eq!(assistant_seq, 2);
assert_ne!(
after, assistant_seq,
"the fixed bug: these must differ here"
);
}
#[test]
fn a_page_folds_into_one_settled_assistant_message() {
let values = vec![
line(1, serde_json::json!({"type": "userMessage", "text": "hi"})),
line(
2,
serde_json::json!({"type": "assistantText", "delta": "hel"}),
),
line(
3,
serde_json::json!({"type": "assistantText", "delta": "lo"}),
),
];
let items = fold_page(&values).unwrap();
assert_eq!(
items,
vec![
TranscriptItem::UserMsg {
seq: 1,
text: "hi".to_string(),
attachments: Vec::new(),
},
TranscriptItem::AssistantMsg {
seq: 2,
text: "hello".to_string(),
settled: false,
},
]
);
}
#[test]
fn an_unparseable_line_fails_the_whole_page() {
let values = vec![serde_json::json!({"seq": 1, "ts": 1.0, "type": "not-a-real-type"})];
let err = fold_page(&values).unwrap_err();
assert!(err.contains("couldn't parse"));
}
}
+134
View File
@@ -0,0 +1,134 @@
//! Where the desktop app keeps the enrollment it should not have to be
//! told about a second time: `client_core::config::EnrolledServer`,
//! persisted at `$XDG_CONFIG_HOME/ai-app-desktop/enrollment.json`,
//! owner-only (0600) -- MACHINE.md's rule for anything holding a bearer
//! token, and the reason `client_core::config`'s own doc comment leaves
//! persistence and file mode to the caller.
//!
//! JSON rather than the project's usual RON: `wg-app-link`'s RON house
//! rules (`format`) are for configs a person hand-edits, and this file
//! never is one -- only this program ever writes or reads it, and
//! `serde_json` is already in the dependency graph through `client-core`,
//! so nothing new is added to reach for it.
use client_core::config::EnrolledServer;
use std::io;
use std::path::{Path, PathBuf};
/// `$XDG_CONFIG_HOME/ai-app-desktop`, falling back to `~/.config` the way
/// the XDG basedir spec says to when the variable is unset -- the same
/// fallback `wg_app_link::xdg::config_home` uses, reimplemented here
/// rather than depended on: that helper lives in the `wg-app-link`
/// submodule, which `server/` needs but this desktop-only crate does not,
/// and pulling in a git submodule for one path join would cost more than
/// it saves.
pub fn config_dir() -> PathBuf {
let base = std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.unwrap_or_else(|| {
let home = std::env::var_os("HOME").expect("HOME must be set");
PathBuf::from(home).join(".config")
});
base.join("ai-app-desktop")
}
fn enrollment_file(dir: &Path) -> PathBuf {
dir.join("enrollment.json")
}
/// Persists `server` under `dir` (`config_dir()` for real use; a tempdir in
/// the tests below), creating it if needed, and sets the file owner-only --
/// it carries a bearer token, the same reason `server/`'s own token store
/// is 0600.
pub fn save_enrollment_in(dir: &Path, server: &EnrolledServer) -> io::Result<()> {
std::fs::create_dir_all(dir)?;
let path = enrollment_file(dir);
let json = serde_json::to_vec_pretty(server)
.expect("EnrolledServer holds nothing that fails to serialise");
std::fs::write(&path, json)?;
#[cfg(unix)]
{
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(&path, std::fs::Permissions::from_mode(0o600))?;
}
Ok(())
}
/// `Ok(None)` when nothing has been enrolled yet, rather than an error --
/// "not enrolled" is an ordinary first-run state, not a failure (UI_RULES'
/// "a deliberate choice is not a problem to report" applies just as well
/// to a file that simply hasn't been written yet).
pub fn load_enrollment_in(dir: &Path) -> io::Result<Option<EnrolledServer>> {
let path = enrollment_file(dir);
match std::fs::read(&path) {
Ok(bytes) => {
let server = serde_json::from_slice(&bytes).map_err(|e| {
io::Error::new(
io::ErrorKind::InvalidData,
format!("{} is not a valid enrollment ({e})", path.display()),
)
})?;
Ok(Some(server))
}
Err(e) if e.kind() == io::ErrorKind::NotFound => Ok(None),
Err(e) => Err(e),
}
}
pub fn save_enrollment(server: &EnrolledServer) -> io::Result<()> {
save_enrollment_in(&config_dir(), server)
}
pub fn load_enrollment() -> io::Result<Option<EnrolledServer>> {
load_enrollment_in(&config_dir())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn a_saved_enrollment_reads_back_the_same() {
let dir = tempfile::tempdir().unwrap();
let server = EnrolledServer {
host: "127.0.0.1".to_string(),
port: 8547,
token: "tok".to_string(),
};
save_enrollment_in(dir.path(), &server).unwrap();
let read_back = load_enrollment_in(dir.path()).unwrap();
assert_eq!(read_back, Some(server));
}
#[test]
fn nothing_saved_yet_is_none_not_an_error() {
let dir = tempfile::tempdir().unwrap();
assert_eq!(load_enrollment_in(dir.path()).unwrap(), None);
}
#[test]
#[cfg(unix)]
fn the_saved_file_is_owner_only() {
use std::os::unix::fs::PermissionsExt;
let dir = tempfile::tempdir().unwrap();
let server = EnrolledServer {
host: "h".to_string(),
port: 1,
token: "t".to_string(),
};
save_enrollment_in(dir.path(), &server).unwrap();
let mode = std::fs::metadata(enrollment_file(dir.path()))
.unwrap()
.permissions()
.mode();
assert_eq!(mode & 0o777, 0o600);
}
#[test]
fn a_corrupt_file_is_named_in_the_error() {
let dir = tempfile::tempdir().unwrap();
std::fs::write(enrollment_file(dir.path()), b"not json").unwrap();
let err = load_enrollment_in(dir.path()).unwrap_err();
assert!(err.to_string().contains("enrollment.json"));
}
}
+95
View File
@@ -0,0 +1,95 @@
//! RUST.md's E4: the transcript screen (`transcript-ui`, I5) in a real
//! winit window on the desktop, with a session list beside it, talking to
//! a real `ai-server` over `client-core`'s REST + SSE clients. See
//! `app.rs`'s module doc for the widget tree and the event flow.
//!
//! Usage:
//!
//! desktop-app --ca /path/to/ca.pem --link 'aiapp://enroll?host=H&port=P&token=T'
//! desktop-app --ca /path/to/ca.pem # after the first run above
//!
//! `--link` is the same text `app/ui-sandbox.sh`'s banner prints and a
//! phone would scan as a QR (DECISIONS.md, 2026-09-05) -- pasted rather
//! than scanned, since a desktop has no camera to assume. It is parsed and
//! saved to `config::save_enrollment` once; later runs read it back and
//! `--link` is only needed again to enrol against a different server. The
//! CA is never persisted -- it is a public certificate whose path a
//! caller is expected to already know (`AGENTS.md`'s "prefer exercising
//! the server directly": the same `certs/ca.pem` a `curl --cacert` call
//! uses).
mod app;
mod config;
use client_core::config::EnrolledServer;
struct Args {
ca_path: std::path::PathBuf,
link: Option<String>,
}
fn parse_args() -> Result<Args, String> {
let mut ca_path = None;
let mut link = None;
let mut args = std::env::args().skip(1);
while let Some(arg) = args.next() {
match arg.as_str() {
"--ca" => {
ca_path = Some(std::path::PathBuf::from(
args.next().ok_or("--ca needs a path")?,
))
}
"--link" => link = Some(args.next().ok_or("--link needs a value")?),
other => return Err(format!("unrecognised argument '{other}'")),
}
}
Ok(Args {
ca_path: ca_path.ok_or(
"--ca PATH is required (the pinned CA's certificate, e.g. \
~/.config/ai-app/certs/ca.pem)",
)?,
link,
})
}
/// What `app.rs`'s `Client::new` needs to talk to the server: the enrolled
/// server (freshly parsed from `--link`, or read back from last time) and
/// the CA's PEM bytes. Loading is a pure function of the process's own
/// argv and config file, so it is safe to call again from `Client::new` --
/// see that call site's comment for why it is not threaded through some
/// other way (`DefaultApp::run()` takes no payload).
fn load_startup_config() -> Result<(EnrolledServer, Vec<u8>), String> {
let args = parse_args()?;
let server = match args.link {
Some(link) => {
let server = EnrolledServer::parse_link(&link)?;
config::save_enrollment(&server)
.map_err(|e| format!("couldn't save the enrollment: {e}"))?;
server
}
None => config::load_enrollment()
.map_err(|e| format!("couldn't read the saved enrollment: {e}"))?
.ok_or_else(|| {
format!(
"no server enrolled yet under {} -- pass --link 'aiapp://enroll?...' \
once (app/ui-sandbox.sh's start banner prints one)",
config::config_dir().display()
)
})?,
};
let ca_pem = std::fs::read(&args.ca_path)
.map_err(|e| format!("couldn't read the CA at {}: {e}", args.ca_path.display()))?;
Ok((server, ca_pem))
}
fn main() {
// Validated once here so a bad `--ca`/`--link` is reported on stderr
// before any window opens; `Client::new` calls this same function
// again once the window exists, so this first call is a fast-fail
// rather than the only place the values come from.
if let Err(e) = load_startup_config() {
eprintln!("desktop-app: {e}");
std::process::exit(2);
}
app::run();
}
+101
View File
@@ -0,0 +1,101 @@
//! (d) of IRIS_TODO.md's "Benchmarks" item: 1,000 image rows, checking that
//! standalone-image bind-group *creation* -- a real `wgpu` resource, unlike
//! the counters in `benches/message_list.rs` -- goes to zero once every
//! image has loaded. This needs an actual `wgpu` device (`GpuTextures`,
//! `UiRenderNode`), so unlike the rest of the suite it cannot run as a
//! plain binary; run it through `iris/run-headless.sh bench_images`, which
//! gives it a real (headless, GPU-accelerated) compositor and surface. See
//! `run-bench.sh` for the wrapper that greps its output into one line.
//!
//! Each `RedrawRequested` prints the frame number and
//! `UiRenderNode::take_image_bind_group_creates()` for that frame, then
//! requests another redraw (nothing else marks the scene dirty, so without
//! this the app would only ever draw once). The first frame is expected to
//! report 1,000 (one create per image, on first load); the steady state
//! IRIS_TODO.md asks this scenario to prove is every frame after settling
//! down to 0.
//!
//! After `SETTLE_FRAMES` it appends one *new* image row (a transcript
//! receiving one more message) and keeps counting -- a chat transcript's
//! real access pattern is "one more image arrives," not "reload the whole
//! list," so the steady-state question that actually matters is the
//! *incremental* cost of that one append, not just whether an untouched
//! scene costs zero. It exits after `FRAMES`.
use iris::prelude::*;
const ROWS: usize = 1000;
const SETTLE_FRAMES: usize = 4;
const FRAMES: usize = 6;
#[derive(DefaultUiState)]
struct State {
ui_state: DefaultUiState,
span: WeakWidget<Span>,
frame: usize,
appended: bool,
}
impl DefaultAppState for State {
fn new(
mut ui_state: DefaultUiState,
rsc: &mut DefaultRsc<Self>,
_: Proxy<Self::Event>,
) -> Self {
let mut span = Span::empty(Dir::DOWN);
for _ in 0..ROWS {
let img = image::DynamicImage::new_rgba8(32, 32);
let widget = image::<DefaultRsc<Self>>(img)(rsc);
let widget = rsc.ui.widgets.add_strong(widget);
span.push(widget.any());
}
let span = rsc.ui.widgets.add_strong(span);
let span_weak = span.weak();
let root = rsc.ui.widgets.add_strong(Scroll::new(span.any(), Axis::Y));
ui_state.set_root(root.any());
Self {
ui_state,
span: span_weak,
frame: 0,
appended: false,
}
}
fn window_event(
&mut self,
event: winit::event::WindowEvent,
rsc: &mut DefaultRsc<Self>,
_render: &mut UiRenderState,
) {
if !matches!(event, winit::event::WindowEvent::RedrawRequested) {
return;
}
self.frame += 1;
let creates = self.ui_state.renderer.ui.take_image_bind_group_creates();
println!(
"BENCH_IMAGES frame={} bind_group_creates={creates}",
self.frame
);
if self.frame == SETTLE_FRAMES && !self.appended {
self.appended = true;
let img = image::DynamicImage::new_rgba8(32, 32);
let widget = image::<DefaultRsc<Self>>(img)(rsc);
let widget = rsc.ui.widgets.add_strong(widget);
rsc.ui
.widgets
.get_mut(&self.span)
.unwrap()
.push(widget.any());
println!("BENCH_IMAGES appended one image after settling");
}
if self.frame < FRAMES {
self.ui_state.window.request_redraw();
} else {
std::process::exit(0);
}
}
}
fn main() {
DefaultApp::<State>::run();
}
+116
View File
@@ -0,0 +1,116 @@
//! RUST.md's I3: `iris::widget::List` with 800 rows of varied-length
//! wrapped text, one in twelve carrying a small image, scrollable with the
//! mouse wheel. Run headless with `iris/run-headless.sh message_list --shot
//! /tmp/message_list.png` -- there is no display on this machine, so that
//! is the only way to see it rendered; `run-tests.sh`/`cargo test` never
//! touch this file.
//!
//! Rows alternate two background tints so a screenshot can show the
//! boundary between adjacent rows even where the text itself wraps to a
//! different number of lines -- exactly the "variable-height rows" I3
//! asks for, and the thing a virtualised list gets wrong first if it is
//! wrong at all (a gap, an overlap, a row the wrong colour). This example
//! is also what found `List::place`'s oversized-background bug (see
//! list.rs's module doc and its `a_fill_shaped_background_is_not_left_
//! oversized` test) -- a plain unit test could have (and now does) catch
//! it directly, but it was this screenshot rendering as a single blank
//! tinted rectangle that pointed at it first.
use iris::prelude::*;
use winit::{dpi::LogicalSize, window::WindowAttributes};
fn main() {
DefaultApp::<State>::run();
}
#[derive(DefaultUiState)]
struct State {
ui_state: DefaultUiState,
}
const ROWS: usize = 800;
const IMAGE_EVERY: usize = 12;
/// Repeats a short sentence a varying number of times per row so real
/// wrapping happens at every row height from one line to several, rather
/// than every row being identically tall (which would render correctly
/// even with a broken height measurement).
fn row_text(i: usize) -> String {
const SENTENCE: &str =
"Iris lays out this row once and moves it on scroll, never re-laying it out. ";
let repeats = 1 + (i * 7) % 5;
format!("Message {i}: {}", SENTENCE.repeat(repeats))
}
/// A small solid-colour square standing in for a real decoded image --
/// what matters for I3 is that a row can carry an `Image` widget at all,
/// not what the picture shows.
fn row_image(i: usize) -> image::DynamicImage {
let hue = ((i * 47) % 255) as u8;
image::RgbaImage::from_pixel(48, 48, image::Rgba([hue, 128, 255 - hue, 255])).into()
}
fn build_row<Rsc: UiRsc + 'static>(rsc: &mut Rsc, i: usize) -> StrongWidget {
let tint = if i.is_multiple_of(2) {
Color::rgb(120, 130, 170)
} else {
Color::rgb(70, 80, 140)
};
let text_color = Color::BLACK;
if i.is_multiple_of(IMAGE_EVERY) {
let text = wtext(row_text(i))
.wrap(true)
.color(text_color)
.add_strong(rsc)
.any();
let img = image::<Rsc>(row_image(i))(rsc);
let img = rsc.widgets_mut().add_strong(img).any();
let mut span = Span::empty(Dir::DOWN);
span.push(text);
span.push(img);
span.pad(8.0).background(rect(tint)).add_strong(rsc).any()
} else {
wtext(row_text(i))
.wrap(true)
.color(text_color)
.pad(8.0)
.background(rect(tint))
.add_strong(rsc)
.any()
}
}
impl DefaultAppState for State {
// A phone-plausible portrait shape (the transcript screen this is
// standing in for). The tiling headless compositor `run-headless.sh`
// uses ignores this and fills its own 1920x1200 output regardless, but
// it's a correct hint for any other backend (a real window manager, or
// android-view) and costs nothing to state.
fn window_attributes() -> WindowAttributes {
WindowAttributes::default().with_inner_size(LogicalSize::new(420.0, 900.0))
}
fn new(
mut ui_state: DefaultUiState,
rsc: &mut DefaultRsc<Self>,
_: Proxy<Self::Event>,
) -> Self {
let mut list = List::new(Axis::Y);
for i in 0..ROWS {
let row = build_row(rsc, i);
list.push_back(ListRow::new(i as u64, row));
}
let root = list
.on(CursorSense::Scroll, |ctx, rsc| {
let delta = ctx.data.scroll_delta.y * 50.0;
ctx.widget(rsc).scroll(delta);
})
.masked()
.background(rect(Color::WHITE))
.add_strong(rsc);
ui_state.set_root(root.any());
Self { ui_state }
}
}
+9 -186
View File
@@ -1,13 +1,14 @@
use std::{cell::RefCell, rc::Rc};
use winit::event::WindowEvent;
use iris::prelude::*;
type ClientRsc = DefaultRsc<Client>;
use winit::event::WindowEvent;
fn main() {
DefaultApp::<Client>::run();
}
/// The tabs example: five demo panes plus a message composer, built by
/// `tabs_ui::build` and driven here through the winit backend. The same
/// widget tree also runs on the android-view backend, through
/// `iris-android-app` -- see RUST.md's I2.
#[derive(DefaultUiState)]
pub struct Client {
ui_state: DefaultUiState,
@@ -20,189 +21,11 @@ impl DefaultAppState for Client {
rsc: &mut DefaultRsc<Self>,
_: Proxy<Self::Event>,
) -> Self {
let rrect = rect(Color::WHITE).radius(20);
let pad_test = (
rrect.color(Color::BLUE),
(
rrect
.color(Color::RED)
.sized((100, 100))
.center()
.width(rest(2)),
(
rrect.color(Color::ORANGE),
rrect.color(Color::LIME).pad(10.0),
)
.span(Dir::RIGHT)
.width(rest(2)),
rrect.color(Color::YELLOW),
)
.span(Dir::RIGHT)
.pad(10)
.width(rest(3)),
)
.span(Dir::RIGHT)
.add(rsc);
let span_test = (
rrect.color(Color::GREEN).width(100),
rrect.color(Color::ORANGE),
rrect.color(Color::CYAN),
rrect.color(Color::BLUE).width(rel(0.5)),
rrect.color(Color::MAGENTA).width(100),
rrect.color(Color::RED).width(100),
)
.span(Dir::LEFT)
.add(rsc);
let span_add = Span::empty(Dir::RIGHT).add(rsc);
let add_button = rect(Color::LIME)
.radius(30)
.on(CursorSense::click(), move |_, rsc| {
let child = image(include_bytes!("assets/sungals.png"))
.center()
.add_strong(rsc);
span_add(rsc).push(child);
})
.sized((150, 150))
.align(Align::BOT_RIGHT);
let del_button = rect(Color::RED)
.radius(30)
.on(CursorSense::click(), move |_, rsc| {
span_add(rsc).pop();
})
.sized((150, 150))
.align(Align::BOT_LEFT);
let span_add_test = (span_add, add_button, del_button).stack().add(rsc);
let btext = |content| wtext(content).size(30);
let text_test = (
btext("this is a").align(Align::LEFT),
btext("teeeeeeeest").align(Align::RIGHT),
btext("okkk\nokkkkkk!").align(Align::LEFT),
btext("hmm"),
btext("a"),
(
btext("'").family(Family::Monospace).align(Align::TOP),
btext("'").family(Family::Monospace),
btext(":gamer mode").family(Family::Monospace),
rect(Color::CYAN).sized((10, 10)).center(),
rect(Color::RED).sized((100, 100)).center(),
rect(Color::PURPLE).sized((50, 50)).align(Align::TOP),
)
.span(Dir::RIGHT)
.center(),
wtext("pretty cool right?").size(50),
)
.span(Dir::DOWN)
.add(rsc);
let texts = Span::empty(Dir::DOWN).gap(10).add(rsc);
let msg_area = texts.scrollable().masked().background(rect(Color::SKY));
let add_text = wtext("add")
.editable(EditMode::MultiLine)
.text_align(Align::LEFT)
.size(30)
.attr::<Selectable>(())
.on(Submit, move |ctx, rsc| {
let w = ctx.widget;
let content = w.edit(rsc).take();
let text = wtext(content)
.editable(EditMode::MultiLine)
.size(30)
.text_align(Align::LEFT)
.wrap(true)
.attr::<Selectable>(());
let msg_box = text
.background(rect(Color::WHITE.darker(0.5)))
.add_strong(rsc);
texts(rsc).push(msg_box);
})
.add(rsc);
let text_edit_scroll = (
msg_area.height(rest(1)),
(
Rect::new(Color::WHITE.darker(0.9)),
(
add_text.width(rest(1)),
Rect::new(Color::GREEN)
.on(CursorSense::click(), move |ctx, rsc: &mut ClientRsc| {
rsc.run_event::<Submit>(add_text, (), ctx.state);
})
.sized((40, 40)),
)
.span(Dir::RIGHT)
.pad(10),
)
.stack()
.size(StackSize::Child(1))
.layer_offset(1)
.align(Align::BOT),
)
.span(Dir::DOWN)
.add(rsc);
let main = WidgetPtr::new().add(rsc);
let vals = Rc::new(RefCell::new((0, Vec::new())));
let mut switch_button = |color, to: WeakWidget, label| {
let to = to.upgrade(rsc);
let vec = &mut vals.borrow_mut().1;
let i = vec.len();
if vec.is_empty() {
vec.push(None);
main(rsc).set(to);
} else {
vec.push(Some(to));
let widgets = tabs_ui::build(rsc, &mut ui_state);
Self {
ui_state,
info: widgets.info,
}
let vals = vals.clone();
let rect = rect(color)
.on(CursorSense::click(), move |ctx, rsc| {
let (prev, vec) = &mut *vals.borrow_mut();
if let Some(h) = vec[i].take() {
vec[*prev] = main(rsc).replace(h);
*prev = i;
}
ctx.widget(rsc).color = color.darker(0.3);
})
.on(
CursorSense::HoverStart | CursorSense::unclick(),
move |ctx, rsc| {
ctx.widget(rsc).color = color.brighter(0.2);
},
)
.on(CursorSense::HoverEnd, move |ctx, rsc| {
ctx.widget(rsc).color = color;
});
(rect, wtext(label).size(30).text_align(Align::CENTER)).stack()
};
let tabs = (
switch_button(Color::RED, pad_test, "pad"),
switch_button(Color::GREEN, span_test, "span"),
switch_button(Color::BLUE, span_add_test, "image span"),
switch_button(Color::MAGENTA, text_test, "text layout"),
switch_button(
Color::YELLOW.mul_rgb(0.5),
text_edit_scroll,
"text edit scroll",
),
)
.span(Dir::RIGHT);
let info = wtext("").add(rsc);
let info_sect = info.pad(10).align(Align::RIGHT);
((tabs.height(40), main.pad(10)).span(Dir::DOWN), info_sect)
.stack()
.set_root(rsc, &mut ui_state);
Self { ui_state, info }
}
fn window_event(
+24
View File
@@ -0,0 +1,24 @@
#!/bin/sh
# Runs iris's on-demand benchmark suite (IRIS_TODO.md's "Benchmarks" item).
# Never run by `cargo test`; run this by hand or before/after a layout
# change. Always release -- see AGENTS.md's own rule against reading a
# frame time from a debug build.
#
# ./run-bench.sh # everything
# ./run-bench.sh list # just the CPU-only message-list scenarios
# ./run-bench.sh images # just the GPU bind-group-creation scenario
set -eu
here=$(cd "$(dirname "$0")" && pwd)
cd "$here"
what="${1:-all}"
if [ "$what" = "all" ] || [ "$what" = "list" ]; then
echo "=== message_list (CPU-only, no window) ==="
cargo bench --bench message_list
fi
if [ "$what" = "all" ] || [ "$what" = "images" ]; then
echo "=== bench_images (real wgpu device, via run-headless.sh) ==="
timeout 60 ./run-headless.sh bench_images --seconds 4 2>&1 | grep "^BENCH_IMAGES"
fi
+21 -2
View File
@@ -4,6 +4,16 @@
# ./run-headless.sh tabs [-- cargo args]
# ./run-headless.sh tabs --shot /tmp/tabs.png --seconds 4
#
# `--bin` runs a real crate binary instead of an example (E4's
# `desktop-app`, which is a window a person runs, not a demo) --
# `cargo build --bin NAME` instead of `--example NAME`, and
# `target/debug/NAME` instead of `target/debug/examples/NAME`. Its own
# argv (the CLI flags a real binary takes, as opposed to `cargo build`'s
# own flags after `--`) comes through `$RUN_HEADLESS_ARGS`, word-split on
# purpose -- an example never needed one, so there was nowhere to plumb it
# through positionally without disturbing the existing `-- cargo args`
# convention above.
#
# The VM has a virtio-gpu render node (Vulkan 1.4 through Venus, GL 4.6
# through virgl), so wgpu runs on the host's real GPU -- what is missing is
# only a compositor to give winit a surface. So: a headless sway, the same
@@ -20,16 +30,18 @@ run="${XDG_RUNTIME_DIR:-/tmp}/iris-headless"
seconds=3
shot=""
example=""
kind=example
while [ $# -gt 0 ]; do
case "$1" in
--shot) shot=$2; shift 2 ;;
--seconds) seconds=$2; shift 2 ;;
--bin) kind=bin; shift ;;
--) shift; break ;;
*) example=$1; shift ;;
esac
done
[ -n "$example" ] || { echo "usage: $0 EXAMPLE [--shot PNG] [--seconds N] [-- cargo args]" >&2; exit 2; }
[ -n "$example" ] || { echo "usage: $0 NAME [--bin] [--shot PNG] [--seconds N] [-- cargo args]" >&2; exit 2; }
mkdir -p "$run"
export SWAYSOCK="$run/sway.sock"
@@ -67,10 +79,17 @@ export WAYLAND_DISPLAY
echo "run-headless: $WAYLAND_DISPLAY (sway $(swaymsg -t get_version --raw | sed -n 's/.*"human_readable":"\([^"]*\)".*/\1/p'))" >&2
cd "$here"
if [ "$kind" = bin ]; then
cargo build --bin "$example" "$@" >&2
bin="$here/target/debug/$example"
else
cargo build --example "$example" "$@" >&2
bin="$here/target/debug/examples/$example"
fi
"$bin" >"$run/$example.log" 2>&1 &
# shellcheck disable=SC2086 -- deliberately word-split: this is the
# binary's own argv, not a single path.
"$bin" ${RUN_HEADLESS_ARGS:-} >"$run/$example.log" 2>&1 &
pid=$!
trap 'kill "$pid" 2>/dev/null || true' EXIT INT TERM
+129
View File
@@ -0,0 +1,129 @@
//! Pass conditions for RUST.md's I4, exercised the same way
//! `layout_tests.rs` exercises LAYOUT.md's: `AccessTree` only touches
//! `Widgets`/`UiRenderState`, neither of which needs a GPU or a window, so
//! it can be driven directly against `layout_tests::TestRsc`.
use crate::layout_tests::TestRsc;
use crate::prelude::*;
#[test]
fn a_named_widget_reaches_the_tree_with_its_role_and_bounds() {
let mut rsc = TestRsc {
ui: UiData::default(),
};
let leaf: WeakWidget<Rect> = rect(UiColor::WHITE).label("Add task").add(&mut rsc);
let root = leaf.upgrade(&mut rsc).any();
let mut render = UiRenderState::new();
render.resize((800.0, 600.0));
render.update(&root, &mut rsc);
let mut access = AccessTree::new();
let update = access
.update(rsc.widgets(), &render, &rsc)
.expect("a first draw with a named widget must produce a tree");
// One node for the widget, one for the synthetic window root.
assert_eq!(update.nodes.len(), 2);
let (_, node) = update
.nodes
.iter()
.find(|(_, n)| n.role() != accesskit::Role::Window)
.expect("the named widget's own node");
assert_eq!(node.label(), Some("Add task"));
assert_eq!(node.role(), accesskit::Role::Unknown);
let bounds = node.bounds().expect("a drawn widget reports its bounds");
let region = render
.window_region(&leaf, &rsc)
.expect("the widget is active after render.update");
assert_eq!(bounds.x0, region.top_left.x as f64);
assert_eq!(bounds.y0, region.top_left.y as f64);
assert_eq!(bounds.x1, region.bot_right.x as f64);
assert_eq!(bounds.y1, region.bot_right.y as f64);
}
#[test]
fn a_widget_with_no_label_never_reaches_the_tree() {
let mut rsc = TestRsc {
ui: UiData::default(),
};
let root = rsc.ui.widgets.add_strong(rect(UiColor::WHITE));
let mut render = UiRenderState::new();
render.resize((800.0, 600.0));
render.update(&root.any(), &mut rsc);
let mut access = AccessTree::new();
assert!(
access.update(rsc.widgets(), &render, &rsc).is_none(),
"no widget was ever `.label()`ed, so there is nothing to report -- \
not even an empty tree change"
);
}
/// LAYOUT.md's "a moved subtree" lesson applies here too: `resolved_region`
/// (which `window_region` sits on) walks the move-offset chain, so a
/// widget moved via `Offset` -- not redrawn from scratch -- must still
/// report where it actually ended up.
#[test]
fn bounds_follow_a_moved_widget_and_updates_stay_incremental() {
let mut rsc = TestRsc {
ui: UiData::default(),
};
let leaf: WeakWidget<Rect> = rect(UiColor::WHITE).label("thing").add(&mut rsc);
let leaf_strong = leaf.upgrade(&mut rsc).any();
let offset = rsc.ui.widgets.add_strong(Offset {
inner: leaf_strong,
amt: UiVec2::ZERO,
});
let offset_id = offset.weak();
let root = offset.any();
let mut render = UiRenderState::new();
render.resize((800.0, 600.0));
render.update(&root, &mut rsc);
let mut access = AccessTree::new();
access
.update(rsc.widgets(), &render, &rsc)
.expect("the first draw is always a change");
assert_eq!(access.take_rebuilds(), 1);
// Unchanged frame: nothing moved, nothing renamed -- `update` must
// report no change, and the rebuild counter (I4's twin of
// `take_counters`) must stay at 0.
render.update(&root, &mut rsc);
assert!(access.update(rsc.widgets(), &render, &rsc).is_none());
assert_eq!(access.take_rebuilds(), 0);
// Move the child via `Offset` (a move-offset write, not necessarily a
// full redraw of the leaf -- see `resolve_move_chain`) and confirm the
// reported bounds shifted by exactly that amount, in exactly one more
// rebuild.
let before = render
.window_region(&leaf, &rsc)
.expect("active before the move");
rsc.ui.widgets.get_mut(&offset_id).unwrap().amt = UiVec2::abs(Vec2::new(50.0, 0.0));
render.update(&root, &mut rsc);
let update = access
.update(rsc.widgets(), &render, &rsc)
.expect("a moved named widget is a change");
assert_eq!(access.take_rebuilds(), 1);
let after = render
.window_region(&leaf, &rsc)
.expect("still active after the move");
// Not asserting the exact delta: `Offset`'s own `amt` -> pixel mapping
// is that widget's business, not this tree's. What I4 owns is that
// `AccessTree` reports whatever `window_region` says *now* -- so the
// node must have moved, and in the direction the offset moved it.
assert!(
after.top_left.x > before.top_left.x,
"the leaf's reported bounds must move right along with its offset"
);
let (_, node) = update
.nodes
.iter()
.find(|(_, n)| n.role() != accesskit::Role::Window)
.unwrap();
let bounds = node.bounds().unwrap();
assert_eq!(bounds.x0, after.top_left.x as f64);
}
+86
View File
@@ -0,0 +1,86 @@
//! I4 (RUST.md): the Android half of the AccessKit push, over
//! `accesskit_android::Adapter` and android-view's
//! `AccessibilityNodeProvider`. Carries E1's mitigation for the adapter's
//! reproducible abort: `accesskit_android`'s `State` (0.4.0 and 0.8.0
//! alike) never moves back to `Inactive` once a client attaches, so once
//! one has, every later `QueuedEvents::raise` reaches
//! `AccessibilityManager.sendAccessibilityEvent` -- which throws if
//! accessibility has since been switched off (or the client detached),
//! and android-view's `panic = "abort"` turns that Java exception into a
//! process kill. `raise_if_enabled` is the gate: ask
//! `AccessibilityManager.isEnabled()` immediately before every `raise`
//! and drop the events instead of calling it when the answer is no. See
//! RUST.md's E1 box for the full repro.
use accesskit::{ActionHandler, ActionRequest, ActivationHandler, TreeUpdate};
use accesskit_android::QueuedEvents;
use android_view::{
View,
jni::{JNIEnv, objects::JObject},
};
use iris_core::{AccessTree, UiRenderState, UiRsc, Widgets};
/// The `ActivationHandler` `accesskit_android::Adapter` asks for its
/// initial tree from -- unlike `accesskit_winit`'s handlers (see
/// `default/access.rs`), this one is only ever invoked synchronously from
/// inside a JNI callback that already holds everything it needs, so it can
/// just borrow `IrisViewPeer`'s own fields for the length of one call
/// rather than going through a channel.
pub(super) struct AndroidAccessSource<'a> {
pub widgets: &'a Widgets,
pub render: &'a UiRenderState,
pub rsc: &'a dyn UiRsc,
}
impl ActivationHandler for AndroidAccessSource<'_> {
fn request_initial_tree(&mut self) -> Option<TreeUpdate> {
Some(AccessTree::build_full(self.widgets, self.render, self.rsc))
}
}
/// Every AccessKit action request is inert here -- see this module's doc
/// comment and `default/access.rs`'s matching handler for why: a screen
/// reader's tap on a named node is a real touch delivered at that node's
/// bounds, which the ordinary pointer path already handles once the
/// bounds `AccessTree` reports are right.
pub(super) struct NullActionHandler;
impl ActionHandler for NullActionHandler {
fn do_action(&mut self, _request: ActionRequest) {}
}
fn is_accessibility_enabled<'local>(env: &mut JNIEnv<'local>, view: &View<'local>) -> bool {
let context = view.context(env);
let name = env.new_string("accessibility").unwrap();
let manager: JObject = env
.call_method(
&context.0,
"getSystemService",
"(Ljava/lang/String;)Ljava/lang/Object;",
&[(&name).into()],
)
.unwrap()
.l()
.unwrap();
if manager.is_null() {
return false;
}
env.call_method(&manager, "isEnabled", "()Z", &[])
.unwrap()
.z()
.unwrap()
}
/// The one place `QueuedEvents::raise` may be called -- see this module's
/// doc comment. Every call site pushes this as a deferred callback rather
/// than calling it inline, matching android-view's own demo: `raise`
/// itself asks not to be called while the caller holds locks a framework
/// callback might, and a deferred callback runs after the current one has
/// returned them.
pub(super) fn raise_if_enabled<'local>(
env: &mut JNIEnv<'local>,
view: &View<'local>,
events: QueuedEvents,
) {
if is_accessibility_enabled(env, view) {
events.raise(env, &view.0);
}
}
+25
View File
@@ -0,0 +1,25 @@
use crate::attr::{FocusHost, recent_click};
use crate::prelude::*;
use super::view::HasAndroidUiState;
impl<T: HasAndroidUiState> FocusHost for T {
fn recent_click(&mut self) -> bool {
recent_click(&mut self.android_state_mut().last_click)
}
fn set_focus(&mut self, id: Option<WeakWidget<TextEdit>>) {
self.android_state_mut().focus = id;
}
fn focus_gained(&mut self, region: Option<PixelRegion>) {
// Showing the keyboard is a JNI call (`InputMethodManager.showSoftInput`),
// and this runs deep inside the platform-agnostic sensor dispatch
// with no `CallbackCtx` in reach -- `IrisViewPeer::after_input`
// (`view.rs`) is what actually makes the call, right after the
// sensor pass that got here returns.
if region.is_some() {
self.android_state_mut().pending_show_keyboard = true;
}
}
}
+272
View File
@@ -0,0 +1,272 @@
//! `InputConnection`, implemented directly against a focused `TextEdit`
//! rather than against a stand-in editor the way android-view's own demo
//! does over its `parley::PlainEditor` -- I1 already put parley behind
//! `TextEdit`, so this is that same bridge, just wired to iris's widget
//! instead of a bespoke one. Follows `demo/src/lib.rs`'s
//! `impl InputConnection for DemoViewPeer`, which is where RUST.md's E1
//! found the shape this needs (`text_before_cursor` is what gets Gboard's
//! suggestion strip to read real words out of the buffer).
//!
//! Two things the demo tracks that this does not, both noted rather than
//! silently dropped: a real "composing region" distinct from the
//! selection (`set_composing_region` here just moves the caret, since
//! `TextEdit` has no third range to hold one), and batch-edit coalescing
//! (`begin`/`end_batch_edit` are no-ops -- a redraw mid-batch costs a frame
//! it does not need to, not correctness).
use crate::prelude::*;
use android_view::{
CAP_MODE_SENTENCES, CallbackCtx, EditorInfo, IME_FLAG_NO_ENTER_ACTION, IME_FLAG_NO_EXTRACT_UI,
IME_FLAG_NO_FULLSCREEN, INPUT_TYPE_CLASS_TEXT, INPUT_TYPE_TEXT_FLAG_AUTO_CORRECT,
INPUT_TYPE_TEXT_FLAG_CAP_SENTENCES, INPUT_TYPE_TEXT_FLAG_MULTI_LINE, InputConnection,
caps_mode,
};
use std::borrow::Cow;
use super::view::{AndroidAppState, IrisViewPeer};
/// Byte offset -> UTF-16 code unit offset, the unit every `InputConnection`
/// method speaks in (Java strings are UTF-16). `TextEdit` is byte-indexed
/// throughout since I1 moved it to parley -- see `edit.rs`'s doc comment on
/// `text()` -- so every crossing of this boundary goes through here rather
/// than through ad hoc counting at each call site.
fn byte_to_utf16(text: &str, byte_idx: usize) -> usize {
text[..byte_idx].encode_utf16().count()
}
fn utf16_to_byte(text: &str, utf16_idx: usize) -> usize {
let mut utf16_len = 0;
for (byte_idx, ch) in text.char_indices() {
if utf16_len >= utf16_idx {
return byte_idx;
}
utf16_len += ch.len_utf16();
}
text.len()
}
impl<State: AndroidAppState> IrisViewPeer<State> {
fn focus(&self) -> Option<WeakWidget<TextEdit>> {
self.state.android_state().focus
}
}
impl<State: AndroidAppState> InputConnection for IrisViewPeer<State> {
fn on_create_input_connection<'local>(
&mut self,
ctx: &mut CallbackCtx<'local>,
out_attrs: &EditorInfo<'local>,
) {
// Set once per `InputConnection`, not per field -- Android calls
// this when the view (not a particular widget) attaches to an
// IME. `MULTI_LINE`/`AUTO_CORRECT`/`CAP_SENTENCES` cover both the
// tabs example's composer and a plain single-line field well
// enough that no per-field variant is worth the extra state yet.
out_attrs.set_input_type(
&mut ctx.env,
INPUT_TYPE_CLASS_TEXT
| INPUT_TYPE_TEXT_FLAG_CAP_SENTENCES
| INPUT_TYPE_TEXT_FLAG_AUTO_CORRECT
| INPUT_TYPE_TEXT_FLAG_MULTI_LINE,
);
out_attrs.set_ime_options(
&mut ctx.env,
IME_FLAG_NO_FULLSCREEN | IME_FLAG_NO_EXTRACT_UI | IME_FLAG_NO_ENTER_ACTION,
);
if let Some(focus) = self.focus() {
let text = &self.rsc[focus];
let sel = text.selection_range().unwrap_or(0..0);
let start = byte_to_utf16(text.text(), sel.start) as i32;
let end = byte_to_utf16(text.text(), sel.end) as i32;
out_attrs.set_initial_sel_start(&mut ctx.env, start);
out_attrs.set_initial_sel_end(&mut ctx.env, end);
let caps = caps_mode(
&mut ctx.env,
text.text(),
start as usize,
CAP_MODE_SENTENCES,
);
out_attrs.set_initial_caps_mode(&mut ctx.env, caps);
}
}
fn text_before_cursor<'slf>(
&'slf mut self,
_ctx: &mut CallbackCtx,
n: i32,
) -> Option<Cow<'slf, str>> {
if n < 0 {
return None;
}
let focus = self.focus()?;
let text = &self.rsc[focus];
let sel = text.selection_range()?;
let end_16 = byte_to_utf16(text.text(), sel.start);
let start_16 = end_16.saturating_sub(n as usize);
let start = utf16_to_byte(text.text(), start_16);
Some(Cow::Borrowed(&text.text()[start..sel.start]))
}
fn text_after_cursor<'slf>(
&'slf mut self,
_ctx: &mut CallbackCtx,
n: i32,
) -> Option<Cow<'slf, str>> {
if n < 0 {
return None;
}
let focus = self.focus()?;
let text = &self.rsc[focus];
let sel = text.selection_range()?;
let len_16 = byte_to_utf16(text.text(), text.text().len());
let start_16 = byte_to_utf16(text.text(), sel.end);
let end_16 = (start_16 + n as usize).min(len_16);
let end = utf16_to_byte(text.text(), end_16);
Some(Cow::Borrowed(&text.text()[sel.end..end]))
}
fn selected_text<'slf>(&'slf mut self, _ctx: &mut CallbackCtx) -> Option<Cow<'slf, str>> {
let focus = self.focus()?;
Some(Cow::Owned(self.rsc[focus].selected_text()?))
}
fn cursor_caps_mode(&mut self, ctx: &mut CallbackCtx, req_modes: u32) -> u32 {
let Some(focus) = self.focus() else {
return 0;
};
let text = &self.rsc[focus];
let Some(caret) = text.caret() else {
return 0;
};
let off = byte_to_utf16(text.text(), caret);
caps_mode(&mut ctx.env, text.text(), off, req_modes)
}
fn delete_surrounding_text(
&mut self,
ctx: &mut CallbackCtx,
before_length: i32,
after_length: i32,
) -> bool {
let Some(focus) = self.focus() else {
return false;
};
let text = &self.rsc[focus];
let Some(sel) = text.selection_range() else {
return false;
};
let content = text.text();
let start_16 =
byte_to_utf16(content, sel.start).saturating_sub(before_length.max(0) as usize);
let len_16 = byte_to_utf16(content, content.len());
let end_16 = (byte_to_utf16(content, sel.end) + after_length.max(0) as usize).min(len_16);
let start = utf16_to_byte(content, start_16);
let end = utf16_to_byte(content, end_16);
focus.edit(&mut self.rsc).delete_byte_range(start, end);
self.after_input(ctx);
true
}
fn delete_surrounding_text_in_code_points(
&mut self,
ctx: &mut CallbackCtx,
before_length: i32,
after_length: i32,
) -> bool {
// Approximated as UTF-16 units rather than Unicode scalar values --
// the two differ only outside the Basic Multilingual Plane, which
// this widget tree does not exercise today. Worth revisiting if a
// field ever needs to edit emoji or other astral-plane text well.
self.delete_surrounding_text(ctx, before_length, after_length)
}
fn set_composing_text(
&mut self,
ctx: &mut CallbackCtx,
text: &str,
_new_cursor_position: i32,
) -> bool {
let Some(focus) = self.focus() else {
return false;
};
// The IME re-sends its whole composition on every keystroke;
// `compose_len` (chars, not bytes -- `TextEditCtx::replace`'s unit)
// is what lets `replace` remove exactly what it inserted last time.
// The same shape as `default::DefaultApp`'s `Ime::Preedit` handling
// for winit.
let compose_len = self.state.android_state().compose_len;
focus.edit(&mut self.rsc).replace(compose_len, text);
self.state.android_state_mut().compose_len = text.chars().count();
self.after_input(ctx);
true
}
fn set_composing_region(&mut self, _ctx: &mut CallbackCtx, _start: i32, _end: i32) -> bool {
// `TextEdit` has no separate composing range to move -- see this
// module's doc comment. Declining (rather than moving the caret,
// which would surprise a caller expecting only a style change)
// is the safer approximation.
false
}
fn finish_composing_text(&mut self, ctx: &mut CallbackCtx) -> bool {
self.state.android_state_mut().compose_len = 0;
self.after_input(ctx);
true
}
fn set_selection(&mut self, ctx: &mut CallbackCtx, start: i32, end: i32) -> bool {
let Some(focus) = self.focus() else {
return false;
};
let text = &self.rsc[focus];
let content = text.text();
// Collapsed to `end`: `TextEditCtx` has no range-selection setter
// yet (nothing before I2 needed one), so an IME-driven selection
// lands the caret at its focus end rather than spanning both.
let byte = utf16_to_byte(content, end.max(0) as usize);
focus.edit(&mut self.rsc).set_cursor_byte(byte);
let _ = start;
self.after_input(ctx);
true
}
fn perform_editor_action(&mut self, _ctx: &mut CallbackCtx, _editor_action: i32) -> bool {
// `IME_FLAG_NO_ENTER_ACTION` above asks the IME not to offer one;
// nothing here needs handling it yet.
false
}
fn begin_batch_edit(&mut self, _ctx: &mut CallbackCtx) -> bool {
true
}
fn end_batch_edit(&mut self, _ctx: &mut CallbackCtx) -> bool {
true
}
fn send_key_event<'local>(
&mut self,
ctx: &mut CallbackCtx<'local>,
event: &android_view::KeyEvent<'local>,
) -> bool {
let key_code = event.key_code(&mut ctx.env);
let handled = super::input::on_key(
&mut self.rsc,
&mut self.state,
&mut ctx.env,
key_code,
event,
);
if handled {
self.after_input(ctx);
}
handled
}
fn request_cursor_updates(&mut self, _ctx: &mut CallbackCtx, _cursor_update_mode: i32) -> bool {
// No cursor-anchor UI to feed -- see RUST.md's I2 notes on what
// this backend does not do yet.
false
}
}
+39
View File
@@ -0,0 +1,39 @@
use crate::prelude::*;
use android_view::{jni::JNIEnv, ndk::event::Keycode};
use super::view::{AndroidAppState, AndroidRsc};
/// Hardware/synthesized key handling for the field that currently has
/// focus. Most typing on Android goes through the IME's `InputConnection`
/// (`android/ime.rs`) instead -- this only sees what a soft keyboard still
/// sends as a real `KeyEvent` in "not fullscreen" mode (Backspace, Enter,
/// the arrow keys on a physical keyboard) plus whatever `unicode_char`
/// reports for a plain key press. Returns whether anything used the event.
pub(super) fn on_key<'local, State: AndroidAppState>(
rsc: &mut AndroidRsc<State>,
state: &mut State,
env: &mut JNIEnv<'local>,
key_code: Keycode,
event: &android_view::KeyEvent<'local>,
) -> bool {
let Some(focus) = state.android_state().focus else {
return false;
};
let mut text = focus.edit(rsc);
match key_code {
Keycode::Del => text.backspace(false),
Keycode::ForwardDel => text.delete(false),
Keycode::DpadLeft => text.motion(Motion::Left, false),
Keycode::DpadRight => text.motion(Motion::Right, false),
Keycode::DpadUp => text.motion(Motion::Up, false),
Keycode::DpadDown => text.motion(Motion::Down, false),
Keycode::MoveHome => text.motion(Motion::LineStart, false),
Keycode::MoveEnd => text.motion(Motion::LineEnd, false),
Keycode::Enter | Keycode::NumpadEnter => text.newline(),
_ => match event.unicode_char(env) {
Some(c) if !c.is_control() => text.insert(&c.to_string()),
_ => return false,
},
}
true
}
+129
View File
@@ -0,0 +1,129 @@
//! Window insets, fed in from outside `ViewPeer`.
//!
//! android-view's registered native methods (`view.rs` in that crate) cover
//! touch, keys, focus, the surface and the IME -- there is nothing for
//! `View.onApplyWindowInsets`, because android-view's own demo does not
//! need it. The back gesture needed no new plumbing at all: with no
//! `OnBackPressedCallback` registered, Android still delivers it as an
//! ordinary `KEYCODE_BACK` `KeyEvent` through the ordinary key path (see
//! `view.rs`'s `on_key_down`), which is the legacy behaviour every app gets
//! by default and is enough for "the back gesture as an event". Insets have
//! no such stand-in, so this module registers one more native method by
//! hand, on the app's own `View` subclass rather than on android-view's.
//!
//! The peer id android-view hands back from `register_view_peer` is opaque
//! outside that crate (`with_peer` is `pub(crate)` there), so there is no
//! way to reach an existing `IrisViewPeer` from a JNI entry point we define
//! ourselves. Instead of forking android-view to add a hook, `new_peer`
//! (`view.rs`) inserts the *same* id into this module's own map, pointing
//! at a plain `Rc<RefCell<Shared>>` cloned into `AndroidUiState` too --
//! so writing here is reading there, with no dependency in either
//! direction on the other's internals.
use android_view::{
View,
jni::{
JNIEnv, NativeMethod,
descriptors::Desc,
objects::JClass,
sys::{jint, jlong},
},
};
use std::{
cell::RefCell,
collections::HashMap,
ffi::c_void,
rc::Rc,
sync::{Mutex, OnceLock},
};
use send_wrapper::SendWrapper;
#[derive(Clone, Copy, Default, Debug, PartialEq, Eq)]
pub struct Insets {
pub left: i32,
pub top: i32,
pub right: i32,
pub bottom: i32,
/// The keyboard's own inset (`WindowInsetsCompat.Type.ime()`), separate
/// from `bottom` (the system bars): a layout wants to know about the
/// keyboard specifically, since it usually means "make room" rather
/// than "stay clear of a corner".
pub ime_bottom: i32,
}
#[derive(Default)]
pub struct Shared {
pub insets: Insets,
}
type SharedMap = HashMap<jlong, SendWrapper<Rc<RefCell<Shared>>>>;
fn map() -> &'static Mutex<SharedMap> {
static MAP: OnceLock<Mutex<SharedMap>> = OnceLock::new();
MAP.get_or_init(Default::default)
}
/// Called from `view::new_peer` with the same id android-view's
/// `register_view_peer` returned, so a later `apply_window_insets` call
/// (keyed on that id by Java, which only ever sees the one long) reaches
/// the same `Shared` cell `AndroidUiState` reads from.
pub(super) fn register(id: jlong, shared: Rc<RefCell<Shared>>) {
map().lock().unwrap().insert(id, SendWrapper::new(shared));
}
extern "system" fn unregister_insets<'local>(
_env: JNIEnv<'local>,
_view: View<'local>,
peer: jlong,
) {
map().lock().unwrap().remove(&peer);
}
extern "system" fn apply_window_insets<'local>(
mut env: JNIEnv<'local>,
view: View<'local>,
peer: jlong,
left: jint,
top: jint,
right: jint,
bottom: jint,
ime_bottom: jint,
) {
if let Some(shared) = map().lock().unwrap().get(&peer) {
shared.borrow_mut().insets = Insets {
left,
top,
right,
bottom,
ime_bottom,
};
}
// Insets can change (the keyboard opening) with no resize and no
// touch, so nothing else here would otherwise ask for a frame.
view.post_frame_callback(&mut env);
}
/// Registers `applyWindowInsetsNative` on the app's own `View` subclass.
/// Called once from `JNI_OnLoad` alongside `android_view::register_view_class`.
pub fn register_native_methods<'local, 'other_local>(
env: &mut JNIEnv<'local>,
class: impl Desc<'local, JClass<'other_local>>,
) {
env.register_native_methods(
class,
&[
NativeMethod {
name: "applyWindowInsetsNative".into(),
sig: "(JIIIII)V".into(),
fn_ptr: apply_window_insets as *mut c_void,
},
NativeMethod {
name: "unregisterInsetsNative".into(),
sig: "(J)V".into(),
fn_ptr: unregister_insets as *mut c_void,
},
],
)
.unwrap();
}
+43
View File
@@ -0,0 +1,43 @@
//! iris's second windowing backend: `android-view` (a `SurfaceView` plus a
//! JNI `ViewPeer`) instead of winit. See RUST.md's I2 for why this exists
//! as a second backend rather than winit's own (unfinished, and blocked on
//! `android-activity`'s backend-feature requirement) Android support, and
//! for the pass condition this was built against.
//!
//! Structured to mirror `default/` module for module: `view.rs` is that
//! module's `app.rs` + `state.rs` combined (android-view has one harness
//! type, `ViewPeer`, where winit splits `ApplicationHandler` from the
//! per-window state), `render.rs` is `render.rs`, `input.rs` is `input.rs`,
//! `attr.rs` is `attr.rs`. `ime.rs` and `insets.rs` have no winit
//! counterpart: winit cannot drive an IME beyond `Ime::Preedit`/`Commit`
//! (RUST.md's E1) and has no concept of Android's window insets at all.
mod access;
mod attr;
mod ime;
mod input;
mod insets;
mod render;
mod view;
pub use insets::Insets;
pub use render::AndroidRenderer;
pub use view::{
AndroidAppState, AndroidRsc, AndroidUiState, HasAndroidUiState, IrisViewPeer, new_peer,
};
/// Registers the extra native methods this backend needs beyond what
/// `android_view::register_view_class` covers (window insets -- see
/// `insets.rs`'s doc comment for why that one could not ride along on an
/// existing android-view callback the way the back gesture does). Call
/// from `JNI_OnLoad` alongside `register_view_class`, on the same `View`
/// subclass.
pub fn register_native_methods<'local, 'other_local>(
env: &mut android_view::jni::JNIEnv<'local>,
class: impl android_view::jni::descriptors::Desc<
'local,
android_view::jni::objects::JClass<'other_local>,
>,
) {
insets::register_native_methods(env, class);
}
+194
View File
@@ -0,0 +1,194 @@
use crate::task::RequestRedraw;
use android_view::{
View,
jni::{JavaVM, objects::GlobalRef},
ndk::native_window::NativeWindow,
};
use iris_core::{UiData, UiRenderNode, UiRenderState};
use pollster::FutureExt;
use wgpu::{
rwh::{DisplayHandle, HandleError, HasDisplayHandle, HasWindowHandle, WindowHandle},
*,
};
pub const CLEAR_COLOR: Color = Color::BLACK;
/// `NativeWindow` (from the surface android-view hands over in
/// `surfaceChanged`) has a window handle but not a display one -- there is
/// exactly one display on Android and `rwh` has a unit variant for it.
/// Mirrors android-view's own demo (`demo/src/lib.rs`'s
/// `AndroidWindowHandle`).
struct AndroidWindowHandle {
window: NativeWindow,
}
impl HasDisplayHandle for AndroidWindowHandle {
fn display_handle(&self) -> Result<DisplayHandle<'_>, HandleError> {
Ok(DisplayHandle::android())
}
}
impl HasWindowHandle for AndroidWindowHandle {
fn window_handle(&self) -> Result<WindowHandle<'_>, HandleError> {
self.window.window_handle()
}
}
/// The android-view surface, unlike winit's window, does not outlive a
/// backgrounding of the activity: `surfaceDestroyed`/`surfaceCreated` (via
/// `SurfaceHolder.Callback`) recreate it, so this holds everything that
/// depends on that surface rather than being built once at startup --
/// `AndroidUiState` holds it as `Option<AndroidRenderer>`, `None` exactly
/// when there is no surface to draw into.
pub struct AndroidRenderer {
surface: Surface<'static>,
device: Device,
queue: Queue,
config: SurfaceConfiguration,
encoder: CommandEncoder,
pub ui: UiRenderNode,
}
impl AndroidRenderer {
pub fn new(window: NativeWindow, width: u32, height: u32) -> Self {
let instance = Instance::new(&InstanceDescriptor {
backends: Backends::PRIMARY,
..Default::default()
});
// SAFETY: the `NativeWindow` outlives the surface built from it --
// android-view drops the old renderer (and this surface with it)
// before handing over a new window, in `surface_changed` below.
let surface = instance
.create_surface(SurfaceTarget::from(AndroidWindowHandle { window }))
.expect("Could not create android surface!");
let adapter = instance
.request_adapter(&RequestAdapterOptions {
power_preference: PowerPreference::default(),
compatible_surface: Some(&surface),
force_fallback_adapter: false,
})
.block_on()
.expect("Could not get adapter!");
// Same request as the winit backend's `UiRenderer::new` -- no
// binding-array features, see TEXTURES.md's "Recommended shape".
let (device, queue) = adapter
.request_device(&DeviceDescriptor {
required_limits: Limits {
max_buffer_size: 1 << 30,
..Default::default()
},
..Default::default()
})
.block_on()
.expect("Could not get device!");
let surface_caps = surface.get_capabilities(&adapter);
let surface_format = surface_caps
.formats
.iter()
.copied()
.find(|f| f.is_srgb())
.unwrap_or(surface_caps.formats[0]);
let config = SurfaceConfiguration {
usage: TextureUsages::RENDER_ATTACHMENT,
format: surface_format,
width,
height,
present_mode: PresentMode::AutoVsync,
alpha_mode: surface_caps.alpha_modes[0],
desired_maximum_frame_latency: 2,
view_formats: vec![],
};
surface.configure(&device, &config);
let encoder = Self::create_encoder(&device);
let ui = UiRenderNode::new(&device, &queue, &config);
Self {
surface,
device,
queue,
config,
encoder,
ui,
}
}
fn create_encoder(device: &Device) -> CommandEncoder {
device.create_command_encoder(&CommandEncoderDescriptor {
label: Some("Render Encoder"),
})
}
pub fn update(&mut self, ui: &mut UiData, render: &mut UiRenderState) {
self.ui.update(&self.device, &self.queue, ui, render);
}
pub fn draw(&mut self) {
let output = self.surface.get_current_texture().unwrap();
let view = output
.texture
.create_view(&TextureViewDescriptor::default());
let mut encoder = std::mem::replace(&mut self.encoder, Self::create_encoder(&self.device));
{
let render_pass = &mut encoder.begin_render_pass(&RenderPassDescriptor {
color_attachments: &[Some(RenderPassColorAttachment {
view: &view,
resolve_target: None,
ops: Operations {
load: LoadOp::Clear(CLEAR_COLOR),
store: StoreOp::Store,
},
depth_slice: None,
})],
..Default::default()
});
self.ui.draw(render_pass);
}
self.queue.submit(std::iter::once(encoder.finish()));
output.present();
}
pub fn size(&self) -> iris_core::util::Vec2 {
(self.config.width, self.config.height).into()
}
pub fn resize(&mut self, width: u32, height: u32) {
self.config.width = width;
self.config.height = height;
self.surface.configure(&self.device, &self.config);
self.ui.resize((width, height), &self.queue);
}
}
/// `Tasks`' redraw handle on Android: a background task finishes on the
/// tokio thread `Tasks::init` spawned, which is not attached to the JVM, so
/// asking for a frame means attaching first. `post_frame_callback` needs a
/// live `View` reference; the global ref is what survives past the JNI call
/// that handed it to us.
pub struct AndroidRedrawHandle {
vm: JavaVM,
view: GlobalRef,
}
impl AndroidRedrawHandle {
pub fn new(vm: JavaVM, view: GlobalRef) -> Self {
Self { vm, view }
}
}
impl RequestRedraw for AndroidRedrawHandle {
fn request_redraw(&self) {
let Ok(mut env) = self.vm.attach_current_thread() else {
return;
};
let local = env.new_local_ref(&self.view).unwrap();
View(local).post_frame_callback(&mut env);
}
}
+543
View File
@@ -0,0 +1,543 @@
use crate::prelude::*;
use crate::task::RequestRedraw;
use accesskit_android::Adapter as AccessAdapter;
use android_view::{
AccessibilityNodeInfo, AccessibilityNodeProvider, Bundle, CallbackCtx, Context,
InputConnection, KeyEvent, MotionEvent, Rect, View, ViewPeer,
jni::{JNIEnv, sys::jint},
ndk::event::{Keycode, MotionAction},
};
// `marker::Sized` explicitly: `crate::prelude::*` below also brings in the
// `Sized` *widget* (`widget::position::sized::Sized`), and an unqualified
// glob import shadows the language prelude -- `default/mod.rs` has the same
// explicit import for the same reason.
use std::{
cell::RefCell,
marker::{PhantomData, Sized},
rc::Rc,
sync::Arc,
time::Instant,
};
use super::{
access::{AndroidAccessSource, NullActionHandler, raise_if_enabled},
insets::{Insets, Shared},
render::{AndroidRedrawHandle, AndroidRenderer},
};
/// The android-view analogue of `default::DefaultUiState`. `renderer` is an
/// `Option` because a `SurfaceView`'s surface does not outlive backgrounding
/// the way a winit `Window` does -- `surfaceDestroyed`/`surfaceCreated` can
/// happen any number of times over the life of one `IrisViewPeer`.
pub struct AndroidUiState {
pub root: Option<StrongWidget>,
pub renderer: Option<AndroidRenderer>,
pub focus: Option<WeakWidget<TextEdit>>,
pub cursor: CursorState,
pub last_click: Instant,
/// The IME preedit's previous length, in `char`s -- the same
/// re-send-the-whole-composition bookkeeping `default::DefaultUiState`
/// keeps for winit's `Ime::Preedit`, since android-view's
/// `setComposingText` has the identical shape (see `android/ime.rs`).
pub compose_len: usize,
/// Set by `attr::FocusHost::focus_gained` when a `TextEdit` is focused;
/// consumed by the touch handler after the sensor pass finishes, since
/// showing the keyboard is a JNI call and `focus_gained` runs deep
/// inside the platform-agnostic sensor dispatch with no `CallbackCtx`
/// in reach.
pub pending_show_keyboard: bool,
/// Window insets, filled in from outside the normal `ViewPeer` callback
/// path -- see `android/insets.rs` for why they need a registry of
/// their own.
shared: Rc<RefCell<Shared>>,
/// I4 (RUST.md): pushed from `IrisViewPeer::render` and consulted by
/// the `AccessibilityNodeProvider` impl below; see `android/access.rs`
/// for the abort mitigation every `raise` on it goes through.
pub access_adapter: AccessAdapter,
/// The AccessKit tree itself -- see `iris_core::AccessTree`'s doc
/// comment.
pub access: AccessTree,
}
impl AndroidUiState {
fn new(shared: Rc<RefCell<Shared>>) -> Self {
Self {
root: None,
renderer: None,
focus: None,
cursor: Default::default(),
last_click: Instant::now(),
compose_len: 0,
pending_show_keyboard: false,
shared,
access_adapter: Default::default(),
access: AccessTree::new(),
}
}
pub fn insets(&self) -> Insets {
self.shared.borrow().insets
}
}
impl HasRoot for AndroidUiState {
fn set_root(&mut self, root: StrongWidget) {
self.root = Some(root);
}
}
pub trait HasAndroidUiState: Sized + 'static {
fn android_state(&self) -> &AndroidUiState;
fn android_state_mut(&mut self) -> &mut AndroidUiState;
}
pub trait AndroidAppState: HasAndroidUiState {
fn new(ui_state: AndroidUiState, rsc: &mut AndroidRsc<Self>) -> Self;
/// The system back gesture/button. `true` means handled -- nothing
/// further happens; `false` lets the activity finish as it would with
/// no view at all. The default declines, since most screens have
/// nothing to intercept it for.
#[allow(unused_variables)]
fn back_pressed(&mut self, rsc: &mut AndroidRsc<Self>, render: &mut UiRenderState) -> bool {
false
}
}
/// The android-view analogue of `default::DefaultRsc` -- identical in
/// substance, since none of `UiRsc`/`HasEvents`/`HasTasks`/`HasWidgetState`
/// mention winit. Kept as a separate type rather than shared code because
/// the two backends' `ViewPeer`/`ApplicationHandler` entry points hold
/// their harness state differently (see RUST.md's I2).
pub struct AndroidRsc<State: 'static> {
pub ui: UiData,
pub events: EventManager<Self>,
pub tasks: Tasks<Self>,
pub state: WidgetState,
_state: PhantomData<State>,
}
impl<State> AndroidRsc<State> {
pub fn create_state<T: 'static>(&mut self, id: impl IdLike, data: T) -> WeakState<T> {
self.state.add(id.id(), data)
}
}
impl<State> UiRsc for AndroidRsc<State> {
fn ui(&self) -> &UiData {
&self.ui
}
fn ui_mut(&mut self) -> &mut UiData {
&mut self.ui
}
fn on_draw(&mut self, active: &ActiveData) {
self.events.draw(active);
}
fn on_undraw(&mut self, active: &ActiveData) {
self.events.undraw(active);
}
fn on_remove(&mut self, id: WidgetId) {
self.events.remove(id);
self.state.remove(id);
}
}
impl<State: 'static> HasState for AndroidRsc<State> {
type State = State;
}
impl<State: 'static> HasEvents for AndroidRsc<State> {
fn events(&self) -> &EventManager<Self> {
&self.events
}
fn events_mut(&mut self) -> &mut EventManager<Self> {
&mut self.events
}
}
impl<State: 'static> HasTasks for AndroidRsc<State> {
fn tasks_mut(&mut self) -> &mut Tasks<Self> {
&mut self.tasks
}
}
impl<State: 'static> HasWidgetState for AndroidRsc<State> {
fn widget_state(&self) -> &WidgetState {
&self.state
}
fn widget_state_mut(&mut self) -> &mut WidgetState {
&mut self.state
}
}
/// The `ViewPeer` android-view dispatches every callback to. One per
/// `RustView` instance; `new_peer` (below) builds it and hands the id to
/// Java the same way android-view's own demo does.
pub struct IrisViewPeer<State: AndroidAppState> {
pub(super) rsc: AndroidRsc<State>,
pub(super) render: UiRenderState,
pub(super) state: State,
task_recv: TaskMsgReceiver<AndroidRsc<State>>,
}
impl<State: 'static, I: RscIdx<AndroidRsc<State>>> std::ops::Index<I> for AndroidRsc<State> {
type Output = I::Output;
fn index(&self, index: I) -> &Self::Output {
index.get(self)
}
}
impl<State: 'static, I: RscIdx<AndroidRsc<State>>> std::ops::IndexMut<I> for AndroidRsc<State> {
fn index_mut(&mut self, index: I) -> &mut Self::Output {
index.get_mut(self)
}
}
impl<State: AndroidAppState> IrisViewPeer<State> {
fn drain_tasks(&mut self) {
while let Ok(update) = self.task_recv.try_recv() {
update(&mut self.state, &mut self.rsc);
}
}
/// Common tail for every callback that might have changed the cursor,
/// the text focus, or the widget tree: run the sensors that touch
/// input feeds, then ask for a frame if the result needs drawing.
/// Mirrors `default::DefaultApp::window_event`'s tail, split across
/// android-view's several entry points instead of winit's one.
pub(super) fn after_input(&mut self, ctx: &mut CallbackCtx) {
let window_size = self.window_size();
let ui_state = self.state.android_state_mut();
let cursor = ui_state.cursor.clone();
let old_focus = ui_state.focus;
self.render
.run_sensors(&mut self.rsc, &mut self.state, cursor, window_size);
let ui_state = self.state.android_state_mut();
if old_focus != ui_state.focus
&& let Some(old) = old_focus
{
old.edit(&mut self.rsc).deselect();
}
if std::mem::take(&mut ui_state.pending_show_keyboard) {
show_soft_input(&mut ctx.env, &ctx.view);
}
let ui_state = self.state.android_state_mut();
ui_state.cursor.end_frame();
if self.render.needs_redraw(&ui_state.root, self.rsc.widgets()) {
ctx.view.post_frame_callback(&mut ctx.env);
}
}
fn window_size(&self) -> Vec2 {
let ui_state = self.state.android_state();
match &ui_state.renderer {
Some(r) => r.size(),
None => Vec2::ZERO,
}
}
/// The `log::debug!` calls here are a live diagnostic for a still-open
/// finding (RUST.md's I2): layout runs and reports the right pixel
/// region for the root (confirmed via `window_region`, logged below),
/// and the clear colour reaches the screen (confirmed by swapping it to
/// magenta and screenshotting), but no primitive ever appears on top of
/// it -- on both the Vulkan/SwiftShader and GLES/virgl backends. Leave
/// these in until that is root-caused; removing them loses the exact
/// evidence a `logcat` capture needs to reproduce the state.
fn render(&mut self, ctx: &mut CallbackCtx) {
let ui_state = self.state.android_state();
if ui_state.renderer.is_none() {
return;
}
log::debug!(
"render(): root={:?} widgets={} active={} root_px={:?} out_size={:?}",
ui_state.root.is_some(),
self.rsc.widgets().len(),
self.render.active_widgets(),
ui_state
.root
.as_ref()
.and_then(|r| self.render.window_region(r, &self.rsc)),
self.window_size(),
);
let ui_state = self.state.android_state_mut();
self.render.update(&ui_state.root, &mut self.rsc);
let ui_state = self.state.android_state_mut();
let Some(renderer) = &mut ui_state.renderer else {
return;
};
renderer.update(&mut self.rsc.ui, &mut self.render);
renderer.draw();
let ui_state = self.state.android_state();
log::debug!(
"render(): after update active={} root_px={:?}",
self.render.active_widgets(),
ui_state
.root
.as_ref()
.and_then(|r| self.render.window_region(r, &self.rsc)),
);
// I4 (RUST.md): only produces a `TreeUpdate` -- and so only queues
// anything to raise -- when the named set actually changed this
// frame; see `AccessTree`'s doc comment. Deferred rather than
// raised inline so it runs after this callback releases whatever
// it's holding, matching android-view's own demo and `raise`'s own
// contract.
let ui_state = self.state.android_state_mut();
if let Some(tree_update) =
ui_state
.access
.update(self.rsc.widgets(), &self.render, &self.rsc)
{
let ui_state = self.state.android_state_mut();
if let Some(events) = ui_state.access_adapter.update_if_active(|| tree_update) {
ctx.push_dynamic_deferred_callback(move |env, view| {
raise_if_enabled(env, view, events);
});
}
}
}
}
fn show_soft_input<'local>(env: &mut JNIEnv<'local>, view: &View<'local>) {
let imm = view.input_method_manager(env);
imm.show_soft_input(env, view, 0);
}
impl<State: AndroidAppState> ViewPeer for IrisViewPeer<State> {
fn on_key_down<'local>(
&mut self,
ctx: &mut CallbackCtx<'local>,
key_code: Keycode,
event: &KeyEvent<'local>,
) -> bool {
self.drain_tasks();
// With no `OnBackPressedCallback` registered on the Java side, the
// system still delivers the back gesture as a synthetic
// `KEYCODE_BACK` through this same path -- the legacy behaviour
// every view-based app gets by default, and enough for "the back
// gesture as an event" without a second JNI registry. See
// `android/insets.rs`'s doc comment for why insets could not take
// the same shortcut.
if key_code == Keycode::Back {
let handled = self.state.back_pressed(&mut self.rsc, &mut self.render);
if handled {
self.after_input(ctx);
}
return handled;
}
let handled = super::input::on_key(
&mut self.rsc,
&mut self.state,
&mut ctx.env,
key_code,
event,
);
if handled {
self.after_input(ctx);
}
handled
}
fn on_touch_event<'local>(
&mut self,
ctx: &mut CallbackCtx<'local>,
event: &MotionEvent<'local>,
) -> bool {
self.drain_tasks();
let action = event.action_masked(&mut ctx.env);
let x = event.x(&mut ctx.env);
let y = event.y(&mut ctx.env);
let ui_state = self.state.android_state_mut();
match action {
MotionAction::Down => {
ui_state.cursor.pos = vec2(x, y);
ui_state.cursor.exists = true;
ui_state.cursor.buttons.left.update(true);
}
MotionAction::Move => {
ui_state.cursor.pos = vec2(x, y);
}
MotionAction::Up | MotionAction::Cancel => {
ui_state.cursor.pos = vec2(x, y);
ui_state.cursor.buttons.left.update(false);
}
_ => return false,
}
self.after_input(ctx);
true
}
fn on_focus_changed<'local>(
&mut self,
ctx: &mut CallbackCtx<'local>,
gain_focus: bool,
_direction: i32,
_previously_focused_rect: Option<&Rect<'local>>,
) {
self.drain_tasks();
if !gain_focus {
let ui_state = self.state.android_state_mut();
if let Some(focus) = ui_state.focus.take() {
focus.edit(&mut self.rsc).deselect();
}
}
self.after_input(ctx);
}
fn on_attached_to_window(&mut self, _ctx: &mut CallbackCtx) {
self.drain_tasks();
}
fn surface_changed<'local>(
&mut self,
ctx: &mut CallbackCtx<'local>,
holder: &android_view::SurfaceHolder<'local>,
_format: i32,
width: i32,
height: i32,
) {
self.drain_tasks();
let window = holder.surface(&mut ctx.env).to_native_window(&mut ctx.env);
// The layout engine's own notion of the canvas size is separate
// from the wgpu surface's -- winit's backend sets it from
// `WindowEvent::Resized`, and there is no equivalent automatic
// trigger here, so this is the one place android-view's surface
// size has to be told to `UiRenderState` too. Missing this drew
// nothing but the clear colour: the widget tree laid out against
// whatever size `UiRenderState::new` starts at instead of the
// surface's real one.
self.render.resize((width as u32, height as u32));
// Drop the old renderer (and the surface it owns) before building
// one from the new window -- see `AndroidRenderer`'s doc comment.
let ui_state = self.state.android_state_mut();
ui_state.renderer = None;
ui_state.renderer = Some(AndroidRenderer::new(window, width as u32, height as u32));
self.render(ctx);
}
fn surface_destroyed<'local>(
&mut self,
_ctx: &mut CallbackCtx<'local>,
_holder: &android_view::SurfaceHolder<'local>,
) {
self.state.android_state_mut().renderer = None;
}
fn do_frame(&mut self, ctx: &mut CallbackCtx, _frame_time_nanos: i64) {
self.drain_tasks();
self.render(ctx);
}
fn as_input_connection(&mut self) -> Option<&mut dyn InputConnection> {
Some(self)
}
fn as_accessibility_node_provider(&mut self) -> Option<&mut dyn AccessibilityNodeProvider> {
Some(self)
}
}
impl<State: AndroidAppState> AccessibilityNodeProvider for IrisViewPeer<State> {
fn create_accessibility_node_info<'local>(
&mut self,
ctx: &mut CallbackCtx<'local>,
virtual_view_id: jint,
) -> AccessibilityNodeInfo<'local> {
let mut source = AndroidAccessSource {
widgets: self.rsc.widgets(),
render: &self.render,
rsc: &self.rsc,
};
let ui_state = self.state.android_state_mut();
AccessibilityNodeInfo(ui_state.access_adapter.create_accessibility_node_info(
&mut source,
&mut ctx.env,
&ctx.view.0,
virtual_view_id,
))
}
fn find_focus<'local>(
&mut self,
ctx: &mut CallbackCtx<'local>,
focus_type: jint,
) -> AccessibilityNodeInfo<'local> {
let mut source = AndroidAccessSource {
widgets: self.rsc.widgets(),
render: &self.render,
rsc: &self.rsc,
};
let ui_state = self.state.android_state_mut();
AccessibilityNodeInfo(ui_state.access_adapter.find_focus(
&mut source,
&mut ctx.env,
&ctx.view.0,
focus_type,
))
}
fn perform_action<'local>(
&mut self,
ctx: &mut CallbackCtx<'local>,
virtual_view_id: jint,
action: jint,
arguments: &Bundle<'local>,
) -> bool {
let Some(action) =
accesskit_android::PlatformAction::from_java(&mut ctx.env, action, &arguments.0)
else {
return false;
};
let ui_state = self.state.android_state_mut();
let Some(events) = ui_state.access_adapter.perform_action(
&mut NullActionHandler,
virtual_view_id,
&action,
) else {
return false;
};
ctx.push_dynamic_deferred_callback(move |env, view| {
raise_if_enabled(env, view, events);
});
true
}
}
/// Registers `IrisViewPeer<State>`'s native methods and builds one on every
/// `newViewPeer` call from Java. `State`'s app crate wraps this in a
/// concrete `extern "system" fn` (a generic function cannot be handed to
/// `register_view_class`, which wants a plain function pointer) -- see
/// `iris/android-app/src/lib.rs`.
pub fn new_peer<'local, State: AndroidAppState>(
env: JNIEnv<'local>,
view: View<'local>,
_context: Context<'local>,
) -> android_view::jni::sys::jlong {
let vm = env.get_java_vm().unwrap();
let global_view = env.new_global_ref(&view.0).unwrap();
let redraw: Arc<dyn RequestRedraw> = Arc::new(AndroidRedrawHandle::new(vm, global_view));
let (tasks, task_recv) = Tasks::init(redraw);
let mut rsc = AndroidRsc {
ui: Default::default(),
events: Default::default(),
tasks,
state: Default::default(),
_state: PhantomData,
};
let shared = Rc::new(RefCell::new(Shared::default()));
let ui_state = AndroidUiState::new(shared.clone());
let state = State::new(ui_state, &mut rsc);
let peer = IrisViewPeer {
rsc,
render: UiRenderState::new(),
state,
task_recv,
};
let id = android_view::register_view_peer(peer);
super::insets::register(id, shared);
id
}
+105
View File
@@ -0,0 +1,105 @@
use crate::prelude::*;
use std::time::{Duration, Instant};
/// What focusing a text field takes from whichever backend is running --
/// tracked here rather than duplicated per backend, since `Selector` and
/// `Selectable` (below) are the *only* thing that decides which `TextEdit`
/// is the IME's target, and both platforms need the same double-click
/// timing and the same "remember which one" bookkeeping. What differs is
/// what happens *after* the focus record is set: winit tells the
/// compositor an IME area (`focus_gained`, in `default/attr.rs`); on
/// android-view a keyboard has to be asked for explicitly, and only from a
/// JNI call this crate cannot make outside a view callback -- so
/// `focus_gained` there (`android/attr.rs`) just raises a flag the next
/// touch callback consumes. See RUST.md's I2.
pub trait FocusHost {
/// True on a click close enough in time to the previous one to grow a
/// selection instead of starting a new one, updating the clock as a
/// side effect the way a real double-click timer does.
fn recent_click(&mut self) -> bool;
fn set_focus(&mut self, id: Option<WeakWidget<TextEdit>>);
/// Called after a `TextEdit` becomes the focus target, with the region
/// it was hit in (`None` when the widget could not be located, which
/// happens for one it was just deselected from).
fn focus_gained(&mut self, region: Option<PixelRegion>);
}
/// Helper shared by every `FocusHost` impl, so the double-click window is
/// one constant rather than one per backend.
pub fn recent_click(last_click: &mut Instant) -> bool {
let now = Instant::now();
let recent = (now - *last_click) < Duration::from_millis(300);
*last_click = now;
recent
}
pub struct Selector;
impl<Rsc: HasEvents, W: Widget + 'static> WidgetAttr<Rsc, W> for Selector
where
Rsc::State: FocusHost,
{
type Input = WeakWidget<TextEdit>;
fn run(rsc: &mut Rsc, container: WeakWidget<W>, id: Self::Input) {
rsc.register_event(container, CursorSense::click_or_drag(), move |ctx, rsc| {
let region = ctx.data.render.window_region(&id, &*rsc).unwrap();
let id_pos = region.top_left;
let container_pos = ctx
.data
.render
.window_region(&container, &*rsc)
.unwrap()
.top_left;
let pos = ctx.data.pos + container_pos - id_pos;
let size = region.size();
select(
rsc,
ctx.data.render,
ctx.state,
id,
pos,
size,
ctx.data.sense.is_dragging(),
);
});
}
}
pub struct Selectable;
impl<Rsc: HasEvents> WidgetAttr<Rsc, TextEdit> for Selectable
where
Rsc::State: FocusHost,
{
type Input = ();
fn run(rsc: &mut Rsc, id: WeakWidget<TextEdit>, _: Self::Input) {
rsc.register_event(id, CursorSense::click_or_drag(), move |ctx, rsc| {
select(
rsc,
ctx.data.render,
ctx.state,
id,
ctx.data.pos,
ctx.data.size,
ctx.data.sense.is_dragging(),
);
});
}
}
fn select(
rsc: &mut impl UiRsc,
render: &UiRenderState,
state: &mut impl FocusHost,
id: WeakWidget<TextEdit>,
pos: Vec2,
size: Vec2,
dragging: bool,
) {
let recent = state.recent_click();
id.edit(rsc).select(pos, size, dragging, recent);
state.set_focus(Some(id));
state.focus_gained(render.window_region(&id, &*rsc));
}
+28
View File
@@ -0,0 +1,28 @@
//! I4 (RUST.md): the desktop half of the AccessKit push, over
//! `accesskit_winit`. `bench-lib.sh`'s tap-by-name goes through the
//! platform's real accessibility tree, so this crate only has to keep that
//! tree in sync with `ui::access::AccessTree`'s output -- nothing here
//! reacts to an AccessKit action request, which is why the three handlers
//! below are inert. See RUST.md's I4 box for why: on Android (and, by the
//! same platform convention, everywhere else) a screen reader's element tap
//! is a real touch delivered at the node's own bounds, not an action
//! request synthesised in-process -- so the ordinary pointer path already
//! handles it once the bounds are right.
use accesskit::{ActionHandler, ActionRequest, ActivationHandler, DeactivationHandler, TreeUpdate};
pub struct NullActivationHandler;
impl ActivationHandler for NullActivationHandler {
fn request_initial_tree(&mut self) -> Option<TreeUpdate> {
None
}
}
pub struct NullActionHandler;
impl ActionHandler for NullActionHandler {
fn do_action(&mut self, _request: ActionRequest) {}
}
pub struct NullDeactivationHandler;
impl DeactivationHandler for NullDeactivationHandler {
fn deactivate_accessibility(&mut self) {}
}
+8 -69
View File
@@ -1,83 +1,22 @@
use crate::prelude::*;
use std::time::{Duration, Instant};
use winit::dpi::{LogicalPosition, LogicalSize};
pub struct Selector;
impl<Rsc: HasEvents, W: Widget + 'static> WidgetAttr<Rsc, W> for Selector
where
Rsc::State: HasDefaultUiState,
{
type Input = WeakWidget<TextEdit>;
fn run(rsc: &mut Rsc, container: WeakWidget<W>, id: Self::Input) {
rsc.register_event(container, CursorSense::click_or_drag(), move |ctx, rsc| {
let region = ctx.data.render.window_region(&id, &*rsc).unwrap();
let id_pos = region.top_left;
let container_pos = ctx
.data
.render
.window_region(&container, &*rsc)
.unwrap()
.top_left;
let pos = ctx.data.pos + container_pos - id_pos;
let size = region.size();
select(
rsc,
ctx.data.render,
ctx.state,
id,
pos,
size,
ctx.data.sense.is_dragging(),
);
});
}
impl<T: HasDefaultUiState> FocusHost for T {
fn recent_click(&mut self) -> bool {
crate::attr::recent_click(&mut self.default_state_mut().last_click)
}
pub struct Selectable;
impl<Rsc: HasEvents> WidgetAttr<Rsc, TextEdit> for Selectable
where
Rsc::State: HasDefaultUiState,
{
type Input = ();
fn run(rsc: &mut Rsc, id: WeakWidget<TextEdit>, _: Self::Input) {
rsc.register_event(id, CursorSense::click_or_drag(), move |ctx, rsc| {
select(
rsc,
ctx.data.render,
ctx.state,
id,
ctx.data.pos,
ctx.data.size,
ctx.data.sense.is_dragging(),
);
});
}
fn set_focus(&mut self, id: Option<WeakWidget<TextEdit>>) {
self.default_state_mut().focus = id;
}
fn select(
rsc: &mut impl UiRsc,
render: &UiRenderState,
state: &mut impl HasDefaultUiState,
id: WeakWidget<TextEdit>,
pos: Vec2,
size: Vec2,
dragging: bool,
) {
let state = state.default_state_mut();
let now = Instant::now();
let recent = (now - state.last_click) < Duration::from_millis(300);
state.last_click = now;
id.edit(rsc).select(pos, size, dragging, recent);
if let Some(region) = render.window_region(&id, &*rsc) {
fn focus_gained(&mut self, region: Option<PixelRegion>) {
let state = self.default_state_mut();
let Some(region) = region else { return };
state.window.set_ime_allowed(true);
state.window.set_ime_cursor_area(
LogicalPosition::<f32>::from(region.top_left.tuple()),
LogicalSize::<f32>::from(region.size().tuple()),
);
}
state.focus = Some(id);
}
-9
View File
@@ -1,9 +0,0 @@
use iris_core::Event;
#[derive(Eq, PartialEq, Hash, Clone)]
pub struct Submit;
impl Event for Submit {}
#[derive(Eq, PartialEq, Hash, Clone)]
pub struct Edited;
impl Event for Edited {}
+46 -42
View File
@@ -11,23 +11,16 @@ use winit::{
window::{Window, WindowAttributes},
};
mod access;
mod app;
mod attr;
mod event;
mod input;
mod render;
mod sense;
mod state;
mod task;
pub use access::*;
pub use app::*;
pub use attr::*;
pub use event::*;
pub use input::*;
pub use render::*;
pub use sense::*;
pub use state::*;
pub use task::*;
pub type Proxy<Event> = EventLoopProxy<Event>;
@@ -40,6 +33,17 @@ pub struct DefaultUiState {
pub window: Arc<Window>,
pub ime: usize,
pub last_click: Instant,
/// I4 (RUST.md): pushed through in `DefaultApp::window_event`'s
/// `RedrawRequested` arm, from `access`'s output. Built in
/// `DefaultApp::new`, which is the only place with the
/// `&ActiveEventLoop` `accesskit_winit::Adapter::with_direct_handlers`
/// needs -- see that constructor's doc comment on why the window must
/// still be invisible when it is called.
pub access_adapter: accesskit_winit::Adapter,
/// The AccessKit tree itself -- see `iris_core::AccessTree`'s doc
/// comment for the flat shape and why it only rebuilds on a real
/// change.
pub access: AccessTree,
}
impl HasRoot for DefaultUiState {
@@ -49,7 +53,7 @@ impl HasRoot for DefaultUiState {
}
impl DefaultUiState {
pub fn new(window: impl Into<Arc<Window>>) -> Self {
pub fn new(window: impl Into<Arc<Window>>, access_adapter: accesskit_winit::Adapter) -> Self {
let window = window.into();
Self {
root: None,
@@ -60,6 +64,8 @@ impl DefaultUiState {
ime: 0,
last_click: Instant::now(),
focus: None,
access_adapter,
access: AccessTree::new(),
}
}
}
@@ -188,10 +194,24 @@ impl<State: DefaultAppState> AppState for DefaultApp<State> {
type Event = State::Event;
fn new(event_loop: &ActiveEventLoop, proxy: EventLoopProxy<Self::Event>) -> Self {
// `accesskit_winit::Adapter::with_direct_handlers` panics if the
// window is already visible when it's built, so the window is
// created hidden and only shown once the adapter exists -- the one
// extra step I4 (RUST.md) needs here. The three handlers are inert
// (see `access.rs`): a screen reader's tap is a real touch at the
// node's bounds, not an action request this process has to answer.
let window = event_loop
.create_window(State::window_attributes())
.create_window(State::window_attributes().with_visible(false))
.unwrap();
let default_state = DefaultUiState::new(window);
let access_adapter = accesskit_winit::Adapter::with_direct_handlers(
event_loop,
&window,
NullActivationHandler,
NullActionHandler,
NullDeactivationHandler,
);
window.set_visible(true);
let default_state = DefaultUiState::new(window, access_adapter);
let (mut rsc, task_recv) = DefaultRsc::init(default_state.window.clone());
let state = State::new(default_state, &mut rsc, proxy);
let render = UiRenderState::new();
@@ -220,6 +240,12 @@ impl<State: DefaultAppState> AppState for DefaultApp<State> {
}
let ui_state = state.default_state_mut();
// Required by `accesskit_winit` on every window event, not just the
// ones this backend otherwise cares about -- some platform adapters
// rely on it to notice activation (a screen reader turning on).
ui_state
.access_adapter
.process_event(&ui_state.window, &event);
let input_changed = ui_state.input.event(&event);
let cursor_state = ui_state.cursor_state().clone();
let old = ui_state.focus;
@@ -242,6 +268,14 @@ impl<State: DefaultAppState> AppState for DefaultApp<State> {
render.update(&ui_state.root, rsc);
ui_state.renderer.update(&mut rsc.ui, render);
ui_state.renderer.draw();
// I4 (RUST.md): only produces a `TreeUpdate` when the named
// set actually changed this frame -- see `AccessTree`'s doc
// comment. `render` reflects the draw that just happened,
// so `resolved_region`/`window_region` inside it report a
// moved subtree's *new* position, not last frame's.
if let Some(tree_update) = ui_state.access.update(rsc.widgets(), render, rsc) {
ui_state.access_adapter.update_if_active(|| tree_update);
}
}
WindowEvent::Resized(size) => {
render.resize((size.width, size.height));
@@ -309,12 +343,6 @@ impl<State: DefaultAppState> AppState for DefaultApp<State> {
}
}
pub trait RscIdx<Rsc> {
type Output;
fn get(self, rsc: &Rsc) -> &Self::Output;
fn get_mut(self, rsc: &mut Rsc) -> &mut Self::Output;
}
impl<State: 'static, I: RscIdx<DefaultRsc<State>>> std::ops::Index<I> for DefaultRsc<State> {
type Output = I::Output;
@@ -328,27 +356,3 @@ impl<State: 'static, I: RscIdx<DefaultRsc<State>>> std::ops::IndexMut<I> for Def
index.get_mut(self)
}
}
impl<W: Widget, Rsc: UiRsc> RscIdx<Rsc> for WeakWidget<W> {
type Output = W;
fn get(self, rsc: &Rsc) -> &Self::Output {
&rsc.ui().widgets[self]
}
fn get_mut(self, rsc: &mut Rsc) -> &mut Self::Output {
&mut rsc.ui_mut().widgets[self]
}
}
impl<T: 'static, Rsc: HasWidgetState> RscIdx<Rsc> for WeakState<T> {
type Output = T;
fn get(self, rsc: &Rsc) -> &Self::Output {
rsc.widget_state().get(self)
}
fn get_mut(self, rsc: &mut Rsc) -> &mut Self::Output {
rsc.widget_state_mut().get_mut(self)
}
}
+7
View File
@@ -1,3 +1,4 @@
use crate::task::RequestRedraw;
use iris_core::{UiData, UiRenderNode, UiRenderState};
use pollster::FutureExt;
use std::sync::Arc;
@@ -6,6 +7,12 @@ use winit::{dpi::PhysicalSize, window::Window};
pub const CLEAR_COLOR: Color = Color::BLACK;
impl RequestRedraw for Window {
fn request_redraw(&self) {
Window::request_redraw(self);
}
}
pub struct UiRenderer {
window: Arc<Window>,
surface: Surface<'static>,
+15 -1
View File
@@ -2,7 +2,21 @@ use iris_core::*;
use iris_macro::*;
use std::sync::Arc;
use crate::default::{TaskCtx, TaskUpdate, Tasks};
use crate::task::{TaskCtx, TaskUpdate, Tasks};
/// A field's Enter key (without a shift, in a multi-line field). Backend
/// input handling raises it directly rather than through `on`, since a
/// field does not know ahead of time whether anything is listening.
#[derive(Eq, PartialEq, Hash, Clone)]
pub struct Submit;
impl Event for Submit {}
/// A field's content changed as a result of input the backend applied
/// directly to it (a keystroke, an IME commit) rather than through a
/// widget event handler.
#[derive(Eq, PartialEq, Hash, Clone)]
pub struct Edited;
impl Event for Edited {}
pub trait Eventable<Rsc: HasEvents, Tag>: WidgetLike<Rsc, Tag> {
fn on<E: EventLike>(
+5 -3
View File
@@ -7,9 +7,11 @@
use crate::prelude::*;
/// The minimal `UiRsc` a test needs: just the shared `UiData`, none of the
/// event/window/state plumbing `DefaultRsc` carries.
struct TestRsc {
ui: UiData,
/// event/window/state plumbing `DefaultRsc` carries. `pub(crate)` so
/// `access_tests.rs` (I4, RUST.md) can reuse it rather than keeping a
/// second copy of the same harness.
pub(crate) struct TestRsc {
pub(crate) ui: UiData,
}
impl UiRsc for TestRsc {
+28 -1
View File
@@ -1,14 +1,33 @@
#![feature(unboxed_closures)]
#![feature(fn_traits)]
#![feature(associated_type_defaults)]
// Only `default::DefaultAppState::Event`'s default uses this; unused (and
// warned about) on the android target, which has no such default.
#![cfg_attr(not(target_os = "android"), feature(associated_type_defaults))]
#![feature(unsize)]
#![feature(option_into_flat_iter)]
#![feature(async_fn_traits)]
// Two windowing backends live side by side, chosen by target rather than by
// feature flag: winit everywhere but Android, android-view on it. They are
// mutually exclusive rather than both-compiled-in because winit's own
// Android support pulls in `android-activity`, which needs one of its
// `game-activity`/`native-activity` features selected -- exactly what
// `iris-core` was kept free of, and android-view is the framework's own
// answer to the same surface on that platform. See RUST.md's I2.
#[cfg(target_os = "android")]
pub mod android;
#[cfg(not(target_os = "android"))]
pub mod default;
pub mod attr;
pub mod event;
pub mod sense;
pub mod state;
pub mod task;
pub mod widget;
#[cfg(test)]
mod access_tests;
#[cfg(test)]
mod layout_tests;
#[cfg(test)]
@@ -19,10 +38,18 @@ pub use iris_macro as macros;
pub mod prelude {
use super::*;
#[cfg(target_os = "android")]
pub use android::*;
#[cfg(not(target_os = "android"))]
pub use default::*;
pub use attr::*;
pub use event::*;
pub use iris_core::*;
pub use iris_macro::*;
pub use sense::*;
pub use state::*;
pub use task::*;
pub use widget::*;
pub use iris_core::util::Vec2;
@@ -2,6 +2,7 @@ use crate::prelude::*;
use std::{
ops::{BitOr, Deref, DerefMut},
rc::Rc,
time::{Duration, Instant},
};
#[derive(Clone, Copy, PartialEq)]
@@ -357,3 +358,241 @@ impl BitOr<CursorSense> for CursorSenses {
self
}
}
/// How long a stationary press has to be held before it is treated as a
/// long-press rather than the start of a pan.
pub const LONG_PRESS: Duration = Duration::from_millis(500);
/// How far a press has to move, in pixels, before it counts as a drag
/// rather than jitter -- for both the pan-vs-select axis test and the
/// "did this actually move" long-press guard.
pub const DRAG_SLOP: f32 = 8.0;
/// What a [`DragArbiter`] decided a frame's drag should mean. `Undecided`
/// means neither a pan nor a selection has committed yet, so the caller
/// should do nothing observable this frame.
#[derive(Debug, Clone, Copy, PartialEq)]
pub enum DragOutcome {
Undecided,
/// Scroll the enclosing list by this many window-space pixels along
/// the drag axis (the delta since the arbiter's last decided frame).
Pan(f32),
/// A selection should begin at the arbiter's press origin.
SelectStart,
/// A selection already underway should extend to the current position.
SelectExtend,
}
#[derive(Clone, Copy, PartialEq)]
enum ArbiterState {
Idle,
Undecided { already_selected: bool },
Panning,
Selecting,
}
/// Decides, one shared instance per gesture surface (a transcript's whole
/// row list here), whether a touch drag that starts on a row's own
/// selectable text is panning the list or extending a text selection --
/// RUST.md's I5 finding that both wanted the same `CursorSense::
/// click_or_drag()` gesture, with the inner text layer winning every frame
/// regardless of which one the reader meant. Decided the way Android
/// itself decides it, so a reader's existing muscle memory carries over:
///
/// - An ordinary vertical drag pans -- checked first, and immediately,
/// so a swipe never waits on the long-press timer.
/// - A stationary press held past [`LONG_PRESS`] starts a selection.
/// Every drag frame after that extends it, whichever direction it goes.
/// - A drag that starts **horizontally** while something is already
/// selected extends that selection right away, skipping the long-press
/// wait -- the "drag the selection handle" gesture a reader reaches for
/// once text is already highlighted.
///
/// Pure state, no rendering or widget access, so it is unit-testable
/// exactly like the rest of this module (`sense_tests.rs`'s style) with a
/// caller-supplied `Instant` rather than a real clock.
pub struct DragArbiter {
state: ArbiterState,
origin: Vec2,
origin_at: Instant,
last: Vec2,
}
impl Default for DragArbiter {
fn default() -> Self {
Self {
state: ArbiterState::Idle,
origin: Vec2::ZERO,
origin_at: Instant::now(),
last: Vec2::ZERO,
}
}
}
impl DragArbiter {
pub fn new() -> Self {
Self::default()
}
/// A fresh press-down at `pos`. `already_selected` is whatever the
/// caller's selection state was *before* this press -- it decides
/// whether an early horizontal move extends that selection instead of
/// waiting for a long-press.
pub fn press_start(&mut self, pos: Vec2, now: Instant, already_selected: bool) {
self.origin = pos;
self.origin_at = now;
self.last = pos;
self.state = ArbiterState::Undecided { already_selected };
}
/// The press continues (still down) at `pos`. Call once per frame
/// while the button/finger is down; returns what this frame means.
pub fn update(&mut self, pos: Vec2, now: Instant) -> DragOutcome {
match self.state {
ArbiterState::Idle => DragOutcome::Undecided,
ArbiterState::Panning => {
let dy = pos.y - self.last.y;
self.last = pos;
DragOutcome::Pan(dy)
}
ArbiterState::Selecting => {
self.last = pos;
DragOutcome::SelectExtend
}
ArbiterState::Undecided { already_selected } => {
let dx = pos.x - self.origin.x;
let dy = pos.y - self.origin.y;
if already_selected && dx.abs() > DRAG_SLOP && dx.abs() > dy.abs() {
self.state = ArbiterState::Selecting;
self.last = pos;
DragOutcome::SelectExtend
} else if dy.abs() > DRAG_SLOP && dy.abs() >= dx.abs() {
self.state = ArbiterState::Panning;
self.last = pos;
DragOutcome::Pan(dy)
} else if now.duration_since(self.origin_at) >= LONG_PRESS
&& dx.abs() <= DRAG_SLOP
&& dy.abs() <= DRAG_SLOP
{
self.state = ArbiterState::Selecting;
self.last = pos;
DragOutcome::SelectStart
} else {
DragOutcome::Undecided
}
}
}
}
/// The press was released -- back to idle for the next one.
pub fn release(&mut self) {
self.state = ArbiterState::Idle;
}
}
#[cfg(test)]
mod drag_arbiter_tests {
use super::*;
fn t(ms: u64) -> Instant {
// A fixed base plus an offset, rather than `Instant::now()` per
// call -- keeps every test's timing deterministic instead of at
// the mercy of how long the test itself took to run.
Instant::now() - Duration::from_secs(3600) + Duration::from_millis(ms)
}
#[test]
fn small_jitter_stays_undecided() {
let mut a = DragArbiter::new();
a.press_start(Vec2::new(0.0, 0.0), t(0), false);
assert_eq!(a.update(Vec2::new(1.0, 1.0), t(10)), DragOutcome::Undecided);
}
#[test]
fn a_vertical_drag_pans_immediately() {
let mut a = DragArbiter::new();
a.press_start(Vec2::new(0.0, 0.0), t(0), false);
assert_eq!(
a.update(Vec2::new(0.0, 20.0), t(10)),
DragOutcome::Pan(20.0)
);
// Subsequent frames keep panning, by the delta since last frame.
assert_eq!(
a.update(Vec2::new(0.0, 35.0), t(20)),
DragOutcome::Pan(15.0)
);
}
#[test]
fn a_horizontal_drag_with_nothing_selected_does_not_select() {
let mut a = DragArbiter::new();
a.press_start(Vec2::new(0.0, 0.0), t(0), false);
// Horizontal movement alone, with no prior selection, is not any
// of the three named gestures -- it stays undecided rather than
// guessing (it will resolve to a long-press-selection if the
// finger then stops moving, or nothing if it lifts).
assert_eq!(
a.update(Vec2::new(20.0, 0.0), t(10)),
DragOutcome::Undecided
);
}
#[test]
fn a_long_press_without_moving_starts_a_selection() {
let mut a = DragArbiter::new();
a.press_start(Vec2::new(5.0, 5.0), t(0), false);
assert_eq!(a.update(Vec2::new(5.0, 5.0), t(10)), DragOutcome::Undecided);
assert_eq!(
a.update(Vec2::new(6.0, 5.0), t(LONG_PRESS.as_millis() as u64 + 1)),
DragOutcome::SelectStart
);
}
#[test]
fn after_a_long_press_any_further_drag_extends() {
let mut a = DragArbiter::new();
a.press_start(Vec2::new(0.0, 0.0), t(0), false);
assert_eq!(
a.update(Vec2::new(0.0, 0.0), t(LONG_PRESS.as_millis() as u64 + 1)),
DragOutcome::SelectStart
);
// Even a vertical move now extends the selection rather than
// panning -- once a selection has started, it owns the gesture
// until release.
assert_eq!(
a.update(Vec2::new(0.0, 40.0), t(600)),
DragOutcome::SelectExtend
);
}
#[test]
fn a_horizontal_drag_on_already_selected_text_extends_immediately() {
let mut a = DragArbiter::new();
a.press_start(Vec2::new(0.0, 0.0), t(0), true);
assert_eq!(
a.update(Vec2::new(20.0, 2.0), t(10)),
DragOutcome::SelectExtend
);
}
#[test]
fn a_vertical_drag_still_pans_even_with_a_prior_selection() {
let mut a = DragArbiter::new();
a.press_start(Vec2::new(0.0, 0.0), t(0), true);
assert_eq!(
a.update(Vec2::new(0.0, 20.0), t(10)),
DragOutcome::Pan(20.0)
);
}
#[test]
fn release_resets_to_idle() {
let mut a = DragArbiter::new();
a.press_start(Vec2::new(0.0, 0.0), t(0), false);
a.update(Vec2::new(0.0, 20.0), t(10));
a.release();
assert_eq!(
a.update(Vec2::new(0.0, 999.0), t(20)),
DragOutcome::Undecided
);
}
}
@@ -1,7 +1,8 @@
use iris_core::{
WidgetId,
UiRsc, WidgetId,
util::{HashMap, HashSet},
};
use iris_core::{WeakWidget, Widget};
use std::{
any::{Any, TypeId},
marker::PhantomData,
@@ -73,3 +74,39 @@ impl<'a, T: 'static> FnOnce<(&'a mut WidgetState,)> for WeakState<T> {
state.get_mut(self)
}
}
/// What `Rsc[weak_handle]` indexes through -- one impl per kind of handle
/// (a widget, a piece of per-widget state), shared by both backends' `Rsc`
/// types since indexing a widget tree has nothing to do with windowing.
/// Each backend still needs its own `Index`/`IndexMut for ItsRsc<State>`
/// (`default/mod.rs`, `android/view.rs`), because a blanket impl over every
/// `I: RscIdx<Rsc>` for every possible `Rsc` would conflict between crates.
pub trait RscIdx<Rsc> {
type Output;
fn get(self, rsc: &Rsc) -> &Self::Output;
fn get_mut(self, rsc: &mut Rsc) -> &mut Self::Output;
}
impl<W: Widget, Rsc: UiRsc> RscIdx<Rsc> for WeakWidget<W> {
type Output = W;
fn get(self, rsc: &Rsc) -> &Self::Output {
&rsc.ui().widgets[self]
}
fn get_mut(self, rsc: &mut Rsc) -> &mut Self::Output {
&mut rsc.ui_mut().widgets[self]
}
}
impl<T: 'static, Rsc: HasWidgetState> RscIdx<Rsc> for WeakState<T> {
type Output = T;
fn get(self, rsc: &Rsc) -> &Self::Output {
rsc.widget_state().get(self)
}
fn get_mut(self, rsc: &mut Rsc) -> &mut Self::Output {
rsc.widget_state_mut().get_mut(self)
}
}
+16 -6
View File
@@ -13,7 +13,17 @@ use tokio::{
unbounded_channel as async_channel,
},
};
use winit::window::Window;
/// What a completed task nudges when it wants its result drawn. Shared
/// between backends rather than typed as `winit::window::Window` directly:
/// android-view has no `Window` at all, and the redraw request there is a
/// JNI call (`View::post_frame_callback`) rather than a method call on a
/// value this crate owns. Each backend supplies its own implementation --
/// `default/render.rs` for winit, `android/render.rs` for android-view --
/// and this module never needs to know which one it is holding.
pub trait RequestRedraw: Send + Sync + 'static {
fn request_redraw(&self);
}
pub type TaskMsgSender<Rsc> = SyncSender<Box<dyn TaskUpdate<Rsc>>>;
pub type TaskMsgReceiver<Rsc> = SyncReceiver<Box<dyn TaskUpdate<Rsc>>>;
@@ -23,7 +33,7 @@ impl<F: FnOnce(&mut Rsc::State, &mut Rsc) + Send, Rsc: HasState> TaskUpdate<Rsc>
pub struct Tasks<Rsc: HasState> {
start: AsyncSender<BoxTask>,
window: Arc<Window>,
redraw: Arc<dyn RequestRedraw>,
msg_send: SyncSender<Box<dyn TaskUpdate<Rsc>>>,
}
@@ -45,7 +55,7 @@ impl<Rsc: HasState + 'static> TaskCtx<Rsc> {
type BoxTask = Pin<Box<dyn Future<Output = ()> + Send>>;
impl<Rsc: HasState> Tasks<Rsc> {
pub fn init(window: Arc<Window>) -> (Self, TaskMsgReceiver<Rsc>) {
pub fn init(redraw: Arc<dyn RequestRedraw>) -> (Self, TaskMsgReceiver<Rsc>) {
let (start, start_recv) = async_channel();
let (msgs, msgs_recv) = sync_channel();
std::thread::spawn(|| {
@@ -56,7 +66,7 @@ impl<Rsc: HasState> Tasks<Rsc> {
Self {
start,
msg_send: msgs,
window,
redraw,
},
msgs_recv,
)
@@ -67,10 +77,10 @@ impl<Rsc: HasState> Tasks<Rsc> {
F::CallOnceFuture: Send,
{
let send = self.msg_send.clone();
let window = self.window.clone();
let redraw = self.redraw.clone();
let _ = self.start.send(Box::pin(async move {
task(TaskCtx::new(send)).await;
window.request_redraw();
redraw.request_redraw();
}));
}
}
File diff suppressed because it is too large. Load diff
+2
View File
@@ -1,4 +1,5 @@
mod image;
mod list;
mod mask;
mod position;
mod ptr;
@@ -7,6 +8,7 @@ mod text;
mod trait_fns;
pub use image::*;
pub use list::*;
pub use mask::*;
pub use position::*;
pub use ptr::*;
+16 -2
View File
@@ -4,6 +4,7 @@ use std::marker::{PhantomData, Sized};
pub struct TextBuilder<State, O = TextOutput, H: WidgetOption<State> = ()> {
pub content: String,
pub attrs: TextAttrs,
pub spans: Vec<SpanStyle>,
pub hint: H,
pub output: O,
state: PhantomData<State>,
@@ -39,10 +40,19 @@ impl<State, O, H: WidgetOption<State>> TextBuilder<State, O, H> {
self.attrs.wrap = wrap;
self
}
/// Per-range style overrides -- I5's inline rich text (bold, italic,
/// inline-code monospace, link colour/underline) within one wrapped
/// paragraph. See `SpanStyle`'s doc for why this exists and what it
/// replaces.
pub fn spans(mut self, spans: Vec<SpanStyle>) -> Self {
self.spans = spans;
self
}
pub fn editable(self, mode: EditMode) -> TextBuilder<State, TextEditOutput, H> {
TextBuilder {
content: self.content,
attrs: self.attrs,
spans: self.spans,
hint: self.hint,
output: TextEditOutput { mode },
state: PhantomData,
@@ -58,6 +68,7 @@ impl<Rsc: UiRsc, O> TextBuilder<Rsc, O> {
TextBuilder {
content: self.content,
attrs: self.attrs,
spans: self.spans,
hint: move |rsc: &mut Rsc| Some(hint.add_strong(rsc).any()),
output: self.output,
state: PhantomData,
@@ -81,7 +92,8 @@ impl<Rsc: UiRsc> TextBuilderOutput<Rsc> for TextOutput {
state: &mut Rsc,
builder: TextBuilder<Rsc, Self, H>,
) -> Self::Output {
let buf = TextBuffer::new(&builder.content);
let mut buf = TextBuffer::new(&builder.content);
buf.set_spans(builder.spans);
let hint = builder.hint.get(state);
let mut text = Text {
content: builder.content.into(),
@@ -103,7 +115,8 @@ impl<State: UiRsc> TextBuilderOutput<State> for TextEditOutput {
state: &mut State,
builder: TextBuilder<State, Self, H>,
) -> Self::Output {
let buf = TextBuffer::new(&builder.content);
let mut buf = TextBuffer::new(&builder.content);
buf.set_spans(builder.spans);
TextEdit::new(
TextView::new(buf, builder.attrs, builder.hint.get(state)),
builder.output.mode,
@@ -125,6 +138,7 @@ pub fn wtext<State>(content: impl Into<String>) -> TextBuilder<State> {
TextBuilder {
content: content.into(),
attrs: TextAttrs::default(),
spans: Vec::new(),
hint: (),
output: TextOutput,
state: PhantomData,
+48
View File
@@ -2,6 +2,7 @@ use crate::prelude::*;
use iris_core::{TextData, UiColor};
use parley::{Affinity, Layout, Selection};
use std::ops::{Deref, DerefMut};
#[cfg(not(target_os = "android"))]
use winit::{
event::KeyEvent,
keyboard::{Key, NamedKey},
@@ -28,6 +29,7 @@ pub struct TextEdit {
/// way to say, since it always denotes some position in the text. A
/// collapsed selection is a caret; an uncollapsed one is a span.
selection: Option<Selection>,
#[cfg_attr(target_os = "android", allow(dead_code))]
history: Vec<(String, Option<Selection>)>,
double_hit: Option<usize>,
pub mode: EditMode,
@@ -57,6 +59,26 @@ impl TextEdit {
}
Some(self.buf.text()[sel.text_range()].to_string())
}
/// The field's content. Byte-indexed, like everything else here since
/// I1 moved to parley -- an IME bridge (`android/ime.rs`) converts to
/// and from UTF-16 code units at its own edge rather than this type
/// knowing about that encoding.
pub fn text(&self) -> &str {
self.view.buf.text()
}
/// The selection as a byte range, collapsed to `caret..caret` when
/// there is no span. `None` when the field is not focused.
pub fn selection_range(&self) -> Option<std::ops::Range<usize>> {
Some(self.selection?.text_range())
}
/// The caret's byte offset -- the focus end of the selection, which is
/// where typing lands regardless of which end of a span it is.
pub fn caret(&self) -> Option<usize> {
Some(self.selection?.focus().index())
}
}
impl Widget for TextEdit {
@@ -92,6 +114,15 @@ impl Widget for TextEdit {
);
used
}
/// I4 (RUST.md): the one override that exists so far -- everything
/// else falls back to `Widget::access_role`'s default `Unknown`.
fn access_role(&self) -> accesskit::Role {
match self.mode {
EditMode::SingleLine => accesskit::Role::TextInput,
EditMode::MultiLine => accesskit::Role::MultilineTextInput,
}
}
}
const CARET_WIDTH: f32 = 1.0;
@@ -115,6 +146,7 @@ impl<'a> TextEditCtx<'a> {
}
/// Keep the selection valid after the text underneath it changed.
#[cfg_attr(target_os = "android", allow(dead_code))]
fn refresh(&mut self) {
if let Some(sel) = self.text.selection {
let layout = self.layout();
@@ -279,6 +311,20 @@ impl<'a> TextEditCtx<'a> {
self.set_caret(start);
}
/// The same range delete, exposed for callers that already have byte
/// offsets in hand rather than a `Motion` -- the IME's
/// `deleteSurroundingText`, which android-view hands over in UTF-16
/// code units that `android/ime.rs` converts before calling this.
pub fn delete_byte_range(&mut self, start: usize, end: usize) {
self.delete_range(start, end);
}
/// Move the caret to a byte offset, collapsing any selection -- the
/// IME's `setSelection`.
pub fn set_cursor_byte(&mut self, index: usize) {
self.set_caret(index);
}
pub fn select_all(&mut self) {
let len = self.text.view.buf.text().len();
if len == 0 {
@@ -336,6 +382,7 @@ impl<'a> TextEditCtx<'a> {
self.text.double_hit = None;
}
#[cfg(not(target_os = "android"))]
pub fn apply_event(&mut self, event: &KeyEvent, modifiers: &Modifiers) -> TextInputResult {
let old = (self.text.view.buf.text().to_string(), self.text.selection);
let mut undo = false;
@@ -352,6 +399,7 @@ impl<'a> TextEditCtx<'a> {
res
}
#[cfg(not(target_os = "android"))]
fn apply_event_inner(
&mut self,
event: &KeyEvent,
+13
View File
@@ -0,0 +1,13 @@
[package]
name = "tabs-ui"
version.workspace = true
edition.workspace = true
# The tabs example's widget tree, factored out of iris/examples/tabs/main.rs
# so it can be built once and driven by either backend: the winit example
# binary, and iris/android-app's cdylib. Its own crate rather than a pub
# module of `iris` because it is demo content, not library surface -- see
# RUST.md's I2.
[dependencies]
iris = { path = ".." }
File renamed without changes.
+212
View File
@@ -0,0 +1,212 @@
//! The tabs example's widget tree -- the five demo panes plus the message
//! composer that exercises `TextEdit`. Factored out of
//! `iris/examples/tabs/main.rs` (I2, RUST.md) so the same UI runs under
//! both backends: the winit example binary calls `build` from
//! `DefaultAppState::new`, and `iris-android-app`'s cdylib calls it from
//! `AndroidAppState::new`. Nothing here mentions either backend by name --
//! it only needs `Rsc: HasEvents` (for `.on(...)`) and `Rsc::State:
//! FocusHost` (for `.attr::<Selectable>(())`), both of which every backend
//! implements.
use iris::prelude::*;
use std::{cell::RefCell, rc::Rc};
pub struct ClientWidgets {
pub info: WeakWidget<Text>,
}
pub fn build<Rsc: HasEvents>(rsc: &mut Rsc, ui_state: &mut impl HasRoot) -> ClientWidgets
where
Rsc::State: FocusHost,
{
let rrect = rect(Color::WHITE).radius(20);
let pad_test = (
rrect.color(Color::BLUE),
(
rrect
.color(Color::RED)
.sized((100, 100))
.center()
.width(rest(2)),
(
rrect.color(Color::ORANGE),
rrect.color(Color::LIME).pad(10.0),
)
.span(Dir::RIGHT)
.width(rest(2)),
rrect.color(Color::YELLOW),
)
.span(Dir::RIGHT)
.pad(10)
.width(rest(3)),
)
.span(Dir::RIGHT)
.add(rsc);
let span_test = (
rrect.color(Color::GREEN).width(100),
rrect.color(Color::ORANGE),
rrect.color(Color::CYAN),
rrect.color(Color::BLUE).width(rel(0.5)),
rrect.color(Color::MAGENTA).width(100),
rrect.color(Color::RED).width(100),
)
.span(Dir::LEFT)
.add(rsc);
let span_add = Span::empty(Dir::RIGHT).add(rsc);
let add_button = rect(Color::LIME)
.radius(30)
.on(CursorSense::click(), move |_, rsc| {
let child = image(include_bytes!("../assets/sungals.png"))
.center()
.add_strong(rsc);
span_add(rsc).push(child);
})
.sized((150, 150))
.align(Align::BOT_RIGHT);
let del_button = rect(Color::RED)
.radius(30)
.on(CursorSense::click(), move |_, rsc| {
span_add(rsc).pop();
})
.sized((150, 150))
.align(Align::BOT_LEFT);
let span_add_test = (span_add, add_button, del_button).stack().add(rsc);
let btext = |content| wtext(content).size(30);
let text_test = (
btext("this is a").align(Align::LEFT),
btext("teeeeeeeest").align(Align::RIGHT),
btext("okkk\nokkkkkk!").align(Align::LEFT),
btext("hmm"),
btext("a"),
(
btext("'").family(Family::Monospace).align(Align::TOP),
btext("'").family(Family::Monospace),
btext(":gamer mode").family(Family::Monospace),
rect(Color::CYAN).sized((10, 10)).center(),
rect(Color::RED).sized((100, 100)).center(),
rect(Color::PURPLE).sized((50, 50)).align(Align::TOP),
)
.span(Dir::RIGHT)
.center(),
wtext("pretty cool right?").size(50),
)
.span(Dir::DOWN)
.add(rsc);
let texts = Span::empty(Dir::DOWN).gap(10).add(rsc);
let msg_area = texts.scrollable().masked().background(rect(Color::SKY));
let add_text = wtext("add")
.editable(EditMode::MultiLine)
.text_align(Align::LEFT)
.size(30)
.attr::<Selectable>(())
.on(Submit, move |ctx, rsc| {
let w = ctx.widget;
let content = w.edit(rsc).take();
let text = wtext(content)
.editable(EditMode::MultiLine)
.size(30)
.text_align(Align::LEFT)
.wrap(true)
.attr::<Selectable>(());
let msg_box = text
.background(rect(Color::WHITE.darker(0.5)))
.add_strong(rsc);
texts(rsc).push(msg_box);
})
.add(rsc);
let text_edit_scroll = (
msg_area.height(rest(1)),
(
Rect::new(Color::WHITE.darker(0.9)),
(
add_text.width(rest(1)),
Rect::new(Color::GREEN)
.on(CursorSense::click(), move |ctx, rsc: &mut Rsc| {
rsc.run_event::<Submit>(add_text, (), ctx.state);
})
.sized((40, 40)),
)
.span(Dir::RIGHT)
.pad(10),
)
.stack()
.size(StackSize::Child(1))
.layer_offset(1)
.align(Align::BOT),
)
.span(Dir::DOWN)
.add(rsc);
let main = WidgetPtr::new().add(rsc);
let vals = Rc::new(RefCell::new((0, Vec::new())));
let mut switch_button = |color, to: WeakWidget, label| {
let to = to.upgrade(rsc);
let vec = &mut vals.borrow_mut().1;
let i = vec.len();
if vec.is_empty() {
vec.push(None);
main(rsc).set(to);
} else {
vec.push(Some(to));
}
let vals = vals.clone();
let rect = rect(color)
.on(CursorSense::click(), move |ctx, rsc| {
let (prev, vec) = &mut *vals.borrow_mut();
if let Some(h) = vec[i].take() {
vec[*prev] = main(rsc).replace(h);
*prev = i;
}
ctx.widget(rsc).color = color.darker(0.3);
})
.on(
CursorSense::HoverStart | CursorSense::unclick(),
move |ctx, rsc| {
ctx.widget(rsc).color = color.brighter(0.2);
},
)
.on(CursorSense::HoverEnd, move |ctx, rsc| {
ctx.widget(rsc).color = color;
})
// I4 (RUST.md): the tabs screen's only named controls, and the
// ones the emulator step at the bottom of that box taps by
// name -- `ui-trace record --do "tap 'pad'"` and so on. `.label`
// slots into this chain like any other widget combinator
// (`RefFnTag` in `core/src/widget/tag.rs`); it does not have to
// be the last thing before `.add`.
.label(label);
(rect, wtext(label).size(30).text_align(Align::CENTER)).stack()
};
let tabs = (
switch_button(Color::RED, pad_test, "pad"),
switch_button(Color::GREEN, span_test, "span"),
switch_button(Color::BLUE, span_add_test, "image span"),
switch_button(Color::MAGENTA, text_test, "text layout"),
switch_button(
Color::YELLOW.mul_rgb(0.5),
text_edit_scroll,
"text edit scroll",
),
)
.span(Dir::RIGHT);
let info = wtext("").add(rsc);
let info_sect = info.pad(10).align(Align::RIGHT);
((tabs.height(40), main.pad(10)).span(Dir::DOWN), info_sect)
.stack()
.set_root(rsc, ui_state);
ClientWidgets { info }
}
+20
View File
@@ -0,0 +1,20 @@
[package]
name = "transcript-ui"
version.workspace = true
edition.workspace = true
# I5 (RUST.md): the transcript screen's widget tree, built the same way
# `tabs-ui` is -- its own crate, generic over `Rsc: HasEvents` +
# `Rsc::State: FocusHost`, so the winit example and an eventual
# `iris-android-app`-style cdylib call the same `build`. See its own module
# doc for the design and RUST.md's I5 box for what is and is not proved yet.
#
# `client-core`/`event-model` by path, real code and not a reimplementation
# -- the same dependency shape E2's uncommitted Masonry experiment used for
# the identical job.
[dependencies]
iris = { path = ".." }
client-core = { path = "../../client-core" }
event-model = { path = "../../event-model" }
pulldown-cmark = { workspace = true }
+122
View File
@@ -0,0 +1,122 @@
//! I5's desktop proof: the transcript screen built from synthetic
//! `client_core::transcript_fold` rows (no network, no server -- see
//! `lib.rs`'s doc for why `transcript-ui` itself never fetches anything),
//! run via `iris/run-headless.sh transcript -- -p transcript-ui` for a
//! screenshot on the winit backend, or `cargo run --example transcript -p
//! transcript-ui` with a real compositor.
//!
//! The rows exercise every one of the seven "hard to get back" behaviours
//! this box's markdown/selection work is meant to show: a heading, bold,
//! italic, an inline code span, a link, a fenced code block (rich inline
//! text), a multi-message conversation (bottom-anchored virtualised list),
//! and a three-call tool run (collapsed by default -- tap it, or drive it
//! with `ui-trace record --do "tap 'Tools'"` on Android, to prove
//! hold-the-edge expand).
use client_core::transcript_fold::{TranscriptItem, TranscriptRow as FoldedRow};
use iris::prelude::*;
fn main() {
DefaultApp::<Client>::run();
}
#[derive(DefaultUiState)]
pub struct Client {
ui_state: DefaultUiState,
#[allow(dead_code)]
screen: transcript_ui::TranscriptScreen,
}
fn msg(seq: u64, from_user: bool, text: &str) -> FoldedRow {
FoldedRow::Single(if from_user {
TranscriptItem::UserMsg {
seq,
text: text.to_string(),
attachments: Vec::new(),
}
} else {
TranscriptItem::AssistantMsg {
seq,
text: text.to_string(),
settled: true,
}
})
}
fn synthetic_rows() -> Vec<FoldedRow> {
vec![
msg(
1,
true,
"Can you show me a **bold** word, some *italic* text, and `inline code`?",
),
msg(
2,
false,
"# Sure\n\nHere's a [link to the repo](https://example.com/ai-app-2) and a fenced block:\n\n```rust\nfn main() {\n println!(\"hi\");\n}\n```",
),
FoldedRow::Tools(vec![
TranscriptItem::ToolRun {
seq: 3,
id: "t1".into(),
run_id: "run1".into(),
tool: "Read".into(),
input: "{\"file\": \"src/main.rs\"}".into(),
output: "fn main() {}\n".into(),
done: true,
asks: Vec::new(),
images: Vec::new(),
},
TranscriptItem::ToolRun {
seq: 4,
id: "t2".into(),
run_id: "run1".into(),
tool: "Edit".into(),
input: "{\"file\": \"src/main.rs\"}".into(),
output: "ok".into(),
done: true,
asks: Vec::new(),
images: Vec::new(),
},
TranscriptItem::ToolRun {
seq: 5,
id: "t3".into(),
run_id: "run1".into(),
tool: "Bash".into(),
input: "cargo build".into(),
output: "Compiling...\nFinished.".into(),
done: true,
asks: Vec::new(),
images: Vec::new(),
},
]),
msg(6, true, "Looks good, thanks!"),
msg(
7,
false,
"You're welcome. Let me know if you'd like anything else.",
),
]
}
impl DefaultAppState for Client {
fn new(
mut ui_state: DefaultUiState,
rsc: &mut DefaultRsc<Self>,
_: Proxy<Self::Event>,
) -> Self {
let screen = transcript_ui::build(rsc, &mut ui_state, synthetic_rows());
// Exercises `push_row`/`ItemKey` beyond construction time, matching
// how a live SSE loop appends -- a row arriving after the screen
// already exists must land at the bottom without disturbing what's
// above it (I3's `push_back`/`snap_end`).
screen.push_row(
rsc,
&FoldedRow::Single(TranscriptItem::CommandRow {
seq: 8,
text: "clear".into(),
}),
);
Self { ui_state, screen }
}
}
+48
View File
@@ -0,0 +1,48 @@
//! The message composer at the bottom of the transcript screen: a
//! multi-line editable field with a natural (not fixed) height, so it
//! grows as typed into -- IRIS_TODO.md's "input box" benchmark case
//! (`iris/benches/message_list.rs` exercises the mechanism in isolation;
//! this wires the same `TextEdit`-with-no-`Sized`-wrapper idiom into the
//! real screen). `lib.rs` gives the transcript `List` `.height(rest(1))`
//! beside this widget in a `Span::down`, so the list's own draw already
//! measures whatever vertical space is left each frame -- nothing here
//! computes a height by hand, and growing this field is exactly the
//! O(1)-move-chain case LAYOUT.md and I3's benchmark already measured.
use iris::prelude::*;
/// `field` is exposed so the caller can read its content on submit
/// (`field.edit(rsc).text()`) and clear it afterward
/// (`field.edit(rsc).set("")`).
pub struct Composer {
pub field: WeakWidget<TextEdit>,
}
/// Returns the composer plus its own bar as a **weak** id -- the caller
/// (`lib.rs::build`) embeds it in the screen's own top-level tuple, whose
/// `set_root` performs the one real strong registration. Calling
/// `.add_strong`/`.upgrade` a second time on an id already strong-owned
/// panics ("was already added", `core/src/widget/like.rs:12`) -- the same
/// mistake this box's `row.rs` first made with its sender-label header, see
/// that file's comment for the fuller account.
pub fn build_composer<Rsc: HasEvents>(rsc: &mut Rsc) -> (Composer, WeakWidget)
where
Rsc::State: FocusHost,
{
let field = wtext("")
.editable(EditMode::MultiLine)
.text_align(Align::LEFT)
.wrap(true)
.size(18)
.color(UiColor::WHITE)
.attr::<Selectable>(())
.label("Message")
.add(rsc);
let bar: WeakWidget = (field.pad(12).width(rest(1)),)
.span(Dir::RIGHT)
.background(rect(UiColor::new(40, 40, 46, 255)))
.add(rsc);
(Composer { field }, bar)
}
+145
View File
@@ -0,0 +1,145 @@
//! The transcript screen, in iris -- RUST.md's I5. Built the same way
//! `tabs-ui` is: its own crate, generic over `Rsc: HasEvents` +
//! `Rsc::State: FocusHost`, so the winit example (`iris/examples/
//! transcript.rs`) and an eventual `iris-android-app`-style cdylib call the
//! same [`build`]. See RUST.md's I5 box for the full account of what is
//! and is not proved yet, and this doc for the shape.
//!
//! ```text
//! +------------------------------------------+
//! | iris::widget::List (transcript_ui::row) | <- .height(rest(1))
//! | row 1: sender label + one TextEdit |
//! | row 2: sender label + one TextEdit |
//! | row 3 (Tools): collapsed/expanded |
//! | ... |
//! +------------------------------------------+
//! | composer bar (transcript_ui::composer) | <- natural height
//! +------------------------------------------+
//! ```
//!
//! **What this crate does not do itself**: fetch anything over the network
//! or read the transcript cache. [`build`] takes an already-folded
//! `Vec<client_core::transcript_fold::TranscriptRow>` and
//! [`TranscriptScreen::push_row`] takes one more as it arrives -- the
//! caller (an app's own `main`, or a future `iris-android-app`-shaped
//! cdylib) owns `client_core::ApiClient`/
//! `event_stream::follow_session_events` and the transcript cache, per the
//! code rules' "ask for the least you need": a widget-tree builder that
//! also knew how to make an HTTPS request would be untestable without a
//! server and unable to be driven by `run-headless.sh` with synthetic rows.
//!
//! **Gap closed, 2026-09-05**: a touch-drag that starts on a row's
//! rendered text used to always begin a cross-row *selection* (`row.rs`'s
//! `CursorSense::click_or_drag()` on each row's `TextEdit`), never a
//! *scroll* of the list, because both wanted the same gesture over the
//! same screen region and `core/src/sense.rs`'s `run_sensors` gave the
//! widget in the *inner* layer (a row's own `TextEdit`) first refusal
//! every frame it was pressed. `row.rs` now routes every row's drag
//! through one shared `iris::sense::DragArbiter`
//! (`Selection::drag`, `selection.rs`), which decides pan vs. select the
//! way Android itself does -- see `DragArbiter`'s own doc and
//! `DECISIONS.md` for the exact rule. `List` scrolls correctly when
//! driven programmatically (I3's benchmark), via the mouse wheel (wired
//! below, `CursorSense::Scroll`), and now via a touch pan starting on a
//! row's own text too.
pub mod composer;
pub mod markdown;
pub mod row;
pub mod selection;
use client_core::transcript_fold::TranscriptRow as FoldedRow;
use iris::prelude::*;
use selection::Selection;
use std::{cell::RefCell, rc::Rc};
pub struct TranscriptScreen {
/// The transcript's own `List` -- exposed so a caller can read
/// `.extent()`/call `.jump_to_end()` etc. directly for anything this
/// crate does not already wrap.
pub list: WeakWidget<List>,
pub composer: composer::Composer,
selection: Rc<RefCell<Selection>>,
}
impl TranscriptScreen {
/// Append one more folded row at the live end of the transcript --
/// what a caller's SSE loop or a sent message calls as new events
/// arrive. `List::push_back` is O(1) and keeps the view pinned to the
/// newest content when it already was (I3).
pub fn push_row<Rsc: HasEvents>(&self, rsc: &mut Rsc, row: &FoldedRow)
where
Rsc::State: FocusHost,
{
let (key, widget) = row::build_row(rsc, self.list, self.selection.clone(), row);
(self.list)(rsc).push_back(ListRow::new(key, widget));
}
/// The concatenated text of whatever is currently selected across one
/// or more rows, `None` if nothing is -- what a copy command reads.
pub fn selected_text(&self, rsc: &mut impl UiRsc) -> Option<String> {
self.selection.borrow().selected_text(rsc)
}
}
pub fn build<Rsc: HasEvents>(
rsc: &mut Rsc,
ui_state: &mut impl HasRoot,
rows: Vec<FoldedRow>,
) -> TranscriptScreen
where
Rsc::State: FocusHost,
{
let (screen, tree) = build_tree(rsc, rows);
ui_state.set_root(tree);
screen
}
/// The same widget tree [`build`] makes, without claiming the window's
/// whole root -- what a caller embedding this screen alongside something
/// else of its own needs (RUST.md's E4: a session list beside the
/// transcript on the desktop). `build` is `build_tree` plus
/// `ui_state.set_root(tree)`; kept as its own function since most callers
/// (the winit example, an eventual Android cdylib) want the screen to *be*
/// the window and don't need the strong handle back.
pub fn build_tree<Rsc: HasEvents>(
rsc: &mut Rsc,
rows: Vec<FoldedRow>,
) -> (TranscriptScreen, StrongWidget)
where
Rsc::State: FocusHost,
{
let selection = Rc::new(RefCell::new(Selection::new()));
let list = List::new(Axis::Y).add(rsc);
for row in &rows {
let (key, widget) = row::build_row(rsc, list, selection.clone(), row);
list(rsc).push_back(ListRow::new(key, widget));
}
// Wheel/trackpad scrolling -- the same idiom `trait_fns.rs`'s
// `scrollable()` uses for `Scroll`, applied directly to `List` since
// `List` already does its own placement and needs no `Scroll` wrapper.
// Real touch-drag panning is the known gap in this module's doc.
list.on(CursorSense::Scroll, |ctx, rsc| {
let delta = ctx.data.scroll_delta.y * 50.0;
ctx.widget(rsc).scroll(delta);
})
.add(rsc);
let (composer, composer_bar) = composer::build_composer(rsc);
let tree = (list.width(rest(1)).height(rest(1)), composer_bar)
.span(Dir::DOWN)
.add_strong(rsc)
.any();
(
TranscriptScreen {
list,
composer,
selection,
},
tree,
)
}
+223
View File
@@ -0,0 +1,223 @@
//! Markdown -> one plain string plus a `Vec<SpanStyle>`, for I5's row
//! builder to hand to a single `TextEdit` (`row.rs`). This is the crate's
//! answer to RUST.md's E2 finding against Masonry ("rich inline text --
//! block-level yes, inline no, and both for the same reason": `TextArea`'s
//! `StyleSet` is one style for the whole editor,
//! `masonry/src/widgets/text_area.rs:43-44`'s `// TODO: RichTextInput`
//! beside it). iris's `SpanStyle` (`core/src/primitive/text.rs`, added for
//! this box) is per-range, so bold/italic/inline-code/links/headings inside
//! one wrapped paragraph render in their own style *and* the paragraph
//! still wraps and selects as one buffer -- there is no second widget per
//! span the way E2's block-level `Prose`-per-heading was.
//!
//! **What this deliberately does not attempt**, each for a reason recorded
//! here rather than silently dropped (see IRIS_TODO.md's dated entries for
//! the same list):
//! - **No background chip behind inline code.** Drawing one needs the
//! glyph run's own geometry (the way `TextEdit::draw`'s selection
//! highlight uses `selection.geometry(layout)`,
//! `iris/src/widget/text/edit.rs:99`), which is `TextEdit`-internal and
//! not exposed to a caller building spans externally. `SpanStyle` gives
//! the code range a monospace family and a dimmer text colour instead --
//! visually distinct, just not chip-shaped.
//! - **A link is styled (colour + underline) but not tappable.** Following
//! it needs the same kind of per-range hit-testing a chip's background
//! would (which byte range did the tap land in, then look up its URL),
//! which is exactly the same missing primitive.
//! - **Tables render as plain paragraphs of their cell text**, no columns.
//! `pulldown_cmark::Tag::Table` is walked but not laid out -- a real grid
//! needs its own widget, out of scope for a row builder.
//! - **A fenced code block's language is not syntax-highlighted.**
//! `client-core::highlight` exists and could feed per-token `SpanStyle`s,
//! but wiring it in is real work belonging to whoever needs it next
//! (IRIS_TODO.md).
//!
//! A heading's `SpanStyle::font_size` override does not also raise its
//! `line_height` (a buffer has one, set from the *base* font size in
//! `TextAttrs`), so a heading's own line looks slightly tighter than a
//! paragraph's -- visible, not incorrect, and not fixed here since it needs
//! `SpanStyle` to carry line-height too, which nothing in this crate needed
//! badly enough yet to justify.
use iris::prelude::*;
use pulldown_cmark::{Event, HeadingLevel, Options, Parser, Tag, TagEnd};
// `UiColor` is `Color<u8>` (`core/src/lib.rs`), not the 0..1 float triples
// its brighter/darker helpers might suggest -- these are plain 0..255 RGB.
pub const CODE_COLOR: UiColor = UiColor::new(140, 217, 242, 255);
pub const LINK_COLOR: UiColor = UiColor::new(140, 190, 255, 255);
const STRIKETHROUGH_COLOR: UiColor = UiColor::new(150, 150, 150, 255);
/// A block-level separator: two blocks never run into each other with no
/// gap, but an empty `out` (the very first block) gets no leading blank.
fn ensure_blank_line(out: &mut String) {
if !out.is_empty() && !out.ends_with("\n\n") {
out.push_str("\n\n");
}
}
fn heading_size(level: HeadingLevel) -> f32 {
match level {
HeadingLevel::H1 => 28.0,
HeadingLevel::H2 => 24.0,
HeadingLevel::H3 => 21.0,
_ => 19.0,
}
}
/// One markdown source string rendered into plain text plus the spans that
/// style it. `base_size` is the row's ordinary paragraph font size, needed
/// only so a heading's override is relative to it rather than a hardcoded
/// absolute the caller cannot retune.
pub fn render_markdown(src: &str, base_size: f32) -> (String, Vec<SpanStyle>) {
let _ = base_size; // headings use fixed sizes today; kept for callers that may want relative sizing later
let mut out = String::new();
let mut spans = Vec::new();
// Stack of start byte offsets for whatever inline/block styling is
// currently open -- pulldown-cmark's `Start`/`End` events are always
// balanced and each `End` already names its own kind (`TagEnd`), so a
// plain offset stack (rather than a tree, or repeating the kind here
// too) is enough.
let mut open: Vec<usize> = Vec::new();
let mut list_depth: u32 = 0;
let parser = Parser::new_ext(src, Options::ENABLE_STRIKETHROUGH | Options::ENABLE_TABLES);
for event in parser {
match event {
Event::Start(tag) => match tag {
Tag::Heading { .. }
| Tag::Emphasis
| Tag::Strong
| Tag::Strikethrough
| Tag::Link { .. } => open.push(out.len()),
Tag::CodeBlock(_) => {
ensure_blank_line(&mut out);
open.push(out.len());
}
Tag::Item => {
out.push_str(&" ".repeat(list_depth.saturating_sub(1) as usize));
out.push_str("\u{2022} ");
}
Tag::List(_) => list_depth += 1,
Tag::Paragraph | Tag::BlockQuote(_) => ensure_blank_line(&mut out),
_ => {}
},
// Only the tag kinds that pushed onto `open` (Start, above) are
// popped here -- `List`/`Item`/`Paragraph`/`BlockQuote`/`Table`
// and friends push nothing, since they need no span, and must
// not touch this stack or they would pop an unrelated styled
// range still open around them.
Event::End(
tag_end @ (TagEnd::Heading(_)
| TagEnd::Emphasis
| TagEnd::Strong
| TagEnd::Strikethrough
| TagEnd::Link
| TagEnd::CodeBlock),
) => {
let Some(start) = open.pop() else {
continue;
};
let range = start..out.len();
if range.is_empty() {
continue;
}
match tag_end {
TagEnd::Heading(level) => {
spans.push(SpanStyle::new(range).font_size(heading_size(level)).bold());
}
TagEnd::Emphasis => spans.push(SpanStyle::new(range).italic()),
TagEnd::Strong => spans.push(SpanStyle::new(range).bold()),
TagEnd::Strikethrough => {
spans.push(SpanStyle::new(range).color(STRIKETHROUGH_COLOR));
}
TagEnd::Link => {
spans.push(SpanStyle::new(range).color(LINK_COLOR).underline());
}
TagEnd::CodeBlock => {
spans.push(
SpanStyle::new(range)
.family(Family::Monospace)
.color(CODE_COLOR),
);
}
_ => unreachable!("filtered by the outer match arm"),
}
}
Event::Text(text) => out.push_str(&text),
// Inline code (single backticks) is one atomic event with no
// `Start`/`End` pair of its own, unlike a fenced block -- so it
// is spanned directly here instead of through the `open` stack.
Event::Code(text) => {
let start = out.len();
out.push_str(&text);
spans.push(
SpanStyle::new(start..out.len())
.family(Family::Monospace)
.color(CODE_COLOR),
);
}
Event::SoftBreak => out.push(' '),
Event::HardBreak => out.push('\n'),
Event::Rule => {
if !out.ends_with('\n') {
out.push('\n');
}
out.push_str("\u{2500}\u{2500}\u{2500}\n");
}
Event::End(TagEnd::List(_)) => list_depth = list_depth.saturating_sub(1),
_ => {}
}
}
(out, spans)
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn plain_paragraph_has_no_spans() {
let (text, spans) = render_markdown("just some words", 16.0);
assert_eq!(text, "just some words");
assert!(spans.is_empty());
}
#[test]
fn bold_and_italic_produce_spans_over_the_right_range() {
let (text, spans) = render_markdown("a **bold** and *italic* word", 16.0);
assert_eq!(text, "a bold and italic word");
let bold = spans.iter().find(|s| s.bold && !s.italic).unwrap();
assert_eq!(&text[bold.range.clone()], "bold");
let italic = spans.iter().find(|s| s.italic).unwrap();
assert_eq!(&text[italic.range.clone()], "italic");
}
#[test]
fn heading_gets_a_bigger_font_size_span() {
let (text, spans) = render_markdown("# A Title\n\nbody text", 16.0);
assert!(text.starts_with("A Title"));
let heading = spans.iter().find(|s| s.font_size.is_some()).unwrap();
assert_eq!(&text[heading.range.clone()], "A Title");
assert_eq!(heading.font_size, Some(28.0));
}
#[test]
fn link_is_styled_and_keeps_its_visible_text() {
let (text, spans) = render_markdown("see [the docs](https://example.com) for more", 16.0);
assert!(text.contains("the docs"));
assert!(
!text.contains("example.com"),
"the URL should not leak into the visible text"
);
let link = spans.iter().find(|s| s.underline).unwrap();
assert_eq!(&text[link.range.clone()], "the docs");
}
#[test]
fn fenced_code_block_is_monospaced() {
let (text, spans) = render_markdown("before\n\n```\nlet x = 1;\n```\n\nafter", 16.0);
let code = spans.iter().find(|s| s.family.is_some()).unwrap();
assert!(text[code.range.clone()].contains("let x = 1;"));
}
}
+301
View File
@@ -0,0 +1,301 @@
//! One `iris::widget::list::ListRow` per folded transcript row
//! (`client_core::transcript_fold::TranscriptRow`). Each row's whole text
//! -- headings, paragraphs, inline styling -- goes through `markdown` into
//! **one** `TextEdit`, which is what makes it one thing `Selection`
//! (`selection.rs`) can select and what lets it wrap and scroll as a
//! single buffer, matching RUST.md's "hard to get back" behaviour 2 (rich
//! inline text) and half of behaviour 1 (selectable within a row; across
//! rows is `selection.rs`'s job).
//!
//! A `TranscriptRow::Tools` (a run of adjacent tool calls, grouped by
//! `client_core::transcript_fold::group_tool_runs`) is the row that proves
//! behaviour 3's "hold the edge nearest the tap" on expand: tapping its
//! header calls `List::note_tap` at the row's own on-screen position
//! (read back from `List::extent`, since the tap event only knows its
//! position *within* this row) before toggling a `WidgetPtr` between the
//! collapsed summary and the full detail -- the same two-step contract
//! `list.rs`'s module doc describes for `AGENTS.md`'s `holdTopEdge`.
use crate::markdown::render_markdown;
use crate::selection::Selection;
use client_core::transcript_fold::{QuestionCard, TranscriptItem, TranscriptRow as FoldedRow};
use iris::prelude::*;
use std::{cell::RefCell, rc::Rc, time::Instant};
/// The paragraph size every row's `TextEdit` is built at; markdown headings
/// inside a row scale relative to a fixed set of sizes rather than this one
/// (`markdown::heading_size`), since a heading is meant to look the same
/// regardless of which row's base size surrounds it.
pub const BASE_SIZE: f32 = 16.0;
/// `ItemKey::Seq` already is the `RowKey` (`u64`) this crate's `List` wants.
/// `ItemKey::RunId` is a string (a tool call's own id), so it is hashed into
/// one -- collisions are not a correctness risk worth guarding against here
/// (a `DefaultHasher` collision across the run ids one session produces is
/// astronomically unlikely, and the consequence of one would only be two
/// tool-call rows sharing a list slot, not data loss), and the high bit is
/// forced on so a hashed key can never collide with a real sequence number
/// (this build never produces 2^63 events).
pub fn row_key(key: &client_core::transcript_fold::ItemKey) -> RowKey {
use client_core::transcript_fold::ItemKey;
use std::hash::{Hash, Hasher};
match key {
ItemKey::Seq(seq) => *seq,
ItemKey::RunId(id) => {
let mut h = std::collections::hash_map::DefaultHasher::new();
id.hash(&mut h);
h.finish() | (1 << 63)
}
}
}
/// The sender label shown above a row's text, and the markdown source to
/// render below it. `None` for a system-style note that has no sender.
fn item_content(item: &TranscriptItem) -> (Option<&str>, String) {
match item {
TranscriptItem::UserMsg { text, .. } => (Some("You"), text.clone()),
TranscriptItem::AssistantMsg { text, .. } => (Some("Claude"), text.clone()),
TranscriptItem::ErrorMsg { message, .. } => (Some("Error"), message.clone()),
TranscriptItem::CommandRow { text, .. } => (Some("Command"), format!("`/{text}`")),
TranscriptItem::PeerNote { from, text, .. } => (Some(from.as_str()), text.clone()),
TranscriptItem::Note { text, .. } => (None, text.clone()),
TranscriptItem::ClearedNote { .. } => (None, "_Context cleared._".to_string()),
TranscriptItem::CompactedNote {
pre_tokens,
post_tokens,
..
} => (
None,
match (pre_tokens, post_tokens) {
(Some(pre), Some(post)) => format!("_Compacted: {pre} -> {post} tokens._"),
_ => "_Compacted._".to_string(),
},
),
TranscriptItem::ImageItem { r#ref, .. } => (None, format!("_[image: {ref}]_")),
TranscriptItem::QuestionCard(card) => (Some("Question"), question_markdown(card)),
TranscriptItem::ToolRun {
tool,
input,
output,
..
} => (Some(tool.as_str()), tool_call_markdown(tool, input, output)),
}
}
fn question_markdown(card: &QuestionCard) -> String {
let mut out = card.prompt.clone();
for opt in &card.options {
out.push_str(&format!("\n- {}", opt.label));
}
out
}
fn tool_call_markdown(tool: &str, input: &str, output: &str) -> String {
let mut out = format!("**{tool}**\n\n```\n{input}\n```");
if !output.is_empty() {
out.push_str(&format!("\n\n```\n{output}\n```"));
}
out
}
/// Build one `TextEdit` from a sender label plus markdown source, register
/// it with `selection` under `key`, and wire the pointer handlers that
/// drive `Selection::drag` -- shared by every row variant below, since a
/// selectable row is always "one TextEdit plus this wiring" regardless of
/// what folded it. `list` is threaded through so that same drag can pan
/// the list instead of selecting, per `Selection::drag`'s own doc.
fn build_text_row<Rsc: HasEvents>(
rsc: &mut Rsc,
list: WeakWidget<List>,
selection: Rc<RefCell<Selection>>,
key: RowKey,
sender: Option<&str>,
markdown_src: &str,
) -> StrongWidget
where
Rsc::State: FocusHost,
{
let (text, spans) = render_markdown(markdown_src, BASE_SIZE);
let field = wtext(text)
.spans(spans)
.editable(EditMode::MultiLine)
.text_align(Align::LEFT)
.wrap(true)
.size(BASE_SIZE)
.color(UiColor::WHITE)
.add(rsc);
selection.borrow_mut().register(key, field);
field
// `| CursorSense::unclick()` on top of the usual click-or-drag set
// -- the arbiter inside `Selection::drag` needs the release too,
// to go back to idle for the next press (`DragArbiter::release`).
.on(
CursorSense::click_or_drag() | CursorSense::unclick(),
move |ctx, rsc| {
selection.borrow_mut().drag(
rsc,
list,
key,
ctx.data.pos,
ctx.data.size,
ctx.data.cursor.pos,
ctx.data.sense,
Instant::now(),
);
},
)
.add(rsc);
// `.add` (weak), not `.add_strong` -- `header` is about to be embedded
// as a child of the `.span(Dir::DOWN)` below, whose own composition is
// what performs the *one* real strong registration each child gets.
// Calling `.add_strong`/`.upgrade` here too, then feeding a `.weak()`
// copy into that composition, tried to strong-register the same id
// twice and panicked with "was already added"
// (`core/src/widget/like.rs:12`) -- found running this crate's own
// `run-headless.sh` example, the first real render of a row.
let header: WeakWidget = match sender {
Some(name) => wtext(name.to_string())
.size(13.0)
.color(UiColor::new(150, 150, 160, 255))
.add(rsc),
None => Span::empty(Dir::DOWN).add(rsc),
};
(header, field.width(rest(1)))
.span(Dir::DOWN)
.gap(4)
.pad(10)
.add_strong(rsc)
.any()
}
fn build_single<Rsc: HasEvents>(
rsc: &mut Rsc,
list: WeakWidget<List>,
selection: Rc<RefCell<Selection>>,
key: RowKey,
item: &TranscriptItem,
) -> StrongWidget
where
Rsc::State: FocusHost,
{
let (sender, markdown_src) = item_content(item);
build_text_row(rsc, list, selection, key, sender, &markdown_src)
}
/// A run of adjacent tool calls: collapsed to a one-line summary by
/// default, expanding in place to every call's own tool/input/output on
/// tap -- see the module doc for the hold-the-edge contract this wires
/// against `list`.
fn build_tools<Rsc: HasEvents>(
rsc: &mut Rsc,
list: WeakWidget<List>,
selection: Rc<RefCell<Selection>>,
key: RowKey,
calls: Vec<TranscriptItem>,
) -> StrongWidget
where
Rsc::State: FocusHost,
{
let expanded = Rc::new(RefCell::new(false));
// `.add_strong` (not `.add`) because nothing else in the tree holds a
// strong reference to this `WidgetPtr` the way a container's own
// `add_strong`-on-its-children does for an ordinary child -- this row
// *is* the top of its own subtree, so it has to own itself.
let ptr_strong = WidgetPtr::new().add_strong(rsc);
let ptr = ptr_strong.weak();
let summary_text = format!("\u{25b8} {} tool calls", calls.len());
let full_text = calls
.iter()
.map(|c| match c {
TranscriptItem::ToolRun {
tool,
input,
output,
..
} => tool_call_markdown(tool, input, output),
other => item_content(other).1,
})
.collect::<Vec<_>>()
.join("\n\n");
fn build_content<Rsc: HasEvents>(
rsc: &mut Rsc,
list: WeakWidget<List>,
selection: Rc<RefCell<Selection>>,
key: RowKey,
expanded: bool,
summary: &str,
full: &str,
) -> StrongWidget
where
Rsc::State: FocusHost,
{
let text = if expanded { full } else { summary };
build_text_row(rsc, list, selection, key, Some("Tools"), text)
}
let content = build_content(
rsc,
list,
selection.clone(),
key,
false,
&summary_text,
&full_text,
);
ptr(rsc).set(content);
ptr.on(CursorSense::click(), move |ctx, rsc| {
// `List::note_tap` wants a viewport-relative position, but the
// click event only knows where inside *this row* it landed
// (`ctx.data.pos`) -- `List::extent` (last frame's on-screen box
// for this row's key) is what turns the two into the position
// `list.rs`'s hold-the-edge layout pass resolves against, per the
// module doc's contract.
let (top, _bottom) = list(rsc).extent(key).unwrap_or((0.0, 0.0));
list(rsc).note_tap(top + ctx.data.pos.y);
let was_expanded = *expanded.borrow();
*expanded.borrow_mut() = !was_expanded;
let content = build_content(
rsc,
list,
selection.clone(),
key,
!was_expanded,
&summary_text,
&full_text,
);
// The old content's `StrongWidget` is freed when this drops --
// the removal half of the row this click just replaced.
let _old = ptr(rsc).replace(content);
})
.add(rsc);
ptr_strong.any()
}
pub fn build_row<Rsc: HasEvents>(
rsc: &mut Rsc,
list: WeakWidget<List>,
selection: Rc<RefCell<Selection>>,
row: &FoldedRow,
) -> (RowKey, StrongWidget)
where
Rsc::State: FocusHost,
{
match row {
FoldedRow::Single(item) => {
let key = row_key(&item.key());
(key, build_single(rsc, list, selection, key, item))
}
FoldedRow::Tools(calls) => {
let key = row_key(&calls[0].key());
(key, build_tools(rsc, list, selection, key, calls.clone()))
}
}
}
+288
View File
@@ -0,0 +1,288 @@
//! Selection spanning multiple transcript rows -- RUST.md's "hard to get
//! back" behaviour 1, and the one E2 found flatly impossible on Masonry:
//! `TextArea` wraps exactly one `parley::PlainEditor`, and there is no
//! `SelectionContainer`-shaped type anywhere in `masonry`/`masonry_core`/
//! `xilem` (RUST.md's E2 box, citing
//! `masonry/src/widgets/text_area.rs:414-459`). Each transcript row here is
//! still its own `TextEdit` (one per row, not one per transcript, since a
//! row is what `List` virtualises), so this is not literally "one
//! `PlainEditor`" either -- iris's answer is a coordinator that drives each
//! visible row's *own* selection primitives (`TextEditCtx::select`/
//! `select_all`/`deselect`, already built for a single field) from one
//! pointer drag that crosses row boundaries, giving the same reader-facing
//! result (a selection that runs from a reply into the tool output beneath
//! it, one copy) without needing a single shared text buffer underneath.
//!
//! Rows are keyed by `RowKey` (`iris::widget::list`), which every real row
//! source (a transcript's sequence number) already assigns in the order the
//! reader reads them in -- so "between the anchor and the current row" is
//! answered by ordinary integer comparison via a `BTreeMap`, not a second
//! copy of the list's own ordering.
//!
//! **Scoped shortcut, recorded rather than hidden**: the anchor row (the
//! one the drag started in) is selected in full (`select_all`) the moment
//! the drag leaves it, rather than "from the click point to whichever edge
//! points away from the drag" -- the exact partial selection would need
//! that row's own laid-out size, which `TextEditCtx` does not expose to a
//! caller outside `iris::widget::text` (`edit.rs`'s `layout()` helper is
//! private). Only the row currently *under the pointer* gets a true partial
//! selection (from its own start or end, per direction, to the pointer's
//! exact point) -- see `extend`. Re-entering the anchor row is still exact,
//! since that branch never goes through the approximation.
use iris::prelude::*;
use std::{collections::BTreeMap, time::Instant};
pub struct Selection {
rows: BTreeMap<RowKey, WeakWidget<TextEdit>>,
anchor: Option<(RowKey, Vec2)>,
/// One arbiter shared by every row's drag handler -- RUST.md's I5
/// gesture conflict (a row's own `click_or_drag()` and a list-level
/// pan wanting the same touch gesture). See `drag` below, and
/// `iris::sense::DragArbiter`'s own doc for the decision itself.
arbiter: DragArbiter,
}
impl Default for Selection {
fn default() -> Self {
Self::new()
}
}
impl Selection {
pub fn new() -> Self {
Self {
rows: BTreeMap::new(),
anchor: None,
arbiter: DragArbiter::new(),
}
}
/// A row's selectable text became visible/known. Every addition here
/// needs its removal (`unregister`) -- called when `List` evicts the
/// row (`pop_front`/`pop_back`), so this map never outgrows however
/// many rows are actually loaded.
pub fn register(&mut self, key: RowKey, text: WeakWidget<TextEdit>) {
self.rows.insert(key, text);
}
pub fn unregister(&mut self, key: RowKey) {
self.rows.remove(&key);
if self.anchor.map(|(k, _)| k) == Some(key) {
self.anchor = None;
}
}
/// A fresh press: clears whatever was selected elsewhere (an ordinary
/// click starts a new selection, it does not extend the old one) and
/// gives `key`'s row a collapsed caret at `pos` -- a plain click that
/// never turns into a drag leaves exactly this and nothing else
/// selected.
pub fn begin(&mut self, ui: &mut impl UiRsc, key: RowKey, pos: Vec2, size: Vec2) {
let rows: Vec<RowKey> = self.rows.keys().copied().collect();
for k in rows {
if k != key
&& let Some(w) = self.rows.get(&k)
{
w.edit(ui).deselect();
}
}
if let Some(w) = self.rows.get(&key) {
w.edit(ui).select(pos, size, false, false);
}
self.anchor = Some((key, pos));
}
/// The drag continues, now over `key`'s row at `pos`. See the module
/// doc for the anchor-row shortcut.
pub fn extend(&mut self, ui: &mut impl UiRsc, key: RowKey, pos: Vec2, size: Vec2) {
let Some((anchor_key, _anchor_pos)) = self.anchor else {
return;
};
if key == anchor_key {
if let Some(w) = self.rows.get(&key) {
w.edit(ui).select(pos, size, true, false);
}
return;
}
let (lo, hi) = if anchor_key < key {
(anchor_key, key)
} else {
(key, anchor_key)
};
let in_range: Vec<RowKey> = self.rows.range(lo..=hi).map(|(&k, _)| k).collect();
for k in &in_range {
let Some(w) = self.rows.get(k).copied() else {
continue;
};
if *k == key {
// The row under the pointer: partial selection from
// whichever of its own edges faces the anchor, extended to
// the exact pointer point.
let start = if key > anchor_key { Vec2::ZERO } else { size };
w.edit(ui).select(start, size, false, false);
w.edit(ui).select(pos, size, true, false);
} else {
w.edit(ui).select_all();
}
}
let outside: Vec<RowKey> = self
.rows
.keys()
.copied()
.filter(|k| *k < lo || *k > hi)
.collect();
for k in outside {
if let Some(w) = self.rows.get(&k) {
w.edit(ui).deselect();
}
}
}
/// Whether any row currently has a non-empty selection -- what a fresh
/// press consults so `drag` knows whether an early horizontal move is
/// "start dragging the selection handle" rather than an ordinary tap.
fn has_selection(&self, ui: &mut impl UiRsc) -> bool {
self.rows
.values()
.any(|w| w.edit(ui).text.selected_text().is_some())
}
/// One row's `CursorSense::click_or_drag()` handler, for every row,
/// routes its raw pointer data through here rather than calling
/// `begin`/`extend` directly -- this is the single place that decides
/// whether the gesture pans `list` or extends a selection, so the
/// decision is made once per gesture rather than independently by
/// whichever row happens to be under the finger this frame (see
/// `DragArbiter`'s own doc for why one shared instance, not one per
/// row, is what makes that consistent as a drag crosses row
/// boundaries).
///
/// `pos_row`/`size` are row-local, as `begin`/`extend` want;
/// `pos_window` is in window space, since a pan's delta has to stay
/// meaningful even when this frame's event landed on a different row
/// than the last one.
#[allow(clippy::too_many_arguments)]
pub fn drag(
&mut self,
ui: &mut impl UiRsc,
list: WeakWidget<List>,
key: RowKey,
pos_row: Vec2,
size: Vec2,
pos_window: Vec2,
sense: CursorSense,
now: Instant,
) {
let outcome = match sense {
CursorSense::PressStart(_) => {
let already_selected = self.has_selection(ui);
self.arbiter.press_start(pos_window, now, already_selected);
self.arbiter.update(pos_window, now)
}
CursorSense::PressEnd(_) => {
self.arbiter.release();
return;
}
_ => self.arbiter.update(pos_window, now),
};
match outcome {
DragOutcome::Undecided => {}
DragOutcome::Pan(dy) => list(ui).scroll(-dy),
DragOutcome::SelectStart => self.begin(ui, key, pos_row, size),
DragOutcome::SelectExtend => self.extend(ui, key, pos_row, size),
}
}
/// The concatenated selected text, in row order, `None` if nothing is
/// selected -- what a copy command reads. Joins with a blank line
/// between rows, matching how the transcript itself separates them.
pub fn selected_text(&self, ui: &mut impl UiRsc) -> Option<String> {
let mut parts = Vec::new();
for w in self.rows.values() {
if let Some(text) = w.edit(ui).text.selected_text() {
parts.push(text);
}
}
if parts.is_empty() {
None
} else {
Some(parts.join("\n\n"))
}
}
}
#[cfg(test)]
mod tests {
use super::*;
// Pure range-membership logic, independent of any widget/render
// machinery (the same reasoning `begin`/`extend` apply per-row) --
// exercised directly so the "which rows fall between anchor and
// current" arithmetic has a test that needs no `UiRenderState`.
fn in_range(anchor: RowKey, current: RowKey, keys: &[RowKey]) -> Vec<RowKey> {
let (lo, hi) = if anchor < current {
(anchor, current)
} else {
(current, anchor)
};
keys.iter()
.copied()
.filter(|k| *k >= lo && *k <= hi)
.collect()
}
#[test]
fn selection_spans_forward_across_rows() {
let keys = [1, 2, 3, 4, 5];
assert_eq!(in_range(2, 4, &keys), vec![2, 3, 4]);
}
#[test]
fn selection_spans_backward_across_rows() {
let keys = [1, 2, 3, 4, 5];
assert_eq!(in_range(4, 2, &keys), vec![2, 3, 4]);
}
#[test]
fn selection_within_one_row_is_just_that_row() {
let keys = [1, 2, 3];
assert_eq!(in_range(2, 2, &keys), vec![2]);
}
struct TestRsc {
ui: UiData,
}
impl UiRsc for TestRsc {
fn ui(&self) -> &UiData {
&self.ui
}
fn ui_mut(&mut self) -> &mut UiData {
&mut self.ui
}
}
#[test]
fn unregister_forgets_the_row_and_clears_a_matching_anchor() {
let mut rsc = TestRsc {
ui: UiData::default(),
};
let field = rsc
.ui
.widgets
.add_strong(TextEdit::new(
TextView::new(TextBuffer::new_empty(), TextAttrs::default(), None),
EditMode::MultiLine,
))
.weak();
let mut sel = Selection::new();
sel.register(5, field);
sel.anchor = Some((5, Vec2::ZERO));
assert_eq!(sel.rows.len(), 1);
sel.unregister(5);
assert!(sel.rows.is_empty());
assert!(sel.anchor.is_none());
}
}
+7
View File
@@ -0,0 +1,7 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "xtask"
version = "0.1.0"
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "xtask"
version = "0.1.0"
edition = "2024"
# E5 (RUST.md): packages app/shellApp into a signed, installable APK without
# Gradle driving the assembly (cargo ndk -> javac -> d8 -> aapt2 -> zipalign
# -> apksigner). No dependencies beyond the standard library: every step
# below is "run this SDK tool with these arguments and check its exit
# status," which needs nothing a crate would help with, and every tool
# invoked is one this project already requires (the NDK, the SDK
# build-tools, the JDK, `cargo ndk`) -- see AGENTS.md's "new dependencies
# need a reason."
[[bin]]
name = "xtask"
path = "src/main.rs"
+484
View File
@@ -0,0 +1,484 @@
//! The pipeline itself: `cargo ndk` -> `javac`/`d8` -> `aapt2` ->
//! `zipalign` -> `apksigner`, with no Gradle driving *this* file's steps.
//!
//! **One disclosed exception**, recorded here rather than left to be
//! rediscovered: step 3 below still runs `./gradlew
//! :shellApp:printRuntimeClasspathJars` once, because `app/shellApp`
//! depends on the `:link` submodule (Kotlin: `ServerStore`/`ServerSettings`,
//! the Keystore-sealed enrollment, RUST.md's E3 entry explains why that
//! code is reused rather than re-derived in Rust) and on
//! `androidx.core:core-ktx` (used at runtime through JNI by
//! `android-shell`'s `notify.rs`, for `NotificationCompat` and friends).
//! Both are ordinary Maven/AAR dependency graphs, and reimplementing a
//! dependency resolver to avoid one Gradle invocation was not a good trade
//! against "smallest honest route" (RUST.md's E5 box) -- especially since
//! that one call also compiles `:link`'s Kotlin as a side effect, using
//! Gradle's own embedded Kotlin compiler. This machine has no standalone
//! `kotlinc` (checked: not on PATH, not under any SDK), so that side
//! effect is what answers E3's open question about `kotlinc` -- see
//! RUST.md's E5 entry for the full account. Nothing past this one call
//! touches Gradle: `javac`, `d8`, `aapt2`, `zipalign` and `apksigner` are
//! invoked directly, and the jars this call resolves are consumed as
//! plain binary inputs to `d8`, exactly like any other pre-built `.jar`.
use std::path::{Path, PathBuf};
use std::process::Command;
use crate::keystore::{self, Signer};
use crate::sdk::{self, Sdk};
use crate::{Fail, Variant};
const APPLICATION_ID: &str = "com.example.aiapp.shell";
pub fn build(variant: Variant, abis: &[String]) -> Result<PathBuf, Fail> {
let repo_root = repo_root()?;
let app_dir = repo_root.join("app");
let shell_app_dir = app_dir.join("shellApp");
let android_shell_dir = repo_root.join("android-shell");
let sdk = sdk::find()?;
sdk::require_ndk_installed(&sdk.root)?;
require_cargo_ndk()?;
let out_dir = repo_root.join("target").join("xtask").join("apk");
std::fs::create_dir_all(&out_dir).map_err(|e| {
Fail::new(
"could not create the xtask output directory",
&e.to_string(),
"check permissions under target/",
)
})?;
println!("==> Building android-shell for {}", abis.join(", "));
build_native_libs(&android_shell_dir, &shell_app_dir, &sdk, abis)?;
println!("==> Resolving the runtime classpath (one Gradle call -- see apk.rs's module doc)");
let classpath_jars = runtime_classpath_jars(&app_dir, &sdk)?;
println!("==> Compiling the Java stub classes");
let ca_pem = pinned_ca_pem()?;
let classes_jar = compile_java(&out_dir, &shell_app_dir, &sdk, &ca_pem)?;
println!("==> Dexing");
let dex_dir = out_dir.join("dex");
dex(&sdk, &classes_jar, &classpath_jars, &dex_dir)?;
println!("==> Linking resources with aapt2");
let base_apk = out_dir.join("base.apk");
aapt2_link(&sdk, &shell_app_dir, &base_apk)?;
println!("==> Merging dex and native libraries");
let merged_apk = out_dir.join("merged.apk");
merge(&base_apk, &dex_dir, &shell_app_dir, abis, &merged_apk)?;
println!(
"==> Aligning and signing ({})",
match variant {
Variant::Release => "release key",
Variant::Debug => "debug key",
}
);
let signer = match variant {
Variant::Release => keystore::release_signer()?,
Variant::Debug => keystore::debug_signer()?,
};
let variant_name = match variant {
Variant::Release => "release",
Variant::Debug => "debug",
};
let signed_apk = out_dir.join(format!("ai-app-shell-{variant_name}.apk"));
align_and_sign(&sdk, &merged_apk, &signed_apk, &signer)?;
// Copied into a Gradle-shaped path (`build/outputs/apk/<mode>/*.apk`
// under this xtask's own directory) as the final step, purely so Dev
// Updater's fixed-pattern APK discovery (`discover.rs`'s
// `APK_PATTERNS`, which has no per-component path override) finds it
// without needing a change on that side -- `.dev-updater.ron`'s
// `shell` component points its `cwd` here. The working files above
// stay under `target/xtask/apk/`, an ordinary build-cache location.
let published_dir = repo_root.join("xtask/build/outputs/apk").join(variant_name);
std::fs::create_dir_all(&published_dir).map_err(|e| {
Fail::new(
"could not create the published APK directory",
&e.to_string(),
"check permissions under xtask/build",
)
})?;
let published_apk = published_dir.join(format!("ai-app-shell-{variant_name}.apk"));
std::fs::copy(&signed_apk, &published_apk).map_err(|e| {
Fail::new(
"could not publish the signed APK",
&e.to_string(),
"check permissions under xtask/build",
)
})?;
Ok(published_apk)
}
fn repo_root() -> Result<PathBuf, Fail> {
// xtask's own Cargo.toml is at <repo_root>/xtask/Cargo.toml.
let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
manifest_dir.parent().map(Path::to_path_buf).ok_or_else(|| {
Fail::new(
"could not find the repo root",
"CARGO_MANIFEST_DIR has no parent",
"run through cargo, not by hand",
)
})
}
fn require_cargo_ndk() -> Result<(), Fail> {
run_checked(
Command::new("cargo").args(["ndk", "--version"]),
"cargo-ndk is not installed",
"cargo install cargo-ndk",
)
.map(|_| ())
}
/// `cargo ndk`'s `-t` target name for each ABI, and `-P 26` -- the API
/// level every other cross-compile in this repo uses (RUST.md: E0, E1, E3,
/// I2), kept consistent here rather than picked fresh.
fn build_native_libs(
crate_dir: &Path,
shell_app_dir: &Path,
sdk: &Sdk,
abis: &[String],
) -> Result<(), Fail> {
let jni_libs = shell_app_dir.join("src/main/jniLibs");
let mut cmd = Command::new("cargo");
cmd.current_dir(crate_dir);
cmd.arg("ndk");
for abi in abis {
cmd.args(["-t", abi]);
}
cmd.args(["-P", "26", "-o"]).arg(&jni_libs);
// Always the release profile for the native library, independent of
// the APK's signing variant -- a debug build's Vulkan object-labelling
// segfaults this emulator's driver (RUST.md's E1 entry), and there is
// no reason for this crate's debug build to be bigger or slower for a
// signing choice that has nothing to do with it.
cmd.args(["build", "--release", "-p", "android-shell"]);
cmd.env("ANDROID_HOME", &sdk.root);
cmd.env("ANDROID_SDK_ROOT", &sdk.root);
run_checked(
&mut cmd,
"cargo ndk build failed",
"see the compiler output above",
)
.map(|_| ())
}
fn runtime_classpath_jars(app_dir: &Path, sdk: &Sdk) -> Result<Vec<PathBuf>, Fail> {
let mut cmd = Command::new(app_dir.join("gradlew"));
cmd.current_dir(app_dir);
cmd.args(["--console=plain", ":shellApp:printRuntimeClasspathJars"]);
cmd.env("ANDROID_HOME", &sdk.root);
cmd.env("ANDROID_SDK_ROOT", &sdk.root);
run_checked(
&mut cmd,
"resolving app/shellApp's dependencies with Gradle failed",
"see the Gradle output above",
)?;
let list_file = app_dir.join("shellApp/build/xtask/runtime-classpath.txt");
let contents = std::fs::read_to_string(&list_file).map_err(|e| {
Fail::new(
"printRuntimeClasspathJars did not produce its output file",
&format!("{}: {e}", list_file.display()),
"check app/shellApp/build.gradle.kts's printRuntimeClasspathJars task",
)
})?;
Ok(contents
.lines()
.filter(|l| !l.is_empty())
.map(PathBuf::from)
.collect())
}
/// The CA this build pins, found the same way `build-apk.sh` and
/// `androidApp`/`shellApp`'s Gradle `generatePinnedCa` tasks do:
/// `$AI_APP_CA`, else `$XDG_CONFIG_HOME/ai-app/certs/ca.pem`.
fn pinned_ca_pem() -> Result<String, Fail> {
let path = std::env::var_os("AI_APP_CA")
.map(PathBuf::from)
.unwrap_or_else(|| {
let config_home = std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.unwrap_or_else(|| {
PathBuf::from(std::env::var_os("HOME").unwrap()).join(".config")
});
config_home.join("ai-app").join("certs").join("ca.pem")
});
let pem = std::fs::read_to_string(&path).map_err(|e| {
Fail::new(
&format!("no CA certificate at {}", path.display()),
&e.to_string(),
"start ai-server (or app/ui-sandbox.sh) once on this machine first -- it generates the CA this build pins",
)
})?;
let pem = pem.trim().to_string();
if !pem.starts_with("-----BEGIN CERTIFICATE-----") {
return Err(Fail::new(
&format!("{} is not a PEM certificate", path.display()),
"missing the BEGIN CERTIFICATE header",
"point AI_APP_CA at a valid one",
));
}
Ok(pem)
}
fn compile_java(
out_dir: &Path,
shell_app_dir: &Path,
sdk: &Sdk,
ca_pem: &str,
) -> Result<PathBuf, Fail> {
let gen_dir = out_dir.join("generated-java");
let package_dir = gen_dir.join("com/example/aiapp/shell");
std::fs::create_dir_all(&package_dir).map_err(|e| {
Fail::new(
"could not create the generated-sources directory",
&e.to_string(),
"check permissions under target/",
)
})?;
// Same shape as shellApp's Gradle `generatePinnedCa` task: the text
// block must start immediately after the opening `"""`, or
// CertificateFactory stops recognising the "-----BEGIN" preamble (a
// real bug this project hit once -- see AGENTS.md's "Things that have
// bitten").
let pinned_ca_java = format!(
"package com.example.aiapp.shell;\n\npublic final class PinnedCa {{\n private PinnedCa() {{}}\n public static final String PINNED_CA_PEM = \"\"\"\n{ca_pem}\"\"\";\n}}\n"
);
std::fs::write(package_dir.join("PinnedCa.java"), pinned_ca_java).map_err(|e| {
Fail::new(
"could not write PinnedCa.java",
&e.to_string(),
"check permissions under target/",
)
})?;
let classes_dir = out_dir.join("classes");
std::fs::create_dir_all(&classes_dir).map_err(|e| {
Fail::new(
"could not create the classes directory",
&e.to_string(),
"check permissions under target/",
)
})?;
let java_dir = shell_app_dir.join("src/main/java/com/example/aiapp/shell");
let mut cmd = Command::new("javac");
cmd.args(["-cp"]).arg(&sdk.android_jar);
cmd.args(["-d"]).arg(&classes_dir);
cmd.arg(java_dir.join("MainActivity.java"));
cmd.arg(java_dir.join("NotificationService.java"));
cmd.arg(package_dir.join("PinnedCa.java"));
run_checked(&mut cmd, "javac failed", "see the compiler output above")?;
let classes_jar = out_dir.join("classes.jar");
let mut cmd = Command::new("jar");
cmd.current_dir(&classes_dir);
cmd.args(["cf"])
.arg(&classes_jar)
.args(["-C", "."])
.arg(".");
run_checked(
&mut cmd,
"jar failed to package the compiled classes",
"see the output above",
)?;
Ok(classes_jar)
}
fn dex(
sdk: &Sdk,
classes_jar: &Path,
classpath_jars: &[PathBuf],
dex_dir: &Path,
) -> Result<(), Fail> {
std::fs::create_dir_all(dex_dir).map_err(|e| {
Fail::new(
"could not create the dex output directory",
&e.to_string(),
"check permissions under target/",
)
})?;
let mut cmd = Command::new(sdk.tool("d8"));
cmd.args(["--release", "--min-api"])
.arg(sdk::MIN_SDK.to_string());
cmd.arg("--lib").arg(&sdk.android_jar);
cmd.arg("--output").arg(dex_dir);
cmd.arg(classes_jar);
cmd.args(classpath_jars);
run_checked(&mut cmd, "d8 failed", "see the compiler output above").map(|_| ())
}
fn aapt2_link(sdk: &Sdk, shell_app_dir: &Path, base_apk: &Path) -> Result<(), Fail> {
let manifest_src = shell_app_dir.join("src/main/AndroidManifest.xml");
let manifest_text = std::fs::read_to_string(&manifest_src).map_err(|e| {
Fail::new(
"could not read the manifest",
&format!("{}: {e}", manifest_src.display()),
"check app/shellApp/src/main/AndroidManifest.xml",
)
})?;
// The checked-in manifest has no `package` attribute -- Gradle injects
// it from `android.namespace` during its own manifest merge, which
// this pipeline does not run. aapt2 needs it to know what package to
// generate resources under.
if manifest_text.contains("package=") {
return Err(Fail::new(
"app/shellApp's manifest already has a package attribute",
"aapt2_link() assumes it doesn't and injects one",
"update aapt2_link() in xtask/src/apk.rs to stop injecting a second one",
));
}
let merged_manifest = manifest_text.replacen(
"<manifest ",
&format!("<manifest package=\"{APPLICATION_ID}\" "),
1,
);
let merged_manifest_path = base_apk.with_file_name("AndroidManifest.merged.xml");
std::fs::write(&merged_manifest_path, merged_manifest).map_err(|e| {
Fail::new(
"could not write the merged manifest",
&e.to_string(),
"check permissions under target/",
)
})?;
let mut cmd = Command::new(sdk.tool("aapt2"));
cmd.args(["link", "-o"]).arg(base_apk);
cmd.args(["--manifest"]).arg(&merged_manifest_path);
cmd.arg("-I").arg(&sdk.android_jar);
cmd.args(["--min-sdk-version", &sdk::MIN_SDK.to_string()]);
cmd.args(["--target-sdk-version", &sdk::COMPILE_SDK.to_string()]);
cmd.args(["--version-code", "1", "--version-name", "1.0"]);
run_checked(&mut cmd, "aapt2 link failed", "see the output above").map(|_| ())
}
fn merge(
base_apk: &Path,
dex_dir: &Path,
shell_app_dir: &Path,
abis: &[String],
merged_apk: &Path,
) -> Result<(), Fail> {
std::fs::copy(base_apk, merged_apk).map_err(|e| {
Fail::new(
"could not copy the base APK",
&e.to_string(),
"check permissions under target/",
)
})?;
let mut cmd = Command::new("jar");
cmd.current_dir(dex_dir);
cmd.args(["uf"])
.arg(std::path::absolute(merged_apk).unwrap_or_else(|_| merged_apk.to_path_buf()));
cmd.args(["classes.dex"]);
run_checked(
&mut cmd,
"jar failed to add classes.dex to the APK",
"see the output above",
)?;
// Android's zip layout wants "lib/<abi>/*.so" at the archive root, but
// cargo ndk's `-o` wrote "jniLibs/<abi>/*.so" (matching the Gradle
// source-set layout it was pointed at) -- so this stages a "lib/"
// directory rather than trying to rename inside the zip.
let stage = merged_apk.with_file_name("lib-stage");
if stage.exists() {
std::fs::remove_dir_all(&stage).ok();
}
for abi in abis {
let so_name = "libandroid_shell.so";
let src = shell_app_dir
.join("src/main/jniLibs")
.join(abi)
.join(so_name);
if !src.is_file() {
return Err(Fail::new(
&format!("no native library built for {abi}"),
&format!("expected {}", src.display()),
"check cargo ndk's output above for that ABI",
));
}
let dest_dir = stage.join("lib").join(abi);
std::fs::create_dir_all(&dest_dir).map_err(|e| {
Fail::new(
"could not stage the native library",
&e.to_string(),
"check permissions under target/",
)
})?;
std::fs::copy(&src, dest_dir.join(so_name)).map_err(|e| {
Fail::new(
"could not stage the native library",
&e.to_string(),
"check permissions under target/",
)
})?;
}
let mut cmd = Command::new("jar");
cmd.current_dir(&stage);
cmd.args(["uf"])
.arg(std::path::absolute(merged_apk).unwrap_or_else(|_| merged_apk.to_path_buf()));
cmd.arg("lib");
run_checked(
&mut cmd,
"jar failed to add the native libraries to the APK",
"see the output above",
)
.map(|_| ())
}
fn align_and_sign(
sdk: &Sdk,
merged_apk: &Path,
final_apk: &Path,
signer: &Signer,
) -> Result<(), Fail> {
let aligned_apk = merged_apk.with_file_name("aligned.apk");
let mut cmd = Command::new(sdk.tool("zipalign"));
cmd.args(["-f", "-p", "4"])
.arg(merged_apk)
.arg(&aligned_apk);
run_checked(&mut cmd, "zipalign failed", "see the output above")?;
let mut cmd = Command::new(sdk.tool("apksigner"));
cmd.args(["sign", "--ks"]).arg(&signer.keystore);
cmd.arg("--ks-pass")
.arg(format!("pass:{}", signer.password));
cmd.arg("--ks-key-alias").arg(&signer.alias);
cmd.arg("--out").arg(final_apk);
cmd.arg(&aligned_apk);
run_checked(
&mut cmd,
"apksigner failed to sign the APK",
"see the output above",
)
.map(|_| ())
}
fn run_checked(cmd: &mut Command, what: &str, fix: &str) -> Result<(), Fail> {
let status = cmd.status().map_err(|e| {
Fail::new(
what,
&format!("could not run {:?}: {e}", cmd.get_program()),
fix,
)
})?;
if status.success() {
Ok(())
} else {
Err(Fail::new(
what,
&format!("{:?} exited with {status}", cmd.get_program()),
fix,
))
}
}
+254
View File
@@ -0,0 +1,254 @@
//! The signing key. Mirrors `app/build-apk.sh`'s exact logic for the
//! release key -- same env vars, same path, same generation recipe -- so
//! the two tools sign with the *same* key and their outputs can
//! `adb install -r` over each other. That is the whole point of E5's pass
//! condition: the key has to be identical, not merely present.
use std::path::PathBuf;
use std::process::Command;
use crate::Fail;
pub struct Signer {
pub keystore: PathBuf,
pub password: String,
pub alias: String,
}
/// The release key at `$AI_APP_KEYSTORE` or
/// `$XDG_CONFIG_HOME/ai-app/release.jks` (`~/.config/ai-app/release.jks` by
/// default) -- generated with `keytool` if it doesn't exist yet, exactly as
/// `build-apk.sh` does, so either tool can run first on a fresh machine.
pub fn release_signer() -> Result<Signer, Fail> {
let keystore = std::env::var_os("AI_APP_KEYSTORE")
.map(PathBuf::from)
.unwrap_or_else(|| {
let config_home = std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.unwrap_or_else(|| {
PathBuf::from(std::env::var_os("HOME").unwrap()).join(".config")
});
config_home.join("ai-app").join("release.jks")
});
let alias = "ai-app".to_string();
let password_file = keystore.with_extension("jks.password");
if keystore.is_file() {
let password = std::fs::read_to_string(&password_file)
.map_err(|e| {
Fail::new(
"release key exists but its password file is unreadable",
&format!("{}: {e}", password_file.display()),
"restore the password file, or delete both and let this regenerate them",
)
})?
.trim()
.to_string();
return Ok(Signer {
keystore,
password,
alias,
});
}
let keytool = which_keytool()?;
if let Some(parent) = keystore.parent() {
std::fs::create_dir_all(parent).map_err(|e| {
Fail::new(
"could not create the keystore's directory",
&format!("{}: {e}", parent.display()),
"check permissions on that path",
)
})?;
}
let password = random_password();
write_owner_only(&password_file, format!("{password}\n").as_bytes())?;
let status = Command::new(&keytool)
.args(["-genkeypair", "-keystore"])
.arg(&keystore)
.args([
"-alias",
&alias,
"-keyalg",
"RSA",
"-keysize",
"2048",
"-validity",
"10000",
])
.args(["-storepass", &password, "-keypass", &password])
.args(["-dname", "CN=ai-app"])
.status()
.map_err(|e| {
Fail::new(
"failed to run keytool",
&format!("{}: {e}", keytool.display()),
"set JAVA_HOME to the JDK Gradle uses",
)
})?;
if !status.success() {
return Err(Fail::new(
"keytool exited with an error while generating the release key",
&format!("status: {status}"),
"check the keytool output above",
));
}
// Owner-only, matching build-apk.sh -- this key is what the phone
// recognises the app by, so it never goes in the repo and it stays
// unreadable to anything else on this machine.
set_owner_only(&keystore)?;
Ok(Signer {
keystore,
password,
alias,
})
}
/// The conventional Android debug key (`~/.android/debug.keystore`,
/// well-known password `android`, alias `androiddebugkey`) -- generated on
/// first use exactly the way Android Studio and Gradle's own debug signing
/// config do, so a `--debug` build here needs no setup and never touches
/// the real release key.
pub fn debug_signer() -> Result<Signer, Fail> {
let home = PathBuf::from(std::env::var_os("HOME").ok_or_else(|| {
Fail::new(
"no $HOME set",
"the debug keystore lives under ~/.android",
"set $HOME",
)
})?);
let keystore = home.join(".android").join("debug.keystore");
let alias = "androiddebugkey".to_string();
let password = "android".to_string();
if !keystore.is_file() {
let keytool = which_keytool()?;
std::fs::create_dir_all(keystore.parent().unwrap()).map_err(|e| {
Fail::new(
"could not create ~/.android",
&format!("{e}"),
"check permissions on your home directory",
)
})?;
let status = Command::new(&keytool)
.args(["-genkeypair", "-keystore"])
.arg(&keystore)
.args([
"-alias",
&alias,
"-keyalg",
"RSA",
"-keysize",
"2048",
"-validity",
"10000",
])
.args(["-storepass", &password, "-keypass", &password])
.args(["-dname", "CN=Android Debug,O=Android,C=US"])
.status()
.map_err(|e| {
Fail::new(
"failed to run keytool",
&format!("{}: {e}", keytool.display()),
"set JAVA_HOME to the JDK Gradle uses",
)
})?;
if !status.success() {
return Err(Fail::new(
"keytool exited with an error while generating the debug key",
&format!("status: {status}"),
"check the keytool output above",
));
}
}
Ok(Signer {
keystore,
password,
alias,
})
}
fn which_keytool() -> Result<PathBuf, Fail> {
if let Some(java_home) = std::env::var_os("JAVA_HOME") {
let candidate = PathBuf::from(java_home).join("bin").join("keytool");
if candidate.is_file() {
return Ok(candidate);
}
}
if Command::new("keytool").arg("-help").output().is_ok() {
return Ok(PathBuf::from("keytool"));
}
Err(Fail::new(
"no keytool available to generate the release key",
"checked $JAVA_HOME/bin/keytool and keytool on PATH",
"set JAVA_HOME to the JDK Gradle uses, or set AI_APP_KEYSTORE to an existing key",
))
}
fn random_password() -> String {
// No dependency on `rand`: /dev/urandom is what build-apk.sh's `head -c
// 24 /dev/urandom | base64` reads too, so this reproduces exactly the
// same recipe without shelling out to head/base64/tr for it.
let mut bytes = [0u8; 24];
std::fs::File::open("/dev/urandom")
.and_then(|mut f| std::io::Read::read_exact(&mut f, &mut bytes))
.expect("/dev/urandom must be readable to generate a signing key password");
base64_no_padding(&bytes)
}
fn base64_no_padding(bytes: &[u8]) -> String {
const ALPHABET: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
let mut out = String::new();
for chunk in bytes.chunks(3) {
let b0 = chunk[0] as u32;
let b1 = *chunk.get(1).unwrap_or(&0) as u32;
let b2 = *chunk.get(2).unwrap_or(&0) as u32;
let n = (b0 << 16) | (b1 << 8) | b2;
out.push(ALPHABET[(n >> 18 & 0x3f) as usize] as char);
out.push(ALPHABET[(n >> 12 & 0x3f) as usize] as char);
if chunk.len() > 1 {
out.push(ALPHABET[(n >> 6 & 0x3f) as usize] as char);
}
if chunk.len() > 2 {
out.push(ALPHABET[(n & 0x3f) as usize] as char);
}
}
// build-apk.sh strips '/', '+' and '=' from its password (tr -d
// '/+='), so the value never needs quoting when it is passed as a
// command-line argument later.
out.retain(|c| c != '/' && c != '+' && c != '=');
out
}
#[cfg(unix)]
fn write_owner_only(path: &std::path::Path, contents: &[u8]) -> Result<(), Fail> {
use std::os::unix::fs::OpenOptionsExt;
std::fs::OpenOptions::new()
.write(true)
.create(true)
.truncate(true)
.mode(0o600)
.open(path)
.and_then(|mut f| std::io::Write::write_all(&mut f, contents))
.map_err(|e| {
Fail::new(
"could not write the keystore password file",
&format!("{}: {e}", path.display()),
"check permissions on that directory",
)
})
}
#[cfg(unix)]
fn set_owner_only(path: &std::path::Path) -> Result<(), Fail> {
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)).map_err(|e| {
Fail::new(
"could not restrict the keystore's permissions",
&format!("{}: {e}", path.display()),
"chmod 600 it by hand",
)
})
}
+115
View File
@@ -0,0 +1,115 @@
//! `cargo xtask apk` -- E5 (RUST.md): packages `app/shellApp` into a
//! signed, installable APK with no Gradle in the packaging step itself.
//! `cargo ndk` cross-compiles `android-shell`; `javac`/`d8` turn its two
//! Java stub classes (plus the generated pinned-CA constant) into dex;
//! `aapt2` compiles the manifest into `resources.arsc`; the dex and native
//! libraries are merged into that base APK with `jar`; `zipalign` and
//! `apksigner` finish it. See `apk.rs`'s module doc for what "no Gradle in
//! the packaging step" does and does not cover -- one disclosed exception.
//!
//! Usage: `cargo xtask apk [--release|--debug] [--abi ABI]...`
mod apk;
mod keystore;
mod sdk;
use std::fmt;
use std::process::ExitCode;
/// A failure a person acts on: what went wrong, what this process actually
/// saw, and the next thing to try. Matches CODE_RULES's "a failure message
/// names the thing, the cause, and the fix."
pub struct Fail {
what: String,
cause: String,
fix: String,
}
impl Fail {
pub fn new(what: &str, cause: &str, fix: &str) -> Self {
Fail {
what: what.to_string(),
cause: cause.to_string(),
fix: fix.to_string(),
}
}
}
impl fmt::Display for Fail {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"{}\n cause: {}\n fix: {}",
self.what, self.cause, self.fix
)
}
}
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum Variant {
/// Signed with `~/.config/ai-app/release.jks`, the same key
/// `build-apk.sh` uses for `androidApp` -- what E5's pass condition
/// needs, since installing over an existing app requires a matching
/// signature.
Release,
/// Signed with the standard Android debug keystore
/// (`~/.android/debug.keystore`, well-known password, generated if
/// missing the same way Gradle would), for a fast local loop that
/// doesn't touch the real signing key.
Debug,
}
fn main() -> ExitCode {
let args: Vec<String> = std::env::args().skip(1).collect();
let Some(("apk", rest)) = args.split_first().map(|(cmd, rest)| (cmd.as_str(), rest)) else {
eprintln!("usage: cargo xtask apk [release|debug] [--abi ABI]...");
return ExitCode::FAILURE;
};
let mut variant = Variant::Release;
let mut abis: Vec<String> = Vec::new();
let mut i = 0;
while i < rest.len() {
match rest[i].as_str() {
// Bare "release"/"debug" is `.dev-updater.ron`'s interface
// (`ByMode::One` appends the chosen mode as the build
// command's last argument -- the same convention
// `app/build-apk.sh`'s `${1:-release}` uses); the `--`-prefixed
// spellings are for typing this by hand.
"release" | "--release" => variant = Variant::Release,
"debug" | "--debug" => variant = Variant::Debug,
"--abi" => {
i += 1;
match rest.get(i) {
Some(abi) => abis.push(abi.clone()),
None => {
eprintln!("--abi needs a value (e.g. arm64-v8a, x86_64)");
return ExitCode::FAILURE;
}
}
}
other => {
eprintln!("unknown argument: {other}");
return ExitCode::FAILURE;
}
}
i += 1;
}
if abis.is_empty() {
// arm64-v8a for a real phone, x86_64 for this machine's emulator --
// the two ABIs every other experiment in RUST.md has actually run
// on. `--abi` overrides either way.
abis = vec!["arm64-v8a".to_string(), "x86_64".to_string()];
}
match apk::build(variant, &abis) {
Ok(path) => {
println!("==> Built {}", path.display());
ExitCode::SUCCESS
}
Err(fail) => {
eprintln!("xtask: {fail}");
ExitCode::FAILURE
}
}
}
Loaded 100 of 101 files, more files were not shown because too many files have changed in this diff. Show more