iris: a phone-shaped desktop window, driven by the same touch recordings

Layer 2 of docs/RUST.md's "Three test layers":

    ./run-headless.sh phone --phone --shot /tmp/p.png -- -p transcript-fixture

opens `transcript-fixture`'s screen -- the same fixture and the same
fold the headless tests and the Android bench use -- in a window at the
phone's own 1080x2424 and `content_scale` 2.55, and screenshots it. 15
seconds, warm. `--replay FILE` drives one of the `.touch` recordings
into it and writes `<shot>-before.png` too, so "the list moved" is two
pictures: the flick carries it back about seven turns of the fixture.

Two things this needed.

**The desktop backend now lays out in physical pixels with a density,
exactly as Android does** (`default::content_scale`, overridable with
`IRIS_SCALE`, which is how `--phone` hands it the phone's). It used to
divide winit's coordinates into a separate "logical" space, which left
`UiRenderState::resize` (physical, from `WindowEvent::Resized`) and the
window uniform (logical) disagreeing on any display whose scale factor
is not 1.0, and rasterised glyphs at one resolution to show them at
another. At 1.0 -- every display here -- the numbers are unchanged, and
the `tabs` screenshot is identical.

**`rig-input`'s `replay-touch`** puts a gesture on screen. This
machine's compositor has no pointer to move: sway runs on the headless
backend with no input devices, so `swaymsg seat - cursor press` reports
success and `swaymsg -t get_seats` shows `capabilities: 0`. wlroots 0.19
dropped `WLR_HEADLESS_INPUTS` and ydotool's uinput device would be
ignored by a compositor not reading libinput, so the virtual-pointer
protocol is what is left. It parses the *same* `TouchScript` the
harness does, so one recording drives both layers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Fable 5.1 committed 2026-09-07 12:38:19 -04:00
1 parent e430880cde
commit 232de0ec53
10 files changed
+437 -62

No files matched your search

+25 -16
View File
@@ -965,9 +965,9 @@ dependencies = [
[[package]] [[package]]
name = "dlib" name = "dlib"
version = "0.5.2" version = "0.5.3"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "330c60081dcc4c72131f8eb70510f1ac07223e5d4163db481a04a0befcffa412" checksum = "ab8ecd87370524b461f8557c119c405552c396ed91fc0a8eec68679eab26f94a"
dependencies = [ dependencies = [
"libloading", "libloading",
] ]
@@ -2875,9 +2875,9 @@ checksum = "a993555f31e5a609f617c12db6250dedcac1b0a85076912c436e6fc9b2c8e6a3"
[[package]] [[package]]
name = "quick-xml" name = "quick-xml"
version = "0.38.4" version = "0.41.0"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b66c2058c55a409d601666cffe35f04333cf1013010882cec174a7467cd4e21c" checksum = "e660451e55124f798a69a5af3f49ccfbefbd41910eefd25caf2393e1f3473ec1"
dependencies = [ dependencies = [
"memchr", "memchr",
] ]
@@ -3058,6 +3058,15 @@ version = "0.8.52"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0c6a884d2998352bb4daf0183589aec883f16a6da1f4dde84d8e2e9a5409a1ce" checksum = "0c6a884d2998352bb4daf0183589aec883f16a6da1f4dde84d8e2e9a5409a1ce"
[[package]]
name = "rig-input"
version = "0.1.0"
dependencies = [
"iris",
"wayland-client",
"wayland-protocols-wlr",
]
[[package]] [[package]]
name = "ring" name = "ring"
version = "0.17.14" version = "0.17.14"
@@ -3906,9 +3915,9 @@ dependencies = [
[[package]] [[package]]
name = "wayland-backend" name = "wayland-backend"
version = "0.3.12" version = "0.3.17"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fee64194ccd96bf648f42a65a7e589547096dfa702f7cadef84347b66ad164f9" checksum = "38a91b4eaddff87b1cd1074985e3713da4af2c49742d1b356b2c01670a67a078"
dependencies = [ dependencies = [
"cc", "cc",
"downcast-rs", "downcast-rs",
@@ -3920,9 +3929,9 @@ dependencies = [
[[package]] [[package]]
name = "wayland-client" name = "wayland-client"
version = "0.31.12" version = "0.31.15"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b8e6faa537fbb6c186cb9f1d41f2f811a4120d1b57ec61f50da451a0c5122bec" checksum = "e3c36a0f861ad76d0901f2800b46321410d9f73f2ea88aac0650d86c32688073"
dependencies = [ dependencies = [
"bitflags 2.10.0", "bitflags 2.10.0",
"rustix 1.1.3", "rustix 1.1.3",
@@ -3954,9 +3963,9 @@ dependencies = [
[[package]] [[package]]
name = "wayland-protocols" name = "wayland-protocols"
version = "0.32.10" version = "0.32.13"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "baeda9ffbcfc8cd6ddaade385eaf2393bd2115a69523c735f12242353c3df4f3" checksum = "23d0c813de3daa2ed6520af85a3bd49b0e722a3078506899aa9686fea58dc4b6"
dependencies = [ dependencies = [
"bitflags 2.10.0", "bitflags 2.10.0",
"wayland-backend", "wayland-backend",
@@ -3979,9 +3988,9 @@ dependencies = [
[[package]] [[package]]
name = "wayland-protocols-wlr" name = "wayland-protocols-wlr"
version = "0.3.10" version = "0.3.12"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e9597cdf02cf0c34cd5823786dce6b5ae8598f05c2daf5621b6e178d4f7345f3" checksum = "eb04e52f7836d7c7976c78ca0250d61e33873c34156a2a1fc9474828ec268234"
dependencies = [ dependencies = [
"bitflags 2.10.0", "bitflags 2.10.0",
"wayland-backend", "wayland-backend",
@@ -3992,9 +4001,9 @@ dependencies = [
[[package]] [[package]]
name = "wayland-scanner" name = "wayland-scanner"
version = "0.31.8" version = "0.31.11"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5423e94b6a63e68e439803a3e153a9252d5ead12fd853334e2ad33997e3889e3" checksum = "338e30461b3a2b67d70eb30a6d89f8e0c93a833e07d2ae89085cd070c4a00ac0"
dependencies = [ dependencies = [
"proc-macro2", "proc-macro2",
"quick-xml", "quick-xml",
@@ -4003,9 +4012,9 @@ dependencies = [
[[package]] [[package]]
name = "wayland-sys" name = "wayland-sys"
version = "0.31.8" version = "0.31.11"
source = "registry+https://github.com/rust-lang/crates.io-index" source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1e6dbfc3ac5ef974c92a2235805cc0114033018ae1290a72e474aa8b28cbbdfd" checksum = "d8eab23fefc9e41f8e841df4a9c707e8a8c4ed26e944ef69297184de2785e3be"
dependencies = [ dependencies = [
"dlib", "dlib",
"log", "log",
+9 -1
View File
@@ -89,7 +89,15 @@ name = "message_list"
harness = false harness = false
[workspace] [workspace]
members = ["core", "macro", "tabs-ui", "transcript-ui", "transcript-fixture", "desktop-app"] members = [
"core",
"macro",
"tabs-ui",
"transcript-ui",
"transcript-fixture",
"rig-input",
"desktop-app",
]
# android-app pulls in android-view, which needs the NDK sysroot to link # android-app pulls in android-view, which needs the NDK sysroot to link
# -- excluded so `cargo build --workspace --all-targets` on the host stays # -- excluded so `cargo build --workspace --all-targets` on the host stays
# buildable. Cross-compile it from its own directory (its own single-crate # buildable. Cross-compile it from its own directory (its own single-crate
+32
View File
@@ -0,0 +1,32 @@
[package]
name = "rig-input"
version.workspace = true
edition.workspace = true
# Layer 2's input half (docs/RUST.md's "Three test layers"): replays one
# of the `.touch` files the headless tests use into whatever window is
# under a Wayland compositor, so the *same recording* drives the
# assertion layer and the layer a person looks at.
#
# It exists because this machine's compositor has no pointer to move.
# `run-headless.sh` starts sway on the headless backend with no input
# devices at all (`WLR_LIBINPUT_NO_DEVICES=1`, `LIBSEAT_BACKEND=noop`),
# so `swaymsg seat - cursor press` reports success and nothing reaches
# the client -- `swaymsg -t get_seats` shows `capabilities: 0`. wlroots
# 0.19 dropped `WLR_HEADLESS_INPUTS`, and ydotool's uinput device would
# be ignored by a compositor that is not reading libinput. The
# virtual-pointer protocol is what is left, and it is a client protocol,
# so it needs no devices and no root.
# Named for what it does rather than for the crate, since the crate may
# grow a keyboard replay beside it.
[[bin]]
name = "replay-touch"
path = "src/main.rs"
[dependencies]
# `TouchScript` -- the same parser the harness uses, so a file that
# replays here and one that replays headless can never disagree.
iris = { path = ".." }
wayland-client = "0.31.15"
wayland-protocols-wlr = { version = "0.3.12", features = ["client"] }
+164
View File
@@ -0,0 +1,164 @@
//! Replays a `.touch` file into the compositor as a left-button drag --
//! see this crate's `Cargo.toml` for why it exists rather than
//! `swaymsg seat - cursor`.
//!
//! WAYLAND_DISPLAY=… replay-touch WIDTH HEIGHT FILE
//!
//! `WIDTH`/`HEIGHT` are the output's own size, because the virtual
//! pointer protocol positions absolutely against an extent rather than
//! in pixels; passing the output size makes a script's coordinates mean
//! the same pixels they mean in the headless tests.
//!
//! Replayed in real time (the sleeps between samples are the gaps in the
//! file), because winit has no timestamp on a pointer event and dates
//! each one when it arrives -- so a 20ms flick has to actually take
//! 20ms here, unlike layer 1 where the sample carries its own time.
use iris::harness::{TouchAction, TouchScript};
use std::time::Duration;
use wayland_client::protocol::wl_pointer::ButtonState;
use wayland_client::protocol::{wl_registry, wl_seat};
use wayland_client::{Connection, Dispatch, QueueHandle, delegate_noop};
use wayland_protocols_wlr::virtual_pointer::v1::client::{
zwlr_virtual_pointer_manager_v1::ZwlrVirtualPointerManagerV1,
zwlr_virtual_pointer_v1::ZwlrVirtualPointerV1,
};
/// `linux/input-event-codes.h`. The protocol takes the kernel's own
/// button code, not a wayland enum.
const BTN_LEFT: u32 = 0x110;
/// How long the pointer sits at the gesture's first position before the
/// script starts -- see the comment at the pre-step in `main`.
const SETTLE: Duration = Duration::from_millis(200);
#[derive(Default)]
struct Globals {
seat: Option<wl_seat::WlSeat>,
manager: Option<ZwlrVirtualPointerManagerV1>,
}
impl Dispatch<wl_registry::WlRegistry, ()> for Globals {
fn event(
state: &mut Self,
registry: &wl_registry::WlRegistry,
event: wl_registry::Event,
_: &(),
_: &Connection,
qh: &QueueHandle<Self>,
) {
let wl_registry::Event::Global {
name,
interface,
version,
} = event
else {
return;
};
match interface.as_str() {
"wl_seat" => {
state.seat = Some(registry.bind(name, version.min(7), qh, ()));
}
"zwlr_virtual_pointer_manager_v1" => {
state.manager = Some(registry.bind(name, version.min(2), qh, ()));
}
_ => {}
}
}
}
delegate_noop!(Globals: ignore wl_seat::WlSeat);
delegate_noop!(Globals: ZwlrVirtualPointerManagerV1);
delegate_noop!(Globals: ZwlrVirtualPointerV1);
fn main() {
let args: Vec<String> = std::env::args().skip(1).collect();
let [width, height, path] = args.as_slice() else {
eprintln!("usage: replay-touch WIDTH HEIGHT FILE");
std::process::exit(2);
};
let (width, height) = (parse(width, "WIDTH"), parse(height, "HEIGHT"));
let text = std::fs::read_to_string(path)
.unwrap_or_else(|e| fail(&format!("could not read {path}: {e}")));
let script = TouchScript::parse(&text).unwrap_or_else(|e| fail(&e));
let conn = Connection::connect_to_env().unwrap_or_else(|e| {
fail(&format!(
"no wayland display ({e}); is WAYLAND_DISPLAY set?"
))
});
let mut queue = conn.new_event_queue();
let qh = queue.handle();
let display = conn.display();
display.get_registry(&qh, ());
let mut globals = Globals::default();
queue
.roundtrip(&mut globals)
.unwrap_or_else(|e| fail(&format!("wayland roundtrip failed: {e}")));
let manager = globals.manager.as_ref().unwrap_or_else(|| {
fail(
"this compositor does not offer zwlr_virtual_pointer_manager_v1, so a pointer cannot \
be synthesised; sway and every wlroots compositor do",
)
});
let pointer = manager.create_virtual_pointer(globals.seat.as_ref(), &qh, ());
// Put the pointer where the gesture starts and let the compositor
// settle before anything is pressed. Without this the press is
// dropped: sway has just learned about this pointer, and a button
// sent in the same breath as the motion that first puts it over a
// window arrives before there is a focused surface to send it to --
// winit sees `CursorEntered`, the moves and the *release*, never the
// press, so the gesture reads as a hover and nothing scrolls. Found
// by printing winit's own events; the settle is what fixed it.
if let Some(first) = script.samples.first() {
pointer.motion_absolute(0, first.pos.x as u32, first.pos.y as u32, width, height);
pointer.frame();
conn.flush()
.unwrap_or_else(|e| fail(&format!("flush: {e}")));
std::thread::sleep(SETTLE);
}
let mut previous = 0;
for sample in &script.samples {
std::thread::sleep(Duration::from_millis(sample.t_ms - previous));
previous = sample.t_ms;
let t = sample.t_ms as u32;
pointer.motion_absolute(t, sample.pos.x as u32, sample.pos.y as u32, width, height);
// One frame per sample, so the compositor delivers them as
// separate pointer frames rather than coalescing the whole
// gesture -- the shape the file recorded is the point.
pointer.frame();
// The button goes in a frame of its own, *after* the motion has
// been committed. Sent in the same frame as the motion that
// first puts the pointer over the window, sway drops it: the
// client sees `CursorEntered` and the moves but never a
// `MouseInput { state: Pressed }`, so the whole gesture reads as
// a hover and nothing scrolls. Found exactly that way, by
// printing winit's events.
let state = match sample.action {
TouchAction::Down => Some(ButtonState::Pressed),
TouchAction::Up | TouchAction::Cancel => Some(ButtonState::Released),
TouchAction::Move => None,
};
if let Some(state) = state {
pointer.button(t, BTN_LEFT, state);
pointer.frame();
}
conn.flush()
.unwrap_or_else(|e| fail(&format!("flush: {e}")));
}
pointer.destroy();
conn.flush().ok();
}
fn parse(text: &str, what: &str) -> u32 {
text.parse()
.unwrap_or_else(|_| fail(&format!("{what} is not a whole number: {text:?}")))
}
fn fail(message: &str) -> ! {
eprintln!("replay-touch: {message}");
std::process::exit(1);
}
+65 -1
View File
@@ -3,6 +3,25 @@
# #
# ./run-headless.sh tabs [-- cargo args] # ./run-headless.sh tabs [-- cargo args]
# ./run-headless.sh tabs --shot /tmp/tabs.png --seconds 4 # ./run-headless.sh tabs --shot /tmp/tabs.png --seconds 4
# ./run-headless.sh phone --phone --shot /tmp/p.png -- -p transcript-fixture
# ./run-headless.sh phone --phone --replay transcript-fixture/touch/flick-120hz.touch \
# --shot /tmp/p.png -- -p transcript-fixture
#
# `--phone` is layer 2 of docs/RUST.md's "Three test layers": the output
# and the window take Iris's phone's own size and density (1080x2424 at
# `content_scale` 2.55, from docs/bench/iris-phone-v2-2026-09-06.md,
# carried in `transcript_fixture::PHONE_*`), and `IRIS_SCALE` hands that
# density to iris the way `DisplayMetrics.density` does on Android
# (`iris::default::content_scale`). So a screenshot from here and one
# from the phone are the same layout at the same density, and what
# differs is only the renderer. Without it the output stays desktop-
# shaped, which is what every other example wants.
#
# `--replay FILE` drives one of the `.touch` recordings the headless
# tests use (`iris/transcript-fixture/touch/`) into the window through
# `rig-input`'s `replay-touch` -- one recording, both layers. With
# `--shot` it also writes `<shot>-before.png` from just before the
# gesture, since "the list moved" is a claim about two pictures.
# #
# `--bin` runs a real crate binary instead of an example (E4's # `--bin` runs a real crate binary instead of an example (E4's
# `desktop-app`, which is a window a person runs, not a demo) -- # `desktop-app`, which is a window a person runs, not a demo) --
@@ -29,19 +48,31 @@ here=$(cd "$(dirname "$0")" && pwd)
run="${XDG_RUNTIME_DIR:-/tmp}/iris-headless" run="${XDG_RUNTIME_DIR:-/tmp}/iris-headless"
seconds=3 seconds=3
shot="" shot=""
replay=""
example="" example=""
kind=example kind=example
phone=no
# The phone Iris runs the bench on. Not typed from memory: these are
# `transcript_fixture::PHONE_WIDTH`/`PHONE_HEIGHT`/`PHONE_SCALE`, which
# in turn come from her own reports -- keep the three in step.
PHONE_MODE=1080x2424@120Hz
PHONE_SCALE=2.55
DESKTOP_MODE=1920x1200@60Hz
while [ $# -gt 0 ]; do while [ $# -gt 0 ]; do
case "$1" in case "$1" in
--shot) shot=$2; shift 2 ;; --shot) shot=$2; shift 2 ;;
--seconds) seconds=$2; shift 2 ;; --seconds) seconds=$2; shift 2 ;;
--bin) kind=bin; shift ;; --bin) kind=bin; shift ;;
--phone) phone=yes; shift ;;
--replay) replay=$2; shift 2 ;;
--) shift; break ;; --) shift; break ;;
*) example=$1; shift ;; *) example=$1; shift ;;
esac esac
done done
[ -n "$example" ] || { echo "usage: $0 NAME [--bin] [--shot PNG] [--seconds N] [-- cargo args]" >&2; exit 2; } [ -n "$example" ] || { echo "usage: $0 NAME [--bin] [--phone] [--replay TOUCH] [--shot PNG] [--seconds N] [-- cargo args]" >&2; exit 2; }
[ -z "$replay" ] || [ -f "$replay" ] || { echo "run-headless: no touch script at $replay" >&2; exit 2; }
mkdir -p "$run" mkdir -p "$run"
export SWAYSOCK="$run/sway.sock" export SWAYSOCK="$run/sway.sock"
@@ -78,6 +109,27 @@ export WAYLAND_DISPLAY
echo "run-headless: $WAYLAND_DISPLAY (sway $(swaymsg -t get_version --raw | sed -n 's/.*"human_readable":"\([^"]*\)".*/\1/p'))" >&2 echo "run-headless: $WAYLAND_DISPLAY (sway $(swaymsg -t get_version --raw | sed -n 's/.*"human_readable":"\([^"]*\)".*/\1/p'))" >&2
# Set every run rather than only when it changes: this compositor is
# reused across runs (see the socket comment above), so a desktop-shaped
# run after a phone-shaped one would otherwise inherit the phone's output
# and silently screenshot the wrong size.
if [ "$phone" = yes ]; then
mode=$PHONE_MODE
export IRIS_SCALE="$PHONE_SCALE"
echo "run-headless: phone-shaped output $PHONE_MODE at IRIS_SCALE=$PHONE_SCALE" >&2
else
mode=$DESKTOP_MODE
fi
swaymsg output HEADLESS-1 mode "$mode" >/dev/null
# The extent `replay-touch` positions against, so a script's coordinates
# are the output's own pixels.
out_w=${mode%x*}
out_h=${mode#*x}; out_h=${out_h%@*}
# Built before the app starts, so a compile error is not reported as a
# window that failed to move.
[ -z "$replay" ] || cargo build --bin replay-touch -p rig-input >&2
cd "$here" cd "$here"
if [ "$kind" = bin ]; then if [ "$kind" = bin ]; then
cargo build --bin "$example" "$@" >&2 cargo build --bin "$example" "$@" >&2
@@ -111,6 +163,18 @@ while [ $i -lt "$((seconds * 2))" ]; do
i=$((i + 1)); sleep 0.5 i=$((i + 1)); sleep 0.5
done done
if [ -n "$replay" ] && kill -0 "$pid" 2>/dev/null; then
if [ -n "$shot" ]; then
grim "${shot%.png}-before.png"
echo "run-headless: wrote ${shot%.png}-before.png (before the gesture)" >&2
fi
"$here/target/debug/replay-touch" "$out_w" "$out_h" "$replay"
# A fling outlives the finger: the gesture's own last sample is not
# when the list stops. Long enough for Android's spline to settle
# (`FlingCalculator::duration` tops out around a second and a half).
sleep 2
fi
if kill -0 "$pid" 2>/dev/null; then if kill -0 "$pid" 2>/dev/null; then
[ -n "$shot" ] && grim "$shot" && echo "run-headless: wrote $shot" >&2 [ -n "$shot" ] && grim "$shot" && echo "run-headless: wrote $shot" >&2
kill "$pid" 2>/dev/null || true kill "$pid" 2>/dev/null || true
+5 -3
View File
@@ -1,5 +1,5 @@
use crate::prelude::*; use crate::prelude::*;
use winit::dpi::{LogicalPosition, LogicalSize}; use winit::dpi::{PhysicalPosition, PhysicalSize};
impl<T: HasDefaultUiState> FocusHost for T { impl<T: HasDefaultUiState> FocusHost for T {
fn recent_click(&mut self) -> bool { fn recent_click(&mut self) -> bool {
@@ -18,9 +18,11 @@ impl<T: HasDefaultUiState> FocusHost for T {
let state = self.default_state_mut(); let state = self.default_state_mut();
let Some(region) = region else { return }; let Some(region) = region else { return };
state.window.set_ime_allowed(true); state.window.set_ime_allowed(true);
// Physical, like everything else this backend hands winit --
// `default::content_scale`.
state.window.set_ime_cursor_area( state.window.set_ime_cursor_area(
LogicalPosition::<f32>::from(region.top_left.tuple()), PhysicalPosition::<f32>::from(region.top_left.tuple()),
LogicalSize::<f32>::from(region.size().tuple()), PhysicalSize::<f32>::from(region.size().tuple()),
); );
} }
} }
+12 -17
View File
@@ -17,15 +17,14 @@ pub struct Input {
} }
impl Input { impl Input {
/// `scale_factor` converts winit's physical-pixel event coordinates /// winit's pointer coordinates are physical pixels, which is the
/// into the same logical units `UiRenderNode`'s window uniform now uses /// space the whole tree is laid out and hit-tested in -- see
/// (`default::render::UiRenderer::new`'s doc comment) -- without it, /// `default::content_scale`. Nothing is converted here; `dp(...)`
/// a cursor position and the widget tree it's tested against would be /// resolves against the density at layout time instead.
/// in two different units on any monitor whose scale factor isn't 1.0. pub fn event(&mut self, event: &WindowEvent) -> bool {
pub fn event(&mut self, event: &WindowEvent, scale_factor: f32) -> bool {
match event { match event {
WindowEvent::CursorMoved { position, .. } => { WindowEvent::CursorMoved { position, .. } => {
self.cursor.pos = Vec2::new(position.x as f32, position.y as f32) / scale_factor; self.cursor.pos = Vec2::new(position.x as f32, position.y as f32);
self.cursor.exists = true; self.cursor.exists = true;
self.cursor.time = Instant::now(); self.cursor.time = Instant::now();
} }
@@ -43,9 +42,7 @@ impl Input {
WindowEvent::MouseWheel { delta, .. } => { WindowEvent::MouseWheel { delta, .. } => {
let mut delta = match *delta { let mut delta = match *delta {
MouseScrollDelta::LineDelta(x, y) => Vec2::new(x, y), MouseScrollDelta::LineDelta(x, y) => Vec2::new(x, y),
MouseScrollDelta::PixelDelta(pos) => { MouseScrollDelta::PixelDelta(pos) => Vec2::new(pos.x as f32, pos.y as f32),
Vec2::new(pos.x as f32, pos.y as f32) / scale_factor
}
}; };
if delta.x == 0.0 && self.modifiers.shift { if delta.x == 0.0 && self.modifiers.shift {
delta.x = delta.y; delta.x = delta.y;
@@ -83,14 +80,12 @@ impl Input {
} }
impl DefaultUiState { impl DefaultUiState {
/// Physical pixels, matching `WindowEvent::Resized` (what
/// `UiRenderState::resize` is given) and the swapchain -- see
/// `default::content_scale`.
pub fn window_size(&self) -> Vec2 { pub fn window_size(&self) -> Vec2 {
let window = self.renderer.window(); let size = self.renderer.window().inner_size();
let size = window.inner_size(); Vec2::new(size.width as f32, size.height as f32)
let scale_factor = window.scale_factor() as f32;
Vec2::new(
size.width as f32 / scale_factor,
size.height as f32 / scale_factor,
)
} }
pub fn cursor_state(&self) -> &CursorState { pub fn cursor_state(&self) -> &CursorState {
+42 -3
View File
@@ -25,6 +25,38 @@ pub use render::*;
pub type Proxy<Event> = EventLoopProxy<Event>; pub type Proxy<Event> = EventLoopProxy<Event>;
/// The desktop's `content_scale`: physical pixels per dp, the same
/// quantity Android reads from `DisplayMetrics.density` and feeds to
/// `UiRenderState::set_density` (`android::view::AndroidUiState::
/// content_scale`'s field comment). Everything in this backend is
/// physical pixels -- the window size, the pointer, the widget tree --
/// and `dp(...)` is what resolves against this at layout time, exactly
/// as on the phone. That is a correction from an earlier version that
/// divided winit's coordinates into a separate "logical" space instead:
/// it left `UiRenderState::resize` (physical, from `WindowEvent::
/// Resized`) and the window uniform (logical) disagreeing on any
/// display whose scale factor is not 1.0, and it rasterised glyphs at
/// one resolution to display them at another -- the blur the phone's own
/// stopgap produced before `dp` existed.
///
/// **`IRIS_SCALE` overrides it**, which is how a phone-shaped desktop
/// window runs the phone's density (`run-headless.sh --phone`,
/// docs/RUST.md's layer 2). An unparsable value is a typo in a command
/// somebody just typed, so it says so and uses the window's own answer
/// rather than silently laying out at the wrong density.
pub fn content_scale(window: &Window) -> f32 {
match std::env::var("IRIS_SCALE") {
Err(_) => window.scale_factor() as f32,
Ok(text) => match text.trim().parse::<f32>() {
Ok(scale) if scale > 0.0 => scale,
_ => {
log::warn!("IRIS_SCALE={text:?} is not a positive number; using the window's own");
window.scale_factor() as f32
}
},
}
}
pub struct DefaultUiState { pub struct DefaultUiState {
pub root: Option<StrongWidget>, pub root: Option<StrongWidget>,
pub renderer: UiRenderer, pub renderer: UiRenderer,
@@ -214,8 +246,16 @@ impl<State: DefaultAppState> AppState for DefaultApp<State> {
window.set_visible(true); window.set_visible(true);
let default_state = DefaultUiState::new(window, access_adapter); let default_state = DefaultUiState::new(window, access_adapter);
let (mut rsc, task_recv) = DefaultRsc::init(default_state.window.clone()); let (mut rsc, task_recv) = DefaultRsc::init(default_state.window.clone());
// Both copies of the density, set before the first widget is
// built so text shapes at the right size on the opening frame --
// the same pair `android::view::new_peer` sets from
// `content_scale`. See `iris_core::TextData::density` for why the
// shaper keeps its own.
let scale = content_scale(default_state.window.as_ref());
rsc.ui.text.density = scale;
let state = State::new(default_state, &mut rsc, proxy); let state = State::new(default_state, &mut rsc, proxy);
let render = UiRenderState::new(); let mut render = UiRenderState::new();
render.set_density(scale);
Self { Self {
rsc, rsc,
state, state,
@@ -247,8 +287,7 @@ impl<State: DefaultAppState> AppState for DefaultApp<State> {
ui_state ui_state
.access_adapter .access_adapter
.process_event(&ui_state.window, &event); .process_event(&ui_state.window, &event);
let scale_factor = ui_state.renderer.window().scale_factor() as f32; let input_changed = ui_state.input.event(&event);
let input_changed = ui_state.input.event(&event, scale_factor);
let cursor_state = ui_state.cursor_state().clone(); let cursor_state = ui_state.cursor_state().clone();
let old = ui_state.focus; let old = ui_state.focus;
if cursor_state.buttons.left.is_start() { if cursor_state.buttons.left.is_start() {
+10 -21
View File
@@ -66,13 +66,11 @@ impl UiRenderer {
self.config.width = size.width; self.config.width = size.width;
self.config.height = size.height; self.config.height = size.height;
self.surface.configure(&self.device, &self.config); self.surface.configure(&self.device, &self.config);
// Logical, matching `new`'s own seed -- see the comment there. // Physical, matching `new`'s own seed -- see the comment there.
let scale_factor = self.window.scale_factor() as f32; self.ui.resize(
let logical = Vec2::new( Vec2::new(size.width as f32, size.height as f32),
size.width as f32 / scale_factor, &self.queue,
size.height as f32 / scale_factor,
); );
self.ui.resize(logical, &self.queue);
} }
fn create_encoder(device: &Device) -> CommandEncoder { fn create_encoder(device: &Device) -> CommandEncoder {
@@ -162,21 +160,12 @@ impl UiRenderer {
// by:" chain as the message, since `UiRenderNode::new` returns it // by:" chain as the message, since `UiRenderNode::new` returns it
// rather than letting wgpu's own default handler panic first (see // rather than letting wgpu's own default handler panic first (see
// that function's doc comment). // that function's doc comment).
// Logical size (physical / `scale_factor`), matching what the // Physical size, the same units the swapchain, `WindowEvent::
// Android backend now reports too (`android::render:: // Resized`, the pointer and the widget tree all use -- see
// AndroidRenderer::new`, `content_scale`) -- the swapchain still // `default::content_scale` for why this backend stopped dividing
// configures at the real physical resolution above; only the // into a separate logical space, and what disagreed while it did.
// window uniform layout/hit-testing agree on is scaled. Without let physical_size = Vec2::new(size.width as f32, size.height as f32);
// this a window on any monitor whose scale factor isn't 1.0 would let ui = UiRenderNode::new(&device, &queue, &config, physical_size)
// have the identical "everything too small" bug RUST.md's P0 box
// found on Iris's phone, just never noticed here because this
// crate's own dev monitors happen to run at 1.0.
let scale_factor = window.scale_factor() as f32;
let logical_size = Vec2::new(
size.width as f32 / scale_factor,
size.height as f32 / scale_factor,
);
let ui = UiRenderNode::new(&device, &queue, &config, logical_size)
.expect("Could not create iris render node!"); .expect("Could not create iris render node!");
Self { Self {
+73
View File
@@ -0,0 +1,73 @@
//! Layer 2 of docs/RUST.md's "Three test layers": the fixture-backed
//! transcript screen in a phone-shaped window, for looking at.
//!
//! iris/run-headless.sh phone --phone --shot /tmp/phone.png -- -p transcript-fixture
//!
//! `--phone` sets the headless sway output to the phone's own 1080x2424
//! and exports `IRIS_SCALE=2.55`, so this draws at the density Iris's
//! phone reports (`transcript_fixture::PHONE_SCALE`) rather than the
//! desktop's 1.0 -- same screen, same fixture and the same folding as
//! the Android bench and the headless tests, so what differs between a
//! screenshot here and one from the phone is the renderer, never the
//! data.
//!
//! No server: `transcript-fixture` embeds the transcript. Colour,
//! spacing, type and anything a person has to *see* is answered here;
//! anything with an assertion behind it belongs in `tests/
//! phone_screen.rs` one layer down.
use iris::prelude::*;
use winit::{dpi::PhysicalSize, window::WindowAttributes};
fn main() {
DefaultApp::<Client>::run();
}
#[derive(DefaultUiState)]
pub struct Client {
ui_state: DefaultUiState,
#[allow(dead_code)]
screen: Option<transcript_ui::TranscriptScreen>,
}
impl DefaultAppState for Client {
fn window_attributes() -> WindowAttributes {
WindowAttributes::default()
.with_title("iris transcript (bench fixture)")
.with_inner_size(PhysicalSize::new(
transcript_fixture::PHONE_WIDTH,
transcript_fixture::PHONE_HEIGHT,
))
}
fn new(
mut ui_state: DefaultUiState,
rsc: &mut DefaultRsc<Self>,
_: Proxy<Self::Event>,
) -> Self {
let screen = match transcript_fixture::open(rsc, &mut ui_state) {
Ok(opened) => {
// A fling coasts only while something asks for the next
// frame; on the desktop that is the window's own redraw
// request (`List::fling`'s doc).
let handle = rsc.tasks.redraw_handle();
(opened.screen.list)(rsc).set_redraw_handle(handle);
Some(opened.screen)
}
// On screen rather than a panic: this window exists to be
// looked at, and "the fixture stopped folding" is something
// to read, not a process that vanished (UI_RULES.md).
Err(message) => {
let text = wtext(format!("Couldn't fold the bench fixture: {message}"))
.color(Color::WHITE)
.wrap(true)
.pad(dp(16))
.add_strong(rsc)
.any();
ui_state.set_root(text);
None
}
};
Self { ui_state, screen }
}
}