iris: I4 -- accessibility names via AccessKit

Builds one flat AccessKit tree (iris_core::ui::access::AccessTree) from
iris's own widget tree: a synthetic Role::Window root with every named
widget as a direct child, names from the existing `.label()`, roles from
a new Widget::access_role() (default Unknown, TextEdit overrides to
TextInput/MultilineTextInput), bounds from UiRenderState::window_region
so a moved subtree reports where it actually is. Modular the way input's
sense registry is: Widgets gained one HashSet<WidgetId> ("named"),
populated only by .label()/set_label and drained by free_next (the
existing removal path), and AccessTree walks only that set -- a widget
nobody named costs it nothing. Updates only when the named set's name,
role or bounds actually changed, with a rebuild counter mirroring
take_counters (confirmed 1/0/1 across first-draw/unchanged/moved in
access_tests.rs).

Pushed through accesskit_winit on the desktop (DefaultApp::new now
creates the window hidden, builds the adapter, then shows it, per that
constructor's requirement) and accesskit_android on Android
(IrisViewPeer now implements AccessibilityNodeProvider). Both action
handlers are inert on purpose: AGENTS.md's tap-by-name is a real touch
at the node's bounds, not an AccessKit action request, so the ordinary
pointer path already answers it once bounds are right. E1's
detach-abort mitigation is carried into android/access.rs's
raise_if_enabled, which gates every QueuedEvents::raise on
AccessibilityManager.isEnabled().

tabs-ui's five switch buttons now carry .label()s matching their
on-screen text, giving both the desktop run and the emulator step real
names to find.

Verified on host: cargo fmt/build/clippy/test all clean (28 tests, 3
new), cargo ndk build+clippy clean for iris and iris-android-app,
run-headless.sh tabs --shot byte-identical to I2's prior screenshot
(27266 bytes). Not run: the emulator step (ui-trace tap-by-name against
iris-android-app), held by another session this pass -- exact commands
recorded in RUST.md's I4 box.

Co-Authored-By: Claude Sonnet <noreply@anthropic.com>
This commit is contained in:
irisandClaude Sonnet committed 2026-09-05 07:05:41 -04:00
1 parent 8adda94a7a
commit 4cfe0ef6e6
22 files changed
+2251 -22

No files matched your search

+1
View File
@@ -10,3 +10,4 @@ image = { workspace = true }
parley = { workspace = true }
swash = { workspace = true }
fxhash = { workspace = true }
accesskit = { workspace = true }
+1 -1
View File
@@ -421,7 +421,7 @@ impl Display for UiRegion {
}
}
#[derive(Debug)]
#[derive(Debug, Clone, Copy, PartialEq)]
pub struct PixelRegion {
pub top_left: Vec2,
pub bot_right: Vec2,
+152
View File
@@ -0,0 +1,152 @@
//! I4 (RUST.md): an AccessKit tree built from iris's own widget tree,
//! shared by both backends -- `android/view.rs` pushes its `TreeUpdate`s
//! through `accesskit_android::Adapter`, `default/mod.rs` through
//! `accesskit_winit::Adapter`. Kept modular the way input's sense registry
//! is: `Widgets::named()` is a side set populated only by `.label()`, so a
//! widget nobody named is never visited here at all, not even to decide it
//! has no name.
//!
//! The tree itself is deliberately flat -- one synthetic `Role::Window`
//! root with every named widget as a direct child, in no particular order.
//! iris's actual widget nesting (a label three `Span`s deep inside a
//! `Scroll`) carries no accessibility meaning of its own here: nothing
//! upstream of a named leaf needs a node, since a screen reader's own
//! traversal (and uiautomator's tap-by-name, the pass condition this was
//! built for) works from each node's on-screen bounds rather than from
//! tree structure. Mirroring the real widget tree exactly would also mean
//! rebuilding intermediate nodes whenever *any* container above a named
//! widget resizes, which is most frames -- the flat shape is what keeps
//! rebuilds tied to "a name, a role or a position actually changed".
use crate::{PixelRegion, UiRenderState, UiRsc, WidgetId, Widgets, util::HashMap};
use accesskit::{Node, NodeId, Rect, Role, TreeId, TreeInfo, TreeUpdate};
/// Reserved for the synthetic root; every real widget's `SlotId::as_u64`
/// starts at 1, so this can never collide with one (see that method's
/// doc comment).
const WINDOW_NODE: NodeId = NodeId(0);
fn node_id(id: WidgetId) -> NodeId {
NodeId(id.as_u64())
}
#[derive(Clone, PartialEq)]
struct Entry {
name: String,
role: Role,
bounds: PixelRegion,
}
fn entry_node(entry: &Entry) -> Node {
let mut node = Node::new(entry.role);
node.set_label(entry.name.clone());
node.set_bounds(Rect {
x0: entry.bounds.top_left.x as f64,
y0: entry.bounds.top_left.y as f64,
x1: entry.bounds.bot_right.x as f64,
y1: entry.bounds.bot_right.y as f64,
});
node
}
/// Owns the last tree pushed out, so `update` can tell "nothing
/// accessibility-relevant changed" from "something did" without asking
/// the platform adapter to diff two `Node`s itself. One of these per
/// window/view -- `default::DefaultUiState` and `android::AndroidUiState`
/// each keep one.
#[derive(Default)]
pub struct AccessTree {
known: HashMap<WidgetId, Entry>,
/// `TreeUpdate`s actually produced since the last `take_rebuilds` --
/// the AccessKit-tree twin of `UiRenderState::take_counters`. Should
/// stay at 0 across an unchanged frame and move by exactly 1 when a
/// named widget's position, name or role changes, however many other
/// widgets are on screen; see `iris/src/access_tests.rs`.
rebuilds: u64,
}
impl AccessTree {
pub fn new() -> Self {
Self::default()
}
fn collect(
widgets: &Widgets,
render: &UiRenderState,
rsc: &dyn UiRsc,
) -> HashMap<WidgetId, Entry> {
let mut current = HashMap::default();
for id in widgets.named() {
let Some(bounds) = render.window_region(&id, rsc) else {
continue;
};
let Some(widget) = widgets.get_dyn(id) else {
continue;
};
current.insert(
id,
Entry {
name: widgets.label(id).clone(),
role: widget.access_role(),
bounds,
},
);
}
current
}
/// Walks `widgets.named()`, looks up each one's current screen bounds
/// via `render.window_region` (which resolves the same move-chain
/// `resolved_region` does, so a moved subtree reports where it
/// actually is), and returns a full `TreeUpdate` if and only if that
/// set differs from the last call -- added, removed, renamed, or
/// moved/resized. A widget that is named but not currently active
/// (not drawn this frame) is left out, the same as one never named at
/// all.
pub fn update(
&mut self,
widgets: &Widgets,
render: &UiRenderState,
rsc: &dyn UiRsc,
) -> Option<TreeUpdate> {
let current = Self::collect(widgets, render, rsc);
if current == self.known {
return None;
}
self.known = current.clone();
self.rebuilds += 1;
Some(build_update(&current))
}
/// The unconditional twin of `update`, for a platform adapter's
/// activation handler (`android/access.rs`'s `AndroidAccessSource`) --
/// AccessKit asks for a full tree the first time a client attaches,
/// which is exactly the case `update`'s diff-against-`known` is not
/// meant to answer (it may have already sent this same snapshot to a
/// client that has since detached and reattached).
pub fn build_full(widgets: &Widgets, render: &UiRenderState, rsc: &dyn UiRsc) -> TreeUpdate {
build_update(&Self::collect(widgets, render, rsc))
}
/// Reads and zeroes the rebuild counter, the same call shape as
/// `UiRenderState::take_counters`.
pub fn take_rebuilds(&mut self) -> u64 {
std::mem::take(&mut self.rebuilds)
}
}
fn build_update(current: &HashMap<WidgetId, Entry>) -> TreeUpdate {
let mut window = Node::new(Role::Window);
let mut nodes = Vec::with_capacity(current.len() + 1);
for (&id, entry) in current {
window.push_child(node_id(id));
nodes.push((node_id(id), entry_node(entry)));
}
nodes.push((WINDOW_NODE, window));
TreeUpdate {
nodes,
tree: Some(TreeInfo::new(WINDOW_NODE)),
tree_id: TreeId::ROOT,
focus: WINDOW_NODE,
}
}
+2
View File
@@ -2,10 +2,12 @@ use crate::{
Mask, MoveOffset, TextData, Textures, WeakWidget, WidgetId, Widgets, util::TrackedArena,
};
mod access;
mod active;
mod painter;
mod render_state;
pub use access::*;
pub use active::*;
pub use painter::Painter;
pub use render_state::*;
+11
View File
@@ -4,6 +4,17 @@ pub struct SlotId {
genr: u32,
}
impl SlotId {
/// A stable, collision-free `u64` encoding of this id -- for a caller
/// (accesskit's `NodeId`, today) that wants a flat integer key rather
/// than the two `u32`s. `idx` is offset by one so no real id ever
/// encodes to 0, which callers can then reserve for their own
/// out-of-band root/window node.
pub fn as_u64(&self) -> u64 {
((self.idx as u64) + 1) << 32 | self.genr as u64
}
}
pub struct SlotVec<T> {
data: Vec<(u32, Option<T>)>,
free: Vec<u32>,
+12
View File
@@ -29,6 +29,18 @@ pub trait Widget: Any {
fn is_size_independent(&self) -> bool {
false
}
/// What kind of control this is, for the AccessKit tree `ui::access`
/// builds (RUST.md's I4). Only consulted for a widget that also has an
/// explicit `.label()` -- an unnamed widget is never visited by that
/// tree at all, named or not, so the default here costs nothing except
/// at the handful of call sites that opt in. Default `Unknown` (a
/// generic control with no more specific semantics); a widget with a
/// real platform equivalent -- `TextEdit`'s `MultilineTextInput` --
/// overrides it.
fn access_role(&self) -> accesskit::Role {
accesskit::Role::Unknown
}
}
impl Widget for () {
+20 -2
View File
@@ -11,6 +11,11 @@ pub struct Widgets {
send: Sender<WidgetId>,
recv: Receiver<WidgetId>,
pub(crate) waiting: HashSet<WidgetId>,
/// Every widget that has ever been given an explicit `.label()` --
/// `ui::access::AccessTree` walks exactly this set, not the whole
/// arena, so a widget nobody named costs it nothing. Symmetric with
/// `free_next` below, which is this set's one removal path.
named: HashSet<WidgetId>,
}
impl Widgets {
@@ -20,6 +25,7 @@ impl Widgets {
needs_redraw: Default::default(),
vec: Default::default(),
waiting: Default::default(),
named: Default::default(),
send,
recv,
}
@@ -95,9 +101,20 @@ impl Widgets {
&self.data(id.id()).unwrap().label
}
/// useful for debugging
/// Also the one place a widget opts into `ui::access`'s AccessKit tree
/// (RUST.md's I4) -- see `named`'s doc comment.
pub fn set_label(&mut self, id: impl IdLike, label: String) {
self.data_mut(id.id()).unwrap().label = label;
let id = id.id();
self.data_mut(id).unwrap().label = label;
self.named.insert(id);
}
/// Every widget with an explicit name, for `ui::access::AccessTree` to
/// walk. Order is unspecified; `AccessTree` doesn't need one; a screen
/// reader's own traversal is worked out by uiautomator from each
/// node's on-screen bounds instead.
pub fn named(&self) -> impl Iterator<Item = WidgetId> + '_ {
self.named.iter().copied()
}
pub fn data_mut(&mut self, id: impl IdLike) -> Option<&mut WidgetData> {
@@ -107,6 +124,7 @@ impl Widgets {
pub fn free_next(&mut self) -> Option<WidgetId> {
let next = self.recv.try_recv().ok()?;
self.vec.free(next);
self.named.remove(&next);
Some(next)
}