Describe a tree before building it, so a failing seed can be reduced
The oracle grew its trees from a seed and the shrinker grew its own, with every scenario written out on each side. So a failure the oracle found could not be handed to the shrinker: there was no tree to pass it, only a seed, and a seed cannot be made smaller. The shrinker could only grow its own trees and hope to meet the same shape, which it does not -- 20,000 of its trees never reproduced what the oracle's seed 18 shows at depth 6. `iris::random` now answers with a `Plan`: `plan(seed, depth, &edits)` draws one out of the random stream and `build(rsc, &plan)` makes the widgets, where `grow` did both at once. Every draw happens in the order it always has, so a seed still means the tree it meant -- checked by running the oracle at 1000 seeds of depth 6 before and after and getting the same three failures with the same boxes. `Plan::smaller` reduces one, `Plan::edited` applies an `Edits` to a tree that already exists, and `tests/scenario/` holds the fifteen cases both rigs now run over the same trees. A span keeps the order it holds its children in apart from the children themselves, so detaching, attaching and reordering leave the widgets made in the same order and two builds still line up index for index. `Tree::detached` is gone: `Spanned::spares` is everything made for a span that it does not hold, which is what both of those were. `tests/cases/plan.rs` pins the three properties the rest rests on: editing a plan is growing one with those edits, every simplification is smaller than what it came from, and reducing ends. The second caught this change's own defect, where dropping a side of a `Branch` duplicated another and grew the tree by four widgets. What it found, on its first run: `SHRINK_SEED=18 SHRINK_DEPTH=6 SHRINK_CASE=repaint-some` reduces 277 widgets to 5. A scroll inside a scroll, the inner one owning a movable region, and only the text at the bottom marked for redraw -- and the span lands 24px out, which is exactly the sized child's height. `git bisect` names `95fb4f9`, where `Masked` began reporting its box rather than its inner's size, so what the outer scroll is told its content measures now depends on whether the inner subtree was redrawn this frame. `tests/cases/unsettled.rs` has it written out, ignored until it is fixed. Checked: fmt, clippy over all targets with -D warnings, the workspace tests (79 + 11 + 15, one ignored for the defect above), and the 100-seed oracle over all fifteen cases at depth 4. The shrinker at 400 seeds of depth 5 now fails, which it did not before running the oracle's trees and cases: seeds 2 and 288 on region-node and 174 and 175 on repaint-some are unreduced leads. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
iris-aiandClaude Opus 5 committed
1 parent
4febabfd2e
commit
98d4e98a29
7 files changed
+1469
-1290
No files matched your search
@@ -0,0 +1,459 @@
|
||||
//! The scenarios both fuzzers run, over the tree a [`Plan`] describes.
|
||||
//!
|
||||
//! One implementation rather than two. The oracle grew its trees from a seed
|
||||
//! and the shrinker grew its own, with every scenario written out on each
|
||||
//! side, so a failure the oracle found could not be handed to the shrinker:
|
||||
//! there was no tree to pass it, only a seed, and a seed cannot be made
|
||||
//! smaller. Both take a plan now, so whatever finds a counterexample can also
|
||||
//! reduce it.
|
||||
//!
|
||||
//! Each target compiles this for itself, so what only one of them calls is
|
||||
//! dead code in the other.
|
||||
#![allow(dead_code)]
|
||||
|
||||
use iris::harness::Harness;
|
||||
use iris::prelude::*;
|
||||
use iris::random::{Aligns, Edits, Kind, Lens, Plan, Rng, SpanEdit, Tree, build};
|
||||
use std::collections::HashMap;
|
||||
|
||||
/// A seed per thread but one, since a seed grows, lays out and drops its tree
|
||||
/// alone. A failing seed still shrinks and panics on its own thread.
|
||||
pub fn over_seeds(seeds: Vec<u64>, run: impl Fn(u64) + Sync) {
|
||||
let threads =
|
||||
std::thread::available_parallelism().map_or(1, |n| n.get().saturating_sub(1).max(1));
|
||||
let chunk = seeds.len().div_ceil(threads).max(1);
|
||||
std::thread::scope(|scope| {
|
||||
for part in seeds.chunks(chunk) {
|
||||
let run = &run;
|
||||
scope.spawn(move || part.iter().for_each(|&seed| run(seed)));
|
||||
}
|
||||
});
|
||||
}
|
||||
|
||||
pub fn env<T: std::str::FromStr>(name: &str, fallback: T) -> T {
|
||||
std::env::var(name)
|
||||
.ok()
|
||||
.and_then(|v| v.parse().ok())
|
||||
.unwrap_or(fallback)
|
||||
}
|
||||
|
||||
/// The window a tree is grown in, and the one a resize takes it to.
|
||||
const OUTER: (f32, f32) = (1920.0, 1200.0);
|
||||
const INNER: (f32, f32) = (640.0, 900.0);
|
||||
const STILL: (f32, f32) = (900.0, 1200.0);
|
||||
|
||||
/// The same box, to a step of the grid per level of nesting between the two
|
||||
/// ways of reaching it. A move, a repaint and a row of shares land on the
|
||||
/// same number; what is left is a box centred in a fraction of its parent
|
||||
/// against the same box centred in its own pixels. A step is a thousandth of
|
||||
/// a pixel, where this was a twentieth of one before any of it was on a grid.
|
||||
const AGREE_STEPS: i32 = 2;
|
||||
|
||||
/// A way of changing what a span holds. Each is a shape worth its own case:
|
||||
/// taking a child out of the middle is not the same as emptying a span, and
|
||||
/// adding one is not the same as adding three.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub enum Shuffle {
|
||||
/// Every other child, so what is left is interleaved with what went.
|
||||
EveryOther,
|
||||
/// Everything but the first, which is the last step before empty.
|
||||
AllButFirst,
|
||||
/// Three more on the end at once.
|
||||
AddThree,
|
||||
/// The first out and three more on, so the count moves both ways.
|
||||
SwapForThree,
|
||||
/// One out of the middle and one on the end.
|
||||
TradeOne,
|
||||
}
|
||||
|
||||
impl Shuffle {
|
||||
fn of(self, grown: usize) -> SpanEdit {
|
||||
let all = |step: usize, from: usize| (from..grown).step_by(step).collect();
|
||||
match self {
|
||||
Self::EveryOther => SpanEdit {
|
||||
detach: all(2, 0),
|
||||
attach: 0,
|
||||
},
|
||||
Self::AllButFirst => SpanEdit {
|
||||
detach: all(1, 1),
|
||||
attach: 0,
|
||||
},
|
||||
Self::AddThree => SpanEdit {
|
||||
detach: Vec::new(),
|
||||
attach: 3,
|
||||
},
|
||||
Self::SwapForThree => SpanEdit {
|
||||
detach: vec![0],
|
||||
attach: 3,
|
||||
},
|
||||
Self::TradeOne => SpanEdit {
|
||||
detach: vec![grown / 2],
|
||||
attach: 1,
|
||||
},
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// What a warm tree is put through before it is compared with a cold one
|
||||
/// grown the way it was left.
|
||||
#[derive(Clone, Copy, Debug, PartialEq)]
|
||||
pub enum Case {
|
||||
/// Nothing changes, so no box may either. What this exercises is the
|
||||
/// order a frame settles a dirty set in.
|
||||
Repaint,
|
||||
/// Every fifth widget rather than all of them: marking all of them
|
||||
/// redraws the whole tree, which is a cold start reached the long way,
|
||||
/// where the mixed case leaves a redrawn subtree beside a retained one.
|
||||
RepaintSome,
|
||||
Resize,
|
||||
ResizeRepaint,
|
||||
/// A resize and then a size change, so a retained answer is asked to
|
||||
/// survive two different kinds of invalidation in a row.
|
||||
ResizeSize,
|
||||
/// A few declared sizes.
|
||||
Size,
|
||||
/// Every declared size at once, so every reader of a size has a changed
|
||||
/// descendant in the same frame and the whole dirty set settles together.
|
||||
EverySize,
|
||||
Align,
|
||||
/// Giving a widget a movable region of its own, or taking it away, is a
|
||||
/// structural change: every primitive under it changes which chain
|
||||
/// resolves it.
|
||||
RegionNode,
|
||||
/// The same children in a different order, which moves every one of them
|
||||
/// without changing what any of them is.
|
||||
Reorder,
|
||||
Shuffle(Shuffle),
|
||||
}
|
||||
|
||||
pub const ALL: [Case; 15] = [
|
||||
Case::Repaint,
|
||||
Case::RepaintSome,
|
||||
Case::Resize,
|
||||
Case::ResizeRepaint,
|
||||
Case::ResizeSize,
|
||||
Case::Size,
|
||||
Case::EverySize,
|
||||
Case::Align,
|
||||
Case::RegionNode,
|
||||
Case::Reorder,
|
||||
Case::Shuffle(Shuffle::EveryOther),
|
||||
Case::Shuffle(Shuffle::AllButFirst),
|
||||
Case::Shuffle(Shuffle::AddThree),
|
||||
Case::Shuffle(Shuffle::SwapForThree),
|
||||
Case::Shuffle(Shuffle::TradeOne),
|
||||
];
|
||||
|
||||
impl Case {
|
||||
/// The name `CASE` selects it by, and the one a failure prints.
|
||||
pub fn name(self) -> &'static str {
|
||||
match self {
|
||||
Self::Repaint => "repaint",
|
||||
Self::RepaintSome => "repaint-some",
|
||||
Self::Resize => "resize",
|
||||
Self::ResizeRepaint => "resize-repaint",
|
||||
Self::ResizeSize => "resize-size",
|
||||
Self::Size => "size",
|
||||
Self::EverySize => "every-size",
|
||||
Self::Align => "align",
|
||||
Self::RegionNode => "region-node",
|
||||
Self::Reorder => "reorder",
|
||||
Self::Shuffle(Shuffle::EveryOther) => "shuffle-every-other",
|
||||
Self::Shuffle(Shuffle::AllButFirst) => "shuffle-all-but-first",
|
||||
Self::Shuffle(Shuffle::AddThree) => "shuffle-add-three",
|
||||
Self::Shuffle(Shuffle::SwapForThree) => "shuffle-swap-for-three",
|
||||
Self::Shuffle(Shuffle::TradeOne) => "shuffle-trade-one",
|
||||
}
|
||||
}
|
||||
|
||||
pub fn named(name: &str) -> Option<Self> {
|
||||
ALL.into_iter().find(|case| case.name() == name)
|
||||
}
|
||||
|
||||
/// Grown in the first, compared in the second.
|
||||
fn window(self) -> ((f32, f32), (f32, f32)) {
|
||||
match self {
|
||||
Self::Resize | Self::ResizeRepaint | Self::ResizeSize => (OUTER, INNER),
|
||||
_ => (STILL, STILL),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn mark(warm: &mut Harness, tree: &Tree, step: usize) {
|
||||
for &id in tree.ids.iter().step_by(step) {
|
||||
warm.rsc.widgets_mut().get_dyn_mut(id);
|
||||
}
|
||||
}
|
||||
|
||||
fn a_len(rng: &mut Rng) -> Option<LayoutLen> {
|
||||
Some(LayoutLen::px(20.0 + rng.below(180) as f32))
|
||||
}
|
||||
|
||||
fn resize_one(warm: &mut Harness, tree: &Tree, idx: usize, rng: &mut Rng) -> Lens {
|
||||
let lens = [a_len(rng), a_len(rng)];
|
||||
warm.rsc
|
||||
.widgets_mut()
|
||||
.set_size_rules(tree.sized[idx], lens[0], lens[1]);
|
||||
lens
|
||||
}
|
||||
|
||||
fn realign_one(warm: &mut Harness, tree: &Tree, idx: usize, rng: &mut Rng) -> Aligns {
|
||||
let side = |rng: &mut Rng| match rng.below(4) {
|
||||
0 => None,
|
||||
1 => Some(AxisAlign::NEG),
|
||||
2 => Some(AxisAlign::CENTER),
|
||||
_ => Some(AxisAlign::POS),
|
||||
};
|
||||
let align = [side(rng), side(rng)];
|
||||
let id = tree.aligned[idx];
|
||||
for (axis, align) in [Axis::X, Axis::Y].into_iter().zip(align) {
|
||||
warm.rsc
|
||||
.widgets_mut()
|
||||
.set_alignment(id, axis, align.unwrap_or_default());
|
||||
}
|
||||
align
|
||||
}
|
||||
|
||||
/// Every span's children in a different order, said both to the warm tree and
|
||||
/// to the plan the cold one is grown from.
|
||||
fn reorder(warm: &mut Harness, tree: &Tree, plan: &Plan) -> Plan {
|
||||
for span in &tree.spans {
|
||||
let children = &mut warm.rsc[span.id].children;
|
||||
if !children.is_empty() {
|
||||
children.rotate_left(1);
|
||||
}
|
||||
}
|
||||
let mut out = plan.clone();
|
||||
out.walk_mut(&mut |node| {
|
||||
if let Kind::Span { order, .. } = &mut node.kind
|
||||
&& !order.is_empty()
|
||||
{
|
||||
order.rotate_left(1);
|
||||
}
|
||||
});
|
||||
out
|
||||
}
|
||||
|
||||
/// Applies `shuffle` to every third span. What it takes out is given back to
|
||||
/// the span's spares: the last share of a widget must outlive the comparison,
|
||||
/// or its id is handed to something else and the two trees stop lining up.
|
||||
fn reshuffle(warm: &mut Harness, tree: &mut Tree, shuffle: Shuffle) -> HashMap<usize, SpanEdit> {
|
||||
let mut edits = HashMap::new();
|
||||
for (idx, span) in tree.spans.iter_mut().enumerate().step_by(3) {
|
||||
let edit = shuffle.of(span.grown);
|
||||
let mut take = edit.detach.clone();
|
||||
take.sort_unstable();
|
||||
let children = &mut warm.rsc[span.id].children;
|
||||
// Highest first, so an index means the same child however many of its
|
||||
// neighbours are going too.
|
||||
for j in take.into_iter().rev() {
|
||||
if j < children.len() {
|
||||
span.spares.push(children.remove(j));
|
||||
}
|
||||
}
|
||||
let attach = edit.attach.min(span.spares.len());
|
||||
let moved: Vec<_> = span.spares.drain(..attach).collect();
|
||||
warm.rsc[span.id].children.extend(moved);
|
||||
edits.insert(idx, edit);
|
||||
}
|
||||
edits
|
||||
}
|
||||
|
||||
/// Changes the warm tree and answers with the plan a cold tree grown that way
|
||||
/// comes from. Each arm settles its own frame, so a case that changes nothing
|
||||
/// does not get a second one that could settle what the first left.
|
||||
fn change(case: Case, warm: &mut Harness, tree: &mut Tree, plan: &Plan, rng: &mut Rng) -> Plan {
|
||||
let some_sizes = |warm: &mut Harness, tree: &Tree, rng: &mut Rng| {
|
||||
let mut sizes = HashMap::new();
|
||||
for _ in 0..4 {
|
||||
if tree.sized.is_empty() {
|
||||
break;
|
||||
}
|
||||
let idx = rng.below(tree.sized.len());
|
||||
sizes.insert(idx, resize_one(warm, tree, idx, rng));
|
||||
}
|
||||
sizes
|
||||
};
|
||||
let edits = match case {
|
||||
Case::Resize => return plan.clone(),
|
||||
Case::Repaint | Case::ResizeRepaint => {
|
||||
mark(warm, tree, 1);
|
||||
warm.frame();
|
||||
return plan.clone();
|
||||
}
|
||||
Case::RepaintSome => {
|
||||
mark(warm, tree, 5);
|
||||
warm.frame();
|
||||
return plan.clone();
|
||||
}
|
||||
Case::Reorder => {
|
||||
let out = reorder(warm, tree, plan);
|
||||
warm.frame();
|
||||
return out;
|
||||
}
|
||||
Case::Size | Case::ResizeSize => Edits {
|
||||
sizes: some_sizes(warm, tree, rng),
|
||||
..Default::default()
|
||||
},
|
||||
Case::EverySize => Edits {
|
||||
sizes: (0..tree.sized.len())
|
||||
.map(|idx| (idx, resize_one(warm, tree, idx, rng)))
|
||||
.collect(),
|
||||
..Default::default()
|
||||
},
|
||||
Case::Align => Edits {
|
||||
aligns: (0..tree.aligned.len())
|
||||
.step_by(3)
|
||||
.map(|idx| (idx, realign_one(warm, tree, idx, rng)))
|
||||
.collect(),
|
||||
..Default::default()
|
||||
},
|
||||
Case::RegionNode => {
|
||||
let mut nodes = HashMap::new();
|
||||
for idx in (0..tree.nodes.len()).step_by(2) {
|
||||
let id = tree.nodes[idx];
|
||||
let take = !warm.rsc.widgets().is_region_node(id);
|
||||
warm.rsc.widgets_mut().set_region_node(id, take);
|
||||
nodes.insert(idx, take);
|
||||
}
|
||||
Edits {
|
||||
nodes,
|
||||
..Default::default()
|
||||
}
|
||||
}
|
||||
Case::Shuffle(shuffle) => Edits {
|
||||
spans: reshuffle(warm, tree, shuffle),
|
||||
..Default::default()
|
||||
},
|
||||
};
|
||||
warm.frame();
|
||||
plan.edited(&edits)
|
||||
}
|
||||
|
||||
/// What a widget was configured with, so a tree a fuzzer found can be written
|
||||
/// out by hand. A failure is a lead; the fast test that replaces it has to be
|
||||
/// buildable from what the failure printed.
|
||||
fn describe(id: WidgetId, h: &Harness) -> String {
|
||||
let rules = h.rsc.widgets().size_rules(id);
|
||||
let rule = |r: SizeRule| match r.exact() {
|
||||
Some(len) => format!("{len}"),
|
||||
None => "-".into(),
|
||||
};
|
||||
let align = h.rsc.widgets().alignment(id);
|
||||
let side = |a: AxisAlign| {
|
||||
if a == AxisAlign::NEG {
|
||||
"neg".into()
|
||||
} else if a == AxisAlign::CENTER {
|
||||
"mid".into()
|
||||
} else if a == AxisAlign::POS {
|
||||
"pos".into()
|
||||
} else {
|
||||
format!("{:.2}", a.rel())
|
||||
}
|
||||
};
|
||||
// A rule and an alignment are properties of whatever carries them, so
|
||||
// they print with that widget rather than as widgets of their own.
|
||||
let mut out = describe_widget(id, h);
|
||||
if (rules.x, rules.y) != (SizeRule::Free, SizeRule::Free) {
|
||||
out += &format!("[x:{},y:{}]", rule(rules.x), rule(rules.y));
|
||||
}
|
||||
if align != RegionAlign::default() {
|
||||
out += &format!("@{},{}", side(align.x), side(align.y));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn describe_widget(id: WidgetId, h: &Harness) -> String {
|
||||
let label = h.rsc.widgets().label(id).to_string();
|
||||
let Some(widget) = h.rsc.widgets().get_dyn(id) else {
|
||||
return label;
|
||||
};
|
||||
let any: &dyn std::any::Any = widget;
|
||||
if let Some(w) = any.downcast_ref::<Span>() {
|
||||
let sign = if w.dir.sign == Sign::Neg { "-" } else { "+" };
|
||||
return format!(
|
||||
"Span{{dir:{:?}{sign},gap:{},n:{}}}",
|
||||
w.dir.axis,
|
||||
w.gap,
|
||||
w.children.len()
|
||||
);
|
||||
}
|
||||
if let Some(w) = any.downcast_ref::<Pad>() {
|
||||
let p = &w.padding;
|
||||
return format!(
|
||||
"Pad{{l:{},r:{},t:{},b:{}}}",
|
||||
p.left, p.right, p.top, p.bottom
|
||||
);
|
||||
}
|
||||
if let Some(w) = any.downcast_ref::<Stack>() {
|
||||
return format!("Stack{{n:{}}}", w.children.len());
|
||||
}
|
||||
label
|
||||
}
|
||||
|
||||
fn same_region(got: Option<PixelRegion>, want: Option<PixelRegion>) -> bool {
|
||||
match (got, want) {
|
||||
(Some(got), Some(want)) => {
|
||||
let same = |a: Px, b: Px| (a - b).abs() <= Px::STEP.mul_int(AGREE_STEPS);
|
||||
same(got.top_left.x, want.top_left.x)
|
||||
&& same(got.top_left.y, want.top_left.y)
|
||||
&& same(got.bot_right.x, want.bot_right.x)
|
||||
&& same(got.bot_right.y, want.bot_right.y)
|
||||
}
|
||||
(None, None) => true,
|
||||
_ => false,
|
||||
}
|
||||
}
|
||||
|
||||
/// Runs `case` on the tree `plan` describes, warm and cold, and says where
|
||||
/// the two disagree. `seed` chooses only the values a case picks at random,
|
||||
/// so one plan under one case is one comparison however it was reached.
|
||||
pub fn diverges(plan: &Plan, case: Case, seed: u64) -> Option<String> {
|
||||
let (start, end) = case.window();
|
||||
let mut warm = Harness::new(start);
|
||||
let (root, mut tree) = build(&mut warm.rsc, plan);
|
||||
warm.state.root = Some(root);
|
||||
// The frame that makes it warm: without it nothing is retained and the
|
||||
// comparison is two cold starts agreeing with each other.
|
||||
warm.frame();
|
||||
if start != end {
|
||||
warm.resize(end);
|
||||
warm.frame();
|
||||
}
|
||||
let cold_plan = change(case, &mut warm, &mut tree, plan, &mut Rng::new(seed));
|
||||
|
||||
let mut cold = Harness::new(end);
|
||||
let (root, cold_tree) = build(&mut cold.rsc, &cold_plan);
|
||||
cold.state.root = Some(root);
|
||||
cold.frame();
|
||||
|
||||
let mut drawn = 0;
|
||||
for (i, (&w, &c)) in tree.ids.iter().zip(&cold_tree.ids).enumerate() {
|
||||
let (got, want) = (warm.region(&w), cold.region(&c));
|
||||
drawn += got.is_some() as usize;
|
||||
if same_region(got, want) {
|
||||
continue;
|
||||
}
|
||||
// Where two trees disagree is rarely where the cause is, so the
|
||||
// ancestry comes with it, marking the widgets that own a region.
|
||||
let mut chain = Vec::new();
|
||||
let mut at = Some(w);
|
||||
while let Some(id) = at {
|
||||
let active = &warm.render.active[&id];
|
||||
let node = match active.move_idx == active.parent_move {
|
||||
true => "",
|
||||
false => "*",
|
||||
};
|
||||
chain.push(format!("{}{node}", describe(id, &warm)));
|
||||
at = active.parent;
|
||||
}
|
||||
return Some(format!(
|
||||
"widget {i}\n warm {got:?}\n cold {want:?}\n {}",
|
||||
chain.join(" < ")
|
||||
));
|
||||
}
|
||||
match drawn {
|
||||
0 => Some("nothing was drawn".into()),
|
||||
_ => None,
|
||||
}
|
||||
}
|
||||
Reference in new issue
Block a user