ui-trace: add holddrag, a press-hold-then-drag gesture

Neither existing action can produce "hold stationary for LONG_PRESS,
then move without lifting, then release": tap has no hold at all, and
swipe X1 Y1 X2 Y2 MS interpolates motion across its whole duration
starting at t=0, so a long swipe with a short first segment is still
continuous motion throughout, never a hold followed by a drag.

holddrag X1 Y1 X2 Y2 HOLD_MS MOVE_MS extends the same
MotionEvent/injectInputEvent mechanism swipe already uses: DOWN, sleep
HOLD_MS, then MOVE at the same fixed 10ms cadence swipe uses over
MOVE_MS, then UP -- one continuous touch. Additive; existing commands
unchanged.

Verified against a real device (this repo has no unit test harness, so
verification is driving a device, matching its existing posture):
`ui-trace record --do "holddrag 300 1850 300 2050 600 300"` against
ai-app-2's iris transcript screen produced a real long-press-then-drag
selection (confirmed by the app's own logcat and a screenshot showing
the resulting highlighted selection) that neither tap nor swipe could
reach. javac --release 17 -Xlint:all -Werror clean.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Fable 5.1 committed 2026-09-05 15:01:01 -04:00
1 parent 0cae9dc1d9
commit 86cf4e8088
2 files changed
+37 -1

No files matched your search

+2 -1
View File
@@ -367,7 +367,8 @@ def main():
rec.add_argument("-i", "--interval", type=int, default=16,
help="ms between samples (0 = as fast as possible)")
rec.add_argument("--do", action="append", default=[], metavar="ACTION",
help="'wait MS' | 'tap X Y' | 'swipe X1 Y1 X2 Y2 [MS]', repeatable")
help="'wait MS' | 'tap X Y' | 'swipe X1 Y1 X2 Y2 [MS]' | "
"'holddrag X1 Y1 X2 Y2 HOLD_MS MOVE_MS', repeatable")
rec.add_argument("-o", "--out", help="where to save the trace")
rec.add_argument("-m", "--match", help="show a timeline for these afterwards")
rec.add_argument("--field", default="top",
+35
View File
@@ -221,6 +221,9 @@ public final class UiTrace {
case "swipe" -> swipe(Integer.parseInt(step[1]), Integer.parseInt(step[2]),
Integer.parseInt(step[3]), Integer.parseInt(step[4]),
step.length > 5 ? Long.parseLong(step[5]) : 300);
case "holddrag" -> holdDrag(Integer.parseInt(step[1]), Integer.parseInt(step[2]),
Integer.parseInt(step[3]), Integer.parseInt(step[4]),
Long.parseLong(step[5]), Long.parseLong(step[6]));
default -> throw new IllegalArgumentException("unknown action: " + step[0]);
}
}
@@ -345,4 +348,36 @@ public final class UiTrace {
}
send(MotionEvent.ACTION_UP, down, x2, y2);
}
/**
* A press held stationary for `holdMs`, then moved to `(x2, y2)` over `moveMs`, then
* released -- one continuous touch, never lifted between the hold and the move.
*
* Neither existing action can produce this. `tap` has no hold at all, and `swipe`
* interpolates motion across its *whole* duration starting at t=0, so a long `swipe` with a
* short first segment is still continuous motion throughout, never a hold followed by a
* drag. That gap matters for anything that decides pan-vs-select the way Android itself
* does -- a stationary press held past `LONG_PRESS` (500ms) starts a selection, which
* further drag then extends -- because neither of the other two actions can reach the
* "starts a selection" branch at all.
*
* The move phase reuses `swipe`'s own fixed 10ms cadence for the same reason: a velocity
* tracker needs a real sequence of points, not two.
*/
private void holdDrag(int x1, int y1, int x2, int y2, long holdMs, long moveMs) {
long down = SystemClock.uptimeMillis();
send(MotionEvent.ACTION_DOWN, down, x1, y1);
SystemClock.sleep(holdMs);
int steps = (int) Math.max(2, moveMs / 10);
long moveStart = SystemClock.uptimeMillis();
for (int i = 1; i <= steps; i++) {
float part = (float) i / steps;
long due = moveStart + (long) (moveMs * part);
long slack = due - SystemClock.uptimeMillis();
if (slack > 0) SystemClock.sleep(slack);
send(MotionEvent.ACTION_MOVE, down, Math.round(x1 + (x2 - x1) * part),
Math.round(y1 + (y2 - y1) * part));
}
send(MotionEvent.ACTION_UP, down, x2, y2);
}
}