Delete the decisions and design logs; scripts, rigs and xtask off the root

Iris: "remove both decisions and iris.md. I've decided to instead make
decisions when planning with agents rather than after they do things, and
they're both too long for me to wanna read, + don't cover all the
decisions I'll wanna make about the code anyways. I'll just naturally run
into things for now. Todo is important though."

So docs/DECISIONS.md (850 lines) and docs/IRIS.md (1,986) are gone, and
AGENTS.md now says not to start another: raise a choice while planning it
with her, otherwise decide it and put the reasoning at the code it
governs. The TODO lists stay. docs/SUBAGENTS_DECISIONS.md went with them
-- same artefact, same reasoning, and she did not name it, so its six
decisions were folded into docs/SUBAGENTS.md rather than deleted.

Deleting the logs left ~30 citations dangling in code comments and docs.
Each states its reason inline and cited the file only for provenance, so
they now read "decided 2026-09-07" or name the module doc that carries
the reasoning.

The root had six things that were not a program or a document. Moved,
per "I only meant top level sh files":

  run-tests.sh, test-wg-tunnel.sh, wg-setup-host.sh  -> scripts/
  rigs/                                              -> scripts/rigs/
  xtask/                                             -> scripts/xtask/

A project's own scripts stayed with the project: app/*.sh, app-rust/*.sh,
iris/*.sh and server/enroll-link.sh did not move.

`target/` at the root is deleted and cannot come back: there was never a
workspace there, and the 29 MB was only xtask's scratch space, now in
scripts/xtask/target/. `cargo xtask apk` still runs from the repo root
and now publishes to scripts/build/outputs/apk/<mode>/ -- one directory
deep, because that is what Dev Updater's `*/build/outputs/apk/*/*.apk`
discovery pattern needs, and scripts/xtask/build would have been two.

Verified: ./scripts/run-tests.sh and `cd iris && cargo test` green, clippy
and fmt clean everywhere, `cargo xtask apk debug --abi x86_64` builds and
signs an APK carrying lib/x86_64/libai_app.so at the new publish path, and
the repo root is now eleven entries with no build output among them.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Opus 5 committed 2026-09-09 00:16:24 -04:00
1 parent 09778346a0
commit 4ccfda6b8e
44 files changed
+198 -2994

No files matched your search

+1
View File
@@ -0,0 +1 @@
target
+1224
View File
File diff suppressed because it is too large. Load diff
+18
View File
@@ -0,0 +1,18 @@
[package]
name = "gpu-probe"
version = "0.1.0"
edition = "2024"
# Deliberately its own crate rather than a member of iris's workspace: the
# vendored `iris/` tree is meant to stay reconcilable with the iris/iris
# repository, and this is a rig belonging to ai-app.
[dependencies]
# Pinned to what iris asks for, so the answer is about iris rather than
# about a different wgpu.
wgpu = "30.0.1"
pollster = "1.0.1"
# Queried directly, because wgpu and `cmd gpu vkjson` disagreed about
# descriptor indexing in the emulator and only the raw call says which is
# right.
ash = "0.38"
+102
View File
@@ -0,0 +1,102 @@
//! Why a GPU test segfaults *after* it has passed, and what stops it.
//!
//! Measured here 2026-09-08, on this VM's Venus adapter. Destroying the
//! last `VkInstance` makes the Vulkan loader `dlclose` the ICD; Mesa's
//! ICD (`/usr/lib/libvulkan_virtio.so`) registers a `pthread_key_create`
//! destructor pointing into its own text and is not linked `-z nodelete`,
//! so glibc calls that destructor through unmapped memory when the thread
//! that used Vulkan exits. libtest runs every `#[test]` on a spawned
//! thread, which is why it looked like "wgpu crashes on drop": the drop
//! itself completes, and the crash lands as the thread unwinds.
//!
//! The four modes are the experiment, and each is one variable:
//!
//! | mode | what it does | 2026-09-08 |
//! |---|---|---|
//! | `main` | wgpu instance + device on the main thread, dropped | exits 0 |
//! | `thread` | the same on a spawned thread | **SIGSEGV** |
//! | `keep` | the same, but the instance is never dropped | exits 0 |
//! | `raw` | raw Vulkan (`ash`), instance + device, spawned thread | **SIGSEGV** |
//!
//! `raw` is the one that says whose bug it is: no wgpu is involved, so
//! there is nothing for wgpu or a caller to fix in its drop order. `keep`
//! is the fix -- hold one `wgpu::Instance` for the process, which is what
//! wgpu asks for anyway. `iris/tests/mask_sdf.rs` does exactly that.
//!
//! `VK_LOADER_DISABLE_DYNAMIC_LIBRARY_UNLOADING=1` also makes every mode
//! exit cleanly, which is the confirmation that the unload is the
//! mechanism -- but it is an environment variable every caller would have
//! to remember, so it belongs in this comment rather than in a script.
use ash::vk;
use pollster::block_on;
use wgpu::*;
fn main() {
let mode = std::env::args().nth(1).unwrap_or_else(|| "thread".into());
let body = match mode.as_str() {
"main" => return wgpu_open_and_close(false),
"thread" => || wgpu_open_and_close(false),
"keep" => || wgpu_open_and_close(true),
"raw" => raw_vulkan_open_and_close,
other => panic!("unknown mode {other:?}: main | thread | keep | raw"),
};
std::thread::spawn(body).join().expect("the spawned thread");
// Not reached when the thread's exit takes the process with it.
eprintln!("thread joined");
}
/// A wgpu instance and device, opened and closed. `keep_instance` is the
/// fix under test: everything else still drops normally.
fn wgpu_open_and_close(keep_instance: bool) {
let instance = Instance::default();
let adapter =
block_on(instance.request_adapter(&RequestAdapterOptions::default())).expect("no adapter");
let info = adapter.get_info();
eprintln!(
"adapter: {} ({:?}, {})",
info.name, info.backend, info.driver
);
let (device, queue) =
block_on(adapter.request_device(&DeviceDescriptor::default())).expect("no device");
drop(queue);
drop(device);
drop(adapter);
if keep_instance {
std::mem::forget(instance);
} else {
drop(instance);
}
eprintln!("wgpu closed");
}
/// The same shape with no wgpu in it at all, which is what makes this a
/// loader/driver bug rather than a wgpu one.
fn raw_vulkan_open_and_close() {
unsafe {
let entry = ash::Entry::load().expect("vulkan loader");
let app = vk::ApplicationInfo::default().api_version(vk::make_api_version(0, 1, 1, 0));
let instance = entry
.create_instance(
&vk::InstanceCreateInfo::default().application_info(&app),
None,
)
.expect("instance");
let phys = instance.enumerate_physical_devices().expect("devices")[0];
let priorities = [1.0f32];
let queues = [vk::DeviceQueueCreateInfo::default()
.queue_family_index(0)
.queue_priorities(&priorities)];
let device = instance
.create_device(
phys,
&vk::DeviceCreateInfo::default().queue_create_infos(&queues),
None,
)
.expect("device");
device.destroy_device(None);
instance.destroy_instance(None);
}
eprintln!("raw vulkan closed");
}
+164
View File
@@ -0,0 +1,164 @@
//! Ask a device whether it can give iris the GPU it asks for.
//!
//! Until 2026-09-04 iris's renderer bound every texture it had drawn as one
//! binding array and indexed it non-uniformly from the shader, which needed
//! descriptor indexing and a very large per-stage binding-array limit
//! (101,000 elements: 100,000 textures and 1,000 samplers,
//! `UiLimits::default`). That was ordinary on a desktop and, per
//! TEXTURES.md's "iris's binding array does not survive real Android
//! hardware", not available on a real share of Android GPUs -- and it failed
//! outright on this emulator's software Vulkan, which is what this rig
//! caught first. iris now asks for nothing beyond wgpu's own defaults (see
//! `iris/src/default/render.rs`): the glyph atlas is one `texture_2d_array`
//! and a standalone image is its own ordinary bind group, and neither needs
//! descriptor indexing. This rig still asks `request_device` for exactly
//! what iris asks for, so it keeps being the answer to "does iris's actual
//! device request succeed here" rather than a guess from reading the code.
//!
//! It runs as a plain executable with no window and no APK, because
//! `request_adapter` needs no surface -- so it can be pushed to a device with
//! `adb push` and run from `/data/local/tmp`, which is far cheaper than an
//! app. What it therefore cannot answer is anything about presenting to a
//! surface; that is the Android backend's own problem.
mod vk;
use wgpu::*;
/// What `iris/src/default/render.rs` asks `request_device` for, now that the
/// binding array is gone: nothing beyond wgpu's own default feature set.
fn iris_features() -> Features {
Features::empty()
}
/// The one non-default limit iris asks for -- unrelated to the binding array,
/// kept for the big storage buffers behind rects/glyphs.
const IRIS_MAX_BUFFER_SIZE: u64 = 1 << 30;
/// Mirrors `iris_core::device_limits()` (`iris/core/src/render/mod.rs`) --
/// cannot call it directly, since this rig is deliberately its own crate,
/// not a workspace member (this file's own Cargo.toml comment). Keep the
/// two in sync by hand when one changes; this rig's whole purpose is "does
/// the device iris actually builds come back," so a stale copy here would
/// silently stop answering that question. Zeroed rather than left at
/// `Limits::default()`'s desktop-tier values because nothing in iris
/// creates a `ComputePipeline` or a `@compute` shader stage -- found by
/// grepping the whole `iris`/`iris-core` tree before this rig's comment was
/// written -- and the unconditional default request is what crashed
/// `request_device` on the Android emulator's software GL path
/// (`EMU_GPU=software`, `force-gles`: SwiftShader's GL reports itself as
/// OpenGL ES 3.0, which has no compute shaders at all, so the adapter's
/// real limit is 0). The same would happen on a real GLES-3.0-only Android
/// device.
fn iris_limits() -> Limits {
Limits {
max_buffer_size: IRIS_MAX_BUFFER_SIZE,
max_compute_workgroup_storage_size: 0,
max_compute_invocations_per_workgroup: 0,
max_compute_workgroup_size_x: 0,
max_compute_workgroup_size_y: 0,
max_compute_workgroup_size_z: 0,
max_compute_workgroups_per_dimension: 0,
..Default::default()
}
}
fn main() {
vk::report();
let instance = Instance::new(InstanceDescriptor {
backends: Backends::from_env().unwrap_or(Backends::PRIMARY),
..InstanceDescriptor::new_without_display_handle()
});
let adapters = pollster::block_on(instance.enumerate_adapters(Backends::all()));
println!("adapters: {}", adapters.len());
for adapter in &adapters {
let info = adapter.get_info();
println!(
" {:?} {} ({:?})",
info.backend, info.name, info.device_type
);
// Compute is a *downlevel* capability, not a feature: Vulkan
// grants it to any 1.0 device, and GLES only from ES 3.1. So
// "can iris use a compute pass here" is this flag on every
// adapter iris might fall back to, not just the preferred one.
let down = adapter.get_downlevel_capabilities();
let limits = adapter.limits();
println!(
" compute shaders: {} (shader model {:?})",
down.flags.contains(DownlevelFlags::COMPUTE_SHADERS),
down.shader_model
);
println!(
" max compute invocations/workgroup: {}, workgroup storage: {} bytes",
limits.max_compute_invocations_per_workgroup, limits.max_compute_workgroup_storage_size
);
}
let Some(adapter) = pollster::block_on(instance.request_adapter(&RequestAdapterOptions {
power_preference: PowerPreference::default(),
compatible_surface: None,
force_fallback_adapter: false,
..Default::default()
}))
.ok() else {
println!("\nNO ADAPTER");
std::process::exit(1);
};
let info = adapter.get_info();
println!(
"\nchosen: {:?} {} ({:?})",
info.backend, info.name, info.device_type
);
println!("driver: {} {}", info.driver, info.driver_info);
let have = adapter.features();
println!("\nfeatures iris requires:");
let mut missing = Features::empty();
for f in iris_features().iter() {
let ok = have.contains(f);
println!(
" {:60} {}",
format!("{f:?}"),
if ok { "yes" } else { "NO" }
);
if !ok {
missing |= f;
}
}
let limits = adapter.limits();
println!("\nlimits iris requires:");
println!(
" {:52} want {:>7} have {:>7} {}",
"max_buffer_size",
IRIS_MAX_BUFFER_SIZE,
limits.max_buffer_size,
if limits.max_buffer_size >= IRIS_MAX_BUFFER_SIZE {
"ok"
} else {
"TOO SMALL"
}
);
// The question that actually matters: does the device iris builds come
// back, or does wgpu refuse it? With no features and no binding-array
// limits requested, this is expected to succeed everywhere -- this rig
// is what turned that from an assumption into a measurement, first on
// this emulator's software Vulkan.
let wanted = iris_limits();
match pollster::block_on(adapter.request_device(&DeviceDescriptor {
required_features: iris_features(),
required_limits: wanted,
..Default::default()
})) {
Ok(_) => println!("\nIRIS DEVICE: ok"),
Err(e) => println!("\nIRIS DEVICE: FAILED -- {e}"),
}
if !missing.is_empty() {
println!("\nmissing features: {missing:?}");
}
}
+108
View File
@@ -0,0 +1,108 @@
//! The raw Vulkan half of the probe.
//!
//! wgpu reports a feature only after a chain of its own decisions -- which
//! physical device, which API version, which extension list -- so "wgpu says
//! no" and "the driver says no" are different claims. This asks
//! `vkGetPhysicalDeviceFeatures2` itself and prints the inputs to that chain,
//! so a disagreement can be attributed rather than guessed at.
use ash::{Entry, vk};
use std::ffi::CStr;
pub fn report() {
let entry = match unsafe { Entry::load() } {
Ok(e) => e,
Err(e) => {
println!("\nraw vulkan: cannot load loader -- {e}");
return;
}
};
let instance_version = match unsafe { entry.try_enumerate_instance_version() } {
Ok(Some(v)) => v,
Ok(None) => vk::API_VERSION_1_0,
Err(e) => {
println!("\nraw vulkan: enumerate_instance_version failed -- {e}");
return;
}
};
println!("\nraw vulkan:");
println!(" loader instance version: {}", ver(instance_version));
// Ask for the highest instance version the loader admits to: wgpu clamps
// the device version by the instance's, so an instance created at 1.0
// makes a 1.3 device look like 1.0.
let app_info = vk::ApplicationInfo::default().api_version(instance_version);
let create = vk::InstanceCreateInfo::default().application_info(&app_info);
let instance = match unsafe { entry.create_instance(&create, None) } {
Ok(i) => i,
Err(e) => {
println!(" create_instance failed -- {e}");
return;
}
};
let devices = unsafe { instance.enumerate_physical_devices() }.unwrap_or_default();
for phd in devices {
let props = unsafe { instance.get_physical_device_properties(phd) };
let name = unsafe { CStr::from_ptr(props.device_name.as_ptr()) };
println!(" device: {}", name.to_string_lossy());
println!(" device api version: {}", ver(props.api_version));
let exts =
unsafe { instance.enumerate_device_extension_properties(phd) }.unwrap_or_default();
let has_ext = exts.iter().any(|e| {
(unsafe { CStr::from_ptr(e.extension_name.as_ptr()) }) == c"VK_EXT_descriptor_indexing"
});
println!(" VK_EXT_descriptor_indexing advertised: {has_ext}");
println!(" device extensions: {}", exts.len());
let mut indexing = vk::PhysicalDeviceDescriptorIndexingFeatures::default();
let mut features2 = vk::PhysicalDeviceFeatures2::default().push_next(&mut indexing);
unsafe { instance.get_physical_device_features2(phd, &mut features2) };
for (name, v) in [
(
"shaderSampledImageArrayNonUniformIndexing",
indexing.shader_sampled_image_array_non_uniform_indexing,
),
(
"descriptorBindingSampledImageUpdateAfterBind",
indexing.descriptor_binding_sampled_image_update_after_bind,
),
(
"shaderStorageImageArrayNonUniformIndexing",
indexing.shader_storage_image_array_non_uniform_indexing,
),
(
"descriptorBindingStorageImageUpdateAfterBind",
indexing.descriptor_binding_storage_image_update_after_bind,
),
(
"shaderStorageBufferArrayNonUniformIndexing",
indexing.shader_storage_buffer_array_non_uniform_indexing,
),
(
"descriptorBindingStorageBufferUpdateAfterBind",
indexing.descriptor_binding_storage_buffer_update_after_bind,
),
(
"descriptorBindingPartiallyBound",
indexing.descriptor_binding_partially_bound,
),
] {
println!(" {name:48} {}", if v != 0 { "yes" } else { "NO" });
}
}
unsafe { instance.destroy_instance(None) };
}
fn ver(v: u32) -> String {
format!(
"{}.{}.{}",
vk::api_version_major(v),
vk::api_version_minor(v),
vk::api_version_patch(v)
)
}
+133
View File
@@ -0,0 +1,133 @@
// What this VM's virtio-gpu actually offers, asked of the kernel and the
// driver rather than inferred from the host's qemu command line.
//
// cc -O2 -o virtgpu-probe virtgpu-probe.c -I/usr/include/libdrm -ldrm -lvulkan
// ./virtgpu-probe
//
// Written 2026-09-08 for the question "Venus keeps causing problems, is
// there something to do with qemu instead" (docs/RUST.md, "Venus went
// away for an hour"). Three things, each of which was guessed wrong at
// least once before being measured:
//
// 1. Which capsets the host offers. Capset 6 (DRM) is the "native
// context" one -- RADV running in the guest against a passed-through
// DRM context instead of Venus proxying every Vulkan call. Whether
// it is available is a host-side fact this is the only way to read
// from in here.
// 2. Whether the Vulkan device has compute. It does, and assuming it
// did not sent one investigation down the wrong path: the "no
// compute" finding on record is about the *Android emulator's*
// SwiftShader GL path, a different machine entirely.
// 3. Whether plain Vulkan device teardown crashes on this adapter. It
// does not -- which is what makes wgpu's teardown SIGSEGV wgpu's and
// not the driver's, and is the kind of claim that is worthless
// without the negative half.
//
// C rather than a Rust crate on purpose: two of the three are ioctl and
// loader questions that a wgpu-shaped rig cannot ask, and this needs to
// keep working when the thing under suspicion is wgpu itself.
#include <errno.h>
#include <fcntl.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <drm/virtgpu_drm.h>
#include <vulkan/vulkan.h>
#include <xf86drm.h>
static const char *capset_name(int id) {
switch (id) {
case 1: return "VIRGL";
case 2: return "VIRGL2";
case 3: return "GFXSTREAM_VULKAN";
case 4: return "VENUS";
case 5: return "CROSS_DOMAIN";
case 6: return "DRM (native context)";
default: return "unknown";
}
}
static int capsets(void) {
int fd = open("/dev/dri/renderD128", O_RDWR);
if (fd < 0) {
printf("capsets: cannot open /dev/dri/renderD128: %s\n", strerror(errno));
return 1;
}
struct drm_virtgpu_getparam gp;
uint64_t mask = 0;
memset(&gp, 0, sizeof gp);
gp.param = VIRTGPU_PARAM_SUPPORTED_CAPSET_IDs;
gp.value = (uint64_t)(uintptr_t)&mask;
int rc = drmIoctl(fd, DRM_IOCTL_VIRTGPU_GETPARAM, &gp);
close(fd);
if (rc) {
// Not a virtio-gpu at all, or a kernel without the param: say
// which, rather than printing an empty list that reads like "the
// host offers nothing".
printf("capsets: SUPPORTED_CAPSET_IDs unavailable: %s\n", strerror(errno));
return 1;
}
printf("capsets: bitmask 0x%llx\n", (unsigned long long)mask);
for (int i = 1; i <= 8; i++)
if (mask & (1ull << i)) printf(" %d: %s\n", i, capset_name(i));
if (!(mask & (1ull << 6)))
printf(" (no capset 6: native context needs host-side virglrenderer + qemu support)\n");
return 0;
}
static int devices(void) {
VkInstanceCreateInfo ici = {.sType = VK_STRUCTURE_TYPE_INSTANCE_CREATE_INFO};
VkInstance inst;
if (vkCreateInstance(&ici, NULL, &inst) != VK_SUCCESS) {
printf("vulkan: no instance -- the loader found no usable ICD\n");
return 1;
}
uint32_t n = 0;
vkEnumeratePhysicalDevices(inst, &n, NULL);
if (n == 0) {
// The exact state this VM was in for an hour on 2026-09-08.
printf("vulkan: instance ok but ZERO devices -- the host refused a context\n");
vkDestroyInstance(inst, NULL);
return 1;
}
VkPhysicalDevice pd[8];
if (n > 8) n = 8;
vkEnumeratePhysicalDevices(inst, &n, pd);
for (uint32_t i = 0; i < n; i++) {
VkPhysicalDeviceProperties p;
vkGetPhysicalDeviceProperties(pd[i], &p);
printf("vulkan: %s (api %u.%u.%u)\n", p.deviceName, VK_VERSION_MAJOR(p.apiVersion),
VK_VERSION_MINOR(p.apiVersion), VK_VERSION_PATCH(p.apiVersion));
printf(" compute: %u invocations/workgroup, size %u,%u,%u, %u bytes shared\n",
p.limits.maxComputeWorkGroupInvocations, p.limits.maxComputeWorkGroupSize[0],
p.limits.maxComputeWorkGroupSize[1], p.limits.maxComputeWorkGroupSize[2],
p.limits.maxComputeSharedMemorySize);
}
// The negative half of "wgpu's teardown crashes on Venus": five
// devices and the instance, created and destroyed the plain way.
float prio = 1.0f;
VkDeviceQueueCreateInfo q = {.sType = VK_STRUCTURE_TYPE_DEVICE_QUEUE_CREATE_INFO,
.queueFamilyIndex = 0, .queueCount = 1, .pQueuePriorities = &prio};
VkDeviceCreateInfo dci = {.sType = VK_STRUCTURE_TYPE_DEVICE_CREATE_INFO,
.queueCreateInfoCount = 1, .pQueueCreateInfos = &q};
for (int i = 0; i < 5; i++) {
VkDevice dev;
if (vkCreateDevice(pd[0], &dci, NULL, &dev) != VK_SUCCESS) {
printf("teardown: device %d could not be created\n", i);
return 1;
}
vkDestroyDevice(dev, NULL);
}
vkDestroyInstance(inst, NULL);
printf("teardown: 5 devices + instance created and destroyed cleanly\n");
return 0;
}
int main(void) {
int bad = capsets();
bad |= devices();
return bad;
}
+25
View File
@@ -0,0 +1,25 @@
#!/bin/sh
# Runs this repo's Rust tests. Extra arguments are forwarded to each
# `cargo test`, e.g. `scripts/run-tests.sh transcript` to run just the transcript
# tests in every workspace.
#
# Three workspaces, in dependency order:
#
# event-model the wire shape server/ and app-rust/ share, so the two
# agree by construction rather than by review
# server/ the backend's own logic (event normalization, transcript
# cursors, config persistence, token auth)
# app-rust/ the app itself: `client` (highlighter, ANSI parser,
# transcript cache and fold, REST and SSE clients -- see
# docs/CLIENT_CORE.md) and `ui` (the screens, drawn with
# iris), plus the headless harness tests over the bench
# fixture that need no window and no GPU.
#
# `iris/` is the UI framework and has its own tests; run them from there
# (`cd iris && cargo test`). They are not in this loop because iris is not
# about this product and its suite is the slower of the two.
set -eu
cd "$(dirname "$0")/.."
for workspace in event-model server app-rust; do
(cd "$workspace" && cargo test "$@")
done
+133
View File
@@ -0,0 +1,133 @@
#!/bin/sh
# Stands up a real WireGuard tunnel entirely inside this machine, so the
# server's production network posture -- "bind wg0 and nothing else, fail
# closed if it's missing" -- can be exercised without a phone, a router
# port-forward, or any internet exposure.
#
# The shape, all in one kernel:
#
# main netns "phone" netns
# wg0 10.66.0.1 <-- encrypted --> wg1 10.66.0.2
# | |
# veth-srv 10.99.0.1 <--- UDP ---> veth-phone 10.99.0.2
#
# The veth pair stands in for "the internet" carrying WireGuard's UDP; the
# wg interfaces are real, with a real handshake and real keys. 10.66.0.1 is
# deliberately the same address the leaf certificate carries a SAN for
# (certs.rs covers every local address), so a client inside the tunnel
# completes the same pinned-TLS handshake a phone will.
#
# scripts/test-wg-tunnel.sh up create the tunnel (needs sudo)
# scripts/test-wg-tunnel.sh test run the server on wg0 and reach it from "phone"
# scripts/test-wg-tunnel.sh down remove everything it created
#
# Everything here is torn down by `down`: the netns (taking wg1 and the veth
# peer with it), wg0, and the temporary key files.
set -eu
NS=phone
WG_SERVER=wg0
WG_CLIENT=wg1
SERVER_WG_IP=10.66.0.1
CLIENT_WG_IP=10.66.0.2
SERVER_UDP_IP=10.99.0.1
CLIENT_UDP_IP=10.99.0.2
LISTEN_PORT=51820
KEYDIR=/run/ai-app-wg-test
# This script lives in scripts/; everything it names is under the root.
REPO=$(cd "$(dirname "$0")/.." && pwd)
up() {
echo "==> Generating ephemeral keypairs in $KEYDIR"
sudo mkdir -p "$KEYDIR"
sudo sh -c "umask 077; wg genkey > $KEYDIR/server.key; wg genkey > $KEYDIR/client.key"
sudo sh -c "wg pubkey < $KEYDIR/server.key > $KEYDIR/server.pub"
sudo sh -c "wg pubkey < $KEYDIR/client.key > $KEYDIR/client.pub"
echo "==> Creating netns '$NS' and the veth pair that carries the UDP"
sudo ip netns add "$NS"
sudo ip link add veth-srv type veth peer name veth-phone
sudo ip link set veth-phone netns "$NS"
sudo ip addr add "$SERVER_UDP_IP/24" dev veth-srv
sudo ip link set veth-srv up
sudo ip -n "$NS" addr add "$CLIENT_UDP_IP/24" dev veth-phone
sudo ip -n "$NS" link set veth-phone up
sudo ip -n "$NS" link set lo up
echo "==> Creating $WG_SERVER (server side, $SERVER_WG_IP)"
sudo ip link add "$WG_SERVER" type wireguard
sudo sh -c "wg set $WG_SERVER listen-port $LISTEN_PORT private-key $KEYDIR/server.key \
peer \$(cat $KEYDIR/client.pub) allowed-ips $CLIENT_WG_IP/32"
sudo ip addr add "$SERVER_WG_IP/24" dev "$WG_SERVER"
sudo ip link set "$WG_SERVER" up
# Created in the main namespace, then moved: a wireguard interface keeps
# its UDP socket in the namespace it was born in, which is exactly what
# lets the "phone" reach the server's veth address from inside its own.
echo "==> Creating $WG_CLIENT (phone side, $CLIENT_WG_IP) in netns '$NS'"
sudo ip link add "$WG_CLIENT" type wireguard
sudo ip link set "$WG_CLIENT" netns "$NS"
sudo ip netns exec "$NS" sh -c "wg set $WG_CLIENT private-key $KEYDIR/client.key \
peer \$(cat $KEYDIR/server.pub) allowed-ips $SERVER_WG_IP/32 \
endpoint $SERVER_UDP_IP:$LISTEN_PORT persistent-keepalive 5"
sudo ip -n "$NS" addr add "$CLIENT_WG_IP/24" dev "$WG_CLIENT"
sudo ip -n "$NS" link set "$WG_CLIENT" up
echo "==> Forcing a handshake"
sudo ip netns exec "$NS" ping -c 2 -W 3 "$SERVER_WG_IP" >/dev/null 2>&1 || true
sudo wg show "$WG_SERVER" | sed 's/^/ /'
echo "==> Up. wg0 is $SERVER_WG_IP; run '$0 test' next."
}
test_tunnel() {
CERTS="${XDG_CONFIG_HOME:-$HOME/.config}/ai-app/certs"
if [ ! -f "$CERTS/leaf.pem" ]; then
echo "No certificates in $CERTS -- start ai-server once; it makes them." >&2
exit 1
fi
if [ ! -x "$REPO/server/target/debug/ai-server" ]; then
echo "Build the server first: (cd server && cargo build)" >&2
exit 1
fi
echo "==> Starting ai-server with NO --bind (production path: wg0 only)"
setsid nohup "$REPO/server/target/debug/ai-server" \
</dev/null >"$REPO/server/wg-test.log" 2>&1 &
sleep 2
echo "==> Where is it actually listening?"
ss -tlnp 2>/dev/null | grep 8443 | sed 's/^/ /' || echo " (nothing on 8443)"
echo "==> From inside the tunnel: GET /sessions through wg1 -> wg0"
if [ -z "${AI_TOKEN:-}" ]; then
echo " (set AI_TOKEN=<the enrollment token> to test an authorized call;"
echo " without it this only proves reachability + TLS, via a 401)"
fi
sudo ip netns exec "$NS" curl -s -o /dev/null -w " HTTP %{http_code} (TLS ok, pinned CA)\n" \
--cacert "$CERTS/ca.pem" \
${AI_TOKEN:+-H "Authorization: Bearer $AI_TOKEN"} \
"https://$SERVER_WG_IP:8443/sessions" || echo " UNREACHABLE"
echo "==> Handshake counters (proves the traffic really crossed WireGuard)"
sudo wg show "$WG_SERVER" transfer | sed 's/^/ /'
pkill -f "[a]i-server" || true
echo "==> Server stopped."
}
down() {
echo "==> Removing tunnel"
sudo ip netns del "$NS" 2>/dev/null || true
sudo ip link del "$WG_SERVER" 2>/dev/null || true
sudo ip link del veth-srv 2>/dev/null || true
sudo rm -rf "$KEYDIR"
echo "==> Down."
}
case "${1:-}" in
up) up ;;
test) test_tunnel ;;
down) down ;;
*) echo "usage: $0 up|test|down" >&2; exit 1 ;;
esac
+154
View File
@@ -0,0 +1,154 @@
#!/bin/sh
# Sets up the WireGuard tunnel on the BACKEND HOST -- the machine that runs
# ai-server and that the phone dials in to. Run this on the host, not in the
# dev VM (the VM is behind qemu user-mode networking and has no inbound path;
# see AGENTS.md).
#
# sudo WG_ENDPOINT=your-name.duckdns.org scripts/wg-setup-host.sh
#
# What it creates:
# /etc/wireguard/wg0.conf the backend's tunnel: 10.66.0.1, port 51820
# /etc/wireguard/peers/phone.conf the phone's config, shown as a QR to scan
# and brings the interface up with wg-quick. Making it come back after a
# reboot is left to you: that is the one step whose commands differ per init
# system, and this script would only be guessing (the backend host is Gentoo,
# the dev VM is Arch). It prints what to run at the end.
#
# Addressing matches PLAN.md: the phone reaches the backend at 10.66.0.1 from
# everywhere, home or away -- one address in the app, one SAN in the leaf
# certificate, no home/away distinction. The phone's AllowedIPs is only
# 10.66.0.0/24, so this is a split tunnel: the phone's other traffic does not
# route through your house, and nothing here forwards or NATs.
#
# Re-running is safe: existing keys are reused, so the phone's config stays
# valid. Pass WG_NEW_PHONE_KEY=1 to issue a fresh phone keypair, which
# invalidates the old one.
#
# The one thing this cannot do for you: forward UDP 51820 from your router to
# this host. That is the only internet-facing hole, and it is silent to
# unauthenticated packets -- scanners see a closed port.
set -eu
WG_DIR=/etc/wireguard
PEER_DIR="$WG_DIR/peers"
SERVER_IP=10.66.0.1
PHONE_IP=10.66.0.2
SUBNET=10.66.0.0/24
PORT="${WG_PORT:-51820}"
ENDPOINT="${WG_ENDPOINT:-}"
if [ "$(id -u)" -ne 0 ]; then
echo "Run this with sudo -- it writes $WG_DIR and enables a service." >&2
exit 1
fi
for tool in wg wg-quick; do
command -v "$tool" >/dev/null || { echo "$tool not found: install wireguard-tools." >&2; exit 1; }
done
if [ -z "$ENDPOINT" ]; then
echo "Set WG_ENDPOINT to the hostname the phone should dial from outside," >&2
echo "e.g. WG_ENDPOINT=your-name.duckdns.org (a DDNS name, since a home IP" >&2
echo "can change). Then re-run." >&2
exit 1
fi
umask 077
mkdir -p "$PEER_DIR"
# Keys are generated here and never leave, except the phone's -- which is
# what the QR carries. Regenerating the server key would invalidate every
# peer, so it is created once and then reused.
if [ ! -f "$WG_DIR/server.key" ]; then
echo "==> Generating the backend's keypair"
wg genkey > "$WG_DIR/server.key"
wg pubkey < "$WG_DIR/server.key" > "$WG_DIR/server.pub"
else
echo "==> Reusing the backend's existing keypair"
fi
if [ ! -f "$PEER_DIR/phone.key" ] || [ -n "${WG_NEW_PHONE_KEY:-}" ]; then
echo "==> Generating the phone's keypair"
wg genkey > "$PEER_DIR/phone.key"
wg pubkey < "$PEER_DIR/phone.key" > "$PEER_DIR/phone.pub"
else
echo "==> Reusing the phone's existing keypair"
fi
echo "==> Writing $WG_DIR/wg0.conf"
cat > "$WG_DIR/wg0.conf" <<EOF
# Generated by ai-app/scripts/wg-setup-host.sh. The backend binds this interface's
# address and refuses to start without it (see server/src/main.rs).
[Interface]
Address = $SERVER_IP/24
ListenPort = $PORT
PrivateKey = $(cat "$WG_DIR/server.key")
[Peer]
# phone
PublicKey = $(cat "$PEER_DIR/phone.pub")
AllowedIPs = $PHONE_IP/32
EOF
echo "==> Writing $PEER_DIR/phone.conf"
cat > "$PEER_DIR/phone.conf" <<EOF
[Interface]
Address = $PHONE_IP/24
PrivateKey = $(cat "$PEER_DIR/phone.key")
[Peer]
PublicKey = $(cat "$WG_DIR/server.pub")
Endpoint = $ENDPOINT:$PORT
# Split tunnel: only the backend's subnet goes over WireGuard.
AllowedIPs = $SUBNET
# Keeps the mapping alive through home NAT so the backend can reach the
# phone first (needed later for "your turn" push).
PersistentKeepalive = 25
EOF
if wg show wg0 >/dev/null 2>&1; then
# Already up: load the new peers without dropping the interface, so a
# re-run doesn't kill a connected phone mid-session. `wg-quick strip`
# prints the config with the wg-quick-only keys removed, which is what
# `wg syncconf` accepts.
echo "==> wg0 is already up -- reloading its peers in place"
STRIPPED=$(mktemp)
trap 'rm -f "$STRIPPED"' EXIT
wg-quick strip wg0 > "$STRIPPED"
wg syncconf wg0 "$STRIPPED"
else
echo "==> Bringing wg0 up"
wg-quick up wg0
fi
sleep 1
wg show wg0 | sed 's/^/ /'
echo
echo "==> Phone config -- scan this with the WireGuard app (Add > Scan from QR code):"
echo
if command -v qrencode >/dev/null; then
qrencode -t ansiutf8 < "$PEER_DIR/phone.conf"
else
echo " (install qrencode to get a scannable QR; the config is below)"
sed 's/^/ /' "$PEER_DIR/phone.conf"
fi
echo
echo "Still to do, in order:"
echo " 0. Make wg0 come back after a reboot. Left to you rather than"
echo " guessed at, since the command depends on your init system:"
echo " OpenRC: ln -s /etc/init.d/wg-quick /etc/init.d/wg-quick.wg0"
echo " rc-update add wg-quick.wg0 default"
echo " systemd: systemctl enable wg-quick@wg0"
echo " (Gentoo with netifrc instead of wg-quick: configure net.wg0 in"
echo " /etc/conf.d/net -- see the WireGuard page on the Gentoo wiki.)"
echo " 1. Forward UDP $PORT on your router to this host. That is the only"
echo " internet-facing port; it stays silent to unauthenticated packets."
echo " 2. Point $ENDPOINT at your home IP (DDNS client on the router, or a"
echo " curl cron here). WireGuard on the phone resolves this once when the"
echo " tunnel comes up, so after a rare IP change, toggle the tunnel."
echo " 3. Check NAT hairpinning works at home: with the tunnel on and the"
echo " phone on your wifi, 'ping $SERVER_IP' from the phone should answer."
echo " If it doesn't, your router can't hairpin -- turn the tunnel off at"
echo " home, or use a split-DNS entry pointing $ENDPOINT at the LAN IP."
echo " 4. Start the backend here (it binds $SERVER_IP only, and refuses to"
echo " start if wg0 is down):"
echo " cd $(dirname "$(dirname "$(readlink -f "$0")")") && ./server/target/release/ai-server"
echo " Add --rotate-token once to print a fresh enrollment QR for the app."
+7
View File
@@ -0,0 +1,7 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "xtask"
version = "0.1.0"
+16
View File
@@ -0,0 +1,16 @@
[package]
name = "xtask"
version = "0.1.0"
edition = "2024"
# E5 (RUST.md): packages app/shellApp into a signed, installable APK without
# Gradle driving the assembly (cargo ndk -> javac -> d8 -> aapt2 -> zipalign
# -> apksigner). No dependencies beyond the standard library: every step
# below is "run this SDK tool with these arguments and check its exit
# status," which needs nothing a crate would help with, and every tool
# invoked is one this project already requires (the NDK, the SDK
# build-tools, the JDK, `cargo ndk`) -- see AGENTS.md's "new dependencies
# need a reason."
[[bin]]
name = "xtask"
path = "src/main.rs"
+510
View File
@@ -0,0 +1,510 @@
//! The pipeline itself: `cargo ndk` -> `javac`/`d8` -> `aapt2` ->
//! `zipalign` -> `apksigner`, with no Gradle driving *this* file's steps.
//!
//! **One disclosed exception**, recorded here rather than left to be
//! rediscovered: step 3 below still runs `./gradlew
//! :shellApp:printRuntimeClasspathJars` once, because `app/shellApp`
//! depends on the `:link` submodule (Kotlin: `ServerStore`/`ServerSettings`,
//! the Keystore-sealed enrollment, RUST.md's E3 entry explains why that
//! code is reused rather than re-derived in Rust) and on
//! `androidx.core:core-ktx` (used at runtime through JNI by
//! the shell bridge's `notify.rs`, for `NotificationCompat` and friends).
//! Both are ordinary Maven/AAR dependency graphs, and reimplementing a
//! dependency resolver to avoid one Gradle invocation was not a good trade
//! against "smallest honest route" (RUST.md's E5 box) -- especially since
//! that one call also compiles `:link`'s Kotlin as a side effect, using
//! Gradle's own embedded Kotlin compiler. This machine has no standalone
//! `kotlinc` (checked: not on PATH, not under any SDK), so that side
//! effect is what answers E3's open question about `kotlinc` -- see
//! RUST.md's E5 entry for the full account. Nothing past this one call
//! touches Gradle: `javac`, `d8`, `aapt2`, `zipalign` and `apksigner` are
//! invoked directly, and the jars this call resolves are consumed as
//! plain binary inputs to `d8`, exactly like any other pre-built `.jar`.
use std::path::{Path, PathBuf};
use std::process::Command;
use crate::keystore::{self, Signer};
use crate::sdk::{self, Sdk};
use crate::{Fail, Variant};
const APPLICATION_ID: &str = "com.example.aiapp.shell";
pub fn build(variant: Variant, abis: &[String]) -> Result<PathBuf, Fail> {
let repo_root = repo_root()?;
let app_dir = repo_root.join("app");
let shell_app_dir = app_dir.join("shellApp");
// The JNI bridge is the `shell` feature of the one app crate now, not
// a crate of its own -- `--no-default-features` is what keeps iris,
// wgpu and parley out of a `.so` for an app that draws with Compose.
let app_crate_dir = repo_root.join("app-rust");
let sdk = sdk::find()?;
sdk::require_ndk_installed(&sdk.root)?;
require_cargo_ndk()?;
// xtask's own intermediate files, in its own crate's target/ rather
// than a `target/` at the repo root -- there is no workspace there and
// a build directory in the root is not something anybody was looking
// for (Iris, 2026-09-09).
let out_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"))
.join("target")
.join("apk");
std::fs::create_dir_all(&out_dir).map_err(|e| {
Fail::new(
"could not create the xtask output directory",
&e.to_string(),
"check permissions under scripts/xtask/target/",
)
})?;
println!("==> Building the shell JNI bridge for {}", abis.join(", "));
build_native_libs(&app_crate_dir, &shell_app_dir, &sdk, abis)?;
println!("==> Resolving the runtime classpath (one Gradle call -- see apk.rs's module doc)");
let classpath_jars = runtime_classpath_jars(&app_dir, &sdk)?;
println!("==> Compiling the Java stub classes");
let ca_pem = pinned_ca_pem()?;
let classes_jar = compile_java(&out_dir, &shell_app_dir, &sdk, &ca_pem)?;
println!("==> Dexing");
let dex_dir = out_dir.join("dex");
dex(&sdk, &classes_jar, &classpath_jars, &dex_dir)?;
println!("==> Linking resources with aapt2");
let base_apk = out_dir.join("base.apk");
aapt2_link(&sdk, &shell_app_dir, &base_apk)?;
println!("==> Merging dex and native libraries");
let merged_apk = out_dir.join("merged.apk");
merge(&base_apk, &dex_dir, &shell_app_dir, abis, &merged_apk)?;
println!(
"==> Aligning and signing ({})",
match variant {
Variant::Release => "release key",
Variant::Debug => "debug key",
}
);
let signer = match variant {
Variant::Release => keystore::release_signer()?,
Variant::Debug => keystore::debug_signer()?,
};
let variant_name = match variant {
Variant::Release => "release",
Variant::Debug => "debug",
};
let signed_apk = out_dir.join(format!("ai-app-shell-{variant_name}.apk"));
align_and_sign(&sdk, &merged_apk, &signed_apk, &signer)?;
// Copied into a Gradle-shaped path (`build/outputs/apk/<mode>/*.apk`
// under this xtask's own directory) as the final step, purely so Dev
// Updater's fixed-pattern APK discovery (`discover.rs`'s
// `APK_PATTERNS`, which has no per-component path override) finds it
// without needing a change on that side -- `.dev-updater.ron`'s
// `shell` component points its `cwd` here. The working files above
// stay under `scripts/xtask/target/apk/`, an ordinary build-cache location.
// `scripts/build/...`, not `scripts/xtask/build/...`: Dev Updater
// discovers APKs with `*/build/outputs/apk/*/*.apk` from the checkout
// root, which is exactly one directory deep. See `.dev-updater.ron`.
let published_dir = repo_root
.join("scripts/build/outputs/apk")
.join(variant_name);
std::fs::create_dir_all(&published_dir).map_err(|e| {
Fail::new(
"could not create the published APK directory",
&e.to_string(),
"check permissions under scripts/build",
)
})?;
let published_apk = published_dir.join(format!("ai-app-shell-{variant_name}.apk"));
std::fs::copy(&signed_apk, &published_apk).map_err(|e| {
Fail::new(
"could not publish the signed APK",
&e.to_string(),
"check permissions under scripts/build",
)
})?;
Ok(published_apk)
}
fn repo_root() -> Result<PathBuf, Fail> {
// xtask's own Cargo.toml is at <repo_root>/scripts/xtask/Cargo.toml,
// so the root is two levels up (2026-09-09: it used to be one).
let manifest_dir = PathBuf::from(env!("CARGO_MANIFEST_DIR"));
manifest_dir
.parent()
.and_then(Path::parent)
.map(Path::to_path_buf)
.ok_or_else(|| {
Fail::new(
"could not find the repo root",
"CARGO_MANIFEST_DIR has no grandparent",
"run through cargo, not by hand",
)
})
}
fn require_cargo_ndk() -> Result<(), Fail> {
run_checked(
Command::new("cargo").args(["ndk", "--version"]),
"cargo-ndk is not installed",
"cargo install cargo-ndk",
)
.map(|_| ())
}
/// `cargo ndk`'s `-t` target name for each ABI, and `-P 26` -- the API
/// level every other cross-compile in this repo uses (RUST.md: E0, E1, E3,
/// I2), kept consistent here rather than picked fresh.
fn build_native_libs(
crate_dir: &Path,
shell_app_dir: &Path,
sdk: &Sdk,
abis: &[String],
) -> Result<(), Fail> {
let jni_libs = shell_app_dir.join("src/main/jniLibs");
let mut cmd = Command::new("cargo");
cmd.current_dir(crate_dir);
cmd.arg("ndk");
for abi in abis {
cmd.args(["-t", abi]);
}
cmd.args(["-P", "26", "-o"]).arg(&jni_libs);
// Always the release profile for the native library, independent of
// the APK's signing variant -- a debug build's Vulkan object-labelling
// segfaults this emulator's driver (RUST.md's E1 entry), and there is
// no reason for this crate's debug build to be bigger or slower for a
// signing choice that has nothing to do with it.
cmd.args([
"build",
"--release",
"--lib",
"--no-default-features",
"--features",
"shell",
]);
cmd.env("ANDROID_HOME", &sdk.root);
cmd.env("ANDROID_SDK_ROOT", &sdk.root);
run_checked(
&mut cmd,
"cargo ndk build failed",
"see the compiler output above",
)
.map(|_| ())
}
fn runtime_classpath_jars(app_dir: &Path, sdk: &Sdk) -> Result<Vec<PathBuf>, Fail> {
let mut cmd = Command::new(app_dir.join("gradlew"));
cmd.current_dir(app_dir);
cmd.args(["--console=plain", ":shellApp:printRuntimeClasspathJars"]);
cmd.env("ANDROID_HOME", &sdk.root);
cmd.env("ANDROID_SDK_ROOT", &sdk.root);
run_checked(
&mut cmd,
"resolving app/shellApp's dependencies with Gradle failed",
"see the Gradle output above",
)?;
let list_file = app_dir.join("shellApp/build/xtask/runtime-classpath.txt");
let contents = std::fs::read_to_string(&list_file).map_err(|e| {
Fail::new(
"printRuntimeClasspathJars did not produce its output file",
&format!("{}: {e}", list_file.display()),
"check app/shellApp/build.gradle.kts's printRuntimeClasspathJars task",
)
})?;
Ok(contents
.lines()
.filter(|l| !l.is_empty())
.map(PathBuf::from)
.collect())
}
/// The CA this build pins, found the same way `build-apk.sh` and
/// `androidApp`/`shellApp`'s Gradle `generatePinnedCa` tasks do:
/// `$AI_APP_CA`, else `$XDG_CONFIG_HOME/ai-app/certs/ca.pem`.
fn pinned_ca_pem() -> Result<String, Fail> {
let path = std::env::var_os("AI_APP_CA")
.map(PathBuf::from)
.unwrap_or_else(|| {
let config_home = std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.unwrap_or_else(|| {
PathBuf::from(std::env::var_os("HOME").unwrap()).join(".config")
});
config_home.join("ai-app").join("certs").join("ca.pem")
});
let pem = std::fs::read_to_string(&path).map_err(|e| {
Fail::new(
&format!("no CA certificate at {}", path.display()),
&e.to_string(),
"start ai-server (or app/ui-sandbox.sh) once on this machine first -- it generates the CA this build pins",
)
})?;
let pem = pem.trim().to_string();
if !pem.starts_with("-----BEGIN CERTIFICATE-----") {
return Err(Fail::new(
&format!("{} is not a PEM certificate", path.display()),
"missing the BEGIN CERTIFICATE header",
"point AI_APP_CA at a valid one",
));
}
Ok(pem)
}
fn compile_java(
out_dir: &Path,
shell_app_dir: &Path,
sdk: &Sdk,
ca_pem: &str,
) -> Result<PathBuf, Fail> {
let gen_dir = out_dir.join("generated-java");
let package_dir = gen_dir.join("com/example/aiapp/shell");
std::fs::create_dir_all(&package_dir).map_err(|e| {
Fail::new(
"could not create the generated-sources directory",
&e.to_string(),
"check permissions under scripts/xtask/target/",
)
})?;
// Same shape as shellApp's Gradle `generatePinnedCa` task: the text
// block must start immediately after the opening `"""`, or
// CertificateFactory stops recognising the "-----BEGIN" preamble (a
// real bug this project hit once -- see AGENTS.md's "Things that have
// bitten").
let pinned_ca_java = format!(
"package com.example.aiapp.shell;\n\npublic final class PinnedCa {{\n private PinnedCa() {{}}\n public static final String PINNED_CA_PEM = \"\"\"\n{ca_pem}\"\"\";\n}}\n"
);
std::fs::write(package_dir.join("PinnedCa.java"), pinned_ca_java).map_err(|e| {
Fail::new(
"could not write PinnedCa.java",
&e.to_string(),
"check permissions under scripts/xtask/target/",
)
})?;
let classes_dir = out_dir.join("classes");
std::fs::create_dir_all(&classes_dir).map_err(|e| {
Fail::new(
"could not create the classes directory",
&e.to_string(),
"check permissions under scripts/xtask/target/",
)
})?;
let java_dir = shell_app_dir.join("src/main/java/com/example/aiapp/shell");
let mut cmd = Command::new("javac");
cmd.args(["-cp"]).arg(&sdk.android_jar);
cmd.args(["-d"]).arg(&classes_dir);
cmd.arg(java_dir.join("MainActivity.java"));
cmd.arg(java_dir.join("NotificationService.java"));
cmd.arg(package_dir.join("PinnedCa.java"));
run_checked(&mut cmd, "javac failed", "see the compiler output above")?;
let classes_jar = out_dir.join("classes.jar");
let mut cmd = Command::new("jar");
cmd.current_dir(&classes_dir);
cmd.args(["cf"])
.arg(&classes_jar)
.args(["-C", "."])
.arg(".");
run_checked(
&mut cmd,
"jar failed to package the compiled classes",
"see the output above",
)?;
Ok(classes_jar)
}
fn dex(
sdk: &Sdk,
classes_jar: &Path,
classpath_jars: &[PathBuf],
dex_dir: &Path,
) -> Result<(), Fail> {
std::fs::create_dir_all(dex_dir).map_err(|e| {
Fail::new(
"could not create the dex output directory",
&e.to_string(),
"check permissions under scripts/xtask/target/",
)
})?;
let mut cmd = Command::new(sdk.tool("d8"));
cmd.args(["--release", "--min-api"])
.arg(sdk::MIN_SDK.to_string());
cmd.arg("--lib").arg(&sdk.android_jar);
cmd.arg("--output").arg(dex_dir);
cmd.arg(classes_jar);
cmd.args(classpath_jars);
run_checked(&mut cmd, "d8 failed", "see the compiler output above").map(|_| ())
}
fn aapt2_link(sdk: &Sdk, shell_app_dir: &Path, base_apk: &Path) -> Result<(), Fail> {
let manifest_src = shell_app_dir.join("src/main/AndroidManifest.xml");
let manifest_text = std::fs::read_to_string(&manifest_src).map_err(|e| {
Fail::new(
"could not read the manifest",
&format!("{}: {e}", manifest_src.display()),
"check app/shellApp/src/main/AndroidManifest.xml",
)
})?;
// The checked-in manifest has no `package` attribute -- Gradle injects
// it from `android.namespace` during its own manifest merge, which
// this pipeline does not run. aapt2 needs it to know what package to
// generate resources under.
if manifest_text.contains("package=") {
return Err(Fail::new(
"app/shellApp's manifest already has a package attribute",
"aapt2_link() assumes it doesn't and injects one",
"update aapt2_link() in scripts/xtask/src/apk.rs to stop injecting a second one",
));
}
let merged_manifest = manifest_text.replacen(
"<manifest ",
&format!("<manifest package=\"{APPLICATION_ID}\" "),
1,
);
let merged_manifest_path = base_apk.with_file_name("AndroidManifest.merged.xml");
std::fs::write(&merged_manifest_path, merged_manifest).map_err(|e| {
Fail::new(
"could not write the merged manifest",
&e.to_string(),
"check permissions under scripts/xtask/target/",
)
})?;
let mut cmd = Command::new(sdk.tool("aapt2"));
cmd.args(["link", "-o"]).arg(base_apk);
cmd.args(["--manifest"]).arg(&merged_manifest_path);
cmd.arg("-I").arg(&sdk.android_jar);
cmd.args(["--min-sdk-version", &sdk::MIN_SDK.to_string()]);
cmd.args(["--target-sdk-version", &sdk::COMPILE_SDK.to_string()]);
cmd.args(["--version-code", "1", "--version-name", "1.0"]);
run_checked(&mut cmd, "aapt2 link failed", "see the output above").map(|_| ())
}
fn merge(
base_apk: &Path,
dex_dir: &Path,
shell_app_dir: &Path,
abis: &[String],
merged_apk: &Path,
) -> Result<(), Fail> {
std::fs::copy(base_apk, merged_apk).map_err(|e| {
Fail::new(
"could not copy the base APK",
&e.to_string(),
"check permissions under scripts/xtask/target/",
)
})?;
let mut cmd = Command::new("jar");
cmd.current_dir(dex_dir);
cmd.args(["uf"])
.arg(std::path::absolute(merged_apk).unwrap_or_else(|_| merged_apk.to_path_buf()));
cmd.args(["classes.dex"]);
run_checked(
&mut cmd,
"jar failed to add classes.dex to the APK",
"see the output above",
)?;
// Android's zip layout wants "lib/<abi>/*.so" at the archive root, but
// cargo ndk's `-o` wrote "jniLibs/<abi>/*.so" (matching the Gradle
// source-set layout it was pointed at) -- so this stages a "lib/"
// directory rather than trying to rename inside the zip.
let stage = merged_apk.with_file_name("lib-stage");
if stage.exists() {
std::fs::remove_dir_all(&stage).ok();
}
for abi in abis {
let so_name = "libai_app.so";
let src = shell_app_dir
.join("src/main/jniLibs")
.join(abi)
.join(so_name);
if !src.is_file() {
return Err(Fail::new(
&format!("no native library built for {abi}"),
&format!("expected {}", src.display()),
"check cargo ndk's output above for that ABI",
));
}
let dest_dir = stage.join("lib").join(abi);
std::fs::create_dir_all(&dest_dir).map_err(|e| {
Fail::new(
"could not stage the native library",
&e.to_string(),
"check permissions under scripts/xtask/target/",
)
})?;
std::fs::copy(&src, dest_dir.join(so_name)).map_err(|e| {
Fail::new(
"could not stage the native library",
&e.to_string(),
"check permissions under scripts/xtask/target/",
)
})?;
}
let mut cmd = Command::new("jar");
cmd.current_dir(&stage);
cmd.args(["uf"])
.arg(std::path::absolute(merged_apk).unwrap_or_else(|_| merged_apk.to_path_buf()));
cmd.arg("lib");
run_checked(
&mut cmd,
"jar failed to add the native libraries to the APK",
"see the output above",
)
.map(|_| ())
}
fn align_and_sign(
sdk: &Sdk,
merged_apk: &Path,
final_apk: &Path,
signer: &Signer,
) -> Result<(), Fail> {
let aligned_apk = merged_apk.with_file_name("aligned.apk");
let mut cmd = Command::new(sdk.tool("zipalign"));
cmd.args(["-f", "-p", "4"])
.arg(merged_apk)
.arg(&aligned_apk);
run_checked(&mut cmd, "zipalign failed", "see the output above")?;
let mut cmd = Command::new(sdk.tool("apksigner"));
cmd.args(["sign", "--ks"]).arg(&signer.keystore);
cmd.arg("--ks-pass")
.arg(format!("pass:{}", signer.password));
cmd.arg("--ks-key-alias").arg(&signer.alias);
cmd.arg("--out").arg(final_apk);
cmd.arg(&aligned_apk);
run_checked(
&mut cmd,
"apksigner failed to sign the APK",
"see the output above",
)
.map(|_| ())
}
fn run_checked(cmd: &mut Command, what: &str, fix: &str) -> Result<(), Fail> {
let status = cmd.status().map_err(|e| {
Fail::new(
what,
&format!("could not run {:?}: {e}", cmd.get_program()),
fix,
)
})?;
if status.success() {
Ok(())
} else {
Err(Fail::new(
what,
&format!("{:?} exited with {status}", cmd.get_program()),
fix,
))
}
}
+254
View File
@@ -0,0 +1,254 @@
//! The signing key. Mirrors `app/build-apk.sh`'s exact logic for the
//! release key -- same env vars, same path, same generation recipe -- so
//! the two tools sign with the *same* key and their outputs can
//! `adb install -r` over each other. That is the whole point of E5's pass
//! condition: the key has to be identical, not merely present.
use std::path::PathBuf;
use std::process::Command;
use crate::Fail;
pub struct Signer {
pub keystore: PathBuf,
pub password: String,
pub alias: String,
}
/// The release key at `$AI_APP_KEYSTORE` or
/// `$XDG_CONFIG_HOME/ai-app/release.jks` (`~/.config/ai-app/release.jks` by
/// default) -- generated with `keytool` if it doesn't exist yet, exactly as
/// `build-apk.sh` does, so either tool can run first on a fresh machine.
pub fn release_signer() -> Result<Signer, Fail> {
let keystore = std::env::var_os("AI_APP_KEYSTORE")
.map(PathBuf::from)
.unwrap_or_else(|| {
let config_home = std::env::var_os("XDG_CONFIG_HOME")
.map(PathBuf::from)
.unwrap_or_else(|| {
PathBuf::from(std::env::var_os("HOME").unwrap()).join(".config")
});
config_home.join("ai-app").join("release.jks")
});
let alias = "ai-app".to_string();
let password_file = keystore.with_extension("jks.password");
if keystore.is_file() {
let password = std::fs::read_to_string(&password_file)
.map_err(|e| {
Fail::new(
"release key exists but its password file is unreadable",
&format!("{}: {e}", password_file.display()),
"restore the password file, or delete both and let this regenerate them",
)
})?
.trim()
.to_string();
return Ok(Signer {
keystore,
password,
alias,
});
}
let keytool = which_keytool()?;
if let Some(parent) = keystore.parent() {
std::fs::create_dir_all(parent).map_err(|e| {
Fail::new(
"could not create the keystore's directory",
&format!("{}: {e}", parent.display()),
"check permissions on that path",
)
})?;
}
let password = random_password();
write_owner_only(&password_file, format!("{password}\n").as_bytes())?;
let status = Command::new(&keytool)
.args(["-genkeypair", "-keystore"])
.arg(&keystore)
.args([
"-alias",
&alias,
"-keyalg",
"RSA",
"-keysize",
"2048",
"-validity",
"10000",
])
.args(["-storepass", &password, "-keypass", &password])
.args(["-dname", "CN=ai-app"])
.status()
.map_err(|e| {
Fail::new(
"failed to run keytool",
&format!("{}: {e}", keytool.display()),
"set JAVA_HOME to the JDK Gradle uses",
)
})?;
if !status.success() {
return Err(Fail::new(
"keytool exited with an error while generating the release key",
&format!("status: {status}"),
"check the keytool output above",
));
}
// Owner-only, matching build-apk.sh -- this key is what the phone
// recognises the app by, so it never goes in the repo and it stays
// unreadable to anything else on this machine.
set_owner_only(&keystore)?;
Ok(Signer {
keystore,
password,
alias,
})
}
/// The conventional Android debug key (`~/.android/debug.keystore`,
/// well-known password `android`, alias `androiddebugkey`) -- generated on
/// first use exactly the way Android Studio and Gradle's own debug signing
/// config do, so a `--debug` build here needs no setup and never touches
/// the real release key.
pub fn debug_signer() -> Result<Signer, Fail> {
let home = PathBuf::from(std::env::var_os("HOME").ok_or_else(|| {
Fail::new(
"no $HOME set",
"the debug keystore lives under ~/.android",
"set $HOME",
)
})?);
let keystore = home.join(".android").join("debug.keystore");
let alias = "androiddebugkey".to_string();
let password = "android".to_string();
if !keystore.is_file() {
let keytool = which_keytool()?;
std::fs::create_dir_all(keystore.parent().unwrap()).map_err(|e| {
Fail::new(
"could not create ~/.android",
&format!("{e}"),
"check permissions on your home directory",
)
})?;
let status = Command::new(&keytool)
.args(["-genkeypair", "-keystore"])
.arg(&keystore)
.args([
"-alias",
&alias,
"-keyalg",
"RSA",
"-keysize",
"2048",
"-validity",
"10000",
])
.args(["-storepass", &password, "-keypass", &password])
.args(["-dname", "CN=Android Debug,O=Android,C=US"])
.status()
.map_err(|e| {
Fail::new(
"failed to run keytool",
&format!("{}: {e}", keytool.display()),
"set JAVA_HOME to the JDK Gradle uses",
)
})?;
if !status.success() {
return Err(Fail::new(
"keytool exited with an error while generating the debug key",
&format!("status: {status}"),
"check the keytool output above",
));
}
}
Ok(Signer {
keystore,
password,
alias,
})
}
fn which_keytool() -> Result<PathBuf, Fail> {
if let Some(java_home) = std::env::var_os("JAVA_HOME") {
let candidate = PathBuf::from(java_home).join("bin").join("keytool");
if candidate.is_file() {
return Ok(candidate);
}
}
if Command::new("keytool").arg("-help").output().is_ok() {
return Ok(PathBuf::from("keytool"));
}
Err(Fail::new(
"no keytool available to generate the release key",
"checked $JAVA_HOME/bin/keytool and keytool on PATH",
"set JAVA_HOME to the JDK Gradle uses, or set AI_APP_KEYSTORE to an existing key",
))
}
fn random_password() -> String {
// No dependency on `rand`: /dev/urandom is what build-apk.sh's `head -c
// 24 /dev/urandom | base64` reads too, so this reproduces exactly the
// same recipe without shelling out to head/base64/tr for it.
let mut bytes = [0u8; 24];
std::fs::File::open("/dev/urandom")
.and_then(|mut f| std::io::Read::read_exact(&mut f, &mut bytes))
.expect("/dev/urandom must be readable to generate a signing key password");
base64_no_padding(&bytes)
}
fn base64_no_padding(bytes: &[u8]) -> String {
const ALPHABET: &[u8] = b"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/";
let mut out = String::new();
for chunk in bytes.chunks(3) {
let b0 = chunk[0] as u32;
let b1 = *chunk.get(1).unwrap_or(&0) as u32;
let b2 = *chunk.get(2).unwrap_or(&0) as u32;
let n = (b0 << 16) | (b1 << 8) | b2;
out.push(ALPHABET[(n >> 18 & 0x3f) as usize] as char);
out.push(ALPHABET[(n >> 12 & 0x3f) as usize] as char);
if chunk.len() > 1 {
out.push(ALPHABET[(n >> 6 & 0x3f) as usize] as char);
}
if chunk.len() > 2 {
out.push(ALPHABET[(n & 0x3f) as usize] as char);
}
}
// build-apk.sh strips '/', '+' and '=' from its password (tr -d
// '/+='), so the value never needs quoting when it is passed as a
// command-line argument later.
out.retain(|c| c != '/' && c != '+' && c != '=');
out
}
#[cfg(unix)]
fn write_owner_only(path: &std::path::Path, contents: &[u8]) -> Result<(), Fail> {
use std::os::unix::fs::OpenOptionsExt;
std::fs::OpenOptions::new()
.write(true)
.create(true)
.truncate(true)
.mode(0o600)
.open(path)
.and_then(|mut f| std::io::Write::write_all(&mut f, contents))
.map_err(|e| {
Fail::new(
"could not write the keystore password file",
&format!("{}: {e}", path.display()),
"check permissions on that directory",
)
})
}
#[cfg(unix)]
fn set_owner_only(path: &std::path::Path) -> Result<(), Fail> {
use std::os::unix::fs::PermissionsExt;
std::fs::set_permissions(path, std::fs::Permissions::from_mode(0o600)).map_err(|e| {
Fail::new(
"could not restrict the keystore's permissions",
&format!("{}: {e}", path.display()),
"chmod 600 it by hand",
)
})
}
+115
View File
@@ -0,0 +1,115 @@
//! `cargo xtask apk` -- E5 (RUST.md): packages `app/shellApp` into a
//! signed, installable APK with no Gradle in the packaging step itself.
//! `cargo ndk` cross-compiles the app crate's `shell` feature; `javac`/`d8` turn its two
//! Java stub classes (plus the generated pinned-CA constant) into dex;
//! `aapt2` compiles the manifest into `resources.arsc`; the dex and native
//! libraries are merged into that base APK with `jar`; `zipalign` and
//! `apksigner` finish it. See `apk.rs`'s module doc for what "no Gradle in
//! the packaging step" does and does not cover -- one disclosed exception.
//!
//! Usage: `cargo xtask apk [--release|--debug] [--abi ABI]...`
mod apk;
mod keystore;
mod sdk;
use std::fmt;
use std::process::ExitCode;
/// A failure a person acts on: what went wrong, what this process actually
/// saw, and the next thing to try. Matches CODE_RULES's "a failure message
/// names the thing, the cause, and the fix."
pub struct Fail {
what: String,
cause: String,
fix: String,
}
impl Fail {
pub fn new(what: &str, cause: &str, fix: &str) -> Self {
Fail {
what: what.to_string(),
cause: cause.to_string(),
fix: fix.to_string(),
}
}
}
impl fmt::Display for Fail {
fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
write!(
f,
"{}\n cause: {}\n fix: {}",
self.what, self.cause, self.fix
)
}
}
#[derive(Clone, Copy, PartialEq, Eq)]
pub enum Variant {
/// Signed with `~/.config/ai-app/release.jks`, the same key
/// `build-apk.sh` uses for `androidApp` -- what E5's pass condition
/// needs, since installing over an existing app requires a matching
/// signature.
Release,
/// Signed with the standard Android debug keystore
/// (`~/.android/debug.keystore`, well-known password, generated if
/// missing the same way Gradle would), for a fast local loop that
/// doesn't touch the real signing key.
Debug,
}
fn main() -> ExitCode {
let args: Vec<String> = std::env::args().skip(1).collect();
let Some(("apk", rest)) = args.split_first().map(|(cmd, rest)| (cmd.as_str(), rest)) else {
eprintln!("usage: cargo xtask apk [release|debug] [--abi ABI]...");
return ExitCode::FAILURE;
};
let mut variant = Variant::Release;
let mut abis: Vec<String> = Vec::new();
let mut i = 0;
while i < rest.len() {
match rest[i].as_str() {
// Bare "release"/"debug" is `.dev-updater.ron`'s interface
// (`ByMode::One` appends the chosen mode as the build
// command's last argument -- the same convention
// `app/build-apk.sh`'s `${1:-release}` uses); the `--`-prefixed
// spellings are for typing this by hand.
"release" | "--release" => variant = Variant::Release,
"debug" | "--debug" => variant = Variant::Debug,
"--abi" => {
i += 1;
match rest.get(i) {
Some(abi) => abis.push(abi.clone()),
None => {
eprintln!("--abi needs a value (e.g. arm64-v8a, x86_64)");
return ExitCode::FAILURE;
}
}
}
other => {
eprintln!("unknown argument: {other}");
return ExitCode::FAILURE;
}
}
i += 1;
}
if abis.is_empty() {
// arm64-v8a for a real phone, x86_64 for this machine's emulator --
// the two ABIs every other experiment in RUST.md has actually run
// on. `--abi` overrides either way.
abis = vec!["arm64-v8a".to_string(), "x86_64".to_string()];
}
match apk::build(variant, &abis) {
Ok(path) => {
println!("==> Built {}", path.display());
ExitCode::SUCCESS
}
Err(fail) => {
eprintln!("xtask: {fail}");
ExitCode::FAILURE
}
}
}
+130
View File
@@ -0,0 +1,130 @@
//! Finds the Android SDK/NDK pieces the packaging pipeline needs, the same
//! way `app/android-env.sh` and `app/build-apk.sh` do: `$ANDROID_HOME`, then
//! `$ANDROID_SDK_ROOT`, then `~/Android/Sdk`. Kept in one place because
//! every step in `main.rs` needs at least one of these paths, and a
//! mismatch between them (an `android.jar` from one SDK, `d8` from
//! another) fails in ways that point at the wrong cause.
use std::path::{Path, PathBuf};
use crate::Fail;
/// compileSdk / targetSdk, matching `app/shellApp/build.gradle.kts`. Not
/// read from that file -- if the two drift, `android.jar` or a platform
/// tools directory goes missing and the error below names the exact path
/// that wasn't there, which is no harder to act on than a parsed number
/// would have been.
pub const COMPILE_SDK: u32 = 37;
pub const MIN_SDK: u32 = 24;
pub struct Sdk {
pub root: PathBuf,
pub build_tools: PathBuf,
pub android_jar: PathBuf,
}
impl Sdk {
pub fn tool(&self, name: &str) -> PathBuf {
self.build_tools.join(name)
}
}
pub fn find() -> Result<Sdk, Fail> {
let root = std::env::var_os("ANDROID_HOME")
.or_else(|| std::env::var_os("ANDROID_SDK_ROOT"))
.map(PathBuf::from)
.filter(|p| p.is_dir())
.or_else(|| {
let home = std::env::var_os("HOME").map(PathBuf::from)?;
let candidate = home.join("Android/Sdk");
candidate.is_dir().then_some(candidate)
})
.ok_or_else(|| {
Fail::new(
"no Android SDK found",
"checked $ANDROID_HOME, $ANDROID_SDK_ROOT and ~/Android/Sdk",
"set ANDROID_HOME, or run app/android-env.sh once to install one",
)
})?;
let build_tools = latest_build_tools(&root)?;
let android_jar = root
.join("platforms")
.join(format!("android-{COMPILE_SDK}.0"))
.join("android.jar");
let android_jar = if android_jar.is_file() {
android_jar
} else {
// Some installs use the bare "android-37" directory name instead of
// "android-37.0" -- both exist on this machine's SDK depending on
// how the platform was installed, so try the other spelling before
// giving up.
let alt = root
.join("platforms")
.join(format!("android-{COMPILE_SDK}"))
.join("android.jar");
if alt.is_file() {
alt
} else {
return Err(Fail::new(
&format!("no android.jar for API {COMPILE_SDK}"),
&format!("checked {} and {}", android_jar.display(), alt.display()),
&format!("install it: android sdk install \"platforms/android-{COMPILE_SDK}.0\""),
));
}
};
Ok(Sdk {
root,
build_tools,
android_jar,
})
}
fn latest_build_tools(sdk_root: &Path) -> Result<PathBuf, Fail> {
let dir = sdk_root.join("build-tools");
let mut versions: Vec<(Vec<u32>, PathBuf)> = std::fs::read_dir(&dir)
.map_err(|e| {
Fail::new(
"no build-tools directory in the Android SDK",
&format!("{}: {e}", dir.display()),
"install one: android sdk install \"build-tools;37.0.0\"",
)
})?
.filter_map(|entry| entry.ok())
.filter(|entry| entry.path().is_dir())
.filter_map(|entry| {
let name = entry.file_name();
let name = name.to_str()?;
let parts: Vec<u32> = name.split('.').filter_map(|p| p.parse().ok()).collect();
(!parts.is_empty()).then_some((parts, entry.path()))
})
.collect();
versions.sort();
versions.pop().map(|(_, path)| path).ok_or_else(|| {
Fail::new(
"no usable build-tools version found",
&format!("{} has no version-numbered subdirectory", dir.display()),
"install one: android sdk install \"build-tools;37.0.0\"",
)
})
}
/// The NDK version `cargo ndk` should find on its own by scanning
/// `$ANDROID_HOME/ndk/*` -- this just checks one exists, so a missing NDK
/// is reported before `cargo ndk` does it with a less specific message.
pub fn require_ndk_installed(sdk_root: &Path) -> Result<(), Fail> {
let ndk_dir = sdk_root.join("ndk");
let has_one = std::fs::read_dir(&ndk_dir)
.map(|entries| entries.filter_map(|e| e.ok()).any(|e| e.path().is_dir()))
.unwrap_or(false);
if has_one {
Ok(())
} else {
Err(Fail::new(
"no NDK installed under the Android SDK",
&format!("{} has no version subdirectory", ndk_dir.display()),
"install one: android sdk install \"ndk;29.0.14206865\"",
))
}
}