Attach any file, take shares from other apps, and survive a backwards highlight

Attachments were images only. Now any file can be attached: from the file
chooser behind the "+" menu, or from Android's share sheet, which the app
is now in. An image still goes to the model as a picture; anything else is
stored under its own name (`<hex>-<name>`, cleaned by `safe_file_name`)
and the Claude driver ends the message with `Attached file: /abs/path`,
since the CLI reads files by path and a model cannot be shown a trace. The
user-message field is renamed `images` -> `attachments` on both sides,
with a serde alias reading the rows written before. A share arrives before
anyone has said which session it is for, so it is held in AppRoot with a
banner on the list until a session takes it; an open session takes it at
once. Unreadable shares are reported beside the composer, not thrown.

The tool card crashed the app when opened on a command holding a quoted
glob such as `-path '*/.git/*'`: highlights 1.1.0's shell lexer answers
`x '*/a/*'` with a span whose end is before its start, and AnnotatedString
refuses the range. Such spans are dropped; the library is the place for
the fix. The echo driver gains `/bash <command>` so a card with a given
command can be produced on the emulator.

ui-sandbox.sh's token salvage read the tokens block's close only at a line
start, ran past the compact `),],` the server writes, and copied `setups`
into the new config twice, which the server then refused.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
irisandClaude Fable 5.1 committed 2026-09-03 08:15:10 -04:00
1 parent 4bc69e8f9c
commit 6180663f14
25 files changed
+672 -165

No files matched your search

+13
View File
@@ -65,6 +65,19 @@ repo is in PLAN.md's "Backend layout" section.
refuse to resume onto a partial from a different revision, and are refuse to resume onto a partial from a different revision, and are
checked against HuggingFace's published sha256 before the file gets its checked against HuggingFace's published sha256 before the file gets its
real name. real name.
- **Attachments** are one list on a user message (`attachments`, the
ref the files route serves), in two shapes. An image is `<hex>.<ext>`
and goes to the model as an image block. Anything else is
`<hex>-<name>` -- the name it was shared or picked under, cleaned by
`safe_file_name` -- and the Claude driver appends `Attached file:
/abs/path` to the message text, since the CLI reads files by path and
a model cannot be shown a trace. `media::media_type_for` on the server
and `isImageRef` on the phone tell the two apart; keep those lists
level. The phone attaches from the photo picker, the file chooser and
Android's share sheet (`Share.kt`; the manifest's SEND filter), all
through one `attach` path in `SessionScreen`. Files exist only on the
server's machine -- see PLAN.md's "Transport" for what that means for
remote sessions.
- `server/` — Rust backend (`ai-server`). `main.rs` bootstraps (TLS, the - `server/` — Rust backend (`ai-server`). `main.rs` bootstraps (TLS, the
auth layer, token/QR enrollment, wg0 binding), `routes.rs` has the HTTP auth layer, token/QR enrollment, wg0 binding), `routes.rs` has the HTTP
table in its module doc comment, `auth.rs` the bearer-token middleware, table in its module doc comment, `auth.rs` the bearer-token middleware,
+13 -4
View File
@@ -713,11 +713,18 @@ host) and **hosts**. The manager runs at most one llama-server per
forwarded port (`ssh -L`) as well as a spawned process. A transport is forwarded port (`ssh -L`) as well as a spawned process. A transport is
therefore "run this" plus "reach this port", and the second operation is therefore "run this" plus "reach this port", and the second operation is
a no-op locally. a no-op locally.
- Attachments need no file transfer, contrary to what this section said - Images need no file transfer, contrary to what this section said
before: `attachment_block` base64s an uploaded image into the before: `attachment_block` base64s an uploaded image into the
stream-json message itself, and produced images come back the same way stream-json message itself, and produced images come back the same way
for the translator to write out locally. Nothing has to exist on the for the translator to write out locally. Nothing has to exist on the
remote filesystem, so there is no `scp` step to get wrong. remote filesystem, so there is no `scp` step to get wrong.
- **Any other file is told to the session by path** (2026-09-03: a trace,
a log, a zip -- things a model cannot be shown and the CLI can read).
The upload stays under the session's `attachments/` and the message
ends with `Attached file: /abs/path`. That directory exists only on the
machine running this server, so a file attached to a remote (ssh)
session names a path that is not there. Shipping it is not built; the
one host in use runs its sessions locally. Images are unaffected.
### Usage limits (Claude) ### Usage limits (Claude)
@@ -1094,9 +1101,11 @@ window just fills.
with port forward, attachment shipping. *Host config and remote spawn with port forward, attachment shipping. *Host config and remote spawn
done 2026-08-25* (any session of any provider can name a host; the done 2026-08-25* (any session of any provider can name a host; the
command is the identical one wrapped in `ssh -T`, with every argument command is the identical one wrapped in `ssh -T`, with every argument
shell-quoted). Attachment shipping turned out to be unnecessary for the shell-quoted). Attachment shipping turned out to be unnecessary for
Claude driver — images ride the stdio JSONL as base64 in both images — they ride the stdio JSONL as base64 in both directions, so
directions, so nothing needs `scp`. Still outstanding: remote nothing needs `scp` — and became necessary again on 2026-09-03 for
files, which are attached by path (see "Transport" above). Still
outstanding: file shipping for remote sessions, and remote
llama-server with its port forward, which comes with phase 4. llama-server with its port forward, which comes with phase 4.
Two things learned doing it: a remote session inherits ssh's non-login Two things learned doing it: a remote session inherits ssh's non-login
PATH, which is narrower than an interactive shell's (point `command` at PATH, which is narrower than an interactive shell's (point `command` at
@@ -61,6 +61,16 @@
<category android:name="android.intent.category.BROWSABLE" /> <category android:name="android.intent.category.BROWSABLE" />
<data android:scheme="aiapp" android:host="enroll" /> <data android:scheme="aiapp" android:host="enroll" />
</intent-filter> </intent-filter>
<!-- The share sheet: a file, a photo or some text from another app
lands here and is attached to a session (see Share.kt). Any
type, because what a session can be handed is the server's
decision rather than the sheet's. -->
<intent-filter>
<action android:name="android.intent.action.SEND" />
<action android:name="android.intent.action.SEND_MULTIPLE" />
<category android:name="android.intent.category.DEFAULT" />
<data android:mimeType="*/*" />
</intent-filter>
</activity> </activity>
<!-- specialUse rather than dataSync, which is the type this looks <!-- specialUse rather than dataSync, which is the type this looks
@@ -539,17 +539,24 @@ fun setSessionCwd(settings: ServerSettings, sessionId: String, cwd: String) {
) {} ) {}
} }
/** Uploads one picked image; the returned id goes into [sendMessage]. */ /**
* Uploads one attachment; the returned id goes into [sendMessage]. [name] is what the server keeps
* a file under and tells the session; for an image it is ignored, since the model is shown the
* picture rather than told its name.
*/
fun uploadAttachment( fun uploadAttachment(
settings: ServerSettings, settings: ServerSettings,
sessionId: String, sessionId: String,
bytes: ByteArray, bytes: ByteArray,
mime: String, mime: String,
name: String,
): String { ): String {
val boundary = "----aiapp-${System.currentTimeMillis()}" val boundary = "----aiapp-${System.currentTimeMillis()}"
// The header is a line: a quote or a line break in the name would end it early.
val safeName = name.replace(Regex("[\"\r\n]"), "_")
val head = val head =
("--$boundary\r\n" + ("--$boundary\r\n" +
"Content-Disposition: form-data; name=\"file\"; filename=\"image\"\r\n" + "Content-Disposition: form-data; name=\"file\"; filename=\"$safeName\"\r\n" +
"Content-Type: $mime\r\n\r\n") "Content-Type: $mime\r\n\r\n")
.encodeToByteArray() .encodeToByteArray()
val tail = "\r\n--$boundary--\r\n".encodeToByteArray() val tail = "\r\n--$boundary--\r\n".encodeToByteArray()
@@ -62,9 +62,16 @@ private data class FailedOpen(val request: SessionOpenRequest, val message: Stri
* re-reading the stored settings -- a plain `remember` would keep serving the pre-enrollment null. * re-reading the stored settings -- a plain `remember` would keep serving the pre-enrollment null.
* *
* [openRequest] is the session a notification tap asked for, likewise from MainActivity. * [openRequest] is the session a notification tap asked for, likewise from MainActivity.
*
* [shareRequest] is what another app shared in, likewise. It is held here until a session takes it,
* because the share arrives before anyone has said which session it is for.
*/ */
@Composable @Composable
fun AppRoot(settingsVersion: Int, openRequest: SessionOpenRequest?) { fun AppRoot(
settingsVersion: Int,
openRequest: SessionOpenRequest?,
shareRequest: ShareRequest? = null,
) {
val context = LocalContext.current val context = LocalContext.current
val scope = rememberCoroutineScope() val scope = rememberCoroutineScope()
var settings by remember(settingsVersion) { mutableStateOf(loadServerSettings(context)) } var settings by remember(settingsVersion) { mutableStateOf(loadServerSettings(context)) }
@@ -75,6 +82,17 @@ fun AppRoot(settingsVersion: Int, openRequest: SessionOpenRequest?) {
// Bumped whenever another screen changes something the list shows, so // Bumped whenever another screen changes something the list shows, so
// returning to it refetches instead of showing a stale list. // returning to it refetches instead of showing a stale list.
var reloadToken by remember { mutableIntStateOf(0) } var reloadToken by remember { mutableIntStateOf(0) }
// Cleared by the session screen that attached it, not when a newer request arrives: a share
// must be attached exactly once, and only the screen that did it knows that it has.
var share by remember { mutableStateOf<ShareRequest?>(null) }
LaunchedEffect(shareRequest) {
if (shareRequest != null) {
share = shareRequest
// A session already open takes it. Otherwise the list is where the choice is made,
// whatever screen was showing: Spawn and Settings have nowhere to put a file.
if (screen !is Screen.Session) screen = Screen.Main
}
}
// A standing condition rather than a per-request failure, so it is // A standing condition rather than a per-request failure, so it is
// stated once here instead of appended to every error that might be // stated once here instead of appended to every error that might be
@@ -162,6 +180,7 @@ fun AppRoot(settingsVersion: Int, openRequest: SessionOpenRequest?) {
MainScreen( MainScreen(
settings = current, settings = current,
reloadToken = reloadToken, reloadToken = reloadToken,
share = share,
onOpen = { screen = Screen.Session(it) }, onOpen = { screen = Screen.Session(it) },
onSpawn = { screen = Screen.Spawn }, onSpawn = { screen = Screen.Spawn },
onImported = { imported -> onImported = { imported ->
@@ -179,7 +198,13 @@ fun AppRoot(settingsVersion: Int, openRequest: SessionOpenRequest?) {
// row key. Only reachable since a notification can move straight from one session to // row key. Only reachable since a notification can move straight from one session to
// another; every other way here passes through [Screen.Main], which disposes it anyway. // another; every other way here passes through [Screen.Main], which disposes it anyway.
key(here.summary.id) { key(here.summary.id) {
SessionScreen(settings = current, summary = here.summary, onBack = goToMain) SessionScreen(
settings = current,
summary = here.summary,
onBack = goToMain,
share = share,
onShareTaken = { share = null },
)
} }
is Screen.Spawn -> is Screen.Spawn ->
Box(Modifier.imePadding()) { Box(Modifier.imePadding()) {
@@ -0,0 +1,66 @@
package com.example.aiapp
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.padding
import androidx.compose.material3.MaterialTheme
import androidx.compose.material3.Text
import androidx.compose.runtime.Composable
import androidx.compose.ui.Modifier
import androidx.compose.ui.draw.clip
import androidx.compose.ui.text.font.FontFamily
import androidx.compose.ui.text.style.TextOverflow
import androidx.compose.ui.unit.dp
/**
* Whether [ref] names an image the server stored as one -- `<hex>.<extension>`, with an extension
* from the list it writes -- rather than a file kept under its own name. Mirrors the server's
* `media` table, which is the one other place the list lives.
*/
fun isImageRef(ref: String): Boolean = ref.substringAfterLast('.', "") in IMAGE_EXTENSIONS
private val IMAGE_EXTENSIONS = setOf("png", "jpg", "gif", "webp")
/**
* The name a file was attached under: the ref less the hex the server put before it. The hex has no
* dash in it, so the first one is the boundary however many the name has.
*/
fun attachmentName(ref: String): String = ref.substringAfter('-', ref)
/**
* One attachment on a sent message, drawn as what it is: an image inline, a file as its name. A
* file is not fetched -- there is nothing on this phone to open a trace or a log with -- so the
* name is the whole of it.
*/
@Composable
fun Attachment(
settings: ServerSettings,
sessionId: String,
ref: String,
onOpenImage: (String) -> Unit,
) {
if (isImageRef(ref)) SessionImage(settings, sessionId, ref, onOpenImage)
else
FileName(
attachmentName(ref),
Modifier.clip(MaterialTheme.shapes.extraSmall)
.background(rawSurface)
.padding(horizontal = 8.dp, vertical = 4.dp),
)
}
/**
* A file's name, one line, in the face names are read in. Overlong names lose their middle: a name
* is identified by both ends -- what it is at the front, what kind at the back -- and either
* ellipsis alone takes away one of them.
*/
@Composable
fun FileName(name: String, modifier: Modifier = Modifier) {
Text(
name,
modifier = modifier,
style = MaterialTheme.typography.bodySmall,
fontFamily = FontFamily.Monospace,
maxLines = 1,
overflow = TextOverflow.MiddleEllipsis,
)
}
@@ -1,10 +1,12 @@
package com.example.aiapp package com.example.aiapp
import android.content.ContentResolver
import android.content.Context import android.content.Context
import android.graphics.Bitmap import android.graphics.Bitmap
import android.graphics.BitmapFactory import android.graphics.BitmapFactory
import android.graphics.Matrix import android.graphics.Matrix
import android.net.Uri import android.net.Uri
import android.provider.OpenableColumns
import androidx.exifinterface.media.ExifInterface import androidx.exifinterface.media.ExifInterface
import java.io.ByteArrayOutputStream import java.io.ByteArrayOutputStream
import kotlin.math.max import kotlin.math.max
@@ -31,7 +33,66 @@ suspend fun uploadPickedImage(
maxEdge: Int?, maxEdge: Int?,
): String { ): String {
val (bytes, mime) = readForUpload(context, uri, maxEdge) val (bytes, mime) = readForUpload(context, uri, maxEdge)
return uploadAttachment(settings, sessionId, bytes, mime) return uploadAttachment(settings, sessionId, bytes, mime, "image")
}
/**
* Uploads whatever [uri] names, the way its kind needs. An image goes through [uploadPickedImage]
* and is shrunk; anything else goes whole, under the name the other app or the file chooser gave
* it, because the session is told that name rather than shown the bytes.
*/
suspend fun uploadPicked(
context: Context,
settings: ServerSettings,
sessionId: String,
uri: Uri,
maxEdge: Int?,
): String {
val resolver = context.contentResolver
val mime = resolver.getType(uri)
if (mime != null && mime.startsWith("image/")) {
return uploadPickedImage(context, settings, sessionId, uri, maxEdge)
}
val bytes = readAll(resolver, uri)
return uploadAttachment(
settings,
sessionId,
bytes,
mime ?: "application/octet-stream",
displayName(resolver, uri),
)
}
/**
* Everything at [uri], or the failure as the kind the composer reports beside the message.
*
* A share arrives with whatever access the other app granted, and a provider that refuses says so
* with a `SecurityException`; a file gone between the pick and the read is an `IOException`. Both
* are things the reader can act on, so neither is left to end the process.
*/
private fun readAll(resolver: ContentResolver, uri: Uri): ByteArray =
try {
resolver.openInputStream(uri)?.use { it.readBytes() }
?: throw ApiException("couldn't read ${uri.lastPathSegment ?: uri}: nothing there")
} catch (e: SecurityException) {
throw ApiException("couldn't read ${uri.lastPathSegment ?: uri}: no access to it")
} catch (e: java.io.IOException) {
throw ApiException("couldn't read ${uri.lastPathSegment ?: uri}: ${e.message}")
}
/**
* The name a document provider shows for [uri]. The last path segment is the fallback because a
* provider's own id for a file is usually a number, which says nothing to the session.
*/
private fun displayName(resolver: ContentResolver, uri: Uri): String {
resolver.query(uri, arrayOf(OpenableColumns.DISPLAY_NAME), null, null, null)?.use { cursor ->
val column = cursor.getColumnIndex(OpenableColumns.DISPLAY_NAME)
if (column >= 0 && cursor.moveToFirst())
cursor.getString(column)?.let {
return it
}
}
return uri.lastPathSegment ?: "file"
} }
/** /**
@@ -44,9 +105,7 @@ suspend fun uploadPickedImage(
private fun readForUpload(context: Context, uri: Uri, maxEdge: Int?): Pair<ByteArray, String> { private fun readForUpload(context: Context, uri: Uri, maxEdge: Int?): Pair<ByteArray, String> {
val resolver = context.contentResolver val resolver = context.contentResolver
val mime = resolver.getType(uri) ?: "image/jpeg" val mime = resolver.getType(uri) ?: "image/jpeg"
val original = val original = readAll(resolver, uri)
resolver.openInputStream(uri)?.use { it.readBytes() }
?: throw ApiException("couldn't read the picked image")
if (maxEdge == null) return original to mime if (maxEdge == null) return original to mime
val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true } val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
@@ -30,14 +30,15 @@ sealed class SessionEvent {
*/ */
val id: String?, val id: String?,
/** /**
* What was attached to it, by the ref the files route serves. * What was attached to it, by the ref the files route serves: images, and since 2026-09-03
* any file, told apart by [isImageRef].
* *
* On the message rather than beside it: these arrived as separate image events until * On the message rather than beside it: these arrived as separate image events until
* 2026-08-30, which drew somebody's screenshot as a row floating above the bubble that sent * 2026-08-30, which drew somebody's screenshot as a row floating above the bubble that sent
* it, and left this app deciding from adjacency alone which message an image went with -- * it, and left this app deciding from adjacency alone which message an image went with --
* something the sender knew and could simply have said. * something the sender knew and could simply have said.
*/ */
val images: List<String>, val attachments: List<String>,
) : SessionEvent() ) : SessionEvent()
/** /**
@@ -50,7 +51,7 @@ sealed class SessionEvent {
* Resolved by the [UserMessage] carrying the same id, exactly as [CommandQueued] is resolved by * Resolved by the [UserMessage] carrying the same id, exactly as [CommandQueued] is resolved by
* [CommandSent]. * [CommandSent].
*/ */
data class MessageQueued(val id: String, val text: String, val images: List<String>) : data class MessageQueued(val id: String, val text: String, val attachments: List<String>) :
SessionEvent() SessionEvent()
/** /**
@@ -197,13 +198,13 @@ fun parseSeqEvent(json: String): SeqEvent {
SessionEvent.UserMessage( SessionEvent.UserMessage(
body.getString("text"), body.getString("text"),
body.optString("id").ifEmpty { null }, body.optString("id").ifEmpty { null },
body.stringList("images"), body.stringList("attachments"),
) )
"messageQueued" -> "messageQueued" ->
SessionEvent.MessageQueued( SessionEvent.MessageQueued(
body.getString("id"), body.getString("id"),
body.getString("text"), body.getString("text"),
body.stringList("images"), body.stringList("attachments"),
) )
"messageDropped" -> SessionEvent.MessageDropped(body.getString("id")) "messageDropped" -> SessionEvent.MessageDropped(body.getString("id"))
"assistantText" -> SessionEvent.AssistantText(body.getString("delta")) "assistantText" -> SessionEvent.AssistantText(body.getString("delta"))
@@ -37,6 +37,10 @@ class MainActivity : ComponentActivity() {
private var openRequest by mutableStateOf<SessionOpenRequest?>(null) private var openRequest by mutableStateOf<SessionOpenRequest?>(null)
private var opens = 0 private var opens = 0
// What another app shared into this one, for the same reason and with the same serial.
private var shareRequest by mutableStateOf<ShareRequest?>(null)
private var shares = 0
// Registered up front since permission launchers must be registered // Registered up front since permission launchers must be registered
// before the activity reaches STARTED. // before the activity reaches STARTED.
private val requestLocalNetworkPermission = private val requestLocalNetworkPermission =
@@ -139,7 +143,7 @@ class MainActivity : ComponentActivity() {
// scoped to what actually moves. // scoped to what actually moves.
.navigationBarsPadding() .navigationBarsPadding()
) { ) {
AppRoot(settingsVersion, openRequest) AppRoot(settingsVersion, openRequest, shareRequest)
} }
} }
} }
@@ -155,14 +159,21 @@ class MainActivity : ComponentActivity() {
} }
/** /**
* The one place an incoming `aiapp://` URI is sorted into what it means. * The one place an incoming intent is sorted into what it means.
* *
* Two things arrive this way -- an enrollment code and a notification naming a session -- and * Three things arrive this way -- a share from another app, and an `aiapp://` URI that is
* they are told apart by the URI's host rather than by two entry points, so a third kind is a * either an enrollment code or a notification naming a session. The URIs are told apart by host
* branch here rather than another intent to remember to handle. * rather than by two entry points, so a further kind is a branch here rather than another
* intent to remember to handle.
*/ */
private fun handleIntent(intent: Intent?) { private fun handleIntent(intent: Intent?) {
val uri = intent?.data ?: return intent ?: return
sharedContent(intent, shares + 1)?.let { shared ->
shares = shared.serial
shareRequest = shared
return
}
val uri = intent.data ?: return
val sessionId = notifiedSessionId(uri) val sessionId = notifiedSessionId(uri)
if (sessionId != null) { if (sessionId != null) {
opens++ opens++
@@ -1,6 +1,7 @@
package com.example.aiapp package com.example.aiapp
import androidx.activity.compose.BackHandler import androidx.activity.compose.BackHandler
import androidx.compose.foundation.background
import androidx.compose.foundation.layout.Column import androidx.compose.foundation.layout.Column
import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.fillMaxSize import androidx.compose.foundation.layout.fillMaxSize
@@ -46,6 +47,8 @@ private enum class MainTab(val label: String) {
fun MainScreen( fun MainScreen(
settings: ServerSettings, settings: ServerSettings,
reloadToken: Int, reloadToken: Int,
/** What another app shared in and no session has taken yet; see [ShareRequest]. */
share: ShareRequest? = null,
onOpen: (SessionSummary) -> Unit, onOpen: (SessionSummary) -> Unit,
onSpawn: () -> Unit, onSpawn: () -> Unit,
onImported: (SessionSummary) -> Unit, onImported: (SessionSummary) -> Unit,
@@ -106,6 +109,24 @@ fun MainScreen(
GlyphButton(SETTINGS_GLYPH, "Settings", onSettings) GlyphButton(SETTINGS_GLYPH, "Settings", onSettings)
} }
} }
// What is waiting to be attached, and what to do about it. Said here because the list
// below is where the choice is made, and a share that arrived with nothing on screen
// saying so would read as a tap that did nothing.
share?.let {
Text(
it.summary() + " -- open the session it belongs in.",
style = MaterialTheme.typography.bodyMedium,
color = MaterialTheme.colorScheme.onPrimaryContainer,
modifier =
Modifier.fillMaxWidth()
.padding(horizontal = 16.dp, vertical = 8.dp)
.background(
MaterialTheme.colorScheme.primaryContainer,
MaterialTheme.shapes.small,
)
.padding(12.dp),
)
}
// Primary rather than the plain TabRow, which is deprecated in favour of the two that // Primary rather than the plain TabRow, which is deprecated in favour of the two that
// say where they sit: these are the app's top-level destinations. // say where they sit: these are the app's top-level destinations.
PrimaryTabRow(selectedTabIndex = tab.ordinal) { PrimaryTabRow(selectedTabIndex = tab.ordinal) {
@@ -8,8 +8,12 @@ import androidx.compose.foundation.horizontalScroll
import androidx.compose.foundation.layout.Arrangement import androidx.compose.foundation.layout.Arrangement
import androidx.compose.foundation.layout.Box import androidx.compose.foundation.layout.Box
import androidx.compose.foundation.layout.Row import androidx.compose.foundation.layout.Row
import androidx.compose.foundation.layout.Spacer
import androidx.compose.foundation.layout.height
import androidx.compose.foundation.layout.padding import androidx.compose.foundation.layout.padding
import androidx.compose.foundation.layout.size import androidx.compose.foundation.layout.size
import androidx.compose.foundation.layout.width
import androidx.compose.foundation.layout.widthIn
import androidx.compose.foundation.rememberScrollState import androidx.compose.foundation.rememberScrollState
import androidx.compose.foundation.shape.CircleShape import androidx.compose.foundation.shape.CircleShape
import androidx.compose.foundation.shape.RoundedCornerShape import androidx.compose.foundation.shape.RoundedCornerShape
@@ -34,7 +38,8 @@ import androidx.compose.ui.unit.sp
* anywhere else on the screen. * anywhere else on the screen.
* *
* Scrolls sideways rather than wrapping or shrinking: the row keeps one thumbnail size whatever is * Scrolls sideways rather than wrapping or shrinking: the row keeps one thumbnail size whatever is
* in it, so four attachments look like four of the same thing rather than four smaller ones. * in it, so four attachments look like four of the same thing rather than four smaller ones. A file
* is a tile of the same height carrying its name, since a name is all there is to show of it.
*/ */
@Composable @Composable
fun PendingAttachments( fun PendingAttachments(
@@ -49,7 +54,10 @@ fun PendingAttachments(
modifier = modifier.horizontalScroll(rememberScrollState()).padding(bottom = 8.dp), modifier = modifier.horizontalScroll(rememberScrollState()).padding(bottom = 8.dp),
horizontalArrangement = Arrangement.spacedBy(8.dp), horizontalArrangement = Arrangement.spacedBy(8.dp),
) { ) {
refs.forEach { ref -> PendingThumbnail(settings, sessionId, ref) { onRemove(ref) } } refs.forEach { ref ->
if (isImageRef(ref)) PendingThumbnail(settings, sessionId, ref) { onRemove(ref) }
else PendingFile(ref) { onRemove(ref) }
}
} }
} }
@@ -122,4 +130,32 @@ private fun PendingThumbnail(
} }
} }
/**
* One attached file: its name, tap to take it back off. The same height and removal as a thumbnail,
* so a row of mixed attachments is one row; the cross sits after the name because a tile this wide
* has no corner the eye goes to.
*/
@Composable
private fun PendingFile(ref: String, onRemove: () -> Unit) {
val name = attachmentName(ref)
val shape = RoundedCornerShape(8.dp)
Row(
Modifier.height(THUMBNAIL)
.clip(shape)
.border(1.dp, MaterialTheme.colorScheme.outlineVariant, shape)
.background(MaterialTheme.colorScheme.surfaceVariant)
.clickable(onClick = onRemove)
.semantics { contentDescription = "Attached file $name, tap to remove" }
.padding(horizontal = 8.dp),
verticalAlignment = Alignment.CenterVertically,
) {
FileName(name, Modifier.widthIn(max = FILE_TILE_WIDTH))
Spacer(Modifier.width(6.dp))
Glyph(CLOSE_GLYPH, colour = MaterialTheme.colorScheme.onSurface, size = 12.sp)
}
}
private val THUMBNAIL = 64.dp private val THUMBNAIL = 64.dp
/** Wide enough for most names whole; longer ones lose their middle, keeping both ends. */
private val FILE_TILE_WIDTH = 200.dp
@@ -2,6 +2,7 @@ package com.example.aiapp
import android.content.Context import android.content.Context
import android.content.pm.ApplicationInfo import android.content.pm.ApplicationInfo
import android.net.Uri
import android.os.Build import android.os.Build
import android.os.SystemClock import android.os.SystemClock
import android.util.Log import android.util.Log
@@ -228,7 +229,15 @@ private fun Modifier.holdTopEdge(key: Any, held: TopEdgeHold, hold: (Int) -> Uni
// self-correction needs it. // self-correction needs it.
@OptIn(ExperimentalLayoutApi::class) @OptIn(ExperimentalLayoutApi::class)
@Composable @Composable
fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: () -> Unit) { fun SessionScreen(
settings: ServerSettings,
summary: SessionSummary,
onBack: () -> Unit,
/** What another app shared in while this session is the one open; see [ShareRequest]. */
share: ShareRequest? = null,
/** Said once [share] has been attached here, so it is not attached again. */
onShareTaken: () -> Unit = {},
) {
DebugStats.count("session screen recomposed") DebugStats.count("session screen recomposed")
val scope = rememberCoroutineScope() val scope = rememberCoroutineScope()
val topEdgeHeld = remember { TopEdgeHold() } val topEdgeHeld = remember { TopEdgeHold() }
@@ -410,7 +419,7 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
// Waiting, then read. Matched by id: the same message sent twice is two // Waiting, then read. Matched by id: the same message sent twice is two
// bubbles, and clearing by text would take away whichever matched first. // bubbles, and clearing by text would take away whichever matched first.
if (event is SessionEvent.MessageQueued) { if (event is SessionEvent.MessageQueued) {
queued = queued + QueuedMessage(event.id, event.text, event.images) queued = queued + QueuedMessage(event.id, event.text, event.attachments)
} }
if (event is SessionEvent.UserMessage) { if (event is SessionEvent.UserMessage) {
queued = queued.filterNot { it.id == event.id } queued = queued.filterNot { it.id == event.id }
@@ -1076,34 +1085,45 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
act { sendMessage(settings, summary.id, text, attachments) } act { sendMessage(settings, summary.id, text, attachments) }
} }
// The system photo picker; the image uploads as soon as it's chosen, // One path for everything attached, however it arrived: the photo picker, the file chooser
// so Send only has ids to reference. // or another app's share sheet. It uploads as soon as it is chosen, so Send only has ids to
val pickImage = // reference.
rememberLauncherForActivityResult(ActivityResultContracts.PickVisualMedia()) { uri -> fun attach(uri: Uri) {
if (uri != null) { scope.launch {
scope.launch { try {
try { val id =
val id = withContext(Dispatchers.IO) {
withContext(Dispatchers.IO) { // An image is shrunk to what this session's provider takes before it is
// Shrunk to what this session's provider takes before it is // uploaded, so a twelve-megapixel photo does not cross the tunnel to be
// uploaded, so a twelve-megapixel photo does not cross the tunnel // rejected at the far end; a file goes whole -- see `uploadPicked`.
// to be rejected at the far end -- see `uploadPickedImage`. uploadPicked(context, settings, summary.id, uri, summary.maxImageEdge)
uploadPickedImage(
context,
settings,
summary.id,
uri,
summary.maxImageEdge,
)
}
pendingAttachments = pendingAttachments + id
actionError = null
} catch (e: ApiException) {
actionError = e.message
} }
} pendingAttachments = pendingAttachments + id
actionError = null
} catch (e: ApiException) {
actionError = e.message
} }
} }
}
val pickImage =
rememberLauncherForActivityResult(ActivityResultContracts.PickVisualMedia()) { uri ->
uri?.let(::attach)
}
val pickFile =
rememberLauncherForActivityResult(ActivityResultContracts.OpenDocument()) { uri ->
uri?.let(::attach)
}
// What another app shared in, attached the moment this screen has it. Taken off the request
// first, so a recomposition or a return to this screen cannot attach it a second time.
LaunchedEffect(share) {
val incoming = share ?: return@LaunchedEffect
onShareTaken()
incoming.uris.forEach(::attach)
incoming.text?.let { shared ->
input = if (input.isBlank()) shared else input + "\n" + shared
saveDraft(context, summary.id, input)
}
}
// One poll for this machine's limits, read by the two things that show them: the bar under // One poll for this machine's limits, read by the two things that show them: the bar under
// the header, and the colour of the button that opens the dialog. // the header, and the colour of the button that opens the dialog.
@@ -1333,7 +1353,7 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
settings = settings, settings = settings,
sessionId = summary.id, sessionId = summary.id,
text = waiting.text, text = waiting.text,
images = waiting.images, attachments = waiting.attachments,
onOpenImage = ::openImage, onOpenImage = ::openImage,
pending = true, pending = true,
refusal = waiting.refusal, refusal = waiting.refusal,
@@ -1470,7 +1490,7 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
settings = settings, settings = settings,
sessionId = summary.id, sessionId = summary.id,
text = item.text, text = item.text,
images = item.images, attachments = item.attachments,
onOpenImage = ::openImage, onOpenImage = ::openImage,
) )
is TranscriptItem.AssistantMsg -> is TranscriptItem.AssistantMsg ->
@@ -1727,17 +1747,38 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
verticalAlignment = Alignment.CenterVertically, verticalAlignment = Alignment.CenterVertically,
modifier = Modifier.fillMaxWidth(), modifier = Modifier.fillMaxWidth(),
) { ) {
TextButton( // Photo or file, asked here rather than by two buttons: the row is full,
onClick = { // and attaching is one action whichever picker answers it.
pickImage.launch( var attaching by remember { mutableStateOf(false) }
PickVisualMediaRequest( Box {
ActivityResultContracts.PickVisualMedia.ImageOnly // Just "+". The count it used to carry was standing in for showing them.
) TextButton(onClick = { attaching = true }) { Text("+") }
DropdownMenu(
expanded = attaching,
onDismissRequest = { attaching = false },
// See PickerButton: without this the menu opens a status bar's
// height away from the button in an edge-to-edge activity.
properties = PopupProperties(clippingEnabled = false),
) {
DropdownMenuItem(
text = { Text("Photo") },
onClick = {
attaching = false
pickImage.launch(
PickVisualMediaRequest(
ActivityResultContracts.PickVisualMedia.ImageOnly
)
)
},
)
DropdownMenuItem(
text = { Text("File") },
onClick = {
attaching = false
pickFile.launch(arrayOf("*/*"))
},
) )
} }
) {
// Just "+" now. The count was standing in for showing them.
Text("+")
} }
// The settings share what is left after the actions have // The settings share what is left after the actions have
// taken what they need. A Row hands out intrinsic widths in // taken what they need. A Row hands out intrinsic widths in
@@ -1929,9 +1970,9 @@ private fun UserChunkRow(
Text(unit.text, color = MaterialTheme.colorScheme.onPrimaryContainer) Text(unit.text, color = MaterialTheme.colorScheme.onPrimaryContainer)
// The same arrangement [UserBubble] gives them: under the words, on the last slice // The same arrangement [UserBubble] gives them: under the words, on the last slice
// because that is the bubble's bottom. // because that is the bubble's bottom.
unit.images.forEachIndexed { index, ref -> unit.attachments.forEachIndexed { index, ref ->
if (index > 0 || unit.text.isNotEmpty()) Spacer(Modifier.height(4.dp)) if (index > 0 || unit.text.isNotEmpty()) Spacer(Modifier.height(4.dp))
SessionImage(settings, sessionId, ref, onOpenImage) Attachment(settings, sessionId, ref, onOpenImage)
} }
} }
} }
@@ -1957,7 +1998,7 @@ private fun UserBubble(
settings: ServerSettings, settings: ServerSettings,
sessionId: String, sessionId: String,
text: String, text: String,
images: List<String> = emptyList(), attachments: List<String> = emptyList(),
onOpenImage: (String) -> Unit, onOpenImage: (String) -> Unit,
pending: Boolean = false, pending: Boolean = false,
refusal: String? = null, refusal: String? = null,
@@ -2003,9 +2044,9 @@ private fun UserBubble(
// Under the words: what somebody wrote is what the bubble is, and the picture is // Under the words: what somebody wrote is what the bubble is, and the picture is
// what they attached to it. It also keeps the first line of every bubble at the // what they attached to it. It also keeps the first line of every bubble at the
// same place down the transcript, whether or not there is an image in it. // same place down the transcript, whether or not there is an image in it.
images.forEachIndexed { index, ref -> attachments.forEachIndexed { index, ref ->
if (index > 0 || text.isNotEmpty()) Spacer(Modifier.height(4.dp)) if (index > 0 || text.isNotEmpty()) Spacer(Modifier.height(4.dp))
SessionImage(settings, sessionId, ref, onOpenImage) Attachment(settings, sessionId, ref, onOpenImage)
} }
refusal?.let { refusal?.let {
Spacer(Modifier.height(6.dp)) Spacer(Modifier.height(6.dp))
@@ -2030,7 +2071,7 @@ private fun UserBubble(
private data class QueuedMessage( private data class QueuedMessage(
val id: String, val id: String,
val text: String, val text: String,
val images: List<String>, val attachments: List<String>,
val refusal: String? = null, val refusal: String? = null,
) )
@@ -0,0 +1,44 @@
package com.example.aiapp
import android.content.Intent
import android.net.Uri
import androidx.core.content.IntentCompat
/**
* What another app handed this one through the share sheet, waiting to be attached to a session.
*
* Held as the URIs rather than uploaded on arrival, because an upload belongs to a session and the
* share arrives before anyone has said which. [serial] makes two shares of the same thing two
* requests, for the reason [SessionOpenRequest] carries one: equal values would not recompose.
*/
data class ShareRequest(val uris: List<Uri>, val text: String?, val serial: Int)
/** The share in [intent], or null when it is some other intent. */
fun sharedContent(intent: Intent, serial: Int): ShareRequest? {
val uris =
when (intent.action) {
Intent.ACTION_SEND ->
listOfNotNull(
IntentCompat.getParcelableExtra(intent, Intent.EXTRA_STREAM, Uri::class.java)
)
Intent.ACTION_SEND_MULTIPLE ->
IntentCompat.getParcelableArrayListExtra(
intent,
Intent.EXTRA_STREAM,
Uri::class.java,
)
.orEmpty()
else -> return null
}
val text = intent.getStringExtra(Intent.EXTRA_TEXT)?.takeIf { it.isNotBlank() }
if (uris.isEmpty() && text == null) return null
return ShareRequest(uris, text, serial)
}
/** What is waiting, for the banner that says so. */
fun ShareRequest.summary(): String =
when {
uris.size == 1 -> "1 file to attach"
uris.isNotEmpty() -> "${uris.size} files to attach"
else -> "Text to attach"
}
@@ -160,6 +160,15 @@ private fun highlighted(code: String, language: SyntaxLanguage?): AnnotatedStrin
Highlights.Builder(code = code, language = language, theme = theme) Highlights.Builder(code = code, language = language, theme = theme)
.build() .build()
.getHighlights() .getHighlights()
// highlights 1.1.0's shell lexer answers a quoted glob that looks like a
// comment -- `x '*/a/*'` is the smallest input -- with a span whose end is
// before its start, and AnnotatedString refuses such a range. That crashed the
// app the moment a card holding `-path '*/.git/*'` was opened. Dropped rather
// than clamped: a span the lexer got backwards is not one it knows the colour
// of. Delete when snipme/highlights fixes it.
.filter {
it.location.start in 0..it.location.end && it.location.end <= code.length
}
buildAnnotatedString { buildAnnotatedString {
append(code) append(code)
marks.forEach { mark -> marks.forEach { mark ->
@@ -42,7 +42,7 @@ sealed class TranscriptItem {
override val seq: Long, override val seq: Long,
val text: String, val text: String,
/** Refs of what was attached, drawn inside the bubble. */ /** Refs of what was attached, drawn inside the bubble. */
val images: List<String> = emptyList(), val attachments: List<String> = emptyList(),
) : TranscriptItem() ) : TranscriptItem()
data class AssistantMsg( data class AssistantMsg(
@@ -375,7 +375,7 @@ private fun splitRun(tail: List<TranscriptItem>, behind: String?): List<Transcri
fun foldEvent(items: List<TranscriptItem>, entry: SeqEvent): List<TranscriptItem> = fun foldEvent(items: List<TranscriptItem>, entry: SeqEvent): List<TranscriptItem> =
when (val event = entry.event) { when (val event = entry.event) {
is SessionEvent.UserMessage -> is SessionEvent.UserMessage ->
items + TranscriptItem.UserMsg(entry.seq, event.text, event.images) items + TranscriptItem.UserMsg(entry.seq, event.text, event.attachments)
is SessionEvent.AssistantText -> { is SessionEvent.AssistantText -> {
// Deltas accumulate into the message they're streaming, which keeps the seq of the // Deltas accumulate into the message they're streaming, which keeps the seq of the
// first of them: a row whose identity changed with every delta would be a new row on // first of them: a row whose identity changed with every delta would be a new row on
@@ -129,7 +129,7 @@ sealed class TranscriptUnit {
val first: Boolean, val first: Boolean,
val last: Boolean, val last: Boolean,
/** The message's attachments, drawn under the words -- so only the last slice has any. */ /** The message's attachments, drawn under the words -- so only the last slice has any. */
val images: List<String>, val attachments: List<String>,
override val gap: Dp, override val gap: Dp,
) : TranscriptUnit() { ) : TranscriptUnit() {
override val key: Any override val key: Any
@@ -211,7 +211,7 @@ fun transcriptUnits(
chunk, chunk,
first = at == 0, first = at == 0,
last = at == chunks.lastIndex, last = at == chunks.lastIndex,
images = if (at == chunks.lastIndex) item.images else emptyList(), attachments = if (at == chunks.lastIndex) item.attachments else emptyList(),
gap = if (at == 0) rowGap else 0.dp, gap = if (at == 0) rowGap else 0.dp,
) )
} }
+12 -1
View File
@@ -148,7 +148,18 @@ salvaged=""
if [ -f "$ROOT/config.ron" ]; then if [ -f "$ROOT/config.ron" ]; then
salvaged=$(awk ' salvaged=$(awk '
/^tokens: \[/ { in_tokens = 1; next } /^tokens: \[/ { in_tokens = 1; next }
in_tokens && /^\],/ { exit } # The server writes the list back compactly, with the last entry
# and the close on one line: " ),],". Reading the close only
# at a line start ran past it into `setups`, and the salvage then
# carried a second copy of that block into the new config.
in_tokens && /\],/ {
sub(/\],.*/, "")
if ($0 ~ /\),/) {
entry = entry $0 "\n"
if (entry !~ h) entries = entries entry
}
exit
}
in_tokens { in_tokens {
entry = entry $0 "\n" entry = entry $0 "\n"
if ($0 ~ /\),/) { if ($0 ~ /\),/) {
+36 -15
View File
@@ -30,8 +30,8 @@
//! POST /sessions/{id}/command {text} -- /compact, /clear, /rename x, or the dialect's own //! POST /sessions/{id}/command {text} -- /compact, /clear, /rename x, or the dialect's own
//! (starts the process first if it has exited) //! (starts the process first if it has exited)
//! POST /sessions/{id}/compact //! POST /sessions/{id}/compact
//! POST /sessions/{id}/attachments multipart image upload -> {id}, referenced by /message //! POST /sessions/{id}/attachments multipart upload, image or any file -> {id}, referenced by /message
//! GET /sessions/{id}/files/{name} images the session produced or was sent //! GET /sessions/{id}/files/{name} images the session produced, and what it was sent
//! DELETE /sessions/{id} kill process, delete transcript + files //! DELETE /sessions/{id} kill process, delete transcript + files
//! (?deleteForeign=true removes the machine's own copy too) //! (?deleteForeign=true removes the machine's own copy too)
//! POST /sessions/{id}/notify {notify} -- announce this one or not //! POST /sessions/{id}/notify {notify} -- announce this one or not
@@ -110,7 +110,13 @@ pub fn router(manager: Arc<SessionManager>) -> Router {
.route("/notifications", get(notifications)) .route("/notifications", get(notifications))
.route("/sessions/{id}/compact", post(compact)) .route("/sessions/{id}/compact", post(compact))
.route("/sessions/{id}/command", post(command)) .route("/sessions/{id}/command", post(command))
.route("/sessions/{id}/attachments", post(upload_attachment)) .route(
"/sessions/{id}/attachments",
// A trace or a log is bigger than a photo; the cap below is
// for everything else, and the innermost limit is the one
// axum applies.
post(upload_attachment).layer(axum::extract::DefaultBodyLimit::max(ATTACHMENT_LIMIT)),
)
.route("/sessions/{id}/files/{name}", get(serve_file)) .route("/sessions/{id}/files/{name}", get(serve_file))
// Phone photos overflow axum's 2 MB default body cap. // Phone photos overflow axum's 2 MB default body cap.
.layer(axum::extract::DefaultBodyLimit::max(32 * 1024 * 1024)) .layer(axum::extract::DefaultBodyLimit::max(32 * 1024 * 1024))
@@ -1273,8 +1279,14 @@ async fn compact(
Ok(StatusCode::NO_CONTENT) Ok(StatusCode::NO_CONTENT)
} }
/// Accepts one image (any multipart field) and stores it under the /// The most one attachment may be. A day of `perfetto` is under a
/// gigabyte; a phone photo is a few megabytes; this is the room between.
const ATTACHMENT_LIMIT: usize = 1024 * 1024 * 1024;
/// Accepts one file (any multipart field) and stores it under the
/// session; the returned id goes into a later `/message`'s attachmentIds. /// session; the returned id goes into a later `/message`'s attachmentIds.
/// An image is later shown to the model, anything else is named to it by
/// path -- see `ClaudeDriver::send_user_message`.
async fn upload_attachment( async fn upload_attachment(
State(manager): State<Arc<SessionManager>>, State(manager): State<Arc<SessionManager>>,
UrlPath(id): UrlPath<String>, UrlPath(id): UrlPath<String>,
@@ -1286,27 +1298,36 @@ async fn upload_attachment(
.await .await
.map_err(|err| ApiError::BadRequest(format!("bad upload: {err}")))? .map_err(|err| ApiError::BadRequest(format!("bad upload: {err}")))?
.ok_or_else(|| ApiError::BadRequest("no file in the upload".to_string()))?; .ok_or_else(|| ApiError::BadRequest("no file in the upload".to_string()))?;
let content_type = field.content_type().unwrap_or("image/jpeg").to_string(); let content_type = field
.content_type()
.unwrap_or("application/octet-stream")
.to_string();
let file_name = field.file_name().map(str::to_string);
let bytes = field let bytes = field
.bytes() .bytes()
.await .await
.map_err(|err| ApiError::BadRequest(format!("upload read failed: {err}")))?; .map_err(|err| ApiError::BadRequest(format!("upload read failed: {err}")))?;
let name = session let name = session
.save_attachment(&bytes, &content_type) .save_attachment(&bytes, &content_type, file_name.as_deref())
.map_err(bad_request)?; .map_err(bad_request)?;
Ok(axum::Json(serde_json::json!({ "id": name }))) Ok(axum::Json(serde_json::json!({ "id": name })))
} }
/// Serves a session's stored images -- both `files/` (produced by tools) /// Serves a session's stored files -- both `files/` (images produced by
/// and `attachments/` (uploaded from the phone), by the id events and /// tools) and `attachments/` (uploaded from the phone), by the id events
/// uploads reference. /// and uploads reference.
async fn serve_file( async fn serve_file(
State(manager): State<Arc<SessionManager>>, State(manager): State<Arc<SessionManager>>,
UrlPath((id, name)): UrlPath<(String, String)>, UrlPath((id, name)): UrlPath<(String, String)>,
) -> Result<Response, ApiError> { ) -> Result<Response, ApiError> {
// Ids are server-generated hex + extension; anything else (and any // Ids are server-generated -- hex and an extension, or hex and a
// path separator in particular) is refused, not resolved. // cleaned file name (`safe_file_name`); anything else (and any path
if !name.chars().all(|c| c.is_ascii_alphanumeric() || c == '.') || name.contains("..") { // separator in particular) is refused, not resolved.
if !name
.chars()
.all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == '_')
|| name.contains("..")
{
return Err(ApiError::BadRequest("invalid file id".to_string())); return Err(ApiError::BadRequest("invalid file id".to_string()));
} }
let session = lookup(&manager, &id)?; let session = lookup(&manager, &id)?;
@@ -1324,9 +1345,9 @@ async fn serve_file(
let bytes = std::fs::read(path) let bytes = std::fs::read(path)
.with_context(|| format!("read {}", path.display())) .with_context(|| format!("read {}", path.display()))
.map_err(ApiError::Internal)?; .map_err(ApiError::Internal)?;
// Names are server-generated, so an unrecognized extension can only // Every image this server writes has an extension it knows; the rest
// mean a file this server didn't write. // are files attached by name, served as the bytes they are.
let content_type = crate::media::media_type_for(&name).unwrap_or("image/jpeg"); let content_type = crate::media::media_type_for(&name).unwrap_or("application/octet-stream");
Ok(([(axum::http::header::CONTENT_TYPE, content_type)], bytes).into_response()) Ok(([(axum::http::header::CONTENT_TYPE, content_type)], bytes).into_response())
} }
+54 -24
View File
@@ -57,7 +57,7 @@ use serde_json::{Value, json};
use tokio::io::AsyncWriteExt; use tokio::io::AsyncWriteExt;
use tokio::sync::mpsc; use tokio::sync::mpsc;
use super::driver::{Driver, Event, EventSink, ImageRef, SessionStatus, Unqueued}; use super::driver::{AttachmentRef, Driver, Event, EventSink, SessionStatus, Unqueued};
use super::process; use super::process;
use super::transport::{Launch, Streams, Transport}; use super::transport::{Launch, Streams, Transport};
use crate::config::{ProviderConfig, SessionConfig}; use crate::config::{ProviderConfig, SessionConfig};
@@ -133,7 +133,7 @@ struct Queue {
/// Written, not yet announced, oldest first, each with the id of the /// Written, not yet announced, oldest first, each with the id of the
/// `MessageQueued` that told the phone it was waiting -- so the /// `MessageQueued` that told the phone it was waiting -- so the
/// announcement can name which bubble it resolves. /// announcement can name which bubble it resolves.
awaiting: VecDeque<(String, String, Vec<ImageRef>)>, awaiting: VecDeque<(String, String, Vec<AttachmentRef>)>,
/// The process is gone, so nothing can be taken up any more. /// The process is gone, so nothing can be taken up any more.
/// ///
/// Needed because every other way out of a turn is an `Idle` this /// Needed because every other way out of a turn is an `Idle` this
@@ -550,20 +550,35 @@ impl ClaudeDriver {
} }
impl Driver for ClaudeDriver { impl Driver for ClaudeDriver {
fn send_user_message(&self, text: String, images: Vec<ImageRef>) { fn send_user_message(&self, text: String, attachments: Vec<AttachmentRef>) {
let mut content = Vec::new(); let mut content = Vec::new();
for id in &images { // An image goes into the message itself; the model looks at it. Any
match attachment_block(&self.session_dir, id) { // other file stays where the upload put it and the message says
Ok(block) => content.push(block), // where, because the CLI can read a file by path and a model cannot
Err(err) => { // be handed a trace, a log or a zip any other way. Named after the
let _ = self.sink.send(Event::Error { // text, so the words come first, the way they were typed.
message: format!("attachment {id} couldn't be sent: {err:#}"), let mut files = Vec::new();
}); for id in &attachments {
} let sent = if crate::media::media_type_for(id).is_some() {
attachment_block(&self.session_dir, id).map(|block| content.push(block))
} else {
attachment_path(&self.session_dir, id).map(|path| files.push(path))
};
if let Err(err) = sent {
let _ = self.sink.send(Event::Error {
message: format!("attachment {id} couldn't be sent: {err:#}"),
});
} }
} }
if !text.is_empty() { let mut body = text.clone();
content.push(json!({"type": "text", "text": text})); for path in files {
if !body.is_empty() {
body.push_str("\n\n");
}
body.push_str(&format!("Attached file: {}", path.display()));
}
if !body.is_empty() {
content.push(json!({"type": "text", "text": body}));
} }
let line = let line =
json!({"type": "user", "message": {"role": "user", "content": content}}).to_string(); json!({"type": "user", "message": {"role": "user", "content": content}}).to_string();
@@ -591,9 +606,13 @@ impl Driver for ClaudeDriver {
let id = super::random_hex(); let id = super::random_hex();
queue queue
.awaiting .awaiting
.push_back((id.clone(), text.clone(), images.clone())); .push_back((id.clone(), text.clone(), attachments.clone()));
drop(queue); drop(queue);
let _ = self.sink.send(Event::MessageQueued { id, text, images }); let _ = self.sink.send(Event::MessageQueued {
id,
text,
attachments,
});
self.send_line(line); self.send_line(line);
return; return;
} }
@@ -605,7 +624,7 @@ impl Driver for ClaudeDriver {
let _ = self.sink.send(Event::MessageTaken { let _ = self.sink.send(Event::MessageTaken {
id: None, id: None,
text, text,
images, attachments,
}); });
let _ = self.sink.send(Event::Status { let _ = self.sink.send(Event::Status {
state: SessionStatus::Running, state: SessionStatus::Running,
@@ -1087,16 +1106,16 @@ fn proves_a_turn(event: &Event) -> bool {
/// [`translate_line`], and the pair is the whole of the rule -- a steer /// [`translate_line`], and the pair is the whole of the rule -- a steer
/// announced anywhere else lands above output that predates it. /// announced anywhere else lands above output that predates it.
fn announce_steers(queue: &Arc<Mutex<Queue>>, sink: &EventSink) -> bool { fn announce_steers(queue: &Arc<Mutex<Queue>>, sink: &EventSink) -> bool {
let taken: Vec<(String, String, Vec<ImageRef>)> = { let taken: Vec<(String, String, Vec<AttachmentRef>)> = {
let mut queue = queue.lock().unwrap(); let mut queue = queue.lock().unwrap();
queue.awaiting.drain(..).collect() queue.awaiting.drain(..).collect()
}; };
for (id, text, images) in taken { for (id, text, attachments) in taken {
if sink if sink
.send(Event::MessageTaken { .send(Event::MessageTaken {
id: Some(id), id: Some(id),
text, text,
images, attachments,
}) })
.is_err() .is_err()
{ {
@@ -1185,17 +1204,28 @@ pub(super) fn write_resume_token(session_dir: &Path, session_id: &str) {
} }
} }
/// Reads an uploaded attachment into an API image content block. /// Where an uploaded attachment is, as a path the CLI can be told.
fn attachment_block(session_dir: &Path, id: &str) -> Result<Value> { ///
// Ids are server-generated hex (see routes::upload_attachment); the /// Absolute, because the CLI's working directory is the session's and the
// check keeps a crafted "id" from naming an arbitrary file. /// attachments are not in it. Refused rather than resolved when the id is
/// not one this server would have written -- see
/// `SessionManager::save_attachment` -- so a crafted id cannot name a file
/// outside the session.
fn attachment_path(session_dir: &Path, id: &str) -> Result<PathBuf> {
if !id if !id
.chars() .chars()
.all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '-') .all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == '_')
|| id.contains("..")
{ {
anyhow::bail!("invalid attachment id"); anyhow::bail!("invalid attachment id");
} }
let path = session_dir.join("attachments").join(id); let path = session_dir.join("attachments").join(id);
std::fs::canonicalize(&path).with_context(|| format!("find {}", path.display()))
}
/// Reads an uploaded image into an API image content block.
fn attachment_block(session_dir: &Path, id: &str) -> Result<Value> {
let path = attachment_path(session_dir, id)?;
let bytes = std::fs::read(&path).with_context(|| format!("read {}", path.display()))?; let bytes = std::fs::read(&path).with_context(|| format!("read {}", path.display()))?;
use base64::Engine; use base64::Engine;
Ok(json!({ Ok(json!({
+18 -7
View File
@@ -15,6 +15,14 @@ use tokio::sync::mpsc;
/// directions use the one id so the transcript renders them identically. /// directions use the one id so the transcript renders them identically.
pub type ImageRef = String; pub type ImageRef = String;
/// The name an upload from the phone is stored and served under: an image
/// is `<hex>.<extension>` and is an [`ImageRef`] like any other; any other
/// file keeps its own name after the hex, `<hex>-<name>`, because the name
/// is what the reader attached and what the session is told. The two are
/// told apart by `crate::media::media_type_for`, which knows every image
/// extension this server writes.
pub type AttachmentRef = String;
/// One choice offered in answer to a [`Event::Question`]. /// One choice offered in answer to a [`Event::Question`].
/// ///
/// More than a label because the reader is deciding, not confirming: what /// More than a label because the reader is deciding, not confirming: what
@@ -89,8 +97,11 @@ pub enum Event {
/// sent it -- and left the phone to decide, from nothing but /// sent it -- and left the phone to decide, from nothing but
/// adjacency, which message an image belonged to. Belonging is not /// adjacency, which message an image belonged to. Belonging is not
/// something to infer when the sender knew. /// something to infer when the sender knew.
#[serde(default, skip_serializing_if = "Vec::is_empty")] ///
images: Vec<ImageRef>, /// `images` on disk until 2026-09-03, when files joined them;
/// the alias reads the rows written before that.
#[serde(default, alias = "images", skip_serializing_if = "Vec::is_empty")]
attachments: Vec<AttachmentRef>,
}, },
/// A message accepted from the phone that the session cannot read yet. /// A message accepted from the phone that the session cannot read yet.
/// ///
@@ -111,8 +122,8 @@ pub enum Event {
/// Carried for the same reason [`Event::UserMessage`] carries it, /// Carried for the same reason [`Event::UserMessage`] carries it,
/// and it matters more here: a waiting message is on screen for as /// and it matters more here: a waiting message is on screen for as
/// long as the turn runs, so its attachment has nowhere else to be. /// long as the turn runs, so its attachment has nowhere else to be.
#[serde(default, skip_serializing_if = "Vec::is_empty")] #[serde(default, alias = "images", skip_serializing_if = "Vec::is_empty")]
images: Vec<ImageRef>, attachments: Vec<AttachmentRef>,
}, },
/// A message taken out of the queue before the session read it, by /// A message taken out of the queue before the session read it, by
/// somebody tapping the bubble that was waiting for it. /// somebody tapping the bubble that was waiting for it.
@@ -146,8 +157,8 @@ pub enum Event {
id: Option<String>, id: Option<String>,
text: String, text: String,
/// Carried through onto the `UserMessage` with everything else. /// Carried through onto the `UserMessage` with everything else.
#[serde(default, skip_serializing_if = "Vec::is_empty")] #[serde(default, alias = "images", skip_serializing_if = "Vec::is_empty")]
images: Vec<ImageRef>, attachments: Vec<AttachmentRef>,
}, },
/// Streaming assistant text; the phone renders the concatenation as /// Streaming assistant text; the phone renders the concatenation as
/// markdown. /// markdown.
@@ -515,7 +526,7 @@ pub trait Driver: Send + Sync {
/// moment it actually starts reading it: that event is what puts the /// moment it actually starts reading it: that event is what puts the
/// message in the transcript, so a driver that never sends it drops /// message in the transcript, so a driver that never sends it drops
/// the message from the conversation entirely. /// the message from the conversation entirely.
fn send_user_message(&self, text: String, images: Vec<ImageRef>); fn send_user_message(&self, text: String, attachments: Vec<AttachmentRef>);
/// Takes back a message that is still waiting, named by the id its /// Takes back a message that is still waiting, named by the id its
/// [`Event::MessageQueued`] carried. /// [`Event::MessageQueued`] carried.
/// ///
+44 -16
View File
@@ -7,6 +7,8 @@
//! A leading word asks for something more specific: //! A leading word asks for something more specific:
//! //!
//! - `/tool [input]` -- a full tool run, start through end. //! - `/tool [input]` -- a full tool run, start through end.
//! - `/bash [command]` -- a Bash call carrying that command, for what the
//! phone's shell highlighting does to a particular line.
//! - `/tools [n] [gap]` -- n calls back to back, for what a run of them //! - `/tools [n] [gap]` -- n calls back to back, for what a run of them
//! looks like when a screen groups them. `gap` is seconds between one //! looks like when a screen groups them. `gap` is seconds between one
//! call and the next, default none: it is what makes a run *grow* while //! call and the next, default none: it is what makes a run *grow* while
@@ -37,7 +39,7 @@
//! shape a real model's reply arrives in, and the one where the row a //! shape a real model's reply arrives in, and the one where the row a
//! reader is anchored to is the row that keeps changing height. //! reader is anchored to is the row that keeps changing height.
//! - `/mixed N` -- N beats of an interleaved transcript: paragraphs of //! - `/mixed N` -- N beats of an interleaved transcript: paragraphs of
//! different lengths, single tool calls, runs of adjacent ones, images //! different lengths, single tool calls, runs of adjacent ones, attachments
//! and a peer message. Rows of every shape and height the app draws, in //! and a peer message. Rows of every shape and height the app draws, in
//! one session, which is what a scrolling problem needs in order to be //! one session, which is what a scrolling problem needs in order to be
//! reproduced twice the same way. //! reproduced twice the same way.
@@ -55,7 +57,9 @@ use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use std::time::Duration; use std::time::Duration;
use super::driver::{Driver, Event, EventSink, ImageRef, QuestionOption, SessionStatus, Unqueued}; use super::driver::{
AttachmentRef, Driver, Event, EventSink, QuestionOption, SessionStatus, Unqueued,
};
/// Delay between streamed deltas -- long enough that streaming is visibly /// Delay between streamed deltas -- long enough that streaming is visibly
/// streaming in the UI, short enough that tests waiting on a full turn /// streaming in the UI, short enough that tests waiting on a full turn
@@ -94,7 +98,7 @@ pub struct EchoDriver {
/// Held messages with the id of the `MessageQueued` each one announced, /// Held messages with the id of the `MessageQueued` each one announced,
/// so the announcement can say which waiting bubble it resolves. /// so the announcement can say which waiting bubble it resolves.
queued: Arc<Mutex<Vec<Held>>>, queued: Arc<Mutex<Vec<Held>>>,
/// Where `/mixed` writes the images it references, which is the same /// Where `/mixed` writes the attachments it references, which is the same
/// directory the files route serves them from. /// directory the files route serves them from.
session_dir: PathBuf, session_dir: PathBuf,
/// Ids of the questions awaiting an answer, in the order they were /// Ids of the questions awaiting an answer, in the order they were
@@ -249,7 +253,7 @@ impl EchoDriver {
/// transcript, and a command is not -- the manager has already /// transcript, and a command is not -- the manager has already
/// recorded that one was sent, and saying so twice drew the same /// recorded that one was sent, and saying so twice drew the same
/// line in both colours. /// line in both colours.
fn handle(&self, text: String, images: Vec<ImageRef>, announce: bool) { fn handle(&self, text: String, attachments: Vec<AttachmentRef>, announce: bool) {
let sink = self.sink.clone(); let sink = self.sink.clone();
// Mid-turn messages are held rather than answered, the way a real // Mid-turn messages are held rather than answered, the way a real
@@ -265,9 +269,13 @@ impl EchoDriver {
self.queued self.queued
.lock() .lock()
.unwrap() .unwrap()
.push((id.clone(), text.clone(), images.clone())); .push((id.clone(), text.clone(), attachments.clone()));
if announce { if announce {
self.emit(Event::MessageQueued { id, text, images }); self.emit(Event::MessageQueued {
id,
text,
attachments,
});
} }
return; return;
} }
@@ -288,7 +296,7 @@ impl EchoDriver {
self.emit(Event::MessageTaken { self.emit(Event::MessageTaken {
id: None, id: None,
text: text.clone(), text: text.clone(),
images: images.clone(), attachments: attachments.clone(),
}); });
} }
self.emit(Event::Status { self.emit(Event::Status {
@@ -319,7 +327,7 @@ impl EchoDriver {
self.emit(Event::MessageTaken { self.emit(Event::MessageTaken {
id: None, id: None,
text: text.clone(), text: text.clone(),
images: images.clone(), attachments: attachments.clone(),
}); });
} }
self.emit(Event::PeerMessage { self.emit(Event::PeerMessage {
@@ -345,7 +353,7 @@ impl EchoDriver {
self.emit(Event::MessageTaken { self.emit(Event::MessageTaken {
id: None, id: None,
text, text,
images, attachments,
}); });
} }
self.compact(); self.compact();
@@ -357,7 +365,7 @@ impl EchoDriver {
self.emit(Event::MessageTaken { self.emit(Event::MessageTaken {
id: None, id: None,
text, text,
images, attachments,
}); });
} }
self.ask_user_question(); self.ask_user_question();
@@ -427,6 +435,9 @@ impl EchoDriver {
text.strip_prefix("/tool") text.strip_prefix("/tool")
.map(|rest| rest.trim().to_string()) .map(|rest| rest.trim().to_string())
}; };
let run_bash = text
.strip_prefix("/bash")
.map(|rest| rest.trim().to_string());
// Seconds to stay running before answering, default 30. Clamped // Seconds to stay running before answering, default 30. Clamped
// rather than trusted: this is a test affordance, and a session // rather than trusted: this is a test affordance, and a session
// pinned running for an hour by a typo is a worse outcome than a // pinned running for an hour by a typo is a worse outcome than a
@@ -469,7 +480,7 @@ impl EchoDriver {
send(Event::MessageTaken { send(Event::MessageTaken {
id: None, id: None,
text: text.clone(), text: text.clone(),
images: images.clone(), attachments: attachments.clone(),
}); });
} }
send(Event::Status { send(Event::Status {
@@ -585,6 +596,23 @@ impl EchoDriver {
return; return;
} }
if let Some(command) = run_bash {
let id = format!("b-{}", super::random_hex());
send(Event::ToolStart {
id: id.clone(),
tool: "Bash".to_string(),
input: serde_json::json!({
"command": command,
"description": "Run what /bash was given",
}),
});
tokio::time::sleep(DELTA_DELAY).await;
send(Event::ToolEnd {
id,
output: format!("ran: {command}"),
});
}
if let Some(input) = run_tool { if let Some(input) = run_tool {
let id = format!("t-{}", super::random_hex()); let id = format!("t-{}", super::random_hex());
send(Event::ToolStart { send(Event::ToolStart {
@@ -762,7 +790,7 @@ async fn write_beat(sink: &EventSink, session_dir: &Path, beat: usize) {
/// it. All three, because all three are what the `MessageTaken` at the other /// it. All three, because all three are what the `MessageTaken` at the other
/// end owes -- named rather than written out at each of the four places that /// end owes -- named rather than written out at each of the four places that
/// mention it. /// mention it.
type Held = (String, String, Vec<ImageRef>); type Held = (String, String, Vec<AttachmentRef>);
/// A markdown table [columns] wide, with cells too long for one line. /// A markdown table [columns] wide, with cells too long for one line.
/// ///
@@ -832,7 +860,7 @@ fn markdown_table(columns: usize) -> String {
/// of them owes the same answer. /// of them owes the same answer.
fn finish_turn(sink: &EventSink, queued: &Mutex<Vec<Held>>, busy: &AtomicBool) { fn finish_turn(sink: &EventSink, queued: &Mutex<Vec<Held>>, busy: &AtomicBool) {
let held = std::mem::take(&mut *queued.lock().unwrap()); let held = std::mem::take(&mut *queued.lock().unwrap());
for (id, text, images) in held { for (id, text, attachments) in held {
// Announced before it is answered, in that order: a phone showing // Announced before it is answered, in that order: a phone showing
// the message as pending needs the signal that it has been read, // the message as pending needs the signal that it has been read,
// and the answer is meaningless above a message still drawn as // and the answer is meaningless above a message still drawn as
@@ -840,7 +868,7 @@ fn finish_turn(sink: &EventSink, queued: &Mutex<Vec<Held>>, busy: &AtomicBool) {
let _ = sink.send(Event::MessageTaken { let _ = sink.send(Event::MessageTaken {
id: Some(id), id: Some(id),
text: text.clone(), text: text.clone(),
images, attachments,
}); });
let _ = sink.send(Event::AssistantText { let _ = sink.send(Event::AssistantText {
delta: format!("\n(taken from the queue) You said: {text}"), delta: format!("\n(taken from the queue) You said: {text}"),
@@ -874,14 +902,14 @@ impl Driver for EchoDriver {
Unqueued::Dropped Unqueued::Dropped
} }
fn send_user_message(&self, text: String, images: Vec<ImageRef>) { fn send_user_message(&self, text: String, attachments: Vec<AttachmentRef>) {
// Announced, because this is a message: every driver owes exactly // Announced, because this is a message: every driver owes exactly
// one `MessageTaken` per message, and one that quietly vanishes // one `MessageTaken` per message, and one that quietly vanishes
// from the transcript is the thing echo must not model. A command // from the transcript is the thing echo must not model. A command
// owes none -- the manager has already recorded that it was sent, // owes none -- the manager has already recorded that it was sent,
// and announcing it again drew the same line twice, once in each // and announcing it again drew the same line twice, once in each
// colour. // colour.
self.handle(text, images, true); self.handle(text, attachments, true);
} }
/// Echo's commands *are* its messages -- `/tool`, `/slow`, `/ask` -- /// Echo's commands *are* its messages -- `/tool`, `/slow`, `/ask` --
+1 -1
View File
@@ -628,7 +628,7 @@ fn push_user(events: &mut Vec<Event>, content: &Value, session_dir: &std::path::
events.push(Event::UserMessage { events.push(Event::UserMessage {
id: None, id: None,
text, text,
images: Vec::new(), attachments: Vec::new(),
}); });
} }
} }
+16 -16
View File
@@ -33,7 +33,7 @@ use anyhow::{Context, Result, bail};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
use serde_json::json; use serde_json::json;
use super::driver::{Driver, Event, EventSink, ImageRef, SessionStatus}; use super::driver::{AttachmentRef, Driver, Event, EventSink, SessionStatus};
use super::process; use super::process;
use super::transport::{Launch, Streams, Transport}; use super::transport::{Launch, Streams, Transport};
use crate::config::{ProviderConfig, SessionConfig}; use crate::config::{ProviderConfig, SessionConfig};
@@ -322,10 +322,10 @@ fn watch(session_dir: PathBuf, sink: EventSink) {
} }
impl Driver for LlamaDriver { impl Driver for LlamaDriver {
fn send_user_message(&self, text: String, images: Vec<ImageRef>) { fn send_user_message(&self, text: String, attachments: Vec<AttachmentRef>) {
if !images.is_empty() { if !attachments.is_empty() {
let _ = self.sink.send(Event::Error { let _ = self.sink.send(Event::Error {
message: "this model can't be sent images".to_string(), message: "this model can't be sent attachments or files".to_string(),
}); });
} }
let sink = self.sink.clone(); let sink = self.sink.clone();
@@ -344,9 +344,9 @@ impl Driver for LlamaDriver {
let _ = sink.send(Event::MessageTaken { let _ = sink.send(Event::MessageTaken {
id: None, id: None,
text: text.clone(), text: text.clone(),
// Never any: this driver refuses images above, and saying // Never any: this driver refuses attachments above, and
// so is what the refusal above is for. // saying so is what the refusal above is for.
images: Vec::new(), attachments: Vec::new(),
}); });
let _ = sink.send(Event::Status { let _ = sink.send(Event::Status {
state: SessionStatus::Running, state: SessionStatus::Running,
@@ -649,7 +649,7 @@ mod tests {
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "hello".into(), text: "hello".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
Event::AssistantText { Event::AssistantText {
delta: "hi ".into(), delta: "hi ".into(),
@@ -663,7 +663,7 @@ mod tests {
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "again".into(), text: "again".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
Event::AssistantText { Event::AssistantText {
delta: "yes".into(), delta: "yes".into(),
@@ -695,7 +695,7 @@ mod tests {
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "count".into(), text: "count".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
Event::AssistantText { Event::AssistantText {
delta: "one two".into(), delta: "one two".into(),
@@ -721,7 +721,7 @@ mod tests {
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "hello".into(), text: "hello".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
Event::Error { Event::Error {
message: "something went wrong".into(), message: "something went wrong".into(),
@@ -749,7 +749,7 @@ mod tests {
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "the long expensive conversation".into(), text: "the long expensive conversation".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
Event::AssistantText { Event::AssistantText {
delta: "at length".into(), delta: "at length".into(),
@@ -758,7 +758,7 @@ mod tests {
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "a fresh start".into(), text: "a fresh start".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
Event::AssistantText { Event::AssistantText {
delta: "cheaply".into(), delta: "cheaply".into(),
@@ -778,19 +778,19 @@ mod tests {
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "one".into(), text: "one".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
Event::Cleared, Event::Cleared,
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "two".into(), text: "two".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
Event::Cleared, Event::Cleared,
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "three".into(), text: "three".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
]); ]);
let messages = conversation(&path); let messages = conversation(&path);
+65 -11
View File
@@ -33,7 +33,7 @@ use crate::config::{
}; };
use claude::ClaudeDriver; use claude::ClaudeDriver;
use driver::{ use driver::{
Driver, Event, EventSink, ImageRef, SessionCommand, SessionStatus, Unqueued, context_after, AttachmentRef, Driver, Event, EventSink, SessionCommand, SessionStatus, Unqueued, context_after,
}; };
use echo::EchoDriver; use echo::EchoDriver;
use llama::LlamaDriver; use llama::LlamaDriver;
@@ -417,14 +417,14 @@ impl LiveSession {
/// The message is deliberately not recorded here. Sent into a running /// The message is deliberately not recorded here. Sent into a running
/// turn it waits, and writing it down on the way past would put it /// turn it waits, and writing it down on the way past would put it
/// above output that happened before the session ever saw it. /// above output that happened before the session ever saw it.
pub fn send_message(&self, text: String, images: Vec<ImageRef>) { pub fn send_message(&self, text: String, attachments: Vec<AttachmentRef>) {
// The attachments ride *on* the message rather than as `Image` // The attachments ride *on* the message rather than as `Image`
// events emitted just before it. They used to be the latter, which // events emitted just before it. They used to be the latter, which
// drew a person's screenshot as a row floating above the bubble // drew a person's screenshot as a row floating above the bubble
// that sent it, and left the phone inferring from adjacency which // that sent it, and left the phone inferring from adjacency which
// message an image went with -- a thing the sender already knew. // message an image went with -- a thing the sender already knew.
self.ask("take a message", |driver| { self.ask("take a message", |driver| {
driver.send_user_message(text, images) driver.send_user_message(text, attachments)
}); });
} }
@@ -488,11 +488,24 @@ impl LiveSession {
/// Stores one uploaded attachment, returning the id `POST /message` /// Stores one uploaded attachment, returning the id `POST /message`
/// references it by. Removed with the session directory on delete -- /// references it by. Removed with the session directory on delete --
/// the same path out as everything else in it. /// the same path out as everything else in it.
pub fn save_attachment(&self, bytes: &[u8], content_type: &str) -> Result<String> { ///
// An unrecognized type is almost always a phone photo whose /// An image is named `<hex>.<extension>` and nothing else, since the
// content type the picker didn't set; jpg is the useful guess. /// model is shown the picture rather than told its name. Anything else
let extension = crate::media::extension_for(content_type).unwrap_or("jpg"); /// keeps the name it arrived with after the hex: the session is told
let name = format!("{}.{extension}", random_hex()); /// the path, and a trace called `trace-komodo-….perfetto-trace` says
/// more to it than `3f9a…` would. The name is cleaned to characters a
/// path and a URL both take unquoted, and the hex keeps two uploads of
/// the same name apart. `AttachmentRef` documents the two shapes.
pub fn save_attachment(
&self,
bytes: &[u8],
content_type: &str,
file_name: Option<&str>,
) -> Result<AttachmentRef> {
let name = match crate::media::extension_for(content_type) {
Some(extension) => format!("{}.{extension}", random_hex()),
None => format!("{}-{}", random_hex(), safe_file_name(file_name)),
};
let dir = self.dir().join("attachments"); let dir = self.dir().join("attachments");
wg_app_link::private::create_dir(&dir)?; wg_app_link::private::create_dir(&dir)?;
std::fs::write(dir.join(&name), bytes) std::fs::write(dir.join(&name), bytes)
@@ -1442,14 +1455,19 @@ impl SessionManager {
/// Started before the message rather than after, because starting /// Started before the message rather than after, because starting
/// replaces the driver and the driver that takes the message has to be /// replaces the driver and the driver that takes the message has to be
/// the one with a process behind it. /// the one with a process behind it.
pub fn send_message(&self, id: &str, text: String, images: Vec<ImageRef>) -> Result<()> { pub fn send_message(
&self,
id: &str,
text: String,
attachments: Vec<AttachmentRef>,
) -> Result<()> {
// Only `Exited` starts anything -- see `start_if_exited`. A session // Only `Exited` starts anything -- see `start_if_exited`. A session
// this cannot say has exited keeps the behaviour it always had: the // this cannot say has exited keeps the behaviour it always had: the
// message goes to the driver, which answers for it. // message goes to the driver, which answers for it.
self.start_if_exited(id)?; self.start_if_exited(id)?;
self.session(id) self.session(id)
.with_context(|| format!("no session {id}"))? .with_context(|| format!("no session {id}"))?
.send_message(text, images); .send_message(text, attachments);
Ok(()) Ok(())
} }
@@ -1767,6 +1785,34 @@ fn names<'a>(all: impl Iterator<Item = &'a str>) -> String {
/// 8 random bytes, hex -- short enough for a URL, unique enough forever at /// 8 random bytes, hex -- short enough for a URL, unique enough forever at
/// this scale. /// this scale.
/// A file name reduced to what an attachment id may hold: letters, digits,
/// `.`, `-` and `_`, no run of dots that could read as a parent directory,
/// at most [`FILE_NAME_LIMIT`] characters keeping the tail (the extension
/// is what identifies a file), and `file` when nothing usable is left.
fn safe_file_name(name: Option<&str>) -> String {
let cleaned: String = name
.unwrap_or_default()
.chars()
.map(|c| {
if c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == '_' {
c
} else {
'_'
}
})
.collect();
let cleaned = cleaned.replace("..", "_").trim_matches('.').to_string();
if cleaned.is_empty() {
return "file".to_string();
}
let excess = cleaned.chars().count().saturating_sub(FILE_NAME_LIMIT);
cleaned.chars().skip(excess).collect()
}
/// Longer than any name a person types, shorter than what a filesystem
/// refuses once the hex and a dash are in front of it.
const FILE_NAME_LIMIT: usize = 120;
pub fn random_hex() -> String { pub fn random_hex() -> String {
use rand::Rng; use rand::Rng;
let mut bytes = [0u8; 8]; let mut bytes = [0u8; 8];
@@ -2232,7 +2278,15 @@ async fn pump(
// the position is the only way a reader can put it back where it // the position is the only way a reader can put it back where it
// happened -- see `Event::PeerMessage::turn_start`. // happened -- see `Event::PeerMessage::turn_start`.
let event = match event { let event = match event {
Event::MessageTaken { id, text, images } => Event::UserMessage { id, text, images }, Event::MessageTaken {
id,
text,
attachments,
} => Event::UserMessage {
id,
text,
attachments,
},
Event::PeerMessage { from, text, .. } => Event::PeerMessage { Event::PeerMessage { from, text, .. } => Event::PeerMessage {
from, from,
text, text,
+1 -1
View File
@@ -636,7 +636,7 @@ mod tests {
Event::UserMessage { Event::UserMessage {
id: None, id: None,
text: "hi".into(), text: "hi".into(),
images: Vec::new(), attachments: Vec::new(),
}, },
text("hello"), text("hello"),
Event::ToolStart { Event::ToolStart {