Attach any file, take shares from other apps, and survive a backwards highlight
Attachments were images only. Now any file can be attached: from the file chooser behind the "+" menu, or from Android's share sheet, which the app is now in. An image still goes to the model as a picture; anything else is stored under its own name (`<hex>-<name>`, cleaned by `safe_file_name`) and the Claude driver ends the message with `Attached file: /abs/path`, since the CLI reads files by path and a model cannot be shown a trace. The user-message field is renamed `images` -> `attachments` on both sides, with a serde alias reading the rows written before. A share arrives before anyone has said which session it is for, so it is held in AppRoot with a banner on the list until a session takes it; an open session takes it at once. Unreadable shares are reported beside the composer, not thrown. The tool card crashed the app when opened on a command holding a quoted glob such as `-path '*/.git/*'`: highlights 1.1.0's shell lexer answers `x '*/a/*'` with a span whose end is before its start, and AnnotatedString refuses the range. Such spans are dropped; the library is the place for the fix. The echo driver gains `/bash <command>` so a card with a given command can be produced on the emulator. ui-sandbox.sh's token salvage read the tokens block's close only at a line start, ran past the compact `),],` the server writes, and copied `setups` into the new config twice, which the server then refused. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
1 parent
4bc69e8f9c
commit
6180663f14
25 files changed
+653
-146
No files matched your search
@@ -65,6 +65,19 @@ repo is in PLAN.md's "Backend layout" section.
|
|||||||
refuse to resume onto a partial from a different revision, and are
|
refuse to resume onto a partial from a different revision, and are
|
||||||
checked against HuggingFace's published sha256 before the file gets its
|
checked against HuggingFace's published sha256 before the file gets its
|
||||||
real name.
|
real name.
|
||||||
|
- **Attachments** are one list on a user message (`attachments`, the
|
||||||
|
ref the files route serves), in two shapes. An image is `<hex>.<ext>`
|
||||||
|
and goes to the model as an image block. Anything else is
|
||||||
|
`<hex>-<name>` -- the name it was shared or picked under, cleaned by
|
||||||
|
`safe_file_name` -- and the Claude driver appends `Attached file:
|
||||||
|
/abs/path` to the message text, since the CLI reads files by path and
|
||||||
|
a model cannot be shown a trace. `media::media_type_for` on the server
|
||||||
|
and `isImageRef` on the phone tell the two apart; keep those lists
|
||||||
|
level. The phone attaches from the photo picker, the file chooser and
|
||||||
|
Android's share sheet (`Share.kt`; the manifest's SEND filter), all
|
||||||
|
through one `attach` path in `SessionScreen`. Files exist only on the
|
||||||
|
server's machine -- see PLAN.md's "Transport" for what that means for
|
||||||
|
remote sessions.
|
||||||
- `server/` — Rust backend (`ai-server`). `main.rs` bootstraps (TLS, the
|
- `server/` — Rust backend (`ai-server`). `main.rs` bootstraps (TLS, the
|
||||||
auth layer, token/QR enrollment, wg0 binding), `routes.rs` has the HTTP
|
auth layer, token/QR enrollment, wg0 binding), `routes.rs` has the HTTP
|
||||||
table in its module doc comment, `auth.rs` the bearer-token middleware,
|
table in its module doc comment, `auth.rs` the bearer-token middleware,
|
||||||
|
|||||||
@@ -713,11 +713,18 @@ host) and **hosts**. The manager runs at most one llama-server per
|
|||||||
forwarded port (`ssh -L`) as well as a spawned process. A transport is
|
forwarded port (`ssh -L`) as well as a spawned process. A transport is
|
||||||
therefore "run this" plus "reach this port", and the second operation is
|
therefore "run this" plus "reach this port", and the second operation is
|
||||||
a no-op locally.
|
a no-op locally.
|
||||||
- Attachments need no file transfer, contrary to what this section said
|
- Images need no file transfer, contrary to what this section said
|
||||||
before: `attachment_block` base64s an uploaded image into the
|
before: `attachment_block` base64s an uploaded image into the
|
||||||
stream-json message itself, and produced images come back the same way
|
stream-json message itself, and produced images come back the same way
|
||||||
for the translator to write out locally. Nothing has to exist on the
|
for the translator to write out locally. Nothing has to exist on the
|
||||||
remote filesystem, so there is no `scp` step to get wrong.
|
remote filesystem, so there is no `scp` step to get wrong.
|
||||||
|
- **Any other file is told to the session by path** (2026-09-03: a trace,
|
||||||
|
a log, a zip -- things a model cannot be shown and the CLI can read).
|
||||||
|
The upload stays under the session's `attachments/` and the message
|
||||||
|
ends with `Attached file: /abs/path`. That directory exists only on the
|
||||||
|
machine running this server, so a file attached to a remote (ssh)
|
||||||
|
session names a path that is not there. Shipping it is not built; the
|
||||||
|
one host in use runs its sessions locally. Images are unaffected.
|
||||||
|
|
||||||
### Usage limits (Claude)
|
### Usage limits (Claude)
|
||||||
|
|
||||||
@@ -1094,9 +1101,11 @@ window just fills.
|
|||||||
with port forward, attachment shipping. *Host config and remote spawn
|
with port forward, attachment shipping. *Host config and remote spawn
|
||||||
done 2026-08-25* (any session of any provider can name a host; the
|
done 2026-08-25* (any session of any provider can name a host; the
|
||||||
command is the identical one wrapped in `ssh -T`, with every argument
|
command is the identical one wrapped in `ssh -T`, with every argument
|
||||||
shell-quoted). Attachment shipping turned out to be unnecessary for the
|
shell-quoted). Attachment shipping turned out to be unnecessary for
|
||||||
Claude driver — images ride the stdio JSONL as base64 in both
|
images — they ride the stdio JSONL as base64 in both directions, so
|
||||||
directions, so nothing needs `scp`. Still outstanding: remote
|
nothing needs `scp` — and became necessary again on 2026-09-03 for
|
||||||
|
files, which are attached by path (see "Transport" above). Still
|
||||||
|
outstanding: file shipping for remote sessions, and remote
|
||||||
llama-server with its port forward, which comes with phase 4.
|
llama-server with its port forward, which comes with phase 4.
|
||||||
Two things learned doing it: a remote session inherits ssh's non-login
|
Two things learned doing it: a remote session inherits ssh's non-login
|
||||||
PATH, which is narrower than an interactive shell's (point `command` at
|
PATH, which is narrower than an interactive shell's (point `command` at
|
||||||
|
|||||||
@@ -61,6 +61,16 @@
|
|||||||
<category android:name="android.intent.category.BROWSABLE" />
|
<category android:name="android.intent.category.BROWSABLE" />
|
||||||
<data android:scheme="aiapp" android:host="enroll" />
|
<data android:scheme="aiapp" android:host="enroll" />
|
||||||
</intent-filter>
|
</intent-filter>
|
||||||
|
<!-- The share sheet: a file, a photo or some text from another app
|
||||||
|
lands here and is attached to a session (see Share.kt). Any
|
||||||
|
type, because what a session can be handed is the server's
|
||||||
|
decision rather than the sheet's. -->
|
||||||
|
<intent-filter>
|
||||||
|
<action android:name="android.intent.action.SEND" />
|
||||||
|
<action android:name="android.intent.action.SEND_MULTIPLE" />
|
||||||
|
<category android:name="android.intent.category.DEFAULT" />
|
||||||
|
<data android:mimeType="*/*" />
|
||||||
|
</intent-filter>
|
||||||
</activity>
|
</activity>
|
||||||
|
|
||||||
<!-- specialUse rather than dataSync, which is the type this looks
|
<!-- specialUse rather than dataSync, which is the type this looks
|
||||||
|
|||||||
@@ -539,17 +539,24 @@ fun setSessionCwd(settings: ServerSettings, sessionId: String, cwd: String) {
|
|||||||
) {}
|
) {}
|
||||||
}
|
}
|
||||||
|
|
||||||
/** Uploads one picked image; the returned id goes into [sendMessage]. */
|
/**
|
||||||
|
* Uploads one attachment; the returned id goes into [sendMessage]. [name] is what the server keeps
|
||||||
|
* a file under and tells the session; for an image it is ignored, since the model is shown the
|
||||||
|
* picture rather than told its name.
|
||||||
|
*/
|
||||||
fun uploadAttachment(
|
fun uploadAttachment(
|
||||||
settings: ServerSettings,
|
settings: ServerSettings,
|
||||||
sessionId: String,
|
sessionId: String,
|
||||||
bytes: ByteArray,
|
bytes: ByteArray,
|
||||||
mime: String,
|
mime: String,
|
||||||
|
name: String,
|
||||||
): String {
|
): String {
|
||||||
val boundary = "----aiapp-${System.currentTimeMillis()}"
|
val boundary = "----aiapp-${System.currentTimeMillis()}"
|
||||||
|
// The header is a line: a quote or a line break in the name would end it early.
|
||||||
|
val safeName = name.replace(Regex("[\"\r\n]"), "_")
|
||||||
val head =
|
val head =
|
||||||
("--$boundary\r\n" +
|
("--$boundary\r\n" +
|
||||||
"Content-Disposition: form-data; name=\"file\"; filename=\"image\"\r\n" +
|
"Content-Disposition: form-data; name=\"file\"; filename=\"$safeName\"\r\n" +
|
||||||
"Content-Type: $mime\r\n\r\n")
|
"Content-Type: $mime\r\n\r\n")
|
||||||
.encodeToByteArray()
|
.encodeToByteArray()
|
||||||
val tail = "\r\n--$boundary--\r\n".encodeToByteArray()
|
val tail = "\r\n--$boundary--\r\n".encodeToByteArray()
|
||||||
|
|||||||
@@ -62,9 +62,16 @@ private data class FailedOpen(val request: SessionOpenRequest, val message: Stri
|
|||||||
* re-reading the stored settings -- a plain `remember` would keep serving the pre-enrollment null.
|
* re-reading the stored settings -- a plain `remember` would keep serving the pre-enrollment null.
|
||||||
*
|
*
|
||||||
* [openRequest] is the session a notification tap asked for, likewise from MainActivity.
|
* [openRequest] is the session a notification tap asked for, likewise from MainActivity.
|
||||||
|
*
|
||||||
|
* [shareRequest] is what another app shared in, likewise. It is held here until a session takes it,
|
||||||
|
* because the share arrives before anyone has said which session it is for.
|
||||||
*/
|
*/
|
||||||
@Composable
|
@Composable
|
||||||
fun AppRoot(settingsVersion: Int, openRequest: SessionOpenRequest?) {
|
fun AppRoot(
|
||||||
|
settingsVersion: Int,
|
||||||
|
openRequest: SessionOpenRequest?,
|
||||||
|
shareRequest: ShareRequest? = null,
|
||||||
|
) {
|
||||||
val context = LocalContext.current
|
val context = LocalContext.current
|
||||||
val scope = rememberCoroutineScope()
|
val scope = rememberCoroutineScope()
|
||||||
var settings by remember(settingsVersion) { mutableStateOf(loadServerSettings(context)) }
|
var settings by remember(settingsVersion) { mutableStateOf(loadServerSettings(context)) }
|
||||||
@@ -75,6 +82,17 @@ fun AppRoot(settingsVersion: Int, openRequest: SessionOpenRequest?) {
|
|||||||
// Bumped whenever another screen changes something the list shows, so
|
// Bumped whenever another screen changes something the list shows, so
|
||||||
// returning to it refetches instead of showing a stale list.
|
// returning to it refetches instead of showing a stale list.
|
||||||
var reloadToken by remember { mutableIntStateOf(0) }
|
var reloadToken by remember { mutableIntStateOf(0) }
|
||||||
|
// Cleared by the session screen that attached it, not when a newer request arrives: a share
|
||||||
|
// must be attached exactly once, and only the screen that did it knows that it has.
|
||||||
|
var share by remember { mutableStateOf<ShareRequest?>(null) }
|
||||||
|
LaunchedEffect(shareRequest) {
|
||||||
|
if (shareRequest != null) {
|
||||||
|
share = shareRequest
|
||||||
|
// A session already open takes it. Otherwise the list is where the choice is made,
|
||||||
|
// whatever screen was showing: Spawn and Settings have nowhere to put a file.
|
||||||
|
if (screen !is Screen.Session) screen = Screen.Main
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// A standing condition rather than a per-request failure, so it is
|
// A standing condition rather than a per-request failure, so it is
|
||||||
// stated once here instead of appended to every error that might be
|
// stated once here instead of appended to every error that might be
|
||||||
@@ -162,6 +180,7 @@ fun AppRoot(settingsVersion: Int, openRequest: SessionOpenRequest?) {
|
|||||||
MainScreen(
|
MainScreen(
|
||||||
settings = current,
|
settings = current,
|
||||||
reloadToken = reloadToken,
|
reloadToken = reloadToken,
|
||||||
|
share = share,
|
||||||
onOpen = { screen = Screen.Session(it) },
|
onOpen = { screen = Screen.Session(it) },
|
||||||
onSpawn = { screen = Screen.Spawn },
|
onSpawn = { screen = Screen.Spawn },
|
||||||
onImported = { imported ->
|
onImported = { imported ->
|
||||||
@@ -179,7 +198,13 @@ fun AppRoot(settingsVersion: Int, openRequest: SessionOpenRequest?) {
|
|||||||
// row key. Only reachable since a notification can move straight from one session to
|
// row key. Only reachable since a notification can move straight from one session to
|
||||||
// another; every other way here passes through [Screen.Main], which disposes it anyway.
|
// another; every other way here passes through [Screen.Main], which disposes it anyway.
|
||||||
key(here.summary.id) {
|
key(here.summary.id) {
|
||||||
SessionScreen(settings = current, summary = here.summary, onBack = goToMain)
|
SessionScreen(
|
||||||
|
settings = current,
|
||||||
|
summary = here.summary,
|
||||||
|
onBack = goToMain,
|
||||||
|
share = share,
|
||||||
|
onShareTaken = { share = null },
|
||||||
|
)
|
||||||
}
|
}
|
||||||
is Screen.Spawn ->
|
is Screen.Spawn ->
|
||||||
Box(Modifier.imePadding()) {
|
Box(Modifier.imePadding()) {
|
||||||
|
|||||||
@@ -0,0 +1,66 @@
|
|||||||
|
package com.example.aiapp
|
||||||
|
|
||||||
|
import androidx.compose.foundation.background
|
||||||
|
import androidx.compose.foundation.layout.padding
|
||||||
|
import androidx.compose.material3.MaterialTheme
|
||||||
|
import androidx.compose.material3.Text
|
||||||
|
import androidx.compose.runtime.Composable
|
||||||
|
import androidx.compose.ui.Modifier
|
||||||
|
import androidx.compose.ui.draw.clip
|
||||||
|
import androidx.compose.ui.text.font.FontFamily
|
||||||
|
import androidx.compose.ui.text.style.TextOverflow
|
||||||
|
import androidx.compose.ui.unit.dp
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Whether [ref] names an image the server stored as one -- `<hex>.<extension>`, with an extension
|
||||||
|
* from the list it writes -- rather than a file kept under its own name. Mirrors the server's
|
||||||
|
* `media` table, which is the one other place the list lives.
|
||||||
|
*/
|
||||||
|
fun isImageRef(ref: String): Boolean = ref.substringAfterLast('.', "") in IMAGE_EXTENSIONS
|
||||||
|
|
||||||
|
private val IMAGE_EXTENSIONS = setOf("png", "jpg", "gif", "webp")
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The name a file was attached under: the ref less the hex the server put before it. The hex has no
|
||||||
|
* dash in it, so the first one is the boundary however many the name has.
|
||||||
|
*/
|
||||||
|
fun attachmentName(ref: String): String = ref.substringAfter('-', ref)
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One attachment on a sent message, drawn as what it is: an image inline, a file as its name. A
|
||||||
|
* file is not fetched -- there is nothing on this phone to open a trace or a log with -- so the
|
||||||
|
* name is the whole of it.
|
||||||
|
*/
|
||||||
|
@Composable
|
||||||
|
fun Attachment(
|
||||||
|
settings: ServerSettings,
|
||||||
|
sessionId: String,
|
||||||
|
ref: String,
|
||||||
|
onOpenImage: (String) -> Unit,
|
||||||
|
) {
|
||||||
|
if (isImageRef(ref)) SessionImage(settings, sessionId, ref, onOpenImage)
|
||||||
|
else
|
||||||
|
FileName(
|
||||||
|
attachmentName(ref),
|
||||||
|
Modifier.clip(MaterialTheme.shapes.extraSmall)
|
||||||
|
.background(rawSurface)
|
||||||
|
.padding(horizontal = 8.dp, vertical = 4.dp),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* A file's name, one line, in the face names are read in. Overlong names lose their middle: a name
|
||||||
|
* is identified by both ends -- what it is at the front, what kind at the back -- and either
|
||||||
|
* ellipsis alone takes away one of them.
|
||||||
|
*/
|
||||||
|
@Composable
|
||||||
|
fun FileName(name: String, modifier: Modifier = Modifier) {
|
||||||
|
Text(
|
||||||
|
name,
|
||||||
|
modifier = modifier,
|
||||||
|
style = MaterialTheme.typography.bodySmall,
|
||||||
|
fontFamily = FontFamily.Monospace,
|
||||||
|
maxLines = 1,
|
||||||
|
overflow = TextOverflow.MiddleEllipsis,
|
||||||
|
)
|
||||||
|
}
|
||||||
@@ -1,10 +1,12 @@
|
|||||||
package com.example.aiapp
|
package com.example.aiapp
|
||||||
|
|
||||||
|
import android.content.ContentResolver
|
||||||
import android.content.Context
|
import android.content.Context
|
||||||
import android.graphics.Bitmap
|
import android.graphics.Bitmap
|
||||||
import android.graphics.BitmapFactory
|
import android.graphics.BitmapFactory
|
||||||
import android.graphics.Matrix
|
import android.graphics.Matrix
|
||||||
import android.net.Uri
|
import android.net.Uri
|
||||||
|
import android.provider.OpenableColumns
|
||||||
import androidx.exifinterface.media.ExifInterface
|
import androidx.exifinterface.media.ExifInterface
|
||||||
import java.io.ByteArrayOutputStream
|
import java.io.ByteArrayOutputStream
|
||||||
import kotlin.math.max
|
import kotlin.math.max
|
||||||
@@ -31,7 +33,66 @@ suspend fun uploadPickedImage(
|
|||||||
maxEdge: Int?,
|
maxEdge: Int?,
|
||||||
): String {
|
): String {
|
||||||
val (bytes, mime) = readForUpload(context, uri, maxEdge)
|
val (bytes, mime) = readForUpload(context, uri, maxEdge)
|
||||||
return uploadAttachment(settings, sessionId, bytes, mime)
|
return uploadAttachment(settings, sessionId, bytes, mime, "image")
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Uploads whatever [uri] names, the way its kind needs. An image goes through [uploadPickedImage]
|
||||||
|
* and is shrunk; anything else goes whole, under the name the other app or the file chooser gave
|
||||||
|
* it, because the session is told that name rather than shown the bytes.
|
||||||
|
*/
|
||||||
|
suspend fun uploadPicked(
|
||||||
|
context: Context,
|
||||||
|
settings: ServerSettings,
|
||||||
|
sessionId: String,
|
||||||
|
uri: Uri,
|
||||||
|
maxEdge: Int?,
|
||||||
|
): String {
|
||||||
|
val resolver = context.contentResolver
|
||||||
|
val mime = resolver.getType(uri)
|
||||||
|
if (mime != null && mime.startsWith("image/")) {
|
||||||
|
return uploadPickedImage(context, settings, sessionId, uri, maxEdge)
|
||||||
|
}
|
||||||
|
val bytes = readAll(resolver, uri)
|
||||||
|
return uploadAttachment(
|
||||||
|
settings,
|
||||||
|
sessionId,
|
||||||
|
bytes,
|
||||||
|
mime ?: "application/octet-stream",
|
||||||
|
displayName(resolver, uri),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* Everything at [uri], or the failure as the kind the composer reports beside the message.
|
||||||
|
*
|
||||||
|
* A share arrives with whatever access the other app granted, and a provider that refuses says so
|
||||||
|
* with a `SecurityException`; a file gone between the pick and the read is an `IOException`. Both
|
||||||
|
* are things the reader can act on, so neither is left to end the process.
|
||||||
|
*/
|
||||||
|
private fun readAll(resolver: ContentResolver, uri: Uri): ByteArray =
|
||||||
|
try {
|
||||||
|
resolver.openInputStream(uri)?.use { it.readBytes() }
|
||||||
|
?: throw ApiException("couldn't read ${uri.lastPathSegment ?: uri}: nothing there")
|
||||||
|
} catch (e: SecurityException) {
|
||||||
|
throw ApiException("couldn't read ${uri.lastPathSegment ?: uri}: no access to it")
|
||||||
|
} catch (e: java.io.IOException) {
|
||||||
|
throw ApiException("couldn't read ${uri.lastPathSegment ?: uri}: ${e.message}")
|
||||||
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* The name a document provider shows for [uri]. The last path segment is the fallback because a
|
||||||
|
* provider's own id for a file is usually a number, which says nothing to the session.
|
||||||
|
*/
|
||||||
|
private fun displayName(resolver: ContentResolver, uri: Uri): String {
|
||||||
|
resolver.query(uri, arrayOf(OpenableColumns.DISPLAY_NAME), null, null, null)?.use { cursor ->
|
||||||
|
val column = cursor.getColumnIndex(OpenableColumns.DISPLAY_NAME)
|
||||||
|
if (column >= 0 && cursor.moveToFirst())
|
||||||
|
cursor.getString(column)?.let {
|
||||||
|
return it
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return uri.lastPathSegment ?: "file"
|
||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -44,9 +105,7 @@ suspend fun uploadPickedImage(
|
|||||||
private fun readForUpload(context: Context, uri: Uri, maxEdge: Int?): Pair<ByteArray, String> {
|
private fun readForUpload(context: Context, uri: Uri, maxEdge: Int?): Pair<ByteArray, String> {
|
||||||
val resolver = context.contentResolver
|
val resolver = context.contentResolver
|
||||||
val mime = resolver.getType(uri) ?: "image/jpeg"
|
val mime = resolver.getType(uri) ?: "image/jpeg"
|
||||||
val original =
|
val original = readAll(resolver, uri)
|
||||||
resolver.openInputStream(uri)?.use { it.readBytes() }
|
|
||||||
?: throw ApiException("couldn't read the picked image")
|
|
||||||
if (maxEdge == null) return original to mime
|
if (maxEdge == null) return original to mime
|
||||||
|
|
||||||
val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
|
val bounds = BitmapFactory.Options().apply { inJustDecodeBounds = true }
|
||||||
|
|||||||
@@ -30,14 +30,15 @@ sealed class SessionEvent {
|
|||||||
*/
|
*/
|
||||||
val id: String?,
|
val id: String?,
|
||||||
/**
|
/**
|
||||||
* What was attached to it, by the ref the files route serves.
|
* What was attached to it, by the ref the files route serves: images, and since 2026-09-03
|
||||||
|
* any file, told apart by [isImageRef].
|
||||||
*
|
*
|
||||||
* On the message rather than beside it: these arrived as separate image events until
|
* On the message rather than beside it: these arrived as separate image events until
|
||||||
* 2026-08-30, which drew somebody's screenshot as a row floating above the bubble that sent
|
* 2026-08-30, which drew somebody's screenshot as a row floating above the bubble that sent
|
||||||
* it, and left this app deciding from adjacency alone which message an image went with --
|
* it, and left this app deciding from adjacency alone which message an image went with --
|
||||||
* something the sender knew and could simply have said.
|
* something the sender knew and could simply have said.
|
||||||
*/
|
*/
|
||||||
val images: List<String>,
|
val attachments: List<String>,
|
||||||
) : SessionEvent()
|
) : SessionEvent()
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -50,7 +51,7 @@ sealed class SessionEvent {
|
|||||||
* Resolved by the [UserMessage] carrying the same id, exactly as [CommandQueued] is resolved by
|
* Resolved by the [UserMessage] carrying the same id, exactly as [CommandQueued] is resolved by
|
||||||
* [CommandSent].
|
* [CommandSent].
|
||||||
*/
|
*/
|
||||||
data class MessageQueued(val id: String, val text: String, val images: List<String>) :
|
data class MessageQueued(val id: String, val text: String, val attachments: List<String>) :
|
||||||
SessionEvent()
|
SessionEvent()
|
||||||
|
|
||||||
/**
|
/**
|
||||||
@@ -197,13 +198,13 @@ fun parseSeqEvent(json: String): SeqEvent {
|
|||||||
SessionEvent.UserMessage(
|
SessionEvent.UserMessage(
|
||||||
body.getString("text"),
|
body.getString("text"),
|
||||||
body.optString("id").ifEmpty { null },
|
body.optString("id").ifEmpty { null },
|
||||||
body.stringList("images"),
|
body.stringList("attachments"),
|
||||||
)
|
)
|
||||||
"messageQueued" ->
|
"messageQueued" ->
|
||||||
SessionEvent.MessageQueued(
|
SessionEvent.MessageQueued(
|
||||||
body.getString("id"),
|
body.getString("id"),
|
||||||
body.getString("text"),
|
body.getString("text"),
|
||||||
body.stringList("images"),
|
body.stringList("attachments"),
|
||||||
)
|
)
|
||||||
"messageDropped" -> SessionEvent.MessageDropped(body.getString("id"))
|
"messageDropped" -> SessionEvent.MessageDropped(body.getString("id"))
|
||||||
"assistantText" -> SessionEvent.AssistantText(body.getString("delta"))
|
"assistantText" -> SessionEvent.AssistantText(body.getString("delta"))
|
||||||
|
|||||||
@@ -37,6 +37,10 @@ class MainActivity : ComponentActivity() {
|
|||||||
private var openRequest by mutableStateOf<SessionOpenRequest?>(null)
|
private var openRequest by mutableStateOf<SessionOpenRequest?>(null)
|
||||||
private var opens = 0
|
private var opens = 0
|
||||||
|
|
||||||
|
// What another app shared into this one, for the same reason and with the same serial.
|
||||||
|
private var shareRequest by mutableStateOf<ShareRequest?>(null)
|
||||||
|
private var shares = 0
|
||||||
|
|
||||||
// Registered up front since permission launchers must be registered
|
// Registered up front since permission launchers must be registered
|
||||||
// before the activity reaches STARTED.
|
// before the activity reaches STARTED.
|
||||||
private val requestLocalNetworkPermission =
|
private val requestLocalNetworkPermission =
|
||||||
@@ -139,7 +143,7 @@ class MainActivity : ComponentActivity() {
|
|||||||
// scoped to what actually moves.
|
// scoped to what actually moves.
|
||||||
.navigationBarsPadding()
|
.navigationBarsPadding()
|
||||||
) {
|
) {
|
||||||
AppRoot(settingsVersion, openRequest)
|
AppRoot(settingsVersion, openRequest, shareRequest)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -155,14 +159,21 @@ class MainActivity : ComponentActivity() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* The one place an incoming `aiapp://` URI is sorted into what it means.
|
* The one place an incoming intent is sorted into what it means.
|
||||||
*
|
*
|
||||||
* Two things arrive this way -- an enrollment code and a notification naming a session -- and
|
* Three things arrive this way -- a share from another app, and an `aiapp://` URI that is
|
||||||
* they are told apart by the URI's host rather than by two entry points, so a third kind is a
|
* either an enrollment code or a notification naming a session. The URIs are told apart by host
|
||||||
* branch here rather than another intent to remember to handle.
|
* rather than by two entry points, so a further kind is a branch here rather than another
|
||||||
|
* intent to remember to handle.
|
||||||
*/
|
*/
|
||||||
private fun handleIntent(intent: Intent?) {
|
private fun handleIntent(intent: Intent?) {
|
||||||
val uri = intent?.data ?: return
|
intent ?: return
|
||||||
|
sharedContent(intent, shares + 1)?.let { shared ->
|
||||||
|
shares = shared.serial
|
||||||
|
shareRequest = shared
|
||||||
|
return
|
||||||
|
}
|
||||||
|
val uri = intent.data ?: return
|
||||||
val sessionId = notifiedSessionId(uri)
|
val sessionId = notifiedSessionId(uri)
|
||||||
if (sessionId != null) {
|
if (sessionId != null) {
|
||||||
opens++
|
opens++
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package com.example.aiapp
|
package com.example.aiapp
|
||||||
|
|
||||||
import androidx.activity.compose.BackHandler
|
import androidx.activity.compose.BackHandler
|
||||||
|
import androidx.compose.foundation.background
|
||||||
import androidx.compose.foundation.layout.Column
|
import androidx.compose.foundation.layout.Column
|
||||||
import androidx.compose.foundation.layout.Row
|
import androidx.compose.foundation.layout.Row
|
||||||
import androidx.compose.foundation.layout.fillMaxSize
|
import androidx.compose.foundation.layout.fillMaxSize
|
||||||
@@ -46,6 +47,8 @@ private enum class MainTab(val label: String) {
|
|||||||
fun MainScreen(
|
fun MainScreen(
|
||||||
settings: ServerSettings,
|
settings: ServerSettings,
|
||||||
reloadToken: Int,
|
reloadToken: Int,
|
||||||
|
/** What another app shared in and no session has taken yet; see [ShareRequest]. */
|
||||||
|
share: ShareRequest? = null,
|
||||||
onOpen: (SessionSummary) -> Unit,
|
onOpen: (SessionSummary) -> Unit,
|
||||||
onSpawn: () -> Unit,
|
onSpawn: () -> Unit,
|
||||||
onImported: (SessionSummary) -> Unit,
|
onImported: (SessionSummary) -> Unit,
|
||||||
@@ -106,6 +109,24 @@ fun MainScreen(
|
|||||||
GlyphButton(SETTINGS_GLYPH, "Settings", onSettings)
|
GlyphButton(SETTINGS_GLYPH, "Settings", onSettings)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
// What is waiting to be attached, and what to do about it. Said here because the list
|
||||||
|
// below is where the choice is made, and a share that arrived with nothing on screen
|
||||||
|
// saying so would read as a tap that did nothing.
|
||||||
|
share?.let {
|
||||||
|
Text(
|
||||||
|
it.summary() + " -- open the session it belongs in.",
|
||||||
|
style = MaterialTheme.typography.bodyMedium,
|
||||||
|
color = MaterialTheme.colorScheme.onPrimaryContainer,
|
||||||
|
modifier =
|
||||||
|
Modifier.fillMaxWidth()
|
||||||
|
.padding(horizontal = 16.dp, vertical = 8.dp)
|
||||||
|
.background(
|
||||||
|
MaterialTheme.colorScheme.primaryContainer,
|
||||||
|
MaterialTheme.shapes.small,
|
||||||
|
)
|
||||||
|
.padding(12.dp),
|
||||||
|
)
|
||||||
|
}
|
||||||
// Primary rather than the plain TabRow, which is deprecated in favour of the two that
|
// Primary rather than the plain TabRow, which is deprecated in favour of the two that
|
||||||
// say where they sit: these are the app's top-level destinations.
|
// say where they sit: these are the app's top-level destinations.
|
||||||
PrimaryTabRow(selectedTabIndex = tab.ordinal) {
|
PrimaryTabRow(selectedTabIndex = tab.ordinal) {
|
||||||
|
|||||||
@@ -8,8 +8,12 @@ import androidx.compose.foundation.horizontalScroll
|
|||||||
import androidx.compose.foundation.layout.Arrangement
|
import androidx.compose.foundation.layout.Arrangement
|
||||||
import androidx.compose.foundation.layout.Box
|
import androidx.compose.foundation.layout.Box
|
||||||
import androidx.compose.foundation.layout.Row
|
import androidx.compose.foundation.layout.Row
|
||||||
|
import androidx.compose.foundation.layout.Spacer
|
||||||
|
import androidx.compose.foundation.layout.height
|
||||||
import androidx.compose.foundation.layout.padding
|
import androidx.compose.foundation.layout.padding
|
||||||
import androidx.compose.foundation.layout.size
|
import androidx.compose.foundation.layout.size
|
||||||
|
import androidx.compose.foundation.layout.width
|
||||||
|
import androidx.compose.foundation.layout.widthIn
|
||||||
import androidx.compose.foundation.rememberScrollState
|
import androidx.compose.foundation.rememberScrollState
|
||||||
import androidx.compose.foundation.shape.CircleShape
|
import androidx.compose.foundation.shape.CircleShape
|
||||||
import androidx.compose.foundation.shape.RoundedCornerShape
|
import androidx.compose.foundation.shape.RoundedCornerShape
|
||||||
@@ -34,7 +38,8 @@ import androidx.compose.ui.unit.sp
|
|||||||
* anywhere else on the screen.
|
* anywhere else on the screen.
|
||||||
*
|
*
|
||||||
* Scrolls sideways rather than wrapping or shrinking: the row keeps one thumbnail size whatever is
|
* Scrolls sideways rather than wrapping or shrinking: the row keeps one thumbnail size whatever is
|
||||||
* in it, so four attachments look like four of the same thing rather than four smaller ones.
|
* in it, so four attachments look like four of the same thing rather than four smaller ones. A file
|
||||||
|
* is a tile of the same height carrying its name, since a name is all there is to show of it.
|
||||||
*/
|
*/
|
||||||
@Composable
|
@Composable
|
||||||
fun PendingAttachments(
|
fun PendingAttachments(
|
||||||
@@ -49,7 +54,10 @@ fun PendingAttachments(
|
|||||||
modifier = modifier.horizontalScroll(rememberScrollState()).padding(bottom = 8.dp),
|
modifier = modifier.horizontalScroll(rememberScrollState()).padding(bottom = 8.dp),
|
||||||
horizontalArrangement = Arrangement.spacedBy(8.dp),
|
horizontalArrangement = Arrangement.spacedBy(8.dp),
|
||||||
) {
|
) {
|
||||||
refs.forEach { ref -> PendingThumbnail(settings, sessionId, ref) { onRemove(ref) } }
|
refs.forEach { ref ->
|
||||||
|
if (isImageRef(ref)) PendingThumbnail(settings, sessionId, ref) { onRemove(ref) }
|
||||||
|
else PendingFile(ref) { onRemove(ref) }
|
||||||
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -122,4 +130,32 @@ private fun PendingThumbnail(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* One attached file: its name, tap to take it back off. The same height and removal as a thumbnail,
|
||||||
|
* so a row of mixed attachments is one row; the cross sits after the name because a tile this wide
|
||||||
|
* has no corner the eye goes to.
|
||||||
|
*/
|
||||||
|
@Composable
|
||||||
|
private fun PendingFile(ref: String, onRemove: () -> Unit) {
|
||||||
|
val name = attachmentName(ref)
|
||||||
|
val shape = RoundedCornerShape(8.dp)
|
||||||
|
Row(
|
||||||
|
Modifier.height(THUMBNAIL)
|
||||||
|
.clip(shape)
|
||||||
|
.border(1.dp, MaterialTheme.colorScheme.outlineVariant, shape)
|
||||||
|
.background(MaterialTheme.colorScheme.surfaceVariant)
|
||||||
|
.clickable(onClick = onRemove)
|
||||||
|
.semantics { contentDescription = "Attached file $name, tap to remove" }
|
||||||
|
.padding(horizontal = 8.dp),
|
||||||
|
verticalAlignment = Alignment.CenterVertically,
|
||||||
|
) {
|
||||||
|
FileName(name, Modifier.widthIn(max = FILE_TILE_WIDTH))
|
||||||
|
Spacer(Modifier.width(6.dp))
|
||||||
|
Glyph(CLOSE_GLYPH, colour = MaterialTheme.colorScheme.onSurface, size = 12.sp)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
private val THUMBNAIL = 64.dp
|
private val THUMBNAIL = 64.dp
|
||||||
|
|
||||||
|
/** Wide enough for most names whole; longer ones lose their middle, keeping both ends. */
|
||||||
|
private val FILE_TILE_WIDTH = 200.dp
|
||||||
@@ -2,6 +2,7 @@ package com.example.aiapp
|
|||||||
|
|
||||||
import android.content.Context
|
import android.content.Context
|
||||||
import android.content.pm.ApplicationInfo
|
import android.content.pm.ApplicationInfo
|
||||||
|
import android.net.Uri
|
||||||
import android.os.Build
|
import android.os.Build
|
||||||
import android.os.SystemClock
|
import android.os.SystemClock
|
||||||
import android.util.Log
|
import android.util.Log
|
||||||
@@ -228,7 +229,15 @@ private fun Modifier.holdTopEdge(key: Any, held: TopEdgeHold, hold: (Int) -> Uni
|
|||||||
// self-correction needs it.
|
// self-correction needs it.
|
||||||
@OptIn(ExperimentalLayoutApi::class)
|
@OptIn(ExperimentalLayoutApi::class)
|
||||||
@Composable
|
@Composable
|
||||||
fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: () -> Unit) {
|
fun SessionScreen(
|
||||||
|
settings: ServerSettings,
|
||||||
|
summary: SessionSummary,
|
||||||
|
onBack: () -> Unit,
|
||||||
|
/** What another app shared in while this session is the one open; see [ShareRequest]. */
|
||||||
|
share: ShareRequest? = null,
|
||||||
|
/** Said once [share] has been attached here, so it is not attached again. */
|
||||||
|
onShareTaken: () -> Unit = {},
|
||||||
|
) {
|
||||||
DebugStats.count("session screen recomposed")
|
DebugStats.count("session screen recomposed")
|
||||||
val scope = rememberCoroutineScope()
|
val scope = rememberCoroutineScope()
|
||||||
val topEdgeHeld = remember { TopEdgeHold() }
|
val topEdgeHeld = remember { TopEdgeHold() }
|
||||||
@@ -410,7 +419,7 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
|
|||||||
// Waiting, then read. Matched by id: the same message sent twice is two
|
// Waiting, then read. Matched by id: the same message sent twice is two
|
||||||
// bubbles, and clearing by text would take away whichever matched first.
|
// bubbles, and clearing by text would take away whichever matched first.
|
||||||
if (event is SessionEvent.MessageQueued) {
|
if (event is SessionEvent.MessageQueued) {
|
||||||
queued = queued + QueuedMessage(event.id, event.text, event.images)
|
queued = queued + QueuedMessage(event.id, event.text, event.attachments)
|
||||||
}
|
}
|
||||||
if (event is SessionEvent.UserMessage) {
|
if (event is SessionEvent.UserMessage) {
|
||||||
queued = queued.filterNot { it.id == event.id }
|
queued = queued.filterNot { it.id == event.id }
|
||||||
@@ -1076,25 +1085,18 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
|
|||||||
act { sendMessage(settings, summary.id, text, attachments) }
|
act { sendMessage(settings, summary.id, text, attachments) }
|
||||||
}
|
}
|
||||||
|
|
||||||
// The system photo picker; the image uploads as soon as it's chosen,
|
// One path for everything attached, however it arrived: the photo picker, the file chooser
|
||||||
// so Send only has ids to reference.
|
// or another app's share sheet. It uploads as soon as it is chosen, so Send only has ids to
|
||||||
val pickImage =
|
// reference.
|
||||||
rememberLauncherForActivityResult(ActivityResultContracts.PickVisualMedia()) { uri ->
|
fun attach(uri: Uri) {
|
||||||
if (uri != null) {
|
|
||||||
scope.launch {
|
scope.launch {
|
||||||
try {
|
try {
|
||||||
val id =
|
val id =
|
||||||
withContext(Dispatchers.IO) {
|
withContext(Dispatchers.IO) {
|
||||||
// Shrunk to what this session's provider takes before it is
|
// An image is shrunk to what this session's provider takes before it is
|
||||||
// uploaded, so a twelve-megapixel photo does not cross the tunnel
|
// uploaded, so a twelve-megapixel photo does not cross the tunnel to be
|
||||||
// to be rejected at the far end -- see `uploadPickedImage`.
|
// rejected at the far end; a file goes whole -- see `uploadPicked`.
|
||||||
uploadPickedImage(
|
uploadPicked(context, settings, summary.id, uri, summary.maxImageEdge)
|
||||||
context,
|
|
||||||
settings,
|
|
||||||
summary.id,
|
|
||||||
uri,
|
|
||||||
summary.maxImageEdge,
|
|
||||||
)
|
|
||||||
}
|
}
|
||||||
pendingAttachments = pendingAttachments + id
|
pendingAttachments = pendingAttachments + id
|
||||||
actionError = null
|
actionError = null
|
||||||
@@ -1103,6 +1105,24 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
val pickImage =
|
||||||
|
rememberLauncherForActivityResult(ActivityResultContracts.PickVisualMedia()) { uri ->
|
||||||
|
uri?.let(::attach)
|
||||||
|
}
|
||||||
|
val pickFile =
|
||||||
|
rememberLauncherForActivityResult(ActivityResultContracts.OpenDocument()) { uri ->
|
||||||
|
uri?.let(::attach)
|
||||||
|
}
|
||||||
|
// What another app shared in, attached the moment this screen has it. Taken off the request
|
||||||
|
// first, so a recomposition or a return to this screen cannot attach it a second time.
|
||||||
|
LaunchedEffect(share) {
|
||||||
|
val incoming = share ?: return@LaunchedEffect
|
||||||
|
onShareTaken()
|
||||||
|
incoming.uris.forEach(::attach)
|
||||||
|
incoming.text?.let { shared ->
|
||||||
|
input = if (input.isBlank()) shared else input + "\n" + shared
|
||||||
|
saveDraft(context, summary.id, input)
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// One poll for this machine's limits, read by the two things that show them: the bar under
|
// One poll for this machine's limits, read by the two things that show them: the bar under
|
||||||
@@ -1333,7 +1353,7 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
|
|||||||
settings = settings,
|
settings = settings,
|
||||||
sessionId = summary.id,
|
sessionId = summary.id,
|
||||||
text = waiting.text,
|
text = waiting.text,
|
||||||
images = waiting.images,
|
attachments = waiting.attachments,
|
||||||
onOpenImage = ::openImage,
|
onOpenImage = ::openImage,
|
||||||
pending = true,
|
pending = true,
|
||||||
refusal = waiting.refusal,
|
refusal = waiting.refusal,
|
||||||
@@ -1470,7 +1490,7 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
|
|||||||
settings = settings,
|
settings = settings,
|
||||||
sessionId = summary.id,
|
sessionId = summary.id,
|
||||||
text = item.text,
|
text = item.text,
|
||||||
images = item.images,
|
attachments = item.attachments,
|
||||||
onOpenImage = ::openImage,
|
onOpenImage = ::openImage,
|
||||||
)
|
)
|
||||||
is TranscriptItem.AssistantMsg ->
|
is TranscriptItem.AssistantMsg ->
|
||||||
@@ -1727,17 +1747,38 @@ fun SessionScreen(settings: ServerSettings, summary: SessionSummary, onBack: ()
|
|||||||
verticalAlignment = Alignment.CenterVertically,
|
verticalAlignment = Alignment.CenterVertically,
|
||||||
modifier = Modifier.fillMaxWidth(),
|
modifier = Modifier.fillMaxWidth(),
|
||||||
) {
|
) {
|
||||||
TextButton(
|
// Photo or file, asked here rather than by two buttons: the row is full,
|
||||||
|
// and attaching is one action whichever picker answers it.
|
||||||
|
var attaching by remember { mutableStateOf(false) }
|
||||||
|
Box {
|
||||||
|
// Just "+". The count it used to carry was standing in for showing them.
|
||||||
|
TextButton(onClick = { attaching = true }) { Text("+") }
|
||||||
|
DropdownMenu(
|
||||||
|
expanded = attaching,
|
||||||
|
onDismissRequest = { attaching = false },
|
||||||
|
// See PickerButton: without this the menu opens a status bar's
|
||||||
|
// height away from the button in an edge-to-edge activity.
|
||||||
|
properties = PopupProperties(clippingEnabled = false),
|
||||||
|
) {
|
||||||
|
DropdownMenuItem(
|
||||||
|
text = { Text("Photo") },
|
||||||
onClick = {
|
onClick = {
|
||||||
|
attaching = false
|
||||||
pickImage.launch(
|
pickImage.launch(
|
||||||
PickVisualMediaRequest(
|
PickVisualMediaRequest(
|
||||||
ActivityResultContracts.PickVisualMedia.ImageOnly
|
ActivityResultContracts.PickVisualMedia.ImageOnly
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
},
|
||||||
|
)
|
||||||
|
DropdownMenuItem(
|
||||||
|
text = { Text("File") },
|
||||||
|
onClick = {
|
||||||
|
attaching = false
|
||||||
|
pickFile.launch(arrayOf("*/*"))
|
||||||
|
},
|
||||||
|
)
|
||||||
}
|
}
|
||||||
) {
|
|
||||||
// Just "+" now. The count was standing in for showing them.
|
|
||||||
Text("+")
|
|
||||||
}
|
}
|
||||||
// The settings share what is left after the actions have
|
// The settings share what is left after the actions have
|
||||||
// taken what they need. A Row hands out intrinsic widths in
|
// taken what they need. A Row hands out intrinsic widths in
|
||||||
@@ -1929,9 +1970,9 @@ private fun UserChunkRow(
|
|||||||
Text(unit.text, color = MaterialTheme.colorScheme.onPrimaryContainer)
|
Text(unit.text, color = MaterialTheme.colorScheme.onPrimaryContainer)
|
||||||
// The same arrangement [UserBubble] gives them: under the words, on the last slice
|
// The same arrangement [UserBubble] gives them: under the words, on the last slice
|
||||||
// because that is the bubble's bottom.
|
// because that is the bubble's bottom.
|
||||||
unit.images.forEachIndexed { index, ref ->
|
unit.attachments.forEachIndexed { index, ref ->
|
||||||
if (index > 0 || unit.text.isNotEmpty()) Spacer(Modifier.height(4.dp))
|
if (index > 0 || unit.text.isNotEmpty()) Spacer(Modifier.height(4.dp))
|
||||||
SessionImage(settings, sessionId, ref, onOpenImage)
|
Attachment(settings, sessionId, ref, onOpenImage)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1957,7 +1998,7 @@ private fun UserBubble(
|
|||||||
settings: ServerSettings,
|
settings: ServerSettings,
|
||||||
sessionId: String,
|
sessionId: String,
|
||||||
text: String,
|
text: String,
|
||||||
images: List<String> = emptyList(),
|
attachments: List<String> = emptyList(),
|
||||||
onOpenImage: (String) -> Unit,
|
onOpenImage: (String) -> Unit,
|
||||||
pending: Boolean = false,
|
pending: Boolean = false,
|
||||||
refusal: String? = null,
|
refusal: String? = null,
|
||||||
@@ -2003,9 +2044,9 @@ private fun UserBubble(
|
|||||||
// Under the words: what somebody wrote is what the bubble is, and the picture is
|
// Under the words: what somebody wrote is what the bubble is, and the picture is
|
||||||
// what they attached to it. It also keeps the first line of every bubble at the
|
// what they attached to it. It also keeps the first line of every bubble at the
|
||||||
// same place down the transcript, whether or not there is an image in it.
|
// same place down the transcript, whether or not there is an image in it.
|
||||||
images.forEachIndexed { index, ref ->
|
attachments.forEachIndexed { index, ref ->
|
||||||
if (index > 0 || text.isNotEmpty()) Spacer(Modifier.height(4.dp))
|
if (index > 0 || text.isNotEmpty()) Spacer(Modifier.height(4.dp))
|
||||||
SessionImage(settings, sessionId, ref, onOpenImage)
|
Attachment(settings, sessionId, ref, onOpenImage)
|
||||||
}
|
}
|
||||||
refusal?.let {
|
refusal?.let {
|
||||||
Spacer(Modifier.height(6.dp))
|
Spacer(Modifier.height(6.dp))
|
||||||
@@ -2030,7 +2071,7 @@ private fun UserBubble(
|
|||||||
private data class QueuedMessage(
|
private data class QueuedMessage(
|
||||||
val id: String,
|
val id: String,
|
||||||
val text: String,
|
val text: String,
|
||||||
val images: List<String>,
|
val attachments: List<String>,
|
||||||
val refusal: String? = null,
|
val refusal: String? = null,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,44 @@
|
|||||||
|
package com.example.aiapp
|
||||||
|
|
||||||
|
import android.content.Intent
|
||||||
|
import android.net.Uri
|
||||||
|
import androidx.core.content.IntentCompat
|
||||||
|
|
||||||
|
/**
|
||||||
|
* What another app handed this one through the share sheet, waiting to be attached to a session.
|
||||||
|
*
|
||||||
|
* Held as the URIs rather than uploaded on arrival, because an upload belongs to a session and the
|
||||||
|
* share arrives before anyone has said which. [serial] makes two shares of the same thing two
|
||||||
|
* requests, for the reason [SessionOpenRequest] carries one: equal values would not recompose.
|
||||||
|
*/
|
||||||
|
data class ShareRequest(val uris: List<Uri>, val text: String?, val serial: Int)
|
||||||
|
|
||||||
|
/** The share in [intent], or null when it is some other intent. */
|
||||||
|
fun sharedContent(intent: Intent, serial: Int): ShareRequest? {
|
||||||
|
val uris =
|
||||||
|
when (intent.action) {
|
||||||
|
Intent.ACTION_SEND ->
|
||||||
|
listOfNotNull(
|
||||||
|
IntentCompat.getParcelableExtra(intent, Intent.EXTRA_STREAM, Uri::class.java)
|
||||||
|
)
|
||||||
|
Intent.ACTION_SEND_MULTIPLE ->
|
||||||
|
IntentCompat.getParcelableArrayListExtra(
|
||||||
|
intent,
|
||||||
|
Intent.EXTRA_STREAM,
|
||||||
|
Uri::class.java,
|
||||||
|
)
|
||||||
|
.orEmpty()
|
||||||
|
else -> return null
|
||||||
|
}
|
||||||
|
val text = intent.getStringExtra(Intent.EXTRA_TEXT)?.takeIf { it.isNotBlank() }
|
||||||
|
if (uris.isEmpty() && text == null) return null
|
||||||
|
return ShareRequest(uris, text, serial)
|
||||||
|
}
|
||||||
|
|
||||||
|
/** What is waiting, for the banner that says so. */
|
||||||
|
fun ShareRequest.summary(): String =
|
||||||
|
when {
|
||||||
|
uris.size == 1 -> "1 file to attach"
|
||||||
|
uris.isNotEmpty() -> "${uris.size} files to attach"
|
||||||
|
else -> "Text to attach"
|
||||||
|
}
|
||||||
@@ -160,6 +160,15 @@ private fun highlighted(code: String, language: SyntaxLanguage?): AnnotatedStrin
|
|||||||
Highlights.Builder(code = code, language = language, theme = theme)
|
Highlights.Builder(code = code, language = language, theme = theme)
|
||||||
.build()
|
.build()
|
||||||
.getHighlights()
|
.getHighlights()
|
||||||
|
// highlights 1.1.0's shell lexer answers a quoted glob that looks like a
|
||||||
|
// comment -- `x '*/a/*'` is the smallest input -- with a span whose end is
|
||||||
|
// before its start, and AnnotatedString refuses such a range. That crashed the
|
||||||
|
// app the moment a card holding `-path '*/.git/*'` was opened. Dropped rather
|
||||||
|
// than clamped: a span the lexer got backwards is not one it knows the colour
|
||||||
|
// of. Delete when snipme/highlights fixes it.
|
||||||
|
.filter {
|
||||||
|
it.location.start in 0..it.location.end && it.location.end <= code.length
|
||||||
|
}
|
||||||
buildAnnotatedString {
|
buildAnnotatedString {
|
||||||
append(code)
|
append(code)
|
||||||
marks.forEach { mark ->
|
marks.forEach { mark ->
|
||||||
|
|||||||
@@ -42,7 +42,7 @@ sealed class TranscriptItem {
|
|||||||
override val seq: Long,
|
override val seq: Long,
|
||||||
val text: String,
|
val text: String,
|
||||||
/** Refs of what was attached, drawn inside the bubble. */
|
/** Refs of what was attached, drawn inside the bubble. */
|
||||||
val images: List<String> = emptyList(),
|
val attachments: List<String> = emptyList(),
|
||||||
) : TranscriptItem()
|
) : TranscriptItem()
|
||||||
|
|
||||||
data class AssistantMsg(
|
data class AssistantMsg(
|
||||||
@@ -375,7 +375,7 @@ private fun splitRun(tail: List<TranscriptItem>, behind: String?): List<Transcri
|
|||||||
fun foldEvent(items: List<TranscriptItem>, entry: SeqEvent): List<TranscriptItem> =
|
fun foldEvent(items: List<TranscriptItem>, entry: SeqEvent): List<TranscriptItem> =
|
||||||
when (val event = entry.event) {
|
when (val event = entry.event) {
|
||||||
is SessionEvent.UserMessage ->
|
is SessionEvent.UserMessage ->
|
||||||
items + TranscriptItem.UserMsg(entry.seq, event.text, event.images)
|
items + TranscriptItem.UserMsg(entry.seq, event.text, event.attachments)
|
||||||
is SessionEvent.AssistantText -> {
|
is SessionEvent.AssistantText -> {
|
||||||
// Deltas accumulate into the message they're streaming, which keeps the seq of the
|
// Deltas accumulate into the message they're streaming, which keeps the seq of the
|
||||||
// first of them: a row whose identity changed with every delta would be a new row on
|
// first of them: a row whose identity changed with every delta would be a new row on
|
||||||
|
|||||||
@@ -129,7 +129,7 @@ sealed class TranscriptUnit {
|
|||||||
val first: Boolean,
|
val first: Boolean,
|
||||||
val last: Boolean,
|
val last: Boolean,
|
||||||
/** The message's attachments, drawn under the words -- so only the last slice has any. */
|
/** The message's attachments, drawn under the words -- so only the last slice has any. */
|
||||||
val images: List<String>,
|
val attachments: List<String>,
|
||||||
override val gap: Dp,
|
override val gap: Dp,
|
||||||
) : TranscriptUnit() {
|
) : TranscriptUnit() {
|
||||||
override val key: Any
|
override val key: Any
|
||||||
@@ -211,7 +211,7 @@ fun transcriptUnits(
|
|||||||
chunk,
|
chunk,
|
||||||
first = at == 0,
|
first = at == 0,
|
||||||
last = at == chunks.lastIndex,
|
last = at == chunks.lastIndex,
|
||||||
images = if (at == chunks.lastIndex) item.images else emptyList(),
|
attachments = if (at == chunks.lastIndex) item.attachments else emptyList(),
|
||||||
gap = if (at == 0) rowGap else 0.dp,
|
gap = if (at == 0) rowGap else 0.dp,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|||||||
+12
-1
@@ -148,7 +148,18 @@ salvaged=""
|
|||||||
if [ -f "$ROOT/config.ron" ]; then
|
if [ -f "$ROOT/config.ron" ]; then
|
||||||
salvaged=$(awk '
|
salvaged=$(awk '
|
||||||
/^tokens: \[/ { in_tokens = 1; next }
|
/^tokens: \[/ { in_tokens = 1; next }
|
||||||
in_tokens && /^\],/ { exit }
|
# The server writes the list back compactly, with the last entry
|
||||||
|
# and the close on one line: " ),],". Reading the close only
|
||||||
|
# at a line start ran past it into `setups`, and the salvage then
|
||||||
|
# carried a second copy of that block into the new config.
|
||||||
|
in_tokens && /\],/ {
|
||||||
|
sub(/\],.*/, "")
|
||||||
|
if ($0 ~ /\),/) {
|
||||||
|
entry = entry $0 "\n"
|
||||||
|
if (entry !~ h) entries = entries entry
|
||||||
|
}
|
||||||
|
exit
|
||||||
|
}
|
||||||
in_tokens {
|
in_tokens {
|
||||||
entry = entry $0 "\n"
|
entry = entry $0 "\n"
|
||||||
if ($0 ~ /\),/) {
|
if ($0 ~ /\),/) {
|
||||||
|
|||||||
+36
-15
@@ -30,8 +30,8 @@
|
|||||||
//! POST /sessions/{id}/command {text} -- /compact, /clear, /rename x, or the dialect's own
|
//! POST /sessions/{id}/command {text} -- /compact, /clear, /rename x, or the dialect's own
|
||||||
//! (starts the process first if it has exited)
|
//! (starts the process first if it has exited)
|
||||||
//! POST /sessions/{id}/compact
|
//! POST /sessions/{id}/compact
|
||||||
//! POST /sessions/{id}/attachments multipart image upload -> {id}, referenced by /message
|
//! POST /sessions/{id}/attachments multipart upload, image or any file -> {id}, referenced by /message
|
||||||
//! GET /sessions/{id}/files/{name} images the session produced or was sent
|
//! GET /sessions/{id}/files/{name} images the session produced, and what it was sent
|
||||||
//! DELETE /sessions/{id} kill process, delete transcript + files
|
//! DELETE /sessions/{id} kill process, delete transcript + files
|
||||||
//! (?deleteForeign=true removes the machine's own copy too)
|
//! (?deleteForeign=true removes the machine's own copy too)
|
||||||
//! POST /sessions/{id}/notify {notify} -- announce this one or not
|
//! POST /sessions/{id}/notify {notify} -- announce this one or not
|
||||||
@@ -110,7 +110,13 @@ pub fn router(manager: Arc<SessionManager>) -> Router {
|
|||||||
.route("/notifications", get(notifications))
|
.route("/notifications", get(notifications))
|
||||||
.route("/sessions/{id}/compact", post(compact))
|
.route("/sessions/{id}/compact", post(compact))
|
||||||
.route("/sessions/{id}/command", post(command))
|
.route("/sessions/{id}/command", post(command))
|
||||||
.route("/sessions/{id}/attachments", post(upload_attachment))
|
.route(
|
||||||
|
"/sessions/{id}/attachments",
|
||||||
|
// A trace or a log is bigger than a photo; the cap below is
|
||||||
|
// for everything else, and the innermost limit is the one
|
||||||
|
// axum applies.
|
||||||
|
post(upload_attachment).layer(axum::extract::DefaultBodyLimit::max(ATTACHMENT_LIMIT)),
|
||||||
|
)
|
||||||
.route("/sessions/{id}/files/{name}", get(serve_file))
|
.route("/sessions/{id}/files/{name}", get(serve_file))
|
||||||
// Phone photos overflow axum's 2 MB default body cap.
|
// Phone photos overflow axum's 2 MB default body cap.
|
||||||
.layer(axum::extract::DefaultBodyLimit::max(32 * 1024 * 1024))
|
.layer(axum::extract::DefaultBodyLimit::max(32 * 1024 * 1024))
|
||||||
@@ -1273,8 +1279,14 @@ async fn compact(
|
|||||||
Ok(StatusCode::NO_CONTENT)
|
Ok(StatusCode::NO_CONTENT)
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Accepts one image (any multipart field) and stores it under the
|
/// The most one attachment may be. A day of `perfetto` is under a
|
||||||
|
/// gigabyte; a phone photo is a few megabytes; this is the room between.
|
||||||
|
const ATTACHMENT_LIMIT: usize = 1024 * 1024 * 1024;
|
||||||
|
|
||||||
|
/// Accepts one file (any multipart field) and stores it under the
|
||||||
/// session; the returned id goes into a later `/message`'s attachmentIds.
|
/// session; the returned id goes into a later `/message`'s attachmentIds.
|
||||||
|
/// An image is later shown to the model, anything else is named to it by
|
||||||
|
/// path -- see `ClaudeDriver::send_user_message`.
|
||||||
async fn upload_attachment(
|
async fn upload_attachment(
|
||||||
State(manager): State<Arc<SessionManager>>,
|
State(manager): State<Arc<SessionManager>>,
|
||||||
UrlPath(id): UrlPath<String>,
|
UrlPath(id): UrlPath<String>,
|
||||||
@@ -1286,27 +1298,36 @@ async fn upload_attachment(
|
|||||||
.await
|
.await
|
||||||
.map_err(|err| ApiError::BadRequest(format!("bad upload: {err}")))?
|
.map_err(|err| ApiError::BadRequest(format!("bad upload: {err}")))?
|
||||||
.ok_or_else(|| ApiError::BadRequest("no file in the upload".to_string()))?;
|
.ok_or_else(|| ApiError::BadRequest("no file in the upload".to_string()))?;
|
||||||
let content_type = field.content_type().unwrap_or("image/jpeg").to_string();
|
let content_type = field
|
||||||
|
.content_type()
|
||||||
|
.unwrap_or("application/octet-stream")
|
||||||
|
.to_string();
|
||||||
|
let file_name = field.file_name().map(str::to_string);
|
||||||
let bytes = field
|
let bytes = field
|
||||||
.bytes()
|
.bytes()
|
||||||
.await
|
.await
|
||||||
.map_err(|err| ApiError::BadRequest(format!("upload read failed: {err}")))?;
|
.map_err(|err| ApiError::BadRequest(format!("upload read failed: {err}")))?;
|
||||||
let name = session
|
let name = session
|
||||||
.save_attachment(&bytes, &content_type)
|
.save_attachment(&bytes, &content_type, file_name.as_deref())
|
||||||
.map_err(bad_request)?;
|
.map_err(bad_request)?;
|
||||||
Ok(axum::Json(serde_json::json!({ "id": name })))
|
Ok(axum::Json(serde_json::json!({ "id": name })))
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Serves a session's stored images -- both `files/` (produced by tools)
|
/// Serves a session's stored files -- both `files/` (images produced by
|
||||||
/// and `attachments/` (uploaded from the phone), by the id events and
|
/// tools) and `attachments/` (uploaded from the phone), by the id events
|
||||||
/// uploads reference.
|
/// and uploads reference.
|
||||||
async fn serve_file(
|
async fn serve_file(
|
||||||
State(manager): State<Arc<SessionManager>>,
|
State(manager): State<Arc<SessionManager>>,
|
||||||
UrlPath((id, name)): UrlPath<(String, String)>,
|
UrlPath((id, name)): UrlPath<(String, String)>,
|
||||||
) -> Result<Response, ApiError> {
|
) -> Result<Response, ApiError> {
|
||||||
// Ids are server-generated hex + extension; anything else (and any
|
// Ids are server-generated -- hex and an extension, or hex and a
|
||||||
// path separator in particular) is refused, not resolved.
|
// cleaned file name (`safe_file_name`); anything else (and any path
|
||||||
if !name.chars().all(|c| c.is_ascii_alphanumeric() || c == '.') || name.contains("..") {
|
// separator in particular) is refused, not resolved.
|
||||||
|
if !name
|
||||||
|
.chars()
|
||||||
|
.all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == '_')
|
||||||
|
|| name.contains("..")
|
||||||
|
{
|
||||||
return Err(ApiError::BadRequest("invalid file id".to_string()));
|
return Err(ApiError::BadRequest("invalid file id".to_string()));
|
||||||
}
|
}
|
||||||
let session = lookup(&manager, &id)?;
|
let session = lookup(&manager, &id)?;
|
||||||
@@ -1324,9 +1345,9 @@ async fn serve_file(
|
|||||||
let bytes = std::fs::read(path)
|
let bytes = std::fs::read(path)
|
||||||
.with_context(|| format!("read {}", path.display()))
|
.with_context(|| format!("read {}", path.display()))
|
||||||
.map_err(ApiError::Internal)?;
|
.map_err(ApiError::Internal)?;
|
||||||
// Names are server-generated, so an unrecognized extension can only
|
// Every image this server writes has an extension it knows; the rest
|
||||||
// mean a file this server didn't write.
|
// are files attached by name, served as the bytes they are.
|
||||||
let content_type = crate::media::media_type_for(&name).unwrap_or("image/jpeg");
|
let content_type = crate::media::media_type_for(&name).unwrap_or("application/octet-stream");
|
||||||
Ok(([(axum::http::header::CONTENT_TYPE, content_type)], bytes).into_response())
|
Ok(([(axum::http::header::CONTENT_TYPE, content_type)], bytes).into_response())
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -57,7 +57,7 @@ use serde_json::{Value, json};
|
|||||||
use tokio::io::AsyncWriteExt;
|
use tokio::io::AsyncWriteExt;
|
||||||
use tokio::sync::mpsc;
|
use tokio::sync::mpsc;
|
||||||
|
|
||||||
use super::driver::{Driver, Event, EventSink, ImageRef, SessionStatus, Unqueued};
|
use super::driver::{AttachmentRef, Driver, Event, EventSink, SessionStatus, Unqueued};
|
||||||
use super::process;
|
use super::process;
|
||||||
use super::transport::{Launch, Streams, Transport};
|
use super::transport::{Launch, Streams, Transport};
|
||||||
use crate::config::{ProviderConfig, SessionConfig};
|
use crate::config::{ProviderConfig, SessionConfig};
|
||||||
@@ -133,7 +133,7 @@ struct Queue {
|
|||||||
/// Written, not yet announced, oldest first, each with the id of the
|
/// Written, not yet announced, oldest first, each with the id of the
|
||||||
/// `MessageQueued` that told the phone it was waiting -- so the
|
/// `MessageQueued` that told the phone it was waiting -- so the
|
||||||
/// announcement can name which bubble it resolves.
|
/// announcement can name which bubble it resolves.
|
||||||
awaiting: VecDeque<(String, String, Vec<ImageRef>)>,
|
awaiting: VecDeque<(String, String, Vec<AttachmentRef>)>,
|
||||||
/// The process is gone, so nothing can be taken up any more.
|
/// The process is gone, so nothing can be taken up any more.
|
||||||
///
|
///
|
||||||
/// Needed because every other way out of a turn is an `Idle` this
|
/// Needed because every other way out of a turn is an `Idle` this
|
||||||
@@ -550,20 +550,35 @@ impl ClaudeDriver {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl Driver for ClaudeDriver {
|
impl Driver for ClaudeDriver {
|
||||||
fn send_user_message(&self, text: String, images: Vec<ImageRef>) {
|
fn send_user_message(&self, text: String, attachments: Vec<AttachmentRef>) {
|
||||||
let mut content = Vec::new();
|
let mut content = Vec::new();
|
||||||
for id in &images {
|
// An image goes into the message itself; the model looks at it. Any
|
||||||
match attachment_block(&self.session_dir, id) {
|
// other file stays where the upload put it and the message says
|
||||||
Ok(block) => content.push(block),
|
// where, because the CLI can read a file by path and a model cannot
|
||||||
Err(err) => {
|
// be handed a trace, a log or a zip any other way. Named after the
|
||||||
|
// text, so the words come first, the way they were typed.
|
||||||
|
let mut files = Vec::new();
|
||||||
|
for id in &attachments {
|
||||||
|
let sent = if crate::media::media_type_for(id).is_some() {
|
||||||
|
attachment_block(&self.session_dir, id).map(|block| content.push(block))
|
||||||
|
} else {
|
||||||
|
attachment_path(&self.session_dir, id).map(|path| files.push(path))
|
||||||
|
};
|
||||||
|
if let Err(err) = sent {
|
||||||
let _ = self.sink.send(Event::Error {
|
let _ = self.sink.send(Event::Error {
|
||||||
message: format!("attachment {id} couldn't be sent: {err:#}"),
|
message: format!("attachment {id} couldn't be sent: {err:#}"),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
let mut body = text.clone();
|
||||||
|
for path in files {
|
||||||
|
if !body.is_empty() {
|
||||||
|
body.push_str("\n\n");
|
||||||
}
|
}
|
||||||
if !text.is_empty() {
|
body.push_str(&format!("Attached file: {}", path.display()));
|
||||||
content.push(json!({"type": "text", "text": text}));
|
}
|
||||||
|
if !body.is_empty() {
|
||||||
|
content.push(json!({"type": "text", "text": body}));
|
||||||
}
|
}
|
||||||
let line =
|
let line =
|
||||||
json!({"type": "user", "message": {"role": "user", "content": content}}).to_string();
|
json!({"type": "user", "message": {"role": "user", "content": content}}).to_string();
|
||||||
@@ -591,9 +606,13 @@ impl Driver for ClaudeDriver {
|
|||||||
let id = super::random_hex();
|
let id = super::random_hex();
|
||||||
queue
|
queue
|
||||||
.awaiting
|
.awaiting
|
||||||
.push_back((id.clone(), text.clone(), images.clone()));
|
.push_back((id.clone(), text.clone(), attachments.clone()));
|
||||||
drop(queue);
|
drop(queue);
|
||||||
let _ = self.sink.send(Event::MessageQueued { id, text, images });
|
let _ = self.sink.send(Event::MessageQueued {
|
||||||
|
id,
|
||||||
|
text,
|
||||||
|
attachments,
|
||||||
|
});
|
||||||
self.send_line(line);
|
self.send_line(line);
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -605,7 +624,7 @@ impl Driver for ClaudeDriver {
|
|||||||
let _ = self.sink.send(Event::MessageTaken {
|
let _ = self.sink.send(Event::MessageTaken {
|
||||||
id: None,
|
id: None,
|
||||||
text,
|
text,
|
||||||
images,
|
attachments,
|
||||||
});
|
});
|
||||||
let _ = self.sink.send(Event::Status {
|
let _ = self.sink.send(Event::Status {
|
||||||
state: SessionStatus::Running,
|
state: SessionStatus::Running,
|
||||||
@@ -1087,16 +1106,16 @@ fn proves_a_turn(event: &Event) -> bool {
|
|||||||
/// [`translate_line`], and the pair is the whole of the rule -- a steer
|
/// [`translate_line`], and the pair is the whole of the rule -- a steer
|
||||||
/// announced anywhere else lands above output that predates it.
|
/// announced anywhere else lands above output that predates it.
|
||||||
fn announce_steers(queue: &Arc<Mutex<Queue>>, sink: &EventSink) -> bool {
|
fn announce_steers(queue: &Arc<Mutex<Queue>>, sink: &EventSink) -> bool {
|
||||||
let taken: Vec<(String, String, Vec<ImageRef>)> = {
|
let taken: Vec<(String, String, Vec<AttachmentRef>)> = {
|
||||||
let mut queue = queue.lock().unwrap();
|
let mut queue = queue.lock().unwrap();
|
||||||
queue.awaiting.drain(..).collect()
|
queue.awaiting.drain(..).collect()
|
||||||
};
|
};
|
||||||
for (id, text, images) in taken {
|
for (id, text, attachments) in taken {
|
||||||
if sink
|
if sink
|
||||||
.send(Event::MessageTaken {
|
.send(Event::MessageTaken {
|
||||||
id: Some(id),
|
id: Some(id),
|
||||||
text,
|
text,
|
||||||
images,
|
attachments,
|
||||||
})
|
})
|
||||||
.is_err()
|
.is_err()
|
||||||
{
|
{
|
||||||
@@ -1185,17 +1204,28 @@ pub(super) fn write_resume_token(session_dir: &Path, session_id: &str) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Reads an uploaded attachment into an API image content block.
|
/// Where an uploaded attachment is, as a path the CLI can be told.
|
||||||
fn attachment_block(session_dir: &Path, id: &str) -> Result<Value> {
|
///
|
||||||
// Ids are server-generated hex (see routes::upload_attachment); the
|
/// Absolute, because the CLI's working directory is the session's and the
|
||||||
// check keeps a crafted "id" from naming an arbitrary file.
|
/// attachments are not in it. Refused rather than resolved when the id is
|
||||||
|
/// not one this server would have written -- see
|
||||||
|
/// `SessionManager::save_attachment` -- so a crafted id cannot name a file
|
||||||
|
/// outside the session.
|
||||||
|
fn attachment_path(session_dir: &Path, id: &str) -> Result<PathBuf> {
|
||||||
if !id
|
if !id
|
||||||
.chars()
|
.chars()
|
||||||
.all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '-')
|
.all(|c| c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == '_')
|
||||||
|
|| id.contains("..")
|
||||||
{
|
{
|
||||||
anyhow::bail!("invalid attachment id");
|
anyhow::bail!("invalid attachment id");
|
||||||
}
|
}
|
||||||
let path = session_dir.join("attachments").join(id);
|
let path = session_dir.join("attachments").join(id);
|
||||||
|
std::fs::canonicalize(&path).with_context(|| format!("find {}", path.display()))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reads an uploaded image into an API image content block.
|
||||||
|
fn attachment_block(session_dir: &Path, id: &str) -> Result<Value> {
|
||||||
|
let path = attachment_path(session_dir, id)?;
|
||||||
let bytes = std::fs::read(&path).with_context(|| format!("read {}", path.display()))?;
|
let bytes = std::fs::read(&path).with_context(|| format!("read {}", path.display()))?;
|
||||||
use base64::Engine;
|
use base64::Engine;
|
||||||
Ok(json!({
|
Ok(json!({
|
||||||
|
|||||||
@@ -15,6 +15,14 @@ use tokio::sync::mpsc;
|
|||||||
/// directions use the one id so the transcript renders them identically.
|
/// directions use the one id so the transcript renders them identically.
|
||||||
pub type ImageRef = String;
|
pub type ImageRef = String;
|
||||||
|
|
||||||
|
/// The name an upload from the phone is stored and served under: an image
|
||||||
|
/// is `<hex>.<extension>` and is an [`ImageRef`] like any other; any other
|
||||||
|
/// file keeps its own name after the hex, `<hex>-<name>`, because the name
|
||||||
|
/// is what the reader attached and what the session is told. The two are
|
||||||
|
/// told apart by `crate::media::media_type_for`, which knows every image
|
||||||
|
/// extension this server writes.
|
||||||
|
pub type AttachmentRef = String;
|
||||||
|
|
||||||
/// One choice offered in answer to a [`Event::Question`].
|
/// One choice offered in answer to a [`Event::Question`].
|
||||||
///
|
///
|
||||||
/// More than a label because the reader is deciding, not confirming: what
|
/// More than a label because the reader is deciding, not confirming: what
|
||||||
@@ -89,8 +97,11 @@ pub enum Event {
|
|||||||
/// sent it -- and left the phone to decide, from nothing but
|
/// sent it -- and left the phone to decide, from nothing but
|
||||||
/// adjacency, which message an image belonged to. Belonging is not
|
/// adjacency, which message an image belonged to. Belonging is not
|
||||||
/// something to infer when the sender knew.
|
/// something to infer when the sender knew.
|
||||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
///
|
||||||
images: Vec<ImageRef>,
|
/// `images` on disk until 2026-09-03, when files joined them;
|
||||||
|
/// the alias reads the rows written before that.
|
||||||
|
#[serde(default, alias = "images", skip_serializing_if = "Vec::is_empty")]
|
||||||
|
attachments: Vec<AttachmentRef>,
|
||||||
},
|
},
|
||||||
/// A message accepted from the phone that the session cannot read yet.
|
/// A message accepted from the phone that the session cannot read yet.
|
||||||
///
|
///
|
||||||
@@ -111,8 +122,8 @@ pub enum Event {
|
|||||||
/// Carried for the same reason [`Event::UserMessage`] carries it,
|
/// Carried for the same reason [`Event::UserMessage`] carries it,
|
||||||
/// and it matters more here: a waiting message is on screen for as
|
/// and it matters more here: a waiting message is on screen for as
|
||||||
/// long as the turn runs, so its attachment has nowhere else to be.
|
/// long as the turn runs, so its attachment has nowhere else to be.
|
||||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
#[serde(default, alias = "images", skip_serializing_if = "Vec::is_empty")]
|
||||||
images: Vec<ImageRef>,
|
attachments: Vec<AttachmentRef>,
|
||||||
},
|
},
|
||||||
/// A message taken out of the queue before the session read it, by
|
/// A message taken out of the queue before the session read it, by
|
||||||
/// somebody tapping the bubble that was waiting for it.
|
/// somebody tapping the bubble that was waiting for it.
|
||||||
@@ -146,8 +157,8 @@ pub enum Event {
|
|||||||
id: Option<String>,
|
id: Option<String>,
|
||||||
text: String,
|
text: String,
|
||||||
/// Carried through onto the `UserMessage` with everything else.
|
/// Carried through onto the `UserMessage` with everything else.
|
||||||
#[serde(default, skip_serializing_if = "Vec::is_empty")]
|
#[serde(default, alias = "images", skip_serializing_if = "Vec::is_empty")]
|
||||||
images: Vec<ImageRef>,
|
attachments: Vec<AttachmentRef>,
|
||||||
},
|
},
|
||||||
/// Streaming assistant text; the phone renders the concatenation as
|
/// Streaming assistant text; the phone renders the concatenation as
|
||||||
/// markdown.
|
/// markdown.
|
||||||
@@ -515,7 +526,7 @@ pub trait Driver: Send + Sync {
|
|||||||
/// moment it actually starts reading it: that event is what puts the
|
/// moment it actually starts reading it: that event is what puts the
|
||||||
/// message in the transcript, so a driver that never sends it drops
|
/// message in the transcript, so a driver that never sends it drops
|
||||||
/// the message from the conversation entirely.
|
/// the message from the conversation entirely.
|
||||||
fn send_user_message(&self, text: String, images: Vec<ImageRef>);
|
fn send_user_message(&self, text: String, attachments: Vec<AttachmentRef>);
|
||||||
/// Takes back a message that is still waiting, named by the id its
|
/// Takes back a message that is still waiting, named by the id its
|
||||||
/// [`Event::MessageQueued`] carried.
|
/// [`Event::MessageQueued`] carried.
|
||||||
///
|
///
|
||||||
|
|||||||
+44
-16
@@ -7,6 +7,8 @@
|
|||||||
//! A leading word asks for something more specific:
|
//! A leading word asks for something more specific:
|
||||||
//!
|
//!
|
||||||
//! - `/tool [input]` -- a full tool run, start through end.
|
//! - `/tool [input]` -- a full tool run, start through end.
|
||||||
|
//! - `/bash [command]` -- a Bash call carrying that command, for what the
|
||||||
|
//! phone's shell highlighting does to a particular line.
|
||||||
//! - `/tools [n] [gap]` -- n calls back to back, for what a run of them
|
//! - `/tools [n] [gap]` -- n calls back to back, for what a run of them
|
||||||
//! looks like when a screen groups them. `gap` is seconds between one
|
//! looks like when a screen groups them. `gap` is seconds between one
|
||||||
//! call and the next, default none: it is what makes a run *grow* while
|
//! call and the next, default none: it is what makes a run *grow* while
|
||||||
@@ -37,7 +39,7 @@
|
|||||||
//! shape a real model's reply arrives in, and the one where the row a
|
//! shape a real model's reply arrives in, and the one where the row a
|
||||||
//! reader is anchored to is the row that keeps changing height.
|
//! reader is anchored to is the row that keeps changing height.
|
||||||
//! - `/mixed N` -- N beats of an interleaved transcript: paragraphs of
|
//! - `/mixed N` -- N beats of an interleaved transcript: paragraphs of
|
||||||
//! different lengths, single tool calls, runs of adjacent ones, images
|
//! different lengths, single tool calls, runs of adjacent ones, attachments
|
||||||
//! and a peer message. Rows of every shape and height the app draws, in
|
//! and a peer message. Rows of every shape and height the app draws, in
|
||||||
//! one session, which is what a scrolling problem needs in order to be
|
//! one session, which is what a scrolling problem needs in order to be
|
||||||
//! reproduced twice the same way.
|
//! reproduced twice the same way.
|
||||||
@@ -55,7 +57,9 @@ use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
|||||||
use std::sync::{Arc, Mutex};
|
use std::sync::{Arc, Mutex};
|
||||||
use std::time::Duration;
|
use std::time::Duration;
|
||||||
|
|
||||||
use super::driver::{Driver, Event, EventSink, ImageRef, QuestionOption, SessionStatus, Unqueued};
|
use super::driver::{
|
||||||
|
AttachmentRef, Driver, Event, EventSink, QuestionOption, SessionStatus, Unqueued,
|
||||||
|
};
|
||||||
|
|
||||||
/// Delay between streamed deltas -- long enough that streaming is visibly
|
/// Delay between streamed deltas -- long enough that streaming is visibly
|
||||||
/// streaming in the UI, short enough that tests waiting on a full turn
|
/// streaming in the UI, short enough that tests waiting on a full turn
|
||||||
@@ -94,7 +98,7 @@ pub struct EchoDriver {
|
|||||||
/// Held messages with the id of the `MessageQueued` each one announced,
|
/// Held messages with the id of the `MessageQueued` each one announced,
|
||||||
/// so the announcement can say which waiting bubble it resolves.
|
/// so the announcement can say which waiting bubble it resolves.
|
||||||
queued: Arc<Mutex<Vec<Held>>>,
|
queued: Arc<Mutex<Vec<Held>>>,
|
||||||
/// Where `/mixed` writes the images it references, which is the same
|
/// Where `/mixed` writes the attachments it references, which is the same
|
||||||
/// directory the files route serves them from.
|
/// directory the files route serves them from.
|
||||||
session_dir: PathBuf,
|
session_dir: PathBuf,
|
||||||
/// Ids of the questions awaiting an answer, in the order they were
|
/// Ids of the questions awaiting an answer, in the order they were
|
||||||
@@ -249,7 +253,7 @@ impl EchoDriver {
|
|||||||
/// transcript, and a command is not -- the manager has already
|
/// transcript, and a command is not -- the manager has already
|
||||||
/// recorded that one was sent, and saying so twice drew the same
|
/// recorded that one was sent, and saying so twice drew the same
|
||||||
/// line in both colours.
|
/// line in both colours.
|
||||||
fn handle(&self, text: String, images: Vec<ImageRef>, announce: bool) {
|
fn handle(&self, text: String, attachments: Vec<AttachmentRef>, announce: bool) {
|
||||||
let sink = self.sink.clone();
|
let sink = self.sink.clone();
|
||||||
|
|
||||||
// Mid-turn messages are held rather than answered, the way a real
|
// Mid-turn messages are held rather than answered, the way a real
|
||||||
@@ -265,9 +269,13 @@ impl EchoDriver {
|
|||||||
self.queued
|
self.queued
|
||||||
.lock()
|
.lock()
|
||||||
.unwrap()
|
.unwrap()
|
||||||
.push((id.clone(), text.clone(), images.clone()));
|
.push((id.clone(), text.clone(), attachments.clone()));
|
||||||
if announce {
|
if announce {
|
||||||
self.emit(Event::MessageQueued { id, text, images });
|
self.emit(Event::MessageQueued {
|
||||||
|
id,
|
||||||
|
text,
|
||||||
|
attachments,
|
||||||
|
});
|
||||||
}
|
}
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
@@ -288,7 +296,7 @@ impl EchoDriver {
|
|||||||
self.emit(Event::MessageTaken {
|
self.emit(Event::MessageTaken {
|
||||||
id: None,
|
id: None,
|
||||||
text: text.clone(),
|
text: text.clone(),
|
||||||
images: images.clone(),
|
attachments: attachments.clone(),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
self.emit(Event::Status {
|
self.emit(Event::Status {
|
||||||
@@ -319,7 +327,7 @@ impl EchoDriver {
|
|||||||
self.emit(Event::MessageTaken {
|
self.emit(Event::MessageTaken {
|
||||||
id: None,
|
id: None,
|
||||||
text: text.clone(),
|
text: text.clone(),
|
||||||
images: images.clone(),
|
attachments: attachments.clone(),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
self.emit(Event::PeerMessage {
|
self.emit(Event::PeerMessage {
|
||||||
@@ -345,7 +353,7 @@ impl EchoDriver {
|
|||||||
self.emit(Event::MessageTaken {
|
self.emit(Event::MessageTaken {
|
||||||
id: None,
|
id: None,
|
||||||
text,
|
text,
|
||||||
images,
|
attachments,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
self.compact();
|
self.compact();
|
||||||
@@ -357,7 +365,7 @@ impl EchoDriver {
|
|||||||
self.emit(Event::MessageTaken {
|
self.emit(Event::MessageTaken {
|
||||||
id: None,
|
id: None,
|
||||||
text,
|
text,
|
||||||
images,
|
attachments,
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
self.ask_user_question();
|
self.ask_user_question();
|
||||||
@@ -427,6 +435,9 @@ impl EchoDriver {
|
|||||||
text.strip_prefix("/tool")
|
text.strip_prefix("/tool")
|
||||||
.map(|rest| rest.trim().to_string())
|
.map(|rest| rest.trim().to_string())
|
||||||
};
|
};
|
||||||
|
let run_bash = text
|
||||||
|
.strip_prefix("/bash")
|
||||||
|
.map(|rest| rest.trim().to_string());
|
||||||
// Seconds to stay running before answering, default 30. Clamped
|
// Seconds to stay running before answering, default 30. Clamped
|
||||||
// rather than trusted: this is a test affordance, and a session
|
// rather than trusted: this is a test affordance, and a session
|
||||||
// pinned running for an hour by a typo is a worse outcome than a
|
// pinned running for an hour by a typo is a worse outcome than a
|
||||||
@@ -469,7 +480,7 @@ impl EchoDriver {
|
|||||||
send(Event::MessageTaken {
|
send(Event::MessageTaken {
|
||||||
id: None,
|
id: None,
|
||||||
text: text.clone(),
|
text: text.clone(),
|
||||||
images: images.clone(),
|
attachments: attachments.clone(),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
send(Event::Status {
|
send(Event::Status {
|
||||||
@@ -585,6 +596,23 @@ impl EchoDriver {
|
|||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if let Some(command) = run_bash {
|
||||||
|
let id = format!("b-{}", super::random_hex());
|
||||||
|
send(Event::ToolStart {
|
||||||
|
id: id.clone(),
|
||||||
|
tool: "Bash".to_string(),
|
||||||
|
input: serde_json::json!({
|
||||||
|
"command": command,
|
||||||
|
"description": "Run what /bash was given",
|
||||||
|
}),
|
||||||
|
});
|
||||||
|
tokio::time::sleep(DELTA_DELAY).await;
|
||||||
|
send(Event::ToolEnd {
|
||||||
|
id,
|
||||||
|
output: format!("ran: {command}"),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
if let Some(input) = run_tool {
|
if let Some(input) = run_tool {
|
||||||
let id = format!("t-{}", super::random_hex());
|
let id = format!("t-{}", super::random_hex());
|
||||||
send(Event::ToolStart {
|
send(Event::ToolStart {
|
||||||
@@ -762,7 +790,7 @@ async fn write_beat(sink: &EventSink, session_dir: &Path, beat: usize) {
|
|||||||
/// it. All three, because all three are what the `MessageTaken` at the other
|
/// it. All three, because all three are what the `MessageTaken` at the other
|
||||||
/// end owes -- named rather than written out at each of the four places that
|
/// end owes -- named rather than written out at each of the four places that
|
||||||
/// mention it.
|
/// mention it.
|
||||||
type Held = (String, String, Vec<ImageRef>);
|
type Held = (String, String, Vec<AttachmentRef>);
|
||||||
|
|
||||||
/// A markdown table [columns] wide, with cells too long for one line.
|
/// A markdown table [columns] wide, with cells too long for one line.
|
||||||
///
|
///
|
||||||
@@ -832,7 +860,7 @@ fn markdown_table(columns: usize) -> String {
|
|||||||
/// of them owes the same answer.
|
/// of them owes the same answer.
|
||||||
fn finish_turn(sink: &EventSink, queued: &Mutex<Vec<Held>>, busy: &AtomicBool) {
|
fn finish_turn(sink: &EventSink, queued: &Mutex<Vec<Held>>, busy: &AtomicBool) {
|
||||||
let held = std::mem::take(&mut *queued.lock().unwrap());
|
let held = std::mem::take(&mut *queued.lock().unwrap());
|
||||||
for (id, text, images) in held {
|
for (id, text, attachments) in held {
|
||||||
// Announced before it is answered, in that order: a phone showing
|
// Announced before it is answered, in that order: a phone showing
|
||||||
// the message as pending needs the signal that it has been read,
|
// the message as pending needs the signal that it has been read,
|
||||||
// and the answer is meaningless above a message still drawn as
|
// and the answer is meaningless above a message still drawn as
|
||||||
@@ -840,7 +868,7 @@ fn finish_turn(sink: &EventSink, queued: &Mutex<Vec<Held>>, busy: &AtomicBool) {
|
|||||||
let _ = sink.send(Event::MessageTaken {
|
let _ = sink.send(Event::MessageTaken {
|
||||||
id: Some(id),
|
id: Some(id),
|
||||||
text: text.clone(),
|
text: text.clone(),
|
||||||
images,
|
attachments,
|
||||||
});
|
});
|
||||||
let _ = sink.send(Event::AssistantText {
|
let _ = sink.send(Event::AssistantText {
|
||||||
delta: format!("\n(taken from the queue) You said: {text}"),
|
delta: format!("\n(taken from the queue) You said: {text}"),
|
||||||
@@ -874,14 +902,14 @@ impl Driver for EchoDriver {
|
|||||||
Unqueued::Dropped
|
Unqueued::Dropped
|
||||||
}
|
}
|
||||||
|
|
||||||
fn send_user_message(&self, text: String, images: Vec<ImageRef>) {
|
fn send_user_message(&self, text: String, attachments: Vec<AttachmentRef>) {
|
||||||
// Announced, because this is a message: every driver owes exactly
|
// Announced, because this is a message: every driver owes exactly
|
||||||
// one `MessageTaken` per message, and one that quietly vanishes
|
// one `MessageTaken` per message, and one that quietly vanishes
|
||||||
// from the transcript is the thing echo must not model. A command
|
// from the transcript is the thing echo must not model. A command
|
||||||
// owes none -- the manager has already recorded that it was sent,
|
// owes none -- the manager has already recorded that it was sent,
|
||||||
// and announcing it again drew the same line twice, once in each
|
// and announcing it again drew the same line twice, once in each
|
||||||
// colour.
|
// colour.
|
||||||
self.handle(text, images, true);
|
self.handle(text, attachments, true);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// Echo's commands *are* its messages -- `/tool`, `/slow`, `/ask` --
|
/// Echo's commands *are* its messages -- `/tool`, `/slow`, `/ask` --
|
||||||
|
|||||||
@@ -628,7 +628,7 @@ fn push_user(events: &mut Vec<Event>, content: &Value, session_dir: &std::path::
|
|||||||
events.push(Event::UserMessage {
|
events.push(Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text,
|
text,
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+16
-16
@@ -33,7 +33,7 @@ use anyhow::{Context, Result, bail};
|
|||||||
use serde::{Deserialize, Serialize};
|
use serde::{Deserialize, Serialize};
|
||||||
use serde_json::json;
|
use serde_json::json;
|
||||||
|
|
||||||
use super::driver::{Driver, Event, EventSink, ImageRef, SessionStatus};
|
use super::driver::{AttachmentRef, Driver, Event, EventSink, SessionStatus};
|
||||||
use super::process;
|
use super::process;
|
||||||
use super::transport::{Launch, Streams, Transport};
|
use super::transport::{Launch, Streams, Transport};
|
||||||
use crate::config::{ProviderConfig, SessionConfig};
|
use crate::config::{ProviderConfig, SessionConfig};
|
||||||
@@ -322,10 +322,10 @@ fn watch(session_dir: PathBuf, sink: EventSink) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
impl Driver for LlamaDriver {
|
impl Driver for LlamaDriver {
|
||||||
fn send_user_message(&self, text: String, images: Vec<ImageRef>) {
|
fn send_user_message(&self, text: String, attachments: Vec<AttachmentRef>) {
|
||||||
if !images.is_empty() {
|
if !attachments.is_empty() {
|
||||||
let _ = self.sink.send(Event::Error {
|
let _ = self.sink.send(Event::Error {
|
||||||
message: "this model can't be sent images".to_string(),
|
message: "this model can't be sent attachments or files".to_string(),
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
let sink = self.sink.clone();
|
let sink = self.sink.clone();
|
||||||
@@ -344,9 +344,9 @@ impl Driver for LlamaDriver {
|
|||||||
let _ = sink.send(Event::MessageTaken {
|
let _ = sink.send(Event::MessageTaken {
|
||||||
id: None,
|
id: None,
|
||||||
text: text.clone(),
|
text: text.clone(),
|
||||||
// Never any: this driver refuses images above, and saying
|
// Never any: this driver refuses attachments above, and
|
||||||
// so is what the refusal above is for.
|
// saying so is what the refusal above is for.
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
});
|
});
|
||||||
let _ = sink.send(Event::Status {
|
let _ = sink.send(Event::Status {
|
||||||
state: SessionStatus::Running,
|
state: SessionStatus::Running,
|
||||||
@@ -649,7 +649,7 @@ mod tests {
|
|||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "hello".into(),
|
text: "hello".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
Event::AssistantText {
|
Event::AssistantText {
|
||||||
delta: "hi ".into(),
|
delta: "hi ".into(),
|
||||||
@@ -663,7 +663,7 @@ mod tests {
|
|||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "again".into(),
|
text: "again".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
Event::AssistantText {
|
Event::AssistantText {
|
||||||
delta: "yes".into(),
|
delta: "yes".into(),
|
||||||
@@ -695,7 +695,7 @@ mod tests {
|
|||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "count".into(),
|
text: "count".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
Event::AssistantText {
|
Event::AssistantText {
|
||||||
delta: "one two".into(),
|
delta: "one two".into(),
|
||||||
@@ -721,7 +721,7 @@ mod tests {
|
|||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "hello".into(),
|
text: "hello".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
Event::Error {
|
Event::Error {
|
||||||
message: "something went wrong".into(),
|
message: "something went wrong".into(),
|
||||||
@@ -749,7 +749,7 @@ mod tests {
|
|||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "the long expensive conversation".into(),
|
text: "the long expensive conversation".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
Event::AssistantText {
|
Event::AssistantText {
|
||||||
delta: "at length".into(),
|
delta: "at length".into(),
|
||||||
@@ -758,7 +758,7 @@ mod tests {
|
|||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "a fresh start".into(),
|
text: "a fresh start".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
Event::AssistantText {
|
Event::AssistantText {
|
||||||
delta: "cheaply".into(),
|
delta: "cheaply".into(),
|
||||||
@@ -778,19 +778,19 @@ mod tests {
|
|||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "one".into(),
|
text: "one".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
Event::Cleared,
|
Event::Cleared,
|
||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "two".into(),
|
text: "two".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
Event::Cleared,
|
Event::Cleared,
|
||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "three".into(),
|
text: "three".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
]);
|
]);
|
||||||
let messages = conversation(&path);
|
let messages = conversation(&path);
|
||||||
|
|||||||
+65
-11
@@ -33,7 +33,7 @@ use crate::config::{
|
|||||||
};
|
};
|
||||||
use claude::ClaudeDriver;
|
use claude::ClaudeDriver;
|
||||||
use driver::{
|
use driver::{
|
||||||
Driver, Event, EventSink, ImageRef, SessionCommand, SessionStatus, Unqueued, context_after,
|
AttachmentRef, Driver, Event, EventSink, SessionCommand, SessionStatus, Unqueued, context_after,
|
||||||
};
|
};
|
||||||
use echo::EchoDriver;
|
use echo::EchoDriver;
|
||||||
use llama::LlamaDriver;
|
use llama::LlamaDriver;
|
||||||
@@ -417,14 +417,14 @@ impl LiveSession {
|
|||||||
/// The message is deliberately not recorded here. Sent into a running
|
/// The message is deliberately not recorded here. Sent into a running
|
||||||
/// turn it waits, and writing it down on the way past would put it
|
/// turn it waits, and writing it down on the way past would put it
|
||||||
/// above output that happened before the session ever saw it.
|
/// above output that happened before the session ever saw it.
|
||||||
pub fn send_message(&self, text: String, images: Vec<ImageRef>) {
|
pub fn send_message(&self, text: String, attachments: Vec<AttachmentRef>) {
|
||||||
// The attachments ride *on* the message rather than as `Image`
|
// The attachments ride *on* the message rather than as `Image`
|
||||||
// events emitted just before it. They used to be the latter, which
|
// events emitted just before it. They used to be the latter, which
|
||||||
// drew a person's screenshot as a row floating above the bubble
|
// drew a person's screenshot as a row floating above the bubble
|
||||||
// that sent it, and left the phone inferring from adjacency which
|
// that sent it, and left the phone inferring from adjacency which
|
||||||
// message an image went with -- a thing the sender already knew.
|
// message an image went with -- a thing the sender already knew.
|
||||||
self.ask("take a message", |driver| {
|
self.ask("take a message", |driver| {
|
||||||
driver.send_user_message(text, images)
|
driver.send_user_message(text, attachments)
|
||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -488,11 +488,24 @@ impl LiveSession {
|
|||||||
/// Stores one uploaded attachment, returning the id `POST /message`
|
/// Stores one uploaded attachment, returning the id `POST /message`
|
||||||
/// references it by. Removed with the session directory on delete --
|
/// references it by. Removed with the session directory on delete --
|
||||||
/// the same path out as everything else in it.
|
/// the same path out as everything else in it.
|
||||||
pub fn save_attachment(&self, bytes: &[u8], content_type: &str) -> Result<String> {
|
///
|
||||||
// An unrecognized type is almost always a phone photo whose
|
/// An image is named `<hex>.<extension>` and nothing else, since the
|
||||||
// content type the picker didn't set; jpg is the useful guess.
|
/// model is shown the picture rather than told its name. Anything else
|
||||||
let extension = crate::media::extension_for(content_type).unwrap_or("jpg");
|
/// keeps the name it arrived with after the hex: the session is told
|
||||||
let name = format!("{}.{extension}", random_hex());
|
/// the path, and a trace called `trace-komodo-….perfetto-trace` says
|
||||||
|
/// more to it than `3f9a…` would. The name is cleaned to characters a
|
||||||
|
/// path and a URL both take unquoted, and the hex keeps two uploads of
|
||||||
|
/// the same name apart. `AttachmentRef` documents the two shapes.
|
||||||
|
pub fn save_attachment(
|
||||||
|
&self,
|
||||||
|
bytes: &[u8],
|
||||||
|
content_type: &str,
|
||||||
|
file_name: Option<&str>,
|
||||||
|
) -> Result<AttachmentRef> {
|
||||||
|
let name = match crate::media::extension_for(content_type) {
|
||||||
|
Some(extension) => format!("{}.{extension}", random_hex()),
|
||||||
|
None => format!("{}-{}", random_hex(), safe_file_name(file_name)),
|
||||||
|
};
|
||||||
let dir = self.dir().join("attachments");
|
let dir = self.dir().join("attachments");
|
||||||
wg_app_link::private::create_dir(&dir)?;
|
wg_app_link::private::create_dir(&dir)?;
|
||||||
std::fs::write(dir.join(&name), bytes)
|
std::fs::write(dir.join(&name), bytes)
|
||||||
@@ -1442,14 +1455,19 @@ impl SessionManager {
|
|||||||
/// Started before the message rather than after, because starting
|
/// Started before the message rather than after, because starting
|
||||||
/// replaces the driver and the driver that takes the message has to be
|
/// replaces the driver and the driver that takes the message has to be
|
||||||
/// the one with a process behind it.
|
/// the one with a process behind it.
|
||||||
pub fn send_message(&self, id: &str, text: String, images: Vec<ImageRef>) -> Result<()> {
|
pub fn send_message(
|
||||||
|
&self,
|
||||||
|
id: &str,
|
||||||
|
text: String,
|
||||||
|
attachments: Vec<AttachmentRef>,
|
||||||
|
) -> Result<()> {
|
||||||
// Only `Exited` starts anything -- see `start_if_exited`. A session
|
// Only `Exited` starts anything -- see `start_if_exited`. A session
|
||||||
// this cannot say has exited keeps the behaviour it always had: the
|
// this cannot say has exited keeps the behaviour it always had: the
|
||||||
// message goes to the driver, which answers for it.
|
// message goes to the driver, which answers for it.
|
||||||
self.start_if_exited(id)?;
|
self.start_if_exited(id)?;
|
||||||
self.session(id)
|
self.session(id)
|
||||||
.with_context(|| format!("no session {id}"))?
|
.with_context(|| format!("no session {id}"))?
|
||||||
.send_message(text, images);
|
.send_message(text, attachments);
|
||||||
Ok(())
|
Ok(())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1767,6 +1785,34 @@ fn names<'a>(all: impl Iterator<Item = &'a str>) -> String {
|
|||||||
|
|
||||||
/// 8 random bytes, hex -- short enough for a URL, unique enough forever at
|
/// 8 random bytes, hex -- short enough for a URL, unique enough forever at
|
||||||
/// this scale.
|
/// this scale.
|
||||||
|
/// A file name reduced to what an attachment id may hold: letters, digits,
|
||||||
|
/// `.`, `-` and `_`, no run of dots that could read as a parent directory,
|
||||||
|
/// at most [`FILE_NAME_LIMIT`] characters keeping the tail (the extension
|
||||||
|
/// is what identifies a file), and `file` when nothing usable is left.
|
||||||
|
fn safe_file_name(name: Option<&str>) -> String {
|
||||||
|
let cleaned: String = name
|
||||||
|
.unwrap_or_default()
|
||||||
|
.chars()
|
||||||
|
.map(|c| {
|
||||||
|
if c.is_ascii_alphanumeric() || c == '.' || c == '-' || c == '_' {
|
||||||
|
c
|
||||||
|
} else {
|
||||||
|
'_'
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
let cleaned = cleaned.replace("..", "_").trim_matches('.').to_string();
|
||||||
|
if cleaned.is_empty() {
|
||||||
|
return "file".to_string();
|
||||||
|
}
|
||||||
|
let excess = cleaned.chars().count().saturating_sub(FILE_NAME_LIMIT);
|
||||||
|
cleaned.chars().skip(excess).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Longer than any name a person types, shorter than what a filesystem
|
||||||
|
/// refuses once the hex and a dash are in front of it.
|
||||||
|
const FILE_NAME_LIMIT: usize = 120;
|
||||||
|
|
||||||
pub fn random_hex() -> String {
|
pub fn random_hex() -> String {
|
||||||
use rand::Rng;
|
use rand::Rng;
|
||||||
let mut bytes = [0u8; 8];
|
let mut bytes = [0u8; 8];
|
||||||
@@ -2232,7 +2278,15 @@ async fn pump(
|
|||||||
// the position is the only way a reader can put it back where it
|
// the position is the only way a reader can put it back where it
|
||||||
// happened -- see `Event::PeerMessage::turn_start`.
|
// happened -- see `Event::PeerMessage::turn_start`.
|
||||||
let event = match event {
|
let event = match event {
|
||||||
Event::MessageTaken { id, text, images } => Event::UserMessage { id, text, images },
|
Event::MessageTaken {
|
||||||
|
id,
|
||||||
|
text,
|
||||||
|
attachments,
|
||||||
|
} => Event::UserMessage {
|
||||||
|
id,
|
||||||
|
text,
|
||||||
|
attachments,
|
||||||
|
},
|
||||||
Event::PeerMessage { from, text, .. } => Event::PeerMessage {
|
Event::PeerMessage { from, text, .. } => Event::PeerMessage {
|
||||||
from,
|
from,
|
||||||
text,
|
text,
|
||||||
|
|||||||
@@ -636,7 +636,7 @@ mod tests {
|
|||||||
Event::UserMessage {
|
Event::UserMessage {
|
||||||
id: None,
|
id: None,
|
||||||
text: "hi".into(),
|
text: "hi".into(),
|
||||||
images: Vec::new(),
|
attachments: Vec::new(),
|
||||||
},
|
},
|
||||||
text("hello"),
|
text("hello"),
|
||||||
Event::ToolStart {
|
Event::ToolStart {
|
||||||
|
|||||||
Reference in new issue
Block a user