Shrink a photo to what the provider takes, and put it in its own bubble
Sending an image was broken in the way that is hardest to see from the phone: a camera photo is twelve megapixels and several megabytes, the Claude API resizes anything past 1568px on its long edge before looking at it and refuses far larger outright, so the picture was uploaded whole over the tunnel to be thrown away or rejected at the other end. Shrunk on the phone, to a limit the server states. Which number it is comes from the provider's *kind* -- `DriverKind::max_image_edge`, reported on the session row -- because that is where a provider's requirements are known, and a phone carrying its own copy of them would be a second place to update when one changes. `None` where nothing cares, rather than a large number: "no limit" and "a limit that happens to be big" are different answers and only one of them stays true. Doing it before the upload rather than after is the point -- the expensive part on a phone is the tunnel, not the decode -- and an image already inside the limit is uploaded byte for byte rather than being round-tripped through JPEG for nothing. EXIF orientation is applied while scaling. The camera writes which way up the picture is into a tag rather than into the pixels, and re-encoding drops it, so a portrait photo would have arrived at the model on its side with nothing anywhere saying so. **What is attached is now visible before it is sent**, in a row directly above the box it will be sent from: the count on the "+" button said how many and never which, so the only way to find out what you had picked was to send it. It scrolls sideways rather than shrinking, and tapping one takes it back off -- an image picked by mistake could otherwise only be dealt with by sending it. The tile is outlined as well as filled, because most of what gets attached here is a screenshot of a dark app and a cropped one is near-black: without an edge the only thing on screen saying an image was attached was the cross drawn on top of nothing. **And the picture is inside the bubble that sent it.** Attachments used to be their own `Image` events emitted just before the message, which drew somebody's screenshot as a row floating above the bubble and left the phone deciding from adjacency alone which message an image belonged to -- a thing the sender knew and could simply say. `UserMessage`, `MessageQueued` and `MessageTaken` carry the refs now, so a waiting message keeps its picture for as long as the turn runs, and a replay puts it back in the same place. Verified on a real claude-cli session rather than an echo one, since the limit only exists for that kind: a 3000x4000 image arrived as 1176x1568 JPEG -- long edge exactly the limit, aspect ratio intact -- and haiku answered "AI Sessions displays idle Photo", which is what the picture was. No error, and the transcript records the message with `images` on it.
This commit is contained in:
1 parent
5d47a1ec89
commit
b0629f77ca
16 files changed
+529
-70
No files matched your search
@@ -82,7 +82,7 @@ pub struct EchoDriver {
|
||||
busy: Arc<AtomicBool>,
|
||||
/// Held messages with the id of the `MessageQueued` each one announced,
|
||||
/// so the announcement can say which waiting bubble it resolves.
|
||||
queued: Arc<Mutex<Vec<(String, String)>>>,
|
||||
queued: Arc<Mutex<Vec<Held>>>,
|
||||
/// Where `/mixed` writes the images it references, which is the same
|
||||
/// directory the files route serves them from.
|
||||
session_dir: PathBuf,
|
||||
@@ -211,7 +211,7 @@ impl EchoDriver {
|
||||
/// transcript, and a command is not -- the manager has already
|
||||
/// recorded that one was sent, and saying so twice drew the same
|
||||
/// line in both colours.
|
||||
fn handle(&self, text: String, _images: Vec<ImageRef>, announce: bool) {
|
||||
fn handle(&self, text: String, images: Vec<ImageRef>, announce: bool) {
|
||||
let sink = self.sink.clone();
|
||||
|
||||
// Mid-turn messages are held rather than answered, the way a real
|
||||
@@ -224,9 +224,12 @@ impl EchoDriver {
|
||||
// an echo session has to produce the same events or the states
|
||||
// it exists to exercise are not the app's real ones.
|
||||
let id = super::random_hex();
|
||||
self.queued.lock().unwrap().push((id.clone(), text.clone()));
|
||||
self.queued
|
||||
.lock()
|
||||
.unwrap()
|
||||
.push((id.clone(), text.clone(), images.clone()));
|
||||
if announce {
|
||||
self.emit(Event::MessageQueued { id, text });
|
||||
self.emit(Event::MessageQueued { id, text, images });
|
||||
}
|
||||
return;
|
||||
}
|
||||
@@ -242,6 +245,7 @@ impl EchoDriver {
|
||||
self.emit(Event::MessageTaken {
|
||||
id: None,
|
||||
text: text.clone(),
|
||||
images: images.clone(),
|
||||
});
|
||||
}
|
||||
self.emit(Event::PeerMessage {
|
||||
@@ -263,7 +267,11 @@ impl EchoDriver {
|
||||
// this is the typed path onto it.
|
||||
if text.trim() == "/compact" {
|
||||
if announce {
|
||||
self.emit(Event::MessageTaken { id: None, text });
|
||||
self.emit(Event::MessageTaken {
|
||||
id: None,
|
||||
text,
|
||||
images,
|
||||
});
|
||||
}
|
||||
self.compact();
|
||||
return;
|
||||
@@ -271,7 +279,11 @@ impl EchoDriver {
|
||||
|
||||
if text.trim() == "/ask" {
|
||||
if announce {
|
||||
self.emit(Event::MessageTaken { id: None, text });
|
||||
self.emit(Event::MessageTaken {
|
||||
id: None,
|
||||
text,
|
||||
images,
|
||||
});
|
||||
}
|
||||
self.ask_user_question();
|
||||
return;
|
||||
@@ -352,6 +364,7 @@ impl EchoDriver {
|
||||
send(Event::MessageTaken {
|
||||
id: None,
|
||||
text: text.clone(),
|
||||
images: images.clone(),
|
||||
});
|
||||
}
|
||||
send(Event::Status {
|
||||
@@ -584,13 +597,20 @@ async fn write_beat(sink: &EventSink, session_dir: &Path, beat: usize) {
|
||||
tokio::time::sleep(Duration::from_millis(120)).await;
|
||||
}
|
||||
|
||||
/// A message written during a turn and waiting for it to end: the id of the
|
||||
/// `MessageQueued` that announced it, what it said, and what was attached to
|
||||
/// it. All three, because all three are what the `MessageTaken` at the other
|
||||
/// end owes -- named rather than written out at each of the four places that
|
||||
/// mention it.
|
||||
type Held = (String, String, Vec<ImageRef>);
|
||||
|
||||
/// Ending a turn is also when anything held during it is taken up -- the
|
||||
/// moment a real CLI would have injected it. One place, because a turn has
|
||||
/// several ways to end (a reply, an interrupt, a compaction) and every one
|
||||
/// of them owes the same answer.
|
||||
fn finish_turn(sink: &EventSink, queued: &Mutex<Vec<(String, String)>>, busy: &AtomicBool) {
|
||||
fn finish_turn(sink: &EventSink, queued: &Mutex<Vec<Held>>, busy: &AtomicBool) {
|
||||
let held = std::mem::take(&mut *queued.lock().unwrap());
|
||||
for (id, text) in held {
|
||||
for (id, text, images) in held {
|
||||
// Announced before it is answered, in that order: a phone showing
|
||||
// the message as pending needs the signal that it has been read,
|
||||
// and the answer is meaningless above a message still drawn as
|
||||
@@ -598,6 +618,7 @@ fn finish_turn(sink: &EventSink, queued: &Mutex<Vec<(String, String)>>, busy: &A
|
||||
let _ = sink.send(Event::MessageTaken {
|
||||
id: Some(id),
|
||||
text: text.clone(),
|
||||
images,
|
||||
});
|
||||
let _ = sink.send(Event::AssistantText {
|
||||
delta: format!("\n(taken from the queue) You said: {text}"),
|
||||
|
||||
Reference in new issue
Block a user