Say a transcript's size, name a model by something, and ask before a reload
Five things asked for on the phone, and one trap behind the first of them.
A llama prompt carries `<__media__>` where a picture was, and llama.cpp
pairs each marker with a decoded image when it tokenizes -- so a marker in
words nobody attached a picture to fails the turn, and then fails every
later one, since the conversation is folded out of a transcript that holds
it for good. A model saying the marker back is enough to do it. Every
message with words in it now goes through `without_marker`.
A model's `general.name` is filled in by whatever converted the file, and
`convert_hf_to_gguf.py` fills it from the directory it converted: Prism ML's
Bonsai publishes `general.name = "Hf"`, which is unique and so passed the
label cascade and told a reader nothing. A name is now used only where it
shares a word with the repo or the file it came from; one that does not
drops to the file name.
The model settings dialog and the server card said what a save would cost in
a paragraph under the control, read after the decision if at all. Both ask
instead, in the shape the rest of that screen already uses for Stop and
Delete -- and the dialog's question is asked over the edits, so Cancel comes
back to them.
A field's label was `labelMedium` in the variant colour while every setting
beside it was body text, which on one form read as two ranks of setting.
`GET /sessions/{id}`'s `transcriptFile` now carries the file's size, and the
settings screen draws it beside what this phone has cached.
Checked on the emulator against the sandbox: the labels line up, the row
says "14 kB · 14 kB cached", and both confirmations appear over a loaded
Qwen3-0.6B. `cargo test` 275 passed, clippy and fmt clean, lint clean.
This commit is contained in:
1 parent
4b5ed6e398
commit
1aac22bfc9
10 files changed
+291
-52
No files matched your search
+94
-1
@@ -74,7 +74,10 @@ pub struct LocalModel {
|
||||
pub fn labels(models: &[LocalModel]) -> Vec<String> {
|
||||
let rungs = |model: &LocalModel| {
|
||||
[
|
||||
model.name.clone(),
|
||||
model
|
||||
.name
|
||||
.clone()
|
||||
.filter(|name| names_the_file(name, model)),
|
||||
Some(model.file.trim_end_matches(".gguf").to_string()),
|
||||
Some(model.key.clone()),
|
||||
]
|
||||
@@ -106,6 +109,35 @@ pub fn labels(models: &[LocalModel]) -> Vec<String> {
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Whether a model's own `general.name` is a name for *this* model, or the
|
||||
/// converter's working directory wearing the same field.
|
||||
///
|
||||
/// The field is filled in by whatever produced the file, and
|
||||
/// `convert_hf_to_gguf.py` fills it from the directory it converted -- so a
|
||||
/// model converted out of a folder called `hf` publishes `general.name = "hf"`,
|
||||
/// which is unique, passes the cascade above, and tells a reader nothing at
|
||||
/// all. Prism ML's Bonsai is the one here (reported 2026-09-21, drawn as "hf"
|
||||
/// in the model picker).
|
||||
///
|
||||
/// The test is corroboration rather than a list of words to distrust: a real
|
||||
/// name shares something with where the file came from, both being about the
|
||||
/// same model, and a directory name picked up in passing does not. One word in
|
||||
/// common is enough. A name that fails it drops to the next rung, which is the
|
||||
/// file name the reader downloaded.
|
||||
fn names_the_file(name: &str, model: &LocalModel) -> bool {
|
||||
let words = |text: &str| -> Vec<String> {
|
||||
text.split(|c: char| !c.is_ascii_alphanumeric())
|
||||
.filter(|word| word.len() > 1)
|
||||
.map(str::to_ascii_lowercase)
|
||||
.collect()
|
||||
};
|
||||
let from = words(&model.repo);
|
||||
let from_file = words(&model.file);
|
||||
words(name)
|
||||
.iter()
|
||||
.any(|word| from.contains(word) || from_file.contains(word))
|
||||
}
|
||||
|
||||
/// What a download is doing, or did.
|
||||
///
|
||||
/// Flat rather than a tagged enum carrying its message, because the phone
|
||||
@@ -702,3 +734,64 @@ pub fn urlencode(value: &str) -> String {
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
mod tests {
|
||||
use super::*;
|
||||
|
||||
fn model(repo: &str, file: &str, name: Option<&str>) -> LocalModel {
|
||||
LocalModel {
|
||||
key: format!("{repo}/{file}"),
|
||||
repo: repo.to_string(),
|
||||
file: file.to_string(),
|
||||
bytes: 1,
|
||||
name: name.map(str::to_string),
|
||||
}
|
||||
}
|
||||
|
||||
/// The cascade, and the one rung that is not simply "is it unique": a name
|
||||
/// the file it came from says nothing about is the converter's working
|
||||
/// directory rather than the model's name.
|
||||
#[test]
|
||||
fn a_model_is_labelled_by_the_first_thing_that_identifies_it() {
|
||||
let models = [
|
||||
// Its own name, which is what a reader recognises.
|
||||
model(
|
||||
"Qwen/Qwen3-0.6B-GGUF",
|
||||
"Qwen3-0.6B-Q8_0.gguf",
|
||||
Some("Qwen3-0.6B"),
|
||||
),
|
||||
// Two quantisations carry one name, so both fall to the file.
|
||||
model("org/Big-GGUF", "Big-Q4_K_M.gguf", Some("Big")),
|
||||
model("org/Big-GGUF", "Big-Q8_0.gguf", Some("Big")),
|
||||
// `convert_hf_to_gguf.py` naming the directory it converted.
|
||||
model("PrismML/Bonsai-GGUF", "Bonsai-1B-TQ1_0.gguf", Some("hf")),
|
||||
// Nothing to go on but where it came from.
|
||||
model("org/Quiet-GGUF", "weights.gguf", None),
|
||||
];
|
||||
assert_eq!(
|
||||
labels(&models),
|
||||
[
|
||||
"Qwen3-0.6B",
|
||||
"Big-Q4_K_M",
|
||||
"Big-Q8_0",
|
||||
"Bonsai-1B-TQ1_0",
|
||||
"weights",
|
||||
],
|
||||
);
|
||||
}
|
||||
|
||||
/// Two repos holding a file of the same name: the file rung collides as
|
||||
/// well, and the key is what always terminates the cascade.
|
||||
#[test]
|
||||
fn a_file_name_two_repos_share_falls_through_to_the_key() {
|
||||
let models = [
|
||||
model("one/GGUF", "model.gguf", None),
|
||||
model("two/GGUF", "model.gguf", None),
|
||||
];
|
||||
assert_eq!(
|
||||
labels(&models),
|
||||
["one/GGUF/model.gguf", "two/GGUF/model.gguf"]
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -39,8 +39,8 @@
|
||||
//! GET /sessions list (id, provider, title, model, status, last activity)
|
||||
//! GET /sessions/{id} one session, for refetching after a change. Its
|
||||
//! `transcriptFile` is where the record itself is --
|
||||
//! {machine, machineName, path} -- so the explorer can be
|
||||
//! pointed at it
|
||||
//! {machine, machineName, path, bytes?} -- so the explorer
|
||||
//! can be pointed at it and its size said
|
||||
//! POST /sessions spawn {machine, provider, title?, model?, cwd?, params?}
|
||||
//! POST /sessions/order {sessions} -- the list in the order it is drawn in,
|
||||
//! as the reader dragged it; ids left out keep their
|
||||
|
||||
@@ -220,11 +220,46 @@ enum Content {
|
||||
Parts(Vec<Value>),
|
||||
}
|
||||
|
||||
/// llama.cpp's placeholder for a picture, as it appears *inside the prompt
|
||||
/// text* once the server has taken the image out of the request.
|
||||
///
|
||||
/// It is an in-band signal in the band that also carries what everybody typed,
|
||||
/// so it has to be taken back out of anything that is merely words.
|
||||
/// `mtmd_tokenize` splits the rendered prompt on it and pairs each occurrence
|
||||
/// with a decoded image, so one nobody sent an image for fails the turn --
|
||||
/// `number of media markers in text (1) exceeds number of bitmaps (0)`, which
|
||||
/// reaches the phone as "Failed to tokenize prompt" and then fails every later
|
||||
/// message too, the words being in the conversation for good. A model that
|
||||
/// says the marker back is enough to do it.
|
||||
///
|
||||
/// llama.cpp's own default, which is what it uses unless started with
|
||||
/// `--media-marker`; nothing here passes that, and the router writes the preset
|
||||
/// its children are started from (see [`router`]).
|
||||
const MEDIA_MARKER: &str = "<__media__>";
|
||||
|
||||
/// `text` with any [`MEDIA_MARKER`] removed, for content that is words rather
|
||||
/// than a picture.
|
||||
///
|
||||
/// Removed rather than escaped: there is nothing to escape it *to* -- the
|
||||
/// marker is matched literally, and the only form llama.cpp will not read as a
|
||||
/// picture is its absence.
|
||||
fn without_marker(text: impl Into<String>) -> String {
|
||||
let text = text.into();
|
||||
if text.contains(MEDIA_MARKER) {
|
||||
return text.replace(MEDIA_MARKER, "");
|
||||
}
|
||||
text
|
||||
}
|
||||
|
||||
impl Message {
|
||||
/// Words, with anything llama.cpp would read as a picture taken out of
|
||||
/// them -- see [`without_marker`]. Every message with text goes through
|
||||
/// here, which is what makes that structural rather than a rule to
|
||||
/// remember at each call.
|
||||
fn new(role: &str, content: impl Into<String>) -> Self {
|
||||
Self {
|
||||
role: role.to_string(),
|
||||
content: Content::Text(content.into()),
|
||||
content: Content::Text(without_marker(content)),
|
||||
tool_calls: Vec::new(),
|
||||
tool_call_id: None,
|
||||
}
|
||||
@@ -239,7 +274,7 @@ impl Message {
|
||||
/// templates, and only the first has ever been tested by everything else
|
||||
/// here.
|
||||
fn from_user(text: impl Into<String>, images: Vec<Value>) -> Self {
|
||||
let text = text.into();
|
||||
let text = without_marker(text);
|
||||
if images.is_empty() {
|
||||
return Self::new("user", text);
|
||||
}
|
||||
@@ -3252,6 +3287,30 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
/// llama.cpp's own picture placeholder is not content, and words that
|
||||
/// contain it are still words: left in, one of them fails the turn it is
|
||||
/// sent in and every turn after it, because the conversation is folded out
|
||||
/// of a transcript that now holds it for good.
|
||||
#[test]
|
||||
fn a_media_marker_in_words_never_reaches_the_model() {
|
||||
assert_eq!(
|
||||
Message::new("user", format!("what is {MEDIA_MARKER} this?")).content,
|
||||
Content::Text("what is this?".to_string()),
|
||||
);
|
||||
// A tool's output is the other text somebody else wrote.
|
||||
assert_eq!(
|
||||
Message::result_of("call_1", MEDIA_MARKER).content,
|
||||
Content::Text(String::new()),
|
||||
);
|
||||
// Beside a real picture, which is the part that says there is one.
|
||||
let image =
|
||||
json!({"type": "image_url", "image_url": {"url": "data:image/png;base64,UE5H"}});
|
||||
assert_eq!(
|
||||
Message::from_user(MEDIA_MARKER, vec![image.clone()]).content,
|
||||
Content::Parts(vec![image]),
|
||||
);
|
||||
}
|
||||
|
||||
/// A file that cannot be read is left out rather than failing the turn:
|
||||
/// the send path has already said so, and a turn is a worse place to learn
|
||||
/// it.
|
||||
|
||||
@@ -342,6 +342,12 @@ pub struct TranscriptFile {
|
||||
pub machine: String,
|
||||
pub machine_name: String,
|
||||
pub path: String,
|
||||
/// How big the record is, for a reader deciding whether to open it.
|
||||
/// Absent where the file could not be measured, which is a session that
|
||||
/// has not written one yet -- and is deliberately not a zero, since "no
|
||||
/// transcript" and "we could not look" are different answers.
|
||||
#[serde(skip_serializing_if = "Option::is_none")]
|
||||
pub bytes: Option<u64>,
|
||||
}
|
||||
|
||||
/// Where a session directory keeps its transcript.
|
||||
@@ -352,12 +358,12 @@ fn transcript_in(dir: &Path) -> PathBuf {
|
||||
/// Where `id`'s transcript is, said the way the explorer takes it.
|
||||
fn transcript_file(config: &Config, data_dir: &Path, id: &str) -> Option<TranscriptFile> {
|
||||
let machine = config.machine(crate::config::LOCAL_MACHINE_ID)?;
|
||||
let path = transcript_in(&data_dir.join(id));
|
||||
Some(TranscriptFile {
|
||||
machine: machine.id.clone(),
|
||||
machine_name: machine.name.clone(),
|
||||
path: transcript_in(&data_dir.join(id))
|
||||
.to_string_lossy()
|
||||
.into_owned(),
|
||||
bytes: std::fs::metadata(&path).ok().map(|file| file.len()),
|
||||
path: path.to_string_lossy().into_owned(),
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
Reference in new issue
Block a user