Take the resumed total from Content-Range, not Content-Length
On a 206 those two headers answer different questions: Content-Length is the length of the range, so a resume at 162 MB reports 72 MB and a bar drawn from it fills at a third of the model. The arithmetic that was here (`have + length`) happened to be right, but only because the range always starts exactly at what is on disk -- it was correct by coincidence of two things agreeing rather than by asking for the number wanted. Content-Range carries the whole size as its last field and does not care where the range began. Confirmed against HuggingFace: `content-range: bytes 162000000-234074815/234074816` beside `content-length: 72074816`, and a resumed download now reports 234.1 MB rather than 72. Two other things checked rather than assumed, both fine as they stood. Downloads are already single-flight per file -- the check and the insert happen under one lock, keyed by the model, so a second client asking for the same file joins the running download instead of starting a second writer onto the same partial. And HuggingFace's ETag is stable across requests, with no weak prefix or per-edge variation, so the identity check will not discard good partials and refetch gigabytes for nothing. One hypothesis worth recording as false: HF's ETag is not the content sha256 for these files (`db6593d0…` against a published `55e0d0b8…`), so the published-hash check cannot collapse into the identity check. Both earn their place. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017xn8nHw1tw1R6PtiY1eEtw
This commit is contained in:
1 parent
6f149398d0
commit
e50d1a2bbf
1 file changed
+20
-6
+20
-6
@@ -349,15 +349,29 @@ impl ModelStore {
|
||||
etag = etag_of(&response);
|
||||
}
|
||||
|
||||
let length: Option<u64> = response
|
||||
// On a 206, Content-Length is the length of the *range*, not of
|
||||
// the file -- it answers a different question than the one a
|
||||
// progress bar asks, and taken at face value it would fill the bar
|
||||
// at 72 MB of a 234 MB model. The whole size is the last field of
|
||||
// Content-Range (`bytes 162000000-234074815/234074816`), which has
|
||||
// the further merit of not depending on where the range began.
|
||||
let total: Option<u64> = if resumed {
|
||||
response
|
||||
.headers()
|
||||
.get("content-range")
|
||||
.and_then(|v| v.to_str().ok())
|
||||
.and_then(|v| {
|
||||
v.rsplit_once('/')
|
||||
.map(|(_, whole)| whole.trim().to_string())
|
||||
})
|
||||
.and_then(|whole| whole.parse().ok())
|
||||
} else {
|
||||
response
|
||||
.headers()
|
||||
.get("content-length")
|
||||
.and_then(|v| v.to_str().ok()?.parse().ok());
|
||||
let (mut done, total) = if resumed {
|
||||
(have, length.map(|l| have + l))
|
||||
} else {
|
||||
(0, length)
|
||||
.and_then(|v| v.to_str().ok()?.parse().ok())
|
||||
};
|
||||
let mut done = if resumed { have } else { 0 };
|
||||
{
|
||||
let mut p = run.progress.lock().unwrap();
|
||||
p.done = done;
|
||||
|
||||
Reference in new issue
Block a user