Delete a batch of imports in one command, not one connection each
Replaces the connection throttle from the previous commit, which was a workaround: it made a batch slower without changing that N sessions meant N ssh connections, and the cap it picked was a guess at somebody else's sshd config. The batch now goes out as a single invocation. import::delete takes the whole list, the remote script loops over the ids and prints one `<id>\t<state>` line each, and the route settles every row from its own line. So a batch of any size is one connection and cannot exceed MaxStartups however many rows are selected -- and it is faster, since it stopped paying a handshake per session. Each id still reports on its own: deleted, missing, or failed, kept apart because only "failed" is worth retrying. Ids the machine never mentioned -- a connection that dropped part-way -- are reported as unknown rather than defaulting to either answer, and a malformed id fails only itself. Verified end-to-end against a fake ssh that counts connections: a 9-session batch used one, every row settled, both copies of a session recorded under two project directories went, and the id that was not there failed with a message saying so rather than a connection error. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
1 parent
7a1a462caf
commit
dce1799802
4 files changed
+230
-79
No files matched your search
@@ -21,31 +21,12 @@
|
||||
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::process::Stdio;
|
||||
use std::sync::LazyLock;
|
||||
|
||||
use anyhow::{Context, Result};
|
||||
use tokio::process::Child;
|
||||
use tokio::sync::Semaphore;
|
||||
|
||||
use crate::config::SshConfig;
|
||||
|
||||
/// How many [`Transport::capture`] calls may have an ssh connection open at
|
||||
/// once, across every setup.
|
||||
///
|
||||
/// `capture` is what a batch (deleting several imports, listing several
|
||||
/// machines) fans out over -- one child `ssh` process per call, all started
|
||||
/// within the same tick. Nothing here throttled that, so a batch large
|
||||
/// enough to open more connections than the remote sshd's default
|
||||
/// `MaxStartups` (10, before it starts randomly refusing) has some of
|
||||
/// them come back as "Connection closed" -- not a real failure of the
|
||||
/// operation, just too many handshakes landing on the listener at once. Four
|
||||
/// keeps a batch comfortably under that ceiling while still overlapping the
|
||||
/// network round trips. Long-lived processes (`Transport::spawn`, a
|
||||
/// session's own child) don't take a permit: they hold it for the session's
|
||||
/// lifetime rather than for one round trip, which would starve every other
|
||||
/// probe behind it.
|
||||
pub(crate) static CAPTURE_PERMITS: LazyLock<Semaphore> = LazyLock::new(|| Semaphore::new(4));
|
||||
|
||||
/// What a driver needs run in order to exist as a process.
|
||||
///
|
||||
/// Deliberately just the three things every transport can carry. Anything
|
||||
|
||||
Reference in new issue
Block a user