Add Mosh-style predictive local echo for shell sessions#290
Merged
chipsenkbeil merged 18 commits intomasterfrom Mar 15, 2026
Merged
Add Mosh-style predictive local echo for shell sessions#290chipsenkbeil merged 18 commits intomasterfrom
chipsenkbeil merged 18 commits intomasterfrom
Conversation
chipsenkbeil
force-pushed
the
feature/mosh-predictive-echo
branch
from
0095b2c to
0080ccf
Compare
chipsenkbeil
added a commit
that referenced
this pull request
Mar 13, 2026
…290) distant-ssh now reads the system SSH config (`/etc/ssh/ssh_config` on Unix, `%ProgramData%\ssh\ssh_config` on Windows) and merges it with the user config (user takes precedence via `overwrite_if_none`). ProxyCommand values from either config or explicit opts are executed via `sh -c` / `cmd /C`, with stdin/stdout bridged to russh via `connect_stream`. The special value "none" disables an inherited proxy, matching OpenSSH behavior. System-wide known_hosts (`/etc/ssh/ssh_known_hosts{,2}`) are now checked in addition to user paths, and `GlobalKnownHostsFile` from SSH config is respected. `IdentitiesOnly` is also resolved from SSH config as a fallback. A portable `tcp-to-stdio` test helper binary enables integration testing of ProxyCommand without platform-specific tools. @cert-authority known_hosts parsing is deferred (tracked in docs/TODO.md) pending russh host certificate support.
chipsenkbeil
force-pushed
the
feature/mosh-predictive-echo
branch
from
57d361b to
618767d
Compare
chipsenkbeil
commented
Mar 15, 2026
Implement a client-side prediction engine that immediately displays typed characters locally, then confirms or rolls back when the server responds. This reduces perceived latency from ~400ms+ to near-instant on high-latency links. Key components: - PredictionEngine with full epoch system (tentative predictions in password prompts/vim stay hidden until confirmed) - Jacobson/Karels SRTT estimator for adaptive mode decisions - Lightweight CSI parser tracking cursor column from server output - Bulk paste detection (>100 bytes in 10ms) triggers reset - --predict flag on shell/spawn/ssh (default: adaptive) Adaptive thresholds (from Mosh): - SRTT < 30ms: predictions hidden (low latency, not needed) - 30-80ms: predictions shown normally - >= 80ms: predictions shown underlined as visual hint No protocol changes required — purely client-side with timeout-based confirmation heuristics. echo_ack deferred to a future PR.
- Overwrite underlined predictions with plain byte on confirmation instead of suppressing (fixes persistent underlines) - Use non-destructive backspace (space overwrite) instead of ESC[K which destroyed the rest of the line - Detect CSI cursor-movement/erase commands during pending predictions and roll back immediately (fixes vim normal-mode overlay) - Emit SGR reset in rollback and expiry paths to clear residual underline/color state - Write ESC[0m at shutdown to prevent stale SGR attributes
- Backspace only undoes own pending predictions (display_ahead > 0); when no predictions are pending, treat as epoch boundary to avoid visually erasing prompt text - Detect DEC private mode alternate screen (modes 1049/47/1047) and suppress predictions while in alternate screen, fixing vim underscored spaces, hjkl overwrite, and insert-mode cursor jitter - Reorder CSI rollback to fire before the CSI is written to output, ensuring cursor-left cleanup operates from the correct position - Rollback alternate screen enter before the screen-switch CSI so cleanup targets the main screen buffer
After each epoch boundary (Enter, Ctrl-C, ESC), predictions in the new epoch are hidden until the server echoes back at least one character. Password prompts never echo, so nothing is shown — preventing visual password leaks on high-latency connections. Tradeoff: first char after each epoch boundary has server latency (not predicted). All subsequent chars in that epoch predict normally. This matches Mosh's tentative epoch design.
Replace the fragile byte-stream prediction engine with a full framebuffer renderer: vt100 parses server bytes into a virtual screen, ratatui diffs and renders to the local terminal, and predictions are trivial overlays on the vt100 screen state. Rollback is implicit — remove the overlay and the vt100 screen is ground truth. - Add crossterm 0.29, ratatui 0.30, vt100 0.16.2; remove termwiz 0.23.3 - New framebuffer.rs: TerminalFramebuffer (vt100 + ratatui + overlay), PredictionOverlay with epoch tracking and RTT-based confirmation - New keyencode.rs: crossterm KeyEvent → xterm byte sequence encoder - Rewrite terminal.rs: crossterm raw mode/input, framebuffer rendering - Strip predict.rs to PredictMode enum + RttEstimator (from ~2080 to ~180 lines)
ratatui's cell-by-cell diff rendering was fundamentally incompatible with transparent terminal proxying, causing 5 runtime bugs: prompt drift after su+Ctrl-C, no screen clear on startup, vim artifacts, vim prediction artifacts, and arrow keys printing as literal text. Replace with direct byte passthrough: server bytes flow through TerminalSanitizer → vt100 shadow screen → stdout. Predictions are displayed via DECSC/DECRC cursor save/restore with CUP positioning. Alternate screen detection (vt100) suppresses predictions during full-screen apps like vim.
Two bugs fixed: 1. Duplicate text at terminal top: build_prediction_display_bytes() used CUP absolute positioning with vt100 shadow screen coordinates, which don't match the real terminal's cursor position. Replaced with sequential character writes from the saved cursor position (DECSC/DECRC). 2. Permanent prediction suppression after su+Ctrl-C: epoch_counter raced ahead of confirmed_epoch with no recovery mechanism. Added catch-up logic in new_epoch() (when cells empty at boundary) and process_server_output() (after predictions confirmed/expired). Also serialized prediction display writes inside the framebuffer lock to prevent interleaved escape sequences with server output.
Eliminate stdout race condition by having the StdoutFilter write directly to stdout while holding the framebuffer lock, instead of returning bytes for the macro to write after lock release. Add prediction erasure via append_erase/append_predictions helpers. Leave cursor at prediction end (no DECRC) to fix visible cursor lag. Predict backspace erasure (\b \b) for existing text when no pending predictions exist.
Remove backspace prediction (was causing double-delete by emitting \b \b on top of server echo). Defer erase on epoch boundaries so Enter/Escape don't flash (erase happens atomically in render_server_output). Add DECCKM support so arrow keys emit SS3 sequences in top/htop/less. Refactor terminal sanitizer with generic escape parser (CSI/OSC/DCS/SS3) and centralized classify dispatch; now strips kitty keyboard query and DECRQSS that caused neovim garbage.
Predict backspace visually using CUB relative cursor movement and shadow screen character restoration. Uses the same atomic erase-before- server-output pattern as forward predictions: display erases the char instantly, then undo the prediction before server bytes apply, so the server's own \b \b echo operates on a clean display.
Add input_floor tracking to PredictionOverlay that records where user input started within each epoch. Backspace predictions cannot go below this column, preventing visual deletion of the shell prompt.
Predictions in a new epoch (after Enter, Tab, etc.) are now added to cells but NOT displayed until the server echoes a matching character, confirming the epoch. This matches mosh's "prove itself anew" design: - Password prompts: server never echoes → predictions stay hidden - Normal commands: first char has RTT latency, then predictions resume The display gate lives in on_keystroke and render_server_output, not in the overlay's add_prediction. This lets confirm_predictions advance confirmed_epoch via character matching, which is the mechanism that unlocks display for subsequent keystrokes.
Add cross-chunk buffering to TerminalSanitizer so terminal query sequences (DA1, DA2, DSR, etc.) that straddle output chunk boundaries are not leaked to the local terminal. Previously a lone ESC at the end of a chunk would pass through, the continuation bytes would pass through literally, and the local terminal would reassemble and respond — causing crossterm to misparse the response into phantom keystrokes in vim.
…rokes Neovim sends XTGETTCAP (DCS +q) queries during startup to probe terminal capabilities. The sanitizer only stripped DCS $q (DECRQSS), so +q queries leaked to the local terminal. Ghostty responded with DCS 1+r payloads that crossterm misinterpreted as Alt+P plus spurious key events, causing phantom keystrokes in vim. Also adds private DSR (ESC[?5n/ESC[?6n) stripping as defense-in-depth.
Fast mode always predicts and skips epoch confirmation, displaying predictions immediately after epoch boundaries (Enter, Escape, etc.) without waiting for server echo. FastAdaptive combines adaptive RTT gating (≥30ms) with epoch-gate bypass when active. Default remains Adaptive (epoch-gated) for password safety.
Address PR review comments: strip section-separator comments from the match block, keeping only a plain comment on the catch-all arm.
- Use crate:: imports instead of super::super:: in shell.rs - Re-export PredictMode from common.rs module - Simplify PredictMode re-export in cli.rs - Update doc comments to list all 5 predict modes
Replace the single unix-only predict-pty-helper binary with three focused, cross-platform binaries: - pty-echo: byte-by-byte stdin→stdout echo loop - pty-password: password prompt via rpassword + echo loop - pty-interactive: mini-shell with passwd command and Ctrl+C handling Uses rpassword (cross-platform) and ctrlc crate instead of raw libc signal handling, enabling Windows support.
chipsenkbeil
force-pushed
the
feature/mosh-predictive-echo
branch
from
3dfc520 to
f112778
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a client-side predictive echo engine inspired by Mosh that immediately
renders typed characters locally, then confirms or rolls back when the server
responds. On high-latency connections, this reduces perceived per-keystroke
latency from hundreds of milliseconds to near-instant.
Prediction engine
vt100::Parsershadow screen tracks serverstate while a
PredictionOverlaymanages speculative characters at computedcursor positions. Server output is sanitized and passed through to stdout
directly; predictions are rendered/erased via cursor escape sequences around
each server output chunk.
opens a new epoch. Predictions in non-echoing contexts (password prompts,
vim, etc.) stay hidden until the server confirms the epoch, preventing
password character leaks.
forward prediction on the same line, never erasing prompt text. An
input_floortracks where user input began within each epoch.predictions to avoid a cascade of speculative characters.
and display thresholds (underline at >80 ms SRTT).
applications (vim, less, tmux) and resume on exit.
CLI integration
--predictflag ondistant shell,distant spawn --pty, anddistant sshwith five modes:
adaptive(default, activates when SRTT > 30 ms),on(always),
off(never),fast(always, skip epoch confirmation), andfast-adaptive(adaptive + skip epoch confirmation).Terminal sanitizer improvements
TerminalSanitizerwith a proper escape sequence parser thatclassifies CSI, OSC, DCS, SS3, and two-character escapes, replacing the
previous regex/byte-scan approach.
boundaries, preventing phantom keystrokes from partial sequence reassembly.
DECRQSS queries, and private-mode DSR — in addition to the previously
handled DA1/DA2/DA3, DECRQM, XTVERSION, and XTWINOPS sequences.
Key encoder
keyencodemodule convertscrossterm::KeyEventinto Xterm bytesequences with proper DECCKM (application cursor) support, replacing
the previous
termwizdependency for input encoding.Stdout filter refactor
StdoutFilterchanged fromfn(&[u8], &mut Vec<u8>)toBox<dyn Fn(&[u8]) + Send>, allowing the filter to handle stdout writinginternally under a lock for atomic prediction erase/redraw cycles.
Test helper binaries
pty-echo— byte-by-byte echo loop for prediction confirmation testspty-password— password prompt scenario usingrpasswordfor no-echodetection tests
pty-interactive— mini-shell with prompt,exit/passwdcommands, andctrlcsignal handling for prompt boundary testsAll three are cross-platform (replaced the previous unix-only
libc-basedhelper).
Test plan
safety, RTT estimation, CSI parsing, bulk paste detection, alternate screen
suppression, password suppression, stale epoch cleanup, cross-chunk escape
buffering, and key encoding
cargo clippy --all-features --workspace --all-targets— 0 warnings