JVNC Developer & Protocol Reference

For anyone maintaining JVNC or writing a compatible client. Start with Architecture & Code Map for the hot paths and the feature framework; this page is the wire-level reference.

1. Solution layout

Build: dotnet build JVNC.sln -c Release. Package: package.ps1. Installer: Installer\build-installer.ps1. Target net8.0-windows, no NuGet packages. Version in Directory.Build.props.

2. Wire protocol

2.1 Plain mode (all security off)

Standard RFB 3.8: RFB 003.008\n exchange, security type None (1), ClientInit/ServerInit, then normal messages. Encodings offered by the client: JVNC JPEG (private), CopyRect (1), Zlib (6) and Raw (0). Any third-party viewer works (it gets CopyRect too if it advertises it).

2.2 Security negotiation (1.0.5+)

The server always performs the standard RFB 003.008 version exchange, then offers security types: 250 (JVNC, private) always; 1 (None) when no layer is on; otherwise 19 (VeNCrypt) when TLS is on or 2 (VNC Authentication) when it is off — the latter two only if Allow standard VNC viewers is on and neither client certificates nor TOTP are required. After the chosen sub-handshake the server sends the normal u32 SecurityResult, then ClientInit/ServerInit proceed.

• Type 250: one flags byte (bit0 = TLS, bit1 = client certificate required, bit2 = TOTP required, bit3 = password required), then the steps below in order.
• Type 19: VeNCrypt 0.2; the server offers exactly one subtype, X509Vnc (260) when a password is required or X509None (259) otherwise; TLS with the server certificate; then, for X509Vnc, the VNC Auth challenge inside TLS.
• Type 2: 16-byte challenge, 16-byte DES response computed from the viewer password (RFB 7.2.2).

• Legacy: servers before 1.0.5 greeted with JVNC-000.01\n + flags before the RFB version; the client still understands that greeting.

• If TLS: both sides start a TLS 1.2/1.3 handshake on the raw socket (SslStream). The server presents server.pfx; if bit1 is set it requests a client certificate and validates the thumbprint against its approved list. The client validates the server by pinned thumbprint.

• If password: server → 16-byte salt + 16-byte nonce; client → 32-byte HMAC-SHA256(key = PBKDF2-SHA256(password, salt, 100000, 32), msg = nonce); server → u32 status.
• If TOTP: server → 16-byte random nonce; client → 32-byte HMAC-SHA256(key = UTF-8(6-digit code), msg = nonce); server → u32 status (0 ok, 1 fail, then closes). Server accepts codes for the current 30 s window ±1.
• The ordinary RFB 3.8 handshake then runs over the (possibly encrypted) stream.

Failures at any step count towards the per-address lockout (5 → 15 min ban). Before any of this the server applies the IP allow list and the public-address rule.

2.3 JVNC extension messages

Message types 250–253, used only between JVNC client and server, both directions unless noted.

A file is a header followed by data messages until size bytes have arrived. Chunks are separate messages so input and screen updates interleave with transfers.

2.4 Screen updates

• Server sends a full frame on a non-incremental request. On incremental requests it captures every 8 ms until something changed (or 1 s elapsed), diffs 64×64 tiles against the previous frame and sends only changed tiles (merged horizontally). Captures are floored at 8 ms apart.
• Adaptive lossy (1.0.8): the client also advertises private encoding EncodingJvncJpeg (0x4A564E43: u32 len + baseline JPEG, WIC). When a frame's changed area is ≥ 12 % of the requested region (and ≥ 250k px) the server encodes that frame's rects as JPEG (q85, q60 above 2 Mpx) and marks the covered 64×64 tiles lossy. Every frame stamps a per-tile last-change time; lossy tiles untouched for 250 ms are re-sent lossless (≤ 3 Mpx per clean-up frame), either piggy-backed on a small lossless frame or as a clean-up frame while the screen is otherwise idle — so a moving video stays lossy while the rest of the screen is exact. Change detection tolerates per-channel noise ≤ DiffTolerance (default 8) so HDR / dithering flicker does not count as change (RowDiffers: exact SequenceEqual fast path, vectorised tolerance compare only for rows that differ; a tile counts as changed once ≥ DiffMinBytes (12) bytes exceed it, so isolated sparkle is ignored but a caret is not).
• Capture (1.0.24): ScreenCapture.Capture() first asks DesktopDuplicator (DXGI Desktop Duplication, Windows 8+). Per output: AcquireNextFrame(0); a new frame is CopyResourced to a staging texture, mapped and copied into the next DIB buffer; outputs with no new frame are copied from the previous buffer; when no output presented and the cursor is unchanged the call returns false and the update loop just polls again. The first frame after (re)creation is a BitBlt baseline so unpresented outputs still have content. Any failure (ACCESS_LOST on mode change / secure desktop / UAC, rotated display, DuplicateOutput refused) disposes the duplicator and BitBlt serves until a retry (2-5 s) succeeds. The COM calls are raw vtable function pointers (slots and IIDs from the Windows 10 SDK headers) - no NuGet, no COM wrappers. Why: GDI BitBlt on Windows 11 reads the DWM desktop plane; when a browser/video is on a hardware overlay plane (MPO) BitBlt returns the stale/half-updated content under it in alternating rows - the "tearing on every monitor" that DwmFlush never reliably fixed. Turn it off with UseDesktopDuplication: false in server.json to compare.
• CopyRect scroll detection (1.0.23): after the diff, DetectScroll looks at every changed 64-px tile: a fingerprint of its middle row is compared with the previous frame's rows up to 160 above/below (the last accepted shift is tried first); a fingerprint hit is verified by an exact block compare. Matching adjacent tiles with the same shift become one CopyRect (16 bytes on the wire) from (x, y+d); the rest of the run stays a pixel rect. Sources must lie inside the requested region and be neither lossy nor deferred (the client must hold them verbatim). One sign of shift per frame; copies are written before pixel rects, top-down when content moved up and bottom-up when it moved down, so no copy reads a row a previous copy in the same update overwrote. At most 48 tiles per frame pay the full search; the rest only try the frame's shift, so a window drag (no vertical match) costs little. Client CopyWithin walks rows in overlap-safe order.
• Link adaptation (1.0.23): the server treats each FramebufferUpdateRequest as the acknowledgement of the previous frame (one frame in flight), so frame bytes / elapsed for frames ≥ 48 KB is the effective throughput (Session.LinkBytesPerMs, EWMA, starts at 20 MB/s). Per frame it derives a byte budget (80 ms of link time, clamped 192 KB..24 MB) and estimates the frame from measured bytes-per-pixel (LosslessBpp, JpegBpp, EWMA over encoded rects). Over budget, SplitToBudget sorts the rects by distance from the last pointer position, keeps as many as fit, and flags the others in DirtyTiles; Diff treats dirty tiles as changed so they go out with the following requests even if the screen is still. JPEG quality follows the link (85 above ~6 MB/s, 60 above ~2.5 MB/s, else 40) and a lossless frame estimated at more than twice the budget is sent as JPEG instead. Full (non-incremental) frames are never split. LinkCapBytesPerMs in server.json caps the measured rate for testing (0 = off).
• One persistent zlib stream per connection per direction (RFB Zlib encoding). The client must never discard unconsumed compressed bytes between rects: ZLibStream reads ahead and may leave a rect's sync-flush marker unread until the next rect (ChunkSource is append-only for this reason).
• The client requests the next update only after the current one has fully arrived (one frame in flight), decodes into RfbClient.Framebuffer on the receive thread and paints dirty rects on the UI thread at Input priority, capped at ~40 paints/s.
• The mouse cursor is composited into the capture.
• The client asks for the union of the regions it is displaying (SetViewRegion); a single-monitor view therefore costs only that monitor. A pending non-incremental request is never downgraded by a later incremental one (needed when the region changes).
• Audio: AudioCapture (server) = WASAPI loopback of the default render endpoint via raw COM interop, float/PCM mix → linear resample + downmix → 20 ms 16-bit blocks → IMA ADPCM (4:1) → MsgAudioData batched to ~60 ms. WaveOutPlayer (client) = winmm waveOut with per-block headers; late blocks are dropped.

2.5 Input

• Pointer: client coalesces motion to ≤125 events/s; button and wheel changes are sent immediately. Server positions with SetCursorPos (absolute SendInput moves are held at monitor edges by Windows 11's "ease cursor movement between displays") and uses SendInput for buttons/wheel.
• Keys: X11 keysyms. Printable characters are sent as Unicode keysyms and injected with KEYEVENTF_UNICODE; while Ctrl/Alt/Win is held they are mapped to virtual keys via VkKeyScan so hotkeys fire. Non-printable keys map through KeySymToVk, with scan codes and the extended flag where required. On disconnect the server releases everything held.
• Windows key: WinKeyHook (WH_KEYBOARD_LL) in the client forwards Win and Win+key chords and swallows them locally while the client is foreground and the screen image has focus.

3. Concurrency model (server)

Per client: a reader task parses messages and injects input immediately; an update task waits for a pending FramebufferUpdateRequest and does capture/diff/encode/send; all socket writes go through Session.WriteLock. File sends run as background tasks taking the write lock per chunk. Dialogs on either side are shown asynchronously (Dispatcher.InvokeAsync); incoming file data streams to a temp file until the user decides.

4. Testing

A headless stress client (see the session notes / Installer folder history) connects, sweeps the pointer, optionally opens a self-dragging "churn" window and a fragmenting TCP proxy, and reports fps and any receive-loop failure. Security layers are exercised by editing server.json (TLS, client cert, TOTP secret, allow list) and running the harness with JVNC_TOTP / JVNC_TRUST environment variables. Run the churn test for 60 s after any change to encoding or the receive loop.

5. Extending

• New encodings belong in Core constants + RfbServer.Encode + RfbClient.ReadRect.
• New JVNC messages: pick a type in 240–249, add constants to Rfb.cs, parse in both message loops, document them here.
• Keep Win32 interop in JVNC.Server (except DPAPI in Core), UI out of Rfb* classes, and no NuGet.