splitux together

Remote-Play-Together–style online co-op layered on top of splitux. Splitux already turns one machine into N isolated split-screen seats (gamescope + bubblewrap per player, per-instance controller isolation, Goldberg LAN emulation). splitux together adds the missing half: let a remote friend occupy one of those seats from a browser — receiving the game video as encoded frames the browser decodes itself, and sending their gamepad back into a virtual controller that splitux assigns to that instance.

splitux ── launches ──►  N × [ gamescope + bwrap + game ]      (already exists)
   │                            │ each = a capturable surface
   │ assigns uinput pad k ──────┘
   ▼
seat-streamer (one per seat):
   pipewiresrc(gamescope-k) ! vulkanupload ! vulkanh264enc ! h264parse
       ! tee ─┬─► fakesink                     (no video into webrtcbin)
              └─► wcsidecar ── chunks ──┐
   pulsesrc ! opusenc ! webrtcbin ──────┼──► WebRTC: AUDIO + signalling only
                                        ▼
                             WebTransport (QUIC), one session:
                               datagrams  ──► video chunks
                               datagrams  ◄── input state
                               stream     ◄─► control (stats, keyframes, vis)
   └─► writes to uinput pad k  (splitux binds it to instance k)
   ▼
orchestrator: signalling broker + seat registry + static web host
   browser: lists seats, click → VideoDecoder → <canvas> + Gamepad API

There is no datachannel. Video rode an unreliable RTCDataChannel and input rode a reliable one until 2026-09-19; both are WebTransport now and the SCTP association is gone entirely. WebRTC is kept for the Opus track and for ICE, which is what discovers the address the QUIC session is dialed on.

Why this shape

• No Wolf, no Sunshine. Splitux is the multi-seat engine already; layering Wolf on top would mean two seat managers fighting. We only needed the streaming half.
• Vulkan HW encode, not x264. On RDNA4 (RX 9070) there is no VAAPI encode; hardware encode is Vulkan Video (vulkanh264enc). This is essential here because the box is simultaneously running N games — software (x264) encode would steal CPU from the games, while VCN (the encode block) is separate silicon that doesn't touch render performance.
• webrtcbin, not webrtcsink. Each seat is 1 game → 1 player → 1 peer, so webrtcsink's multi-consumer fan-out is unused overhead, and it can't drive vulkanh264enc (no vulkanupload insertion, no congestion-control bitrate support). Plain webrtcbin gives full pipeline control and HW encode.

Validated on this machine (RX 9070 / RADV GFX1201, gst 1.28)

• vulkanh264enc encodes H.264 (constrained-baseline — ideal for universal browser decode). ✔
• vulkanupload ! vulkanh264enc ! h264parse(config-interval=-1) ! rtph264pay produces browser-ready RTP (packetization-mode=1, profile-level-id=42c02a, sprop-parameter-sets) at 1080p60. ✔

Crates

crate	what
crates/seat-streamer	one process per seat: Vulkan encode pipeline, the WebTransport endpoint that carries video and input, webrtcbin for audio, and the persistent uinput virtual gamepad. gst is the offerer; input datagrams → pad.
crates/orchestrator	axum signalling broker (/ws/producer, /ws/viewer), seat registry (GET /api/seats), serves web/.
web/	plain-JS client: WebTransport session, VideoDecoder onto a canvas, Gamepad API, and an RTCPeerConnection answerer for audio.

Dev quickstart (local loopback, video = test pattern)

# 1. orchestrator (serves web + signalling) on :8080
cargo run -p orchestrator -- --web web

# 2. a seat (test source until pipewire capture is wired)
#    needs access to /dev/uinput (see below)
cargo run -p seat-streamer -- --seat seat-1 --name "Player 1 — test" \
    --signalling ws://127.0.0.1:8080/ws/producer

# 3. open http://localhost:8080 , click the seat, plug in a controller

Test benches

scripts/bench-terraria.sh  # POC: launch Terraria in gamescope-splitux + stream it
scripts/harness.sh         # attach to a game you launched in splitux (1 seat per player)
tests/browser/smoke.mjs    # drives a real browser against a live seat

scripts/smoke.sh and its aiortc peer used to sit at the top of that list. They are gone: the peer counted decoded RTP video frames and drove input over the input datachannel, and the seat now sends neither — so the gate could only ever exit non-zero, which is worse than no gate, because it reads as coverage. Replacing it needs a Python WebTransport peer that also speaks the binary input-state wire format, and aiortc has neither half. The browser harnesses under tests/browser/ cover the same ground by driving a real browser, which exercises the real transport for free.

bench-terraria.sh is the self-contained proof: it launches Terraria inside gamescope-splitux the way splitux does on Wayland (default nested-Wayland backend), reads the gamescope PipeWire node id from its log, and streams that node to a browser seat. Open http://localhost:8080 and click "Terraria". Video only for now — remote gamepad→game needs the seat→instance pad binding.

/dev/uinput access: add a udev rule or run the seat with group input.

KERNEL=="uinput", GROUP="input", MODE="0660", OPTIONS+="static_node=uinput"

Status / next

• gst webrtc stack installed, HW-encode + RTP chain validated headlessly
• seat-streamer (webrtcbin offerer, Vulkan, uinput pad)
• orchestrator (signalling + registry + static)
• browser client (answerer + gamepad)
• real capture: pipewire node of each gamescope surface (--source pipewire --pw-node <id>)
• splitux binding: pre-create pads, assign seat→instance deterministically
• VPS edge: nginx game.gabeforge.com + coturn TURN + signalling over wg0