Remote control in the AI era: from pixels to intent

Classic remote control moves pixels one way and keystrokes the other. It assumes a human on each end of the wire, and its quality metrics are latency and frame rate. That model is about to look as quaint as a terminal emulator.

The first shift is already visible: the "user" at the near end is increasingly an agent. An AI that operates a machine does not need a beautiful 4K stream — it needs a semantic one: UI trees, window states, intentions and outcomes. Expect protocols to split into two planes, a thin semantic channel for agents and a rich visual channel that humans open only when they want to look.

The second shift is supervision. When an agent drives, the human’s job moves from doing to approving: consent checkpoints, scoped sessions ("you may work in this project folder until 18:00"), and replayable action logs that read like a flight recorder. Trust becomes a feature you can inspect, not a promise.

The third shift is locality. The cheapest frame is the one never sent: more work will render and even decide on the far side, with the near side receiving summaries and exceptions. Remote control stops being a window and becomes a delegation contract.

None of this kills the old craft — a dropped connection at the wrong moment still ruins a day, and someone still has to move real pixels for real games and real design tools. But the centre of gravity moves from seeing the screen to meaning the action. The tools that win will be the ones that make delegation legible.