Files
Johnathon Selstad 51e7a95f19 feat(video): Add 120hz Support (#1452)
* feat(video): experimental 120 Hz low-latency mode

Add an opt-in setting that swaps the JetKVM default EDID for one
advertising 848x480@120 (preferred) and 1280x720@120. At 120 fps the
per-frame display→encode delay drops from ~16.7 ms to ~8.3 ms, halving
source-side video latency vs the standard 1080p60 path.

The TC358743 capture chip on JetKVM v1 has a hard ~120 Hz vrefresh
ceiling (Toshiba spec is 1080p@60; everything above 60 Hz is
undocumented territory). 144/240 Hz were tested and do not lock — the
chip's internal blocks above the TMDS PHY were never validated past
60 Hz. 120 Hz works reliably across both 480p and 720p; that's what
this EDID advertises.

Wiring:
- internal/native/video.go: new LowLatency120HzEDID constant
  (CVT-RB, EDID 1.4, single base block, no CEA extension)
- config.go: VideoLowLatencyMode bool, with reconciliation on load —
  toggling only swaps EdidString when it currently holds one of the
  well-known JetKVM defaults; user-supplied custom EDIDs are preserved
- jsonrpc.go: getVideoLowLatencyMode / setVideoLowLatencyMode RPCs
- UI: experimental Checkbox in Settings → Video and an extra entry
  in the EDID preset dropdown

Source-side note: enabling the toggle does not switch the source PC's
display mode. The user must manually pick 1280x720@120 or 848x480@120
in their OS display settings; the EDID alone just tells the source
what's allowed.

scripts/edid_gen.py is the generator used to produce the EDID hex.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(video): address PR review on 120 Hz toggle

Single source of truth for the toggle is now `EdidString`. The separate
`VideoLowLatencyMode` config bool is gone, along with the load-time
reconciliation logic that could silently revert the EDID dropdown's
choice on reboot.

Addresses:

- cursor[bot] HIGH "Config reconciliation reverts explicit EDID
  dropdown choices on reboot" — drop `VideoLowLatencyMode` field;
  `rpcGetVideoLowLatencyMode` now derives state from `EdidString`,
  `rpcSetVideoLowLatencyMode` only writes `EdidString`. UI toggle is
  derived from `edid` state. Dropdown ↔ toggle can no longer drift.
- Copilot ui/src/routes/.../video.tsx:30 — same root cause; same fix.
- Copilot ui/src/routes/.../video.tsx:199 — drop the spurious
  `.toUpperCase()` on the matched-EDID value so SelectMenuBasic strict
  equality matches the option's actual `value`.
- Copilot jsonrpc.go:276 + config.go:311 — case-insensitive EDID
  comparisons via `strings.EqualFold`.
- Copilot scripts/edid_gen.py:13 — drop unused `import struct`.
- Copilot scripts/edid_gen.py:63 — `pclk_khz` was holding Hz; rename
  to `pclk_hz` and use a `raw_pclk_hz` intermediate for the pre-quantized
  value. Generator output is byte-identical.
- cursor[bot] LOW en.json:1041 "wrong advice when disabling" — split
  the single `video_low_latency_set_success` (which always told users
  to switch to 120 Hz) into `video_low_latency_enabled` and
  `video_low_latency_disabled`; the disabled message tells the user to
  switch their source back to its usual resolution.

Also: small UX cleanup — extracted `applyEDID(...)` helper so the
toggle and the dropdown don't double-fire success notifications.

Verified locally: `go vet` clean, `tsc --noEmit` clean, `oxlint` 0
errors, `python3 -c "import py_compile; py_compile.compile(...)"` OK,
and the regenerated EDID hex matches `LowLatency120HzEDID` byte for
byte.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(video): replace 120 Hz toggle with four single-mode EDIDs in the dropdown

Drops the experimental "Low Latency 120 Hz Mode" toggle and the
LowLatency120HzEDID bundle (which packed both 848x480@120 and 1280x720@120
DTDs into a single EDID) in favor of four standalone single-mode EDIDs
added directly to the existing EDID dropdown:

  - JetKVM 1280x720 @ 120 Hz (low latency)
  - JetKVM 1280x720 @ 60 Hz
  - JetKVM 848x480  @ 120 Hz (low latency)
  - JetKVM 848x480  @ 60 Hz

Each EDID advertises exactly one DTD, the monitor range descriptor, and
the model name — no CEA extension. Generated by scripts/edid_gen.py.

Why the redesign

The toggle was special-casing a single EDID bundle and trying to keep
two pieces of state (the toggle and the EDID dropdown) in agreement.
The reviewer flagged that the load-time reconciliation could revert
explicit EDID-dropdown choices on reboot, and even the simpler
derive-toggle-state-from-EdidString version was carrying:

  - a separate i18n string set for toggle-on / toggle-off,
  - a Checkbox plus an extra dropdown row labeling the same EDID,
  - special-cased applyEDID plumbing distinct from setEDID,
  - a config-load reconciliation path.

Treating the 120 Hz modes as ordinary EDID choices removes all of
that. Picking a 120 Hz EDID writes through setEDID like every other
entry; the dropdown is the only source of truth.

Bundling 480p120 and 720p120 into one EDID also forced the source PC
to choose between two preferred modes. With separate EDIDs the source
sees exactly one preferred timing per choice.

Changes

internal/native/video.go: add EDID720p120, EDID720p60, EDID480p120,
  EDID480p60. Drop LowLatency120HzEDID.

jsonrpc.go: drop rpcGetVideoLowLatencyMode / rpcSetVideoLowLatencyMode
  and their handler registrations. Drop the now-unused native and
  strings imports.

ui/src/routes/devices.\$id.settings.video.tsx: drop the Checkbox, the
  derived lowLatencyMode flag, the handleLowLatencyChange handler, the
  applyEDID helper, and the warning paragraph. Add four new entries to
  the EDID dropdown.

ui/localization/messages/en.json: drop the five video_low_latency_*
  keys and the single-bundle video_edid_jetkvm_120hz key. Add four new
  per-mode keys; update video_edid_jetkvm_default to spell out the
  resolution so the dropdown is internally consistent.

Verified locally: go vet clean on all packages, tsc --noEmit clean,
oxlint 0 errors, EDID hex round-trips byte-for-byte through
scripts/edid_gen.py.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Drop unused EDID constants; fix CVT-RB v1 vertical timing in edid_gen.py

internal/native/video.go: the four EDID720p120 / EDID720p60 / EDID480p120 /
  EDID480p60 constants were never read — the dropdown carries the hex
  inline. Remove them; nothing else in the Go side referenced them.

scripts/edid_gen.py: CVT-RB v1 specifies a fixed 3-line vertical front
  porch and an aspect-dependent vsync; the back porch absorbs the
  remainder of v_blank. The previous implementation had it backwards
  (back porch fixed at RB_V_BACK_PORCH, front porch = remainder), which
  produced a multi-hundred-line front porch and a 6-line back porch —
  technically a valid frame, but not CVT-RB v1. If the recomputed back
  porch falls below the spec minimum, bump v_blank so it does, and let
  v_total follow.

ui/src/routes/devices.\$id.settings.video.tsx: regenerate the three
  affected EDID hex strings (720p120, 720p60, 480p120). 480p60 is
  unchanged because its v_blank already left front porch = 3 in the
  old code path. All four still lock end-to-end on TC358743 hardware.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Set sRGB color space + 640x480@60 established-timing in generated EDIDs

scripts/edid_gen.py:
  - Feature-support byte (offset 24): 0x0A -> 0x0E. Adds the sRGB
    color-space bit while keeping the existing flags (digital, RGB 4:4:4,
    YCbCr 4:2:2, preferred timing in DTD0). EDID 1.4 already requires
    DTD0 to be the preferred timing; tagging sRGB lets the source treat
    the chromaticity block as authoritative instead of guessing.
  - Established-timings byte (offset 35): 0x00 -> 0x20. Bit 5 advertises
    640x480@60 as a VGA-fallback mode some sources fall back to during
    early-boot / BIOS. The source still prefers DTD0 for the active
    desktop, so this is harmless for the 120 Hz / 60 Hz advertised modes.

ui/src/routes/devices.\$id.settings.video.tsx: regenerate all four
  dropdown EDIDs with the new flag bytes and recomputed checksums.
  Strings are now uppercase to match the rest of the dropdown.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(video): plumb source vrefresh into MPP encoder rate control

The MPP H.264/H.265 rate-control structure has Src/Dst FrameRateNum/Den
fields the firmware never set, so MPP defaulted to fps_fix [30/1].
That made the encoder size its bitrate budget for 30 fps and behave
unpredictably when 120 fps arrived from the capture chip.

Now run_detect_format rounds the v4l2 dv-timings vrefresh to an integer
(stored in detected_fps), and run_video_stream passes that value through
venc_start -> populate_venc_attr where it's written into both Src and
Dst FrameRate fields. GOP is sized to fps/2, which keeps the IDR cadence
at ~0.5s for any source refresh — same WebRTC recovery latency at 60 Hz
and 120 Hz.

run_detect_format now also restarts the streaming pipeline when the
rounded fps changes by more than ±1 fps, so an EDID swap that keeps
resolution but changes refresh (e.g. 720p60 -> 720p120) actually
reconfigures the encoder. The ±1 tolerance absorbs CVT-RB rounding
(119.91 fps and 119.87 fps both round to 120).

Verified on device:

  before: fps fix [30/1] -> fix [30/1] gop i [30]
  after:  fps fix [120/1] -> fix [120/1] gop i [60]

WebRTC inbound-rtp framesPerSecond now sustains ~120 with 0 dropped
packets at 720p120, 19 ms jitter buffer, glass-to-glass delay halved
on a 120 Hz panel.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Collapse 4 single-mode EDIDs into one combined 720p entry

Drop the four single-mode 480p/720p × 60/120 EDIDs and replace them
with one multi-mode JetKVM 720p EDID that advertises both 1280x720@120
(DTD0, preferred) and 1280x720@60 (DTD1). Drop 480p entirely — 848x480
is non-standard and the source PC's display panel UI usually doesn't
expose it.

The source picks 60 Hz vs 120 Hz via OS-side display settings
(`xrandr --rate 60/120` on Linux, Display Settings on Windows) without
needing to swap EDIDs. The encoder-fps plumbing from fa36843 already
reconfigures MPP on each v4l2 source-change event, so rate swaps work
end-to-end against this single EDID.

Validated at edidcraft.com: 0 errors / 0 warnings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Fold low-latency 720p modes into the JetKVM default EDID

The JetKVM v1 default EDID had two empty CTA-extension DTD slots after
its existing data blocks (audio, YCbCr 4:2:2, vendor-specific). Use the
first slot to advertise 1280x720@120 alongside the existing 1080p60
(DTD0, preferred) and 1280x720@60 (DTD1) base-block timings.

Source picks rate via OS display settings (`xrandr --rate 120`, Windows
Display Settings, etc.) — no separate "low latency" EDID needed in the
dropdown. The encoder-fps plumbing from fa36843 already reconfigures
MPP on every v4l2 source-change event, so OS-side rate swaps work
end-to-end against this single combined EDID.

Migration: config.LoadConfig now also rewrites EdidString to the new
default when the user is currently on the previous JetKVM v1 EDID
(without the 720p120 DTD), so existing devices auto-pick up the
high-refresh option on next boot.

Validated at edidcraft.com: 0 errors / 0 warnings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Drop scripts/edid_gen.py — exploration tool, not a build dependency

The Python EDID generator was only used during this PR's development
phase to probe the TC358743 chip's max vrefresh and produce candidate
EDIDs. Now that the only EDID change is one DTD added to the JetKVM
default (a fully-validated 18-byte hex string in
internal/native/video.go and ui/src/routes/devices.\$id.settings.video.tsx),
the generator is no longer referenced by any code path. Drop it from
the PR to keep the diff focused on the runtime change. Available in
git history if anyone wants to extend the EDID set later.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Revert "fix(video): plumb source vrefresh into MPP encoder rate control"

This reverts commit fa36843. Empirically the MPP encoder forwards every
input frame whether or not Src/DstFrameRateNum/Den are set in the rate
control struct — those fields appear to only size the bitrate budget,
not gate frame submission. With this code in, dmesg shows
fps fix [120/1] -> fix [120/1] gop i [60]; with it reverted, dmesg
shows fps fix [30/1] -> fix [30/1] gop i [30]; in both cases WebRTC
inbound-rtp framesPerSecond sustains ~120 at the receiver. Drop the
plumbing to keep the PR's surface area minimal — the EDID-side change
alone is what unlocks 120 Hz end-to-end.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Move 720p120 DTD from CTA extension to base block

NVIDIA's display driver enumerates base-block DTDs reliably but ignores
DTDs in the CTA extension that don't carry a CTA-861 VIC. 1280x720@120
isn't a CTA VIC, so the previous layout (720p120 in CTA-extension first
DTD slot) silently dropped the 120 Hz mode on GeForce hosts — `xrandr`
listed 1080p60 / 720p60 / 640p variants only.

Swap base-block DTD1 from 720p60 to 720p120. 720p60 stays advertised
through the Standard Timings block (0x81C0 at offset 40), which every
driver respects. Also drop the now-redundant 720p120 DTD from the CTA
extension and add the prior CTA-only EDID to the migration list so
existing devices auto-upgrade on next boot.

Validated at edidcraft.com: 0 errors / 0 warnings.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* Reapply "fix(video): plumb source vrefresh into MPP encoder rate control"

This reverts commit 3cb61dfc8f.

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: Adam Shiervani <adam.shiervani@gmail.com>
2026-05-15 07:48:44 +02:00

240 lines
6.6 KiB
Go

package native
import (
"fmt"
"os"
"strings"
"time"
)
const sleepModeFile = "/sys/devices/platform/ff470000.i2c/i2c-4/4-000f/sleep_mode"
// DefaultEDID is the default EDID for the video stream.
// CEA-861 extension with HDMI vendor block, audio support, and JetKVM manufacturer ID.
// Base block DTDs: 1920x1080@60 (preferred, DTD0), 1280x720@120 (DTD1). 1280x720@60
// is still advertised via the Standard Timings block (0x81C0 at offset 40). NVIDIA
// drivers ignore non-VIC DTDs in the CTA extension, so the high-refresh DTD has to
// live in the base block to be picked up by GeForce display settings.
const DefaultEDID = "00FFFFFFFFFFFF0028B4010001EEFFC0302301038047287856EE91A3544C99260F5054000000D1C081C0318001010101010101010101023A801871382D40582C4500C48E2100001E773300A050D02B2030203500122C2100001A000000FD00174C0F5111000A202020202020000000FC004A65744B564D2076310A20202001D5020322D1431004012309070783010000E200CFE40D100401E305000065030C001000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000CF"
var extraLockTimeout = 5 * time.Second
// VideoState is the state of the video stream.
type VideoState struct {
Ready bool `json:"ready"`
Streaming VideoStreamingStatus `json:"streaming"`
Error string `json:"error,omitempty"` //no_signal, no_lock, out_of_range
Width int `json:"width"`
Height int `json:"height"`
FramePerSecond float64 `json:"fps"`
}
func isSleepModeSupported() bool {
_, err := os.Stat(sleepModeFile)
return err == nil
}
const sleepModeWaitTimeout = 3 * time.Second
func (n *Native) waitForVideoStreamingStatus(status VideoStreamingStatus) error {
timeout := time.After(sleepModeWaitTimeout)
ticker := time.NewTicker(100 * time.Millisecond)
defer ticker.Stop()
for {
if videoGetStreamingStatus() == status {
return nil
}
select {
case <-timeout:
return fmt.Errorf("timed out waiting for video streaming status to be %s", status.String())
case <-ticker.C:
}
}
}
// before calling this function, make sure to lock n.videoLock
func (n *Native) setSleepMode(enabled bool) error {
if !n.sleepModeSupported {
return nil
}
bEnabled := "0"
shouldWait := false
if enabled {
bEnabled = "1"
switch videoGetStreamingStatus() {
case VideoStreamingStatusActive:
n.l.Info().Msg("stopping video stream to enable sleep mode")
videoStop()
shouldWait = true
case VideoStreamingStatusStopping:
n.l.Info().Msg("video stream is stopping, will enable sleep mode in a few seconds")
shouldWait = true
}
}
if shouldWait {
if err := n.waitForVideoStreamingStatus(VideoStreamingStatusInactive); err != nil {
return err
}
}
return os.WriteFile(sleepModeFile, []byte(bEnabled), 0644)
}
func (n *Native) getSleepMode() (bool, error) {
if !n.sleepModeSupported {
return false, nil
}
data, err := os.ReadFile(sleepModeFile)
if err == nil {
return strings.TrimSpace(string(data)) == "1", nil
}
return false, nil
}
// VideoSetSleepMode sets the sleep mode for the video stream.
func (n *Native) VideoSetSleepMode(enabled bool) error {
n.videoLock.Lock()
defer n.videoLock.Unlock()
return n.setSleepMode(enabled)
}
// VideoGetSleepMode gets the sleep mode for the video stream.
func (n *Native) VideoGetSleepMode() (bool, error) {
n.videoLock.Lock()
defer n.videoLock.Unlock()
return n.getSleepMode()
}
// VideoSleepModeSupported checks if the sleep mode is supported.
func (n *Native) VideoSleepModeSupported() bool {
return n.sleepModeSupported
}
// useExtraLock uses the extra lock to execute a function.
// if the lock is currently held by another goroutine, returns an error.
//
// it's used to ensure that only one change is made to the video stream at a time.
// as the change usually requires to restart video streaming
// TODO: check video streaming status instead of using a hardcoded timeout
func (n *Native) useExtraLock(fn func() error) error {
if !n.extraLock.TryLock() {
return fmt.Errorf("the previous change hasn't been completed yet")
}
err := fn()
if err == nil {
time.Sleep(extraLockTimeout)
}
n.extraLock.Unlock()
return err
}
// VideoSetQualityFactor sets the quality factor for the video stream.
func (n *Native) VideoSetQualityFactor(factor float64) error {
n.videoLock.Lock()
defer n.videoLock.Unlock()
return n.useExtraLock(func() error {
return videoSetStreamQualityFactor(factor)
})
}
// VideoGetQualityFactor gets the quality factor for the video stream.
func (n *Native) VideoGetQualityFactor() (float64, error) {
n.videoLock.Lock()
defer n.videoLock.Unlock()
return videoGetStreamQualityFactor()
}
// VideoSetCodecType must be called before VideoStart(), not mid-stream.
func (n *Native) VideoSetCodecType(codecType int) error {
n.videoLock.Lock()
defer n.videoLock.Unlock()
return videoSetCodecType(codecType)
}
func (n *Native) VideoGetCodecType() (int, error) {
n.videoLock.Lock()
defer n.videoLock.Unlock()
return videoGetCodecType()
}
// VideoSetEDID sets the EDID for the video stream.
func (n *Native) VideoSetEDID(edid string) error {
n.videoLock.Lock()
defer n.videoLock.Unlock()
if edid == "" {
edid = DefaultEDID
}
return n.useExtraLock(func() error {
return videoSetEDID(edid)
})
}
// VideoGetEDID gets the EDID for the video stream.
func (n *Native) VideoGetEDID() (string, error) {
n.videoLock.Lock()
defer n.videoLock.Unlock()
return videoGetEDID()
}
// VideoLogStatus gets the log status for the video stream.
func (n *Native) VideoLogStatus() (string, error) {
n.videoLock.Lock()
defer n.videoLock.Unlock()
return videoLogStatus(), nil
}
// VideoStop stops the video stream.
func (n *Native) VideoStop() error {
n.videoLock.Lock()
defer n.videoLock.Unlock()
videoStop()
return nil
}
// VideoStart starts the video stream.
func (n *Native) VideoStart() error {
n.videoLock.Lock()
defer n.videoLock.Unlock()
// check if the chip is currently in sleep mode
wasSleeping, _ := n.getSleepMode()
// disable sleep mode before starting video
_ = n.setSleepMode(false)
// when waking from sleep, the capture chip needs time to re-lock the HDMI
// signal before we can start streaming (similar to the delay in useExtraLock)
if wasSleeping {
n.l.Info().Msg("capture chip was sleeping, waiting for signal re-lock")
time.Sleep(extraLockTimeout)
}
videoStart()
return nil
}
// VideoGetStreamingStatus gets the streaming status of the video.
func (n *Native) VideoGetStreamingStatus() VideoStreamingStatus {
n.videoLock.Lock()
defer n.videoLock.Unlock()
return videoGetStreamingStatus()
}