math-pixel/La-Fabrik
1
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fix(handtracking): bump browser camera to 640x480 for detection

Browse Source 
The browser MediaPipe model (hand_landmarker.task float16) was failing
to detect hands at 320x240 — MediaPipe ran for 10s straight without
ever returning a hand. Bumping the requested camera resolution to
640x480 (browser only — backend still ships 320x240 JPEGs over the
WebSocket) makes the model reliably pick hands up.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tom Boullay
2026-06-02 17:39:43 +02:00
parent d9cacdad12
commit 73c6d7d50d
2 changed files with 10 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/data/handTrackingConfig.ts
+6
View File
@@ -1,5 +1,11 @@
export const HAND_TRACKING_FRAME_WIDTH = 320; export const HAND_TRACKING_FRAME_WIDTH = 320;
export const HAND_TRACKING_FRAME_HEIGHT = 240; export const HAND_TRACKING_FRAME_HEIGHT = 240;
// The browser MediaPipe model (hand_landmarker.task float16) is more
// sensitive than the backend Python model and needs a higher-resolution
// frame to detect hands reliably. The backend keeps 320x240 because that
// is the JPEG payload size sent over the WebSocket.
export const HAND_TRACKING_BROWSER_CAMERA_WIDTH = 640;
export const HAND_TRACKING_BROWSER_CAMERA_HEIGHT = 480;
export const HAND_TRACKING_TARGET_FPS = 10; export const HAND_TRACKING_TARGET_FPS = 10;
export const HAND_TRACKING_JPEG_QUALITY = 0.55; export const HAND_TRACKING_JPEG_QUALITY = 0.55;
export const HAND_TRACKING_CAMERA_TIMEOUT_MS = 8_000; export const HAND_TRACKING_CAMERA_TIMEOUT_MS = 8_000;
src/hooks/handTracking/useBrowserHandTracking.ts
+4 -4
View File
@@ -1,7 +1,7 @@
import { useEffect, useRef, useState } from "react"; import { useEffect, useRef, useState } from "react";
import { import {
HAND_TRACKING_FRAME_HEIGHT, HAND_TRACKING_BROWSER_CAMERA_HEIGHT,
HAND_TRACKING_FRAME_WIDTH, HAND_TRACKING_BROWSER_CAMERA_WIDTH,
HAND_TRACKING_RUNTIME_START_DELAY_MS, HAND_TRACKING_RUNTIME_START_DELAY_MS,
HAND_TRACKING_TARGET_FPS, HAND_TRACKING_TARGET_FPS,
} from "@/data/handTrackingConfig"; } from "@/data/handTrackingConfig";
@@ -66,8 +66,8 @@ export function useBrowserHandTracking({
try { try {
const stream = await getCameraStreamWithTimeout({ const stream = await getCameraStreamWithTimeout({
video: { video: {
width: HAND_TRACKING_FRAME_WIDTH, width: HAND_TRACKING_BROWSER_CAMERA_WIDTH,
height: HAND_TRACKING_FRAME_HEIGHT, height: HAND_TRACKING_BROWSER_CAMERA_HEIGHT,
facingMode: "user", facingMode: "user",
}, },
audio: false, audio: false,