Gesture-Controlled Music Player

Gesture-Controlled Music Player

A browser prototype that controls a music player with hand gestures, using webcam hand tracking.

A browser prototype that controls a music player with hand gestures, using webcam hand tracking.

Vibe coding

Gesture Interaction

AI-Assisted Prototyping

Timeline

May 2026

May 2026

Tools

Figma

MediaPipe Hands
Claude (AI-assisted coding)
YouTube IFrame API
BroadcastChannel API

Figma

MediaPipe Hands
Claude (AI-assisted coding)
YouTube IFrame API
BroadcastChannel API

Explorations & Process

When I'm cooking or working out, my hands are too busy or messy to touch a screen.

I wondered: could I play the exact song I want without touching anything at all?

So I built a browser prototype that reads hand gestures through a webcam. An open palm plays or pauses, moving your hand left or right skips tracks, and pointing opens an album carousel where you swipe with two fingers and make a fist to pick a song.

When I'm cooking or working out, my hands are too busy or messy to touch a screen.

I wondered: could I play the exact song I want without touching anything at all?

So I built a browser prototype that reads hand gestures through a webcam. An open palm plays or pauses, moving your hand left or right skips tracks, and pointing opens an album carousel where you swipe with two fingers and make a fist to pick a song.

Layout

TOP

Hand Music Player

Hand Music Player

Hand Music Player

BOTTOM

Gesture CAM

Gesture CAM

Gesture CAM

Gesture Camera

SCREEN ZONES
LeftPrev Track
CenterPlay / Pause Select Track
RightNext Track
GESTURES
  • Open Palm — Hold StillCenter → Play/Pause · Left → Prev · Right → Next
  • Index Finger — HoldCenter → Enter Select Mode (album carousel)
  • Fist — HoldInside Select Mode → Confirm song selection
  • Two Fingers — Swipe Left / RightSpin Carousel

Hand Music Player

MUSIC PLAYER

Can only move to the previous or next song.
Linear navigation only.

GESTURES
  • Open Palm — Hold StillCenter → Play/Pause · Left → Prev · Right → Next
  • Index Finger — HoldCenter → Enter Select Mode (album carousel)
SELECT TRACK ZONE

Jump to any song freely.

GESTURES
  • Two Fingers — Swipe Left / RightSpin Carousel
  • Index Finger — HoldSelect Song
  • Fist — HoldConfirm song selection

Gesture Camera

SCREEN ZONES
LeftPrev Track
CenterPlay / Pause Select Track
RightNext Track
GESTURES
  • Open Palm — Hold StillCenter → Play/Pause · Left → Prev · Right → Next
  • Index Finger — HoldCenter → Enter Select Mode (album carousel)
  • Fist — HoldInside Select Mode → Confirm song selection
  • Two Fingers — Swipe Left / RightSpin Carousel

Hand Music Player

MUSIC PLAYER

Can only move to the previous or next song.
Linear navigation only.

GESTURES
  • Open Palm — Hold StillCenter → Play/Pause · Left → Prev · Right → Next
  • Index Finger — HoldCenter → Enter Select Mode (album carousel)
SELECT TRACK ZONE

Jump to any song freely.

GESTURES
  • Two Fingers — Swipe Left / RightSpin Carousel
  • Index Finger — HoldSelect Song
  • Fist — HoldConfirm song selection

Play

THANK YOU
FOR VIEWING

THANK YOU
FOR VIEWING

THANK YOU
FOR VIEWING