ASL Fingerspelling: Tutor + Video Translator
In progress
Recognises American Sign Language fingerspelling from a webcam or any video, on a laptop CPU with no GPU. A live tutor watches you sign and says exactly what to fix; a translator turns fingerspelling video into subtitles, now with a small sequence model trained on 87 real-world signers. Tested openly on people it has never seen, with every result published, good or bad.
67%letter accuracy on 36 real-world signers it never saw (ChicagoFSWild)
25% → 67%per-frame system vs. sequence model on those signers
1 → 12 / 26letters on a stranger's video after one root-cause fix (the video)
MediaPipeOpenCVNumPyPyTorch (training)CTCPython
Try it in your browser uses your webcam; runs on your device
Test video: Travis Dougherty, ASL demo, by Fuzheado, CC BY-SA 3.0, via Wikimedia Commons. Demo video CC BY-SA 3.0.