Use Case
Ishara is designed to make Indonesian Sign Language (BISINDO) more accessible to the general public through an engaging, gamified learning experience. With XP points, levels, and daily challenges, users are motivated to keep practicing until they can confidently recognize and produce hand gestures.
Key Features
- Real-time gesture detection — a TensorFlow Lite model runs entirely on-device, recognizing hand shapes via the camera with no internet connection required
- Progressive learning modules — content is organized from basic letters and numbers up through simple words and full sentences
- Gamification system — XP points, daily streaks, achievement badges, and a leaderboard to sustain long-term motivation
- Instant feedback — after each detected gesture, users receive immediate right/wrong confirmation along with a correction animation
- Practice & quiz modes — a free exploration mode for open practice and a structured quiz mode for skill evaluation
Technical Challenges
The gesture classification model was trained on a manually curated BISINDO dataset, then converted to .tflite format and optimized with INT8 quantization to achieve low inference latency (<50ms) on mid-range Android devices. MediaPipe handles hand landmark extraction before data is passed to the classifier.