Signify-ISL

Videos

Tech Stack

Python

TensorFlow

JavaScript

HTML5

Flask

NumPy

MediaPipe

PyTorch

Description

Project Title: Signify – Bridging Silence with AI

Overview:

Signify is a revolutionary AI-powered communication platform that translates Indian Sign Language (ISL) into real-time text and speech, empowering the deaf and hard-of-hearing community to interact seamlessly with the world around them. Whether it’s education, everyday conversation, or professional communication, Signify brings inclusive interaction to the forefront.

What Makes Signify Unique?

1. Real-Time ISL to Text & Speech Translation:
Leverages powerful 2D CNN-based deep learning models to interpret sign gestures via webcam and convert them into instantly understandable text and synthesized speech.

2. AI-Powered Video Calling with ISL Support – Built for Education
This is a feature that enables real-time ISL gesture translation during video calls, with live text transcription and multi-user call sharing.
What sets Signify apart is its educational focus: teachers can conduct live virtual classes for deaf students, using sign language or voice, while the platform automatically translates it into text and speech, making inclusive digital learning a reality.

3. Interactive ISL Learning Hub:
Includes an integrated ISL learning module with animated tutorials, practice tests, and progress tracking – making Signify not just a tool but an educational companion.

4. Multilingual Support:
Ensures translations are available in multiple Indian languages, making the platform accessible across diverse linguistic regions.

5. Fallback Mechanism – Robustness Beyond AI

To ensure consistent performance even when the model encounters difficulty (due to poor lighting, occlusion, or background noise), Signify integrates a fallback logic using hand landmark tracking:

If the CNN model fails to classify a gesture, the system captures hand keypoints using MediaPipe.

Based on the relative positions of landmarks (like fingertip distances, angles, and hand orientation), the system infers the likely sign.

This fallback ensures that users still receive accurate text and speech translations even under challenging conditions, increasing reliability and usability.

Tech Stack & Intelligence

Frontend: HTML, CSS, JavaScript
Backend & APIs: Flask, TensorFlow
Model: Custom-trained 2D CNN for gesture classification
Real-time Processing: OpenCV, MediaPipe
Deployment: Vercel + GitHub Integration

"Over 63 million Indians face hearing impairments. Signify gives them a voice—literally."
By enabling barrier-free, real-time communication, Signify contributes to inclusive education, accessibility in workplaces, and a more empathetic society.

Here is the link of our website : https://signify-isl.vercel.app/

Progress During Hackathon

During the hackathon, we made significant strides in building Signify , an inclusive AI-driven platform that translates Indian Sign Language (ISL) into real-time text and speech. We successfully developed and deployed our custom-trained 2D CNN gesture recognition model using OpenCV and MediaPipe, built a multilingual text-to-speech system, and designed a clean, user-friendly interface. We also implemented an interactive ISL learning module to promote self-paced learning. While the video calling feature is in progress, we’ve architected the UI and backend logic to support a future-ready virtual classroom experience — enabling teachers to conduct real-time classes for deaf students. The current version showcases the core innovation live, and we’re confident it’s a strong foundation for transforming communication and education accessibility at scale.

Fundraising Status

We are actively seeking funding to take Signify from a powerful prototype to a scalable, production-ready platform. Initial funds will be used to enhance the video calling feature with full ISL gesture translation, host larger, more accurate deep learning models, and expand support to regional Indian languages. With over 63 million hearing-impaired individuals in India, Signify has the potential to revolutionize accessible education and communication. Funding will also support mobile app development, cloud-based training for high-accuracy models, and collaborations with schools and NGOs to pilot Signify in real-world classrooms. We're committed to making inclusive communication a reality — and with the right support, Signify can become a nationwide educational bridge for the deaf community.