You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+21-2Lines changed: 21 additions & 2 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -38,7 +38,7 @@ flutter run
38
38
```
39
39
40
40
## AI Model
41
-
A Google AI gesture recognition model was used for ASL image recognition. This model was trained on a 2.7GB curated dataset compiled from various Kaggle repositories.
41
+
A Google AI gesture recognition model was used for American Sign Language (ASL) image recognition. This model was trained on a 2.4GB dataset compiled from various Kaggle repositories.
42
42
The model achieves an accuracy of 93.6% and a loss of 0.1563.
43
43
In practice, the lighting of the environment significantly affects the accuracy of classification because the training data primarily consists of images with good to excellent lighting conditions.
44
44
Other Hugging Face alternatives tested were slower and less accurate overall, based on our evaluations. Additionally, the training data predominantly features right-handed photos, so ASL letter recognition is expected to perform better for right-handed gestures.
@@ -49,6 +49,9 @@ A Mistral alternative was tested but found to be less effective. The OpenAI solu
49
49
50
50
## Frontend
51
51
52
+
- the `sign_language_translator/` directory
53
+
54
+
### Structure
52
55
```
53
56
lib/
54
57
├── main.dart
@@ -57,6 +60,22 @@ lib/
57
60
│ ├── camera.dart
58
61
│ └── instructions_pop_up.dart
59
62
└── services/
60
-
└── network.dart
63
+
└── network.dart # handles API requests for image upload and translation retrieval.
61
64
```
62
65
66
+
### Components
67
+
The app consists of one page with the following components:
68
+
- camera, used to capture the signs that will be translated
69
+
- button to change the camera in use (rear/frontal camera)
70
+
- button to start/stop the translation
71
+
- button that opens a pop-up with the usage instructions for the app
72
+
- a text component, where the translation will be displayed
73
+
74
+
75
+
### Translation Workflow
76
+
1. Open the app.
77
+
2. Tap **Start Translation** to begin.
78
+
3. The app captures images periodically and sends them to the backend for processing.
79
+
4. Tap **Stop Translation** to end the process.
80
+
5. View the translation result displayed on the screen.
0 commit comments