Skip to content

Commit

Permalink
feat: implement isRecognitionAvailable() (#28)
Browse files Browse the repository at this point in the history
  • Loading branch information
jamsch authored Sep 30, 2024
1 parent 9f32539 commit 4db9769
Show file tree
Hide file tree
Showing 10 changed files with 77 additions and 28 deletions.
42 changes: 29 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -38,8 +38,9 @@ expo-speech-recognition implements the iOS [`SFSpeechRecognizer`](https://develo
- [getSpeechRecognitionServices()](#getspeechrecognitionservices-string-android-only)
- [getDefaultRecognitionService()](#getdefaultrecognitionservice--packagename-string--android-only)
- [getAssistantService()](#getassistantservice--packagename-string--android-only)
- [isRecognitionAvailable()](#isrecognitionavailable-boolean)
- [supportsOnDeviceRecognition()](#supportsondevicerecognition-boolean)
- [supportsRecording()](#supportsrecording-boolean-android-only)
- [supportsRecording()](#supportsrecording-boolean)
- [androidTriggerOfflineModelDownload()](#androidtriggerofflinemodeldownload-locale-string--promise-status-opened_dialog--download_success--download_canceled-message-string-)
- [setCategoryIOS()](#setcategoryios-void-ios-only)
- [getAudioSessionCategoryAndOptionsIOS()](#getaudiosessioncategoryandoptionsios-ios-only)
Expand Down Expand Up @@ -307,7 +308,7 @@ Events are largely based on the [Web Speech API](https://developer.mozilla.org/e
| ------------- | ------------------------------------------------------------------------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `audiostart` | Audio capturing has started | Includes the `uri` if `recordingOptions.persist` is enabled. |
| `audioend` | Audio capturing has ended | Includes the `uri` if `recordingOptions.persist` is enabled. |
| `end` | Speech recognition service has disconnected. | This should be the last event dispatched. |
| `end` | Speech recognition service has disconnected. | This should always be the last event dispatched, including after errors. |
| `error` | Fired when a speech recognition error occurs. | You'll also receive an `error` event (with code "aborted") when calling `.abort()` |
| `nomatch` | Speech recognition service returns a final result with no significant recognition. | You may have non-final results recognized. This may get emitted after cancellation. |
| `result` | Speech recognition service returns a word or phrase has been positively recognized. | On Android, continous mode runs as a segmented session, meaning when a final result is reached, additional partial and final results will cover a new segment separate from the previous final result. On iOS, you should expect one final result before speech recognition has stopped. |
Expand Down Expand Up @@ -361,7 +362,7 @@ The error code is based on the [Web Speech API error codes](https://developer.mo
If you would like to persist the recognized audio for later use, you can enable the `recordingOptions.persist` option when calling `start()`. Enabling this setting will emit an `{ uri: string }` event object in the `audiostart` and `audioend` events with the local file path.

> [!IMPORTANT]
> This feature is available on Android 13+ and iOS. Call `supportsRecording()` to see if it's available before using this feature.
> This feature is available on Android 13+ and iOS. Call [`supportsRecording()`](#supportsrecording-boolean) to see if it's available before using this feature.
Default audio output formats:

Expand Down Expand Up @@ -838,9 +839,9 @@ Get list of speech recognition services available on the device.
> This only includes services that are listed under `androidSpeechServicePackages` in your app.json as well as the core services listed under `forceQueryable` when running the command: `adb shell dumpsys package queries`
```ts
import { ExpoSpeechRecognitionModule } from "expo-speech-recognition";
import { getSpeechRecognitionServices } from "expo-speech-recognition";

const packages = ExpoSpeechRecognitionModule.getSpeechRecognitionServices();
const packages = getSpeechRecognitionServices();
console.log("Speech recognition services:", packages.join(", "));
// e.g. ["com.google.android.as", "com.google.android.tts", "com.samsung.android.bixby.agent"]
```
Expand All @@ -850,9 +851,9 @@ console.log("Speech recognition services:", packages.join(", "));
Returns the default voice recognition service on the device.

```ts
import { ExpoSpeechRecognitionModule } from "expo-speech-recognition";
import { getDefaultRecognitionService } from "expo-speech-recognition";

const service = ExpoSpeechRecognitionModule.getDefaultRecognitionService();
const service = getDefaultRecognitionService();
console.log("Default recognition service:", service.packageName);
// Usually this is "com.google.android.tts" on Android 13+ and "com.google.android.googlequicksearchbox" on Android <=12.
// For on-device recognition, "com.google.android.as" will likely be used.
Expand All @@ -863,17 +864,32 @@ console.log("Default recognition service:", service.packageName);
Returns the default voice assistant service on the device.

```ts
import { ExpoSpeechRecognitionModule } from "expo-speech-recognition";
import { getAssistantService } from "expo-speech-recognition";

const service = ExpoSpeechRecognitionModule.getAssistantService();
const service = getAssistantService();
console.log("Default assistant service:", service.packageName);
// Usually "com.google.android.googlequicksearchbox" for Google
// or "com.samsung.android.bixby.agent" for Samsung
```

### `isRecognitionAvailable(): boolean`

Whether speech recognition is currently available on the device.

If this method returns false, calling `start()` will fail and emit an error event with the code `service-not-allowed` or `language-not-supported`. You should also ask the user to enable speech recognition in the system settings (i.e, for iOS to enable Siri & Dictation). On Android, you should ask the user to install and enable `com.google.android.tts` (Android 13+) or `com.google.android.googlequicksearchbox` (Android <= 12) as a default voice recognition service.

For Web, this method only checks if the browser has the Web SpeechRecognition API available, however keep in mind that browsers (like Brave) may still have the APIs but not have it implemented yet. Refer to [Platform Compatibility Table](#platform-compatibility-table) for more information. You may want to use a user agent parser to fill in the gaps.

```ts
import { isRecognitionAvailable } from "expo-speech-recognition";

const available = isRecognitionAvailable();
console.log("Speech recognition available:", available);
```

### `supportsOnDeviceRecognition(): boolean`

Whether on-device speech recognition is available on the device.
Whether the device supports on-device speech recognition.

```ts
import { supportsOnDeviceRecognition } from "expo-speech-recognition";
Expand All @@ -882,7 +898,7 @@ const available = supportsOnDeviceRecognition();
console.log("OnDevice recognition available:", available);
```

### `supportsRecording(): boolean` (Android only)
### `supportsRecording(): boolean`

Whether audio recording is supported during speech recognition. This mostly applies to Android devices, to check if it's at least Android 13.

Expand All @@ -902,10 +918,10 @@ You can see which locales are supported and installed on your device by running
To download the offline model for a specific locale, use the `androidTriggerOfflineModelDownload` function.

```ts
import { ExpoSpeechRecognitionModule } from "expo-speech-recognition";
import { androidTriggerOfflineModelDownload } from "expo-speech-recognition";

// Download the offline model for the specified locale
ExpoSpeechRecognitionModule.androidTriggerOfflineModelDownload({
androidTriggerOfflineModelDownload({
locale: "en-US",
})
.then((result) => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -183,6 +183,10 @@ class ExpoSpeechRecognitionModule : Module() {
}
}

Function("isRecognitionAvailable") {
SpeechRecognizer.isRecognitionAvailable(appContext.reactContext!!)
}

Function("supportsRecording") {
Build.VERSION.SDK_INT >= Build.VERSION_CODES.TIRAMISU
}
Expand Down
9 changes: 9 additions & 0 deletions example/App.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -786,6 +786,15 @@ function OtherSettings(props: {
});
}}
/>
<BigButton
title="Call isRecognitionAvailable()"
color="#7C90DB"
onPress={() => {
const isAvailable =
ExpoSpeechRecognitionModule.isRecognitionAvailable();
Alert.alert("isRecognitionAvailable()", isAvailable.toString());
}}
/>
{Platform.OS === "ios" && (
<BigButton
title="Set audio session active state"
Expand Down
6 changes: 5 additions & 1 deletion ios/ExpoSpeechRecognitionModule.swift
Original file line number Diff line number Diff line change
Expand Up @@ -279,7 +279,11 @@ public class ExpoSpeechRecognitionModule: Module {
}

Function("supportsRecording") { () -> Bool in
let recognizer: SFSpeechRecognizer? = SFSpeechRecognizer()
return true
}

Function("isRecognitionAvailable") { () -> Bool in
let recognizer = SFSpeechRecognizer()
return recognizer?.isAvailable ?? false
}

Expand Down
1 change: 0 additions & 1 deletion ios/ExpoSpeechRecognizer.swift
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ enum RecognizerError: Error {
}
}

/// A helper for transcribing speech to text using SFSpeechRecognizer and AVAudioEngine.
actor ExpoSpeechRecognizer: ObservableObject {

private var options: SpeechRecognitionOptions?
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "expo-speech-recognition",
"version": "0.2.20",
"version": "0.2.21",
"description": "Speech Recognition for React Native Expo projects",
"main": "build/index.js",
"types": "build/index.d.ts",
Expand Down
2 changes: 2 additions & 0 deletions src/ExpoSpeechRecognitionModule.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ export const ExpoSpeechRecognitionModule: ExpoSpeechRecognitionModuleType = {
ExpoSpeechRecognitionNativeModule.supportsOnDeviceRecognition(),
supportsRecording: () =>
ExpoSpeechRecognitionNativeModule.supportsRecording(),
isRecognitionAvailable: () =>
ExpoSpeechRecognitionNativeModule.isRecognitionAvailable(),
};

export const ExpoSpeechRecognitionModuleEmitter = new EventEmitter(
Expand Down
7 changes: 7 additions & 0 deletions src/ExpoSpeechRecognitionModule.types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -570,6 +570,13 @@ export interface ExpoSpeechRecognitionModuleType extends NativeModule {
* This mostly applies to Android devices, to check if it's greater than Android 13.
*/
supportsRecording(): boolean;
/**
* Whether on-device speech recognition is available.
*
* If this method returns false, `start()` will fail and emit an error event with the code `service-not-allowed` or `language-not-supported`.
*/
isRecognitionAvailable(): boolean;

/**
* Downloads the offline model for the specified locale.
* Note: this is only supported on Android 13 and above.
Expand Down
20 changes: 8 additions & 12 deletions src/ExpoSpeechRecognitionModule.web.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,31 +82,19 @@ export const ExpoSpeechRecognitionModule: ExpoSpeechRecognitionModuleType = {
);
},
getSpeechRecognitionServices: () => {
console.warn(
"getSpeechRecognitionServices is not supported on web. Returning an empty array.",
);
return [] as string[];
},
getDefaultRecognitionService: () => {
console.warn(
"getDefaultRecognitionService is not supported on web. Returning an empty object.",
);
return {
packageName: "",
};
},
getAssistantService: () => {
console.warn(
"getAssistantService is not supported on web. Returning an empty object.",
);
return {
packageName: "",
};
},
supportsOnDeviceRecognition: () => {
console.warn(
"supportsOnDeviceRecognition is not supported on web. Returning false.",
);
return false;
},
supportsRecording: () => {
Expand Down Expand Up @@ -137,6 +125,13 @@ export const ExpoSpeechRecognitionModule: ExpoSpeechRecognitionModuleType = {
setAudioSessionActiveIOS: () => {
console.warn("setAudioSessionActiveIOS is not supported on web.");
},
isRecognitionAvailable: () => {
const hasSpeechRecognitionAPI =
typeof webkitSpeechRecognition !== "undefined" ||
typeof SpeechRecognition !== "undefined";

return hasSpeechRecognitionAPI;
},
};

/**
Expand Down Expand Up @@ -176,6 +171,7 @@ const webToNativeEventMap: {
start: (ev) => null,
soundend: (ev) => null,
};

export const ExpoSpeechRecognitionModuleEmitter = {
_nativeListeners: new Map() as Map<string, Set<(event: any) => void>>,
_clientListeners: new Map() as Map<
Expand Down
12 changes: 12 additions & 0 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,18 @@ export const getAudioSessionCategoryAndOptionsIOS =
export const setAudioSessionActiveIOS =
ExpoSpeechRecognitionModule.setAudioSessionActiveIOS;

export const androidTriggerOfflineModelDownload =
ExpoSpeechRecognitionModule.androidTriggerOfflineModelDownload;

export const isRecognitionAvailable =
ExpoSpeechRecognitionModule.isRecognitionAvailable;

export const getDefaultRecognitionService =
ExpoSpeechRecognitionModule.getDefaultRecognitionService;

export const getAssistantService =
ExpoSpeechRecognitionModule.getAssistantService;

export const addSpeechRecognitionListener = <
T extends keyof ExpoSpeechRecognitionNativeEventMap,
>(
Expand Down

0 comments on commit 4db9769

Please sign in to comment.