forked from boocmp/go-stt
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Cuda 12.1.1, pipe name randomization.
- Loading branch information
Showing
11 changed files
with
79 additions
and
184 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,22 +1,3 @@ | ||
# stt:git | ||
|
||
[![pypi_status](https://img.shields.io/badge/BentoML-1.1.10-informational)](https://pypi.org/project/BentoML) | ||
[![documentation_status](https://readthedocs.org/projects/bentoml/badge/?version=latest)](https://docs.bentoml.com/) | ||
[![join_slack](https://badgen.net/badge/Join/BentoML%20Slack/cyan?icon=slack)](https://l.bentoml.com/join-slack-swagger) | ||
[![BentoML GitHub Repo](https://img.shields.io/github/stars/bentoml/bentoml?style=social)](https://github.com/bentoml/BentoML) | ||
[![Twitter Follow](https://img.shields.io/twitter/follow/bentomlai?label=Follow%20BentoML&style=social)](https://twitter.com/bentomlai) | ||
|
||
This is a Machine Learning Service created with BentoML. | ||
| InferenceAPI | Input | Output | | ||
| ------------ | ----- | ------ | | ||
| POST [`/process_audio`](#operations-Service_APIs-stt__process_audio) | BytesIOFile | JSON | | ||
|
||
|
||
|
||
|
||
## Help | ||
|
||
* [📖 Documentation](https://docs.bentoml.com/en/latest/): Learn how to use BentoML. | ||
* [💬 Community](https://l.bentoml.com/join-slack-swagger): Join the BentoML Slack community. | ||
* [🐛 GitHub Issues](https://github.com/bentoml/BentoML/issues): Report bugs and feature requests. | ||
* Tip: you can also [customize this README](https://docs.bentoml.com/en/latest/concepts/bento.html#description). |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -46,31 +46,11 @@ info: | |
contact: | ||
email: [email protected] | ||
name: BentoML Team | ||
description: "# stt:None\n\n[![pypi_status](https://img.shields.io/badge/BentoML-1.1.10-informational)](https://pypi.org/project/BentoML)\n\ | ||
[![documentation_status](https://readthedocs.org/projects/bentoml/badge/?version=latest)](https://docs.bentoml.com/)\n\ | ||
[![join_slack](https://badgen.net/badge/Join/BentoML%20Slack/cyan?icon=slack)](https://l.bentoml.com/join-slack-swagger)\n\ | ||
[![BentoML GitHub Repo](https://img.shields.io/github/stars/bentoml/bentoml?style=social)](https://github.com/bentoml/BentoML)\n\ | ||
[![Twitter Follow](https://img.shields.io/twitter/follow/bentomlai?label=Follow%20BentoML&style=social)](https://twitter.com/bentomlai)\n\ | ||
\nThis is a Machine Learning Service created with BentoML.\n| InferenceAPI | Input\ | ||
\ | Output |\n| ------------ | ----- | ------ |\n| POST [`/process_audio`](#operations-Service_APIs-stt__process_audio)\ | ||
\ | BytesIOFile | JSON |\n\n\n\n\n## Help\n\n* [\U0001F4D6 Documentation](https://docs.bentoml.com/en/latest/):\ | ||
\ Learn how to use BentoML.\n* [\U0001F4AC Community](https://l.bentoml.com/join-slack-swagger):\ | ||
\ Join the BentoML Slack community.\n* [\U0001F41B GitHub Issues](https://github.com/bentoml/BentoML/issues):\ | ||
\ Report bugs and feature requests.\n* Tip: you can also [customize this README](https://docs.bentoml.com/en/latest/concepts/bento.html#description).\n" | ||
description: "# stt:None\n\n[![pypi_status](https://img.shields.io/badge/BentoML-1.1.10-informational)](https://pypi.org/project/BentoML)\n | ||
title: stt | ||
version: None | ||
openapi: 3.0.2 | ||
paths: | ||
/healthz: | ||
get: | ||
description: Health check endpoint. Expecting an empty response with status | ||
code <code>200</code> when the service is in health state. The <code>/healthz</code> | ||
endpoint is <b>deprecated</b>. (since Kubernetes v1.16) | ||
responses: | ||
'200': | ||
description: Successful Response | ||
tags: | ||
- Infrastructure | ||
/livez: | ||
get: | ||
description: Health check endpoint for Kubernetes. Healthy endpoint responses | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,5 +1,5 @@ | ||
ctranslate2==3.24.0 | ||
faster_whisper==0.10.1 | ||
ctranslate2 | ||
faster_whisper | ||
fastapi | ||
aiofiles | ||
asyncio |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1 @@ | ||
from os import name as _os_name | ||
|
||
if _os_name == "nt": | ||
from ._win32 import AsyncNamedPipe | ||
else: | ||
from ._posix import AsyncNamedPipe | ||
from ._posix import AsyncChannelWriter, AsyncChannelReader |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,71 +1,92 @@ | ||
import os | ||
import tempfile | ||
import asyncio | ||
import aiofiles | ||
import aiofiles.os | ||
from contextlib import contextmanager | ||
from pathlib import Path | ||
|
||
class AsyncNamedPipe: | ||
def __init__(self, pipe, path, creator = False): | ||
self._pipe = pipe | ||
self._path = path | ||
self._creator = creator | ||
|
||
async def create(path: str, timeout: int = 10000): | ||
class AsyncChannelWriter: | ||
def __init__(self, fd, dir: Path): | ||
self._fd = fd | ||
self._dir = dir | ||
|
||
async def open(pair: str, timeout: int = 10000): | ||
loop = asyncio.get_running_loop() | ||
path = f"/tmp/{path}" | ||
try: | ||
await loop.run_in_executor(None, os.mkfifo, path) | ||
except: | ||
pass | ||
|
||
|
||
try: | ||
dir = await loop.run_in_executor(None, tempfile.mkdtemp, None, pair + "-", Path.home() / "tmp" / "channels") | ||
dir = Path(dir) | ||
await loop.run_in_executor(None, os.mkfifo, dir / "pipe") | ||
async with asyncio.timeout(timeout / 1000.0): | ||
fd = await aiofiles.open(path, "w") | ||
return AsyncNamedPipe(fd, path, True) | ||
fd = await aiofiles.open(dir / "pipe", "w") | ||
return AsyncChannelWriter(fd, dir) | ||
|
||
except asyncio.TimeoutError: | ||
await aiofiles.os.unlink(path) | ||
await aiofiles.os.unlink(dir / "pipe") | ||
await aiofiles.os.rmdir(dir) | ||
raise Exception("No consumer") | ||
|
||
async def open(path: str, timeout: int = 1000): | ||
pipe = None | ||
path = f"/tmp/{path}" | ||
try: | ||
async with asyncio.timeout(timeout / 1000.0): | ||
while not await aiofiles.ospath.exists(path): | ||
await asyncio.sleep(0.1) | ||
pipe = await aiofiles.open(path, "r") | ||
except asyncio.TimeoutError: | ||
raise Exception("No producer") | ||
|
||
return AsyncNamedPipe(pipe, path) | ||
async def close(self): | ||
await self._fd.close() | ||
await aiofiles.os.unlink(self._dir / "pipe") | ||
await aiofiles.os.rmdir(self._dir) | ||
|
||
async def __aenter__(self): | ||
return self | ||
|
||
async def __aexit__(self, *_): | ||
return await self.close() | ||
await self.close() | ||
|
||
def __bool__(self): | ||
return self._pipe is not None | ||
return self._fd is not None | ||
|
||
async def write(self, data): | ||
await self._pipe.write(data) | ||
await self._pipe.flush() | ||
await self._fd.write(data) | ||
await self._fd.flush() | ||
|
||
async def writeline(self, data): | ||
await self._pipe.writelines([data]) | ||
await self._pipe.flush() | ||
await self._fd.writelines([data]) | ||
await self._fd.flush() | ||
|
||
async def read(self, size = -1): | ||
return await self._pipe.read(size) | ||
|
||
async def readline(self): | ||
return await self._pipe.readline() | ||
class AsyncChannelReader: | ||
def __init__(self, fd): | ||
self._fd = fd | ||
|
||
async def open(pair: str, timeout: int = 10000): | ||
fd = None | ||
try: | ||
async with asyncio.timeout(timeout / 1000.0): | ||
channels = Path.home() / "tmp" / "channels" | ||
dirs = None | ||
while not dirs: | ||
dirs = list(filter(lambda d: d.startswith(f"{pair}-"), await aiofiles.os.listdir(channels))) | ||
if not dir: | ||
await asyncio.sleep(0.1) | ||
|
||
path = channels / dirs[0] / "pipe" | ||
while not await aiofiles.ospath.exists(path): | ||
await asyncio.sleep(0.1) | ||
|
||
fd = await aiofiles.open(path, "r") | ||
except asyncio.TimeoutError: | ||
raise Exception("No producer") | ||
|
||
return AsyncChannelReader(fd) | ||
|
||
async def close(self): | ||
if self._creator: | ||
await aiofiles.os.unlink(self._path) | ||
await self._fd.close() | ||
|
||
async def __aenter__(self): | ||
return self | ||
|
||
async def __aexit__(self, *_): | ||
await self.close() | ||
|
||
async def read(self, size = -1): | ||
return await self._fd.read(size) | ||
|
||
async def readline(self): | ||
return await self._fd.readline() | ||
|
||
return await self._pipe.close() |
Oops, something went wrong.