-
Notifications
You must be signed in to change notification settings - Fork 8
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
9fca947
commit 6978761
Showing
1 changed file
with
25 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,25 @@ | ||
# Data Storage and Routing | ||
|
||
## Status | ||
|
||
Proposed | ||
|
||
## Context | ||
|
||
Currently, all encoded files are stored locally in a directory on each worker node. This approach suits our live streaming requirements due to its low latency and direct access. However, the system lacks a defined method for efficiently routing data requests to the appropriate worker, which is crucial for targeting specific live streams. | ||
|
||
## Decision | ||
|
||
To address this, we propose the following: | ||
* Maintain local storage of data in each worker to leverage quick access and response times. | ||
* Implement a monitoring and routing routine that starts with each new transcoding session. | ||
|
||
This routine will: | ||
* Periodically verify (every 5 seconds) if the worker is correctly transcoding the stream. | ||
* Notify a central API with status updates and routing metadata necessary for efficiently directing incoming requests. | ||
* Every key (stream) lasts for 30 seconds, after which it is deleted from the central API. This ensures the system always has the most up-to-date information. | ||
|
||
## Consequences | ||
|
||
* By avoiding the use of services like S3 for data storage, we anticipate significant cost savings without sacrificing performance for low latency streaming. | ||
* Without S3, we need to store which worker is responsible for each stream in a central key-value database to ensure efficient routing keeping resources usage low - probably Redis. |