-
Notifications
You must be signed in to change notification settings - Fork 47
Media API
This page describes proposed changes to the API to add support for uploaded media files (first cut: just images).
A media object is any static file that a user can share with a room. This is meant to cover images and videos, but the concept can be more general than that.
A transcoding is a downloadable version of an uploaded media object. Generally the original upload itself may be available as the trivial identity transcode. Extracted thumbnails (potentially in multiple resolutions) are also transcodings.
A media store is a compartmentalized service that serves uploads and downloads and performs transcoding. Heim must be able to generate upload/download URLs that the media store will serve, and heim must serve a hook for the media store to notify that a transcoding of a media object is available to download. It is the media store's responsibility to encrypt files from private room at rest and manage all transcoding.
An attachment is a relation between a chat message and a media object.
A media event is an asynchronous event from the server to clients to announce when transcodings become available.
A media object begins its life as a creation request from the client. The server generates an HTTP request (method, URL, and headers) for the client to use to upload a file and returns it to the client. The client then transmits the media object's content using that request. Uploads are in plaintext (ideally using an SSL endpoint).
When this upload completes, the media store will persist the file (encrypted, if for a private room). The media store must then send a request to heim to obtain a list of transcodings to perform. At this point heim may note that the original file is available by sending out a media event to subscribed clients (which will be covered shortly). The media store will then schedule (or immediately trigger) a number of transcoding jobs on the uploaded content, corresponding to the requested transcodings returned by heim.
When each transcoding job completes, it persists the resulting file (again, encrypted if for a private room). The media store must then send a request to heim to add the completed transcoding to the media object. Each transcoding that's added triggers a media event.
Media objects exist to be published. Currently the only manner of publishing is by attaching to a message. This is recorded in the database in a table that relates the message to the media object.
When a message with an attachment is delivered to a client, whether through a log reply or a send event, the full media object is delivered. This consists of a description of all the completed transcodings for the object.
Media events allow us to post media objects to a chat before they're ready to be downloaded in the ideal format for the context. For example, the client may display a placeholder for the thumbnail until it receives a media event indicating that a thumbnail download is available.
The originator of the media object will receive all media events for that object (if connected). If the object has been attached to any published messages, then anyone who can receive events about that message will also receive a media event.