This is the CERN Document Server source code overlay.
CDS is a small layer on top of Invenio, a free software suite enabling you to run your own digital library or document repository on the web.
Ensure that the following dependencies are installed with the specified versions:
- Python 3.9
- Node.js v18
- FFmpeg v5.0
- Docker v2 or later: If Docker is not already installed, download and install Docker Desktop from the official Docker website.
To update Python dependencies you need to run npm install in the target deployment environment:
$ docker run -it --platform="linux/amd64" --rm -v $(pwd):/app -w /app \
registry.cern.ch/inveniosoftware/almalinux:1 \
sh -c "dnf install -y openldap-devel && pip install -e . && pip freeze > requirements.new.txt"
Begin by cloning this repository and navigating to the project directory:
git clone https://github.com/CERNDocumentServer/cds-videos.git
cd cds-videos
Use Docker Compose to start the required containers in detached mode:
docker compose up -d
The scripts
folder contains the necessary setup scripts to initialize and configure your instance.
- 1. Bootstrap Script
Initialize the environment by running the bootstrap script:
./scripts/bootstrap
Troubleshooting:
These are the macOS solutions using
brew
for installation.If you encounter the error
pg_config executable not found
, you may need to install PostgreSQL and update the PATH:brew install postgresql@14 export PATH=$PATH:/opt/homebrew/opt/postgresql@14/bin
For errors related to missing
cmake
andninja
toolsERROR: Command errored out with exit status 1 ... "cmake>=3.14" "ninja>=1.5"
:Install
cmake
andninja
with the following command:brew install cmake ninja
If you encounter errors with
cryptography
andOpenSSL
, ensure that OpenSSL version 3 is installed:brew install openssl@3
- 2. Setup Script
Run the setup script to finalize the installation and configuration:
./scripts/setup
Troubleshooting: If you encounter the error
connection to server at "localhost", port 5432 failed: FATAL: role ".." does not exist
, it may indicate an issue with the database role or a port conflict. To diagnose:First, connect to the Docker database container and verify that the expected role exists and the database is working correctly.
docker exec -it <db_container_name> psql -U <username> -d <database>
If the role is present and the database is functional, check for port conflicts on port 5432:
lsof -i :5432
Terminate any conflicting process if found, and restart Docker.
To facilitate local development, open multiple terminal sessions and run the following commands separately:
Start Web Server This command launches the web server:
./scripts/server
Start Celery Workers Celery workers are required for background task processing:
./scripts/celery
Watch Frontend Code This command watches frontend code for changes and rebuilds assets as needed:
./scripts/assets-watch
Running the tests are as simple as:
python setup.py test
or (to also show test coverage)
source run-tests.sh
- Navigate to the
CDS Videos
platform. - Click on your user info in the top-right corner.
- Go to Applications and create a new Personal Access Token.
- Copy the token and store it securely.
If you'd like to use the pre-configured REST API collection in Bruno, ensure you have the application installed. Follow the steps below to set up and use the collection:
Install Bruno:
Visit the official Bruno documentation or repository and install the application.
Import the Collection:
- Download this Bruno collection.
- Open Bruno and import downloaded collection.
- Create an environment for the collection.
- Configure the environment by adding a variable named
baseURl
. Set its value to your API base URL (e.g.,http://localhost:5000
).
Configure Authentication in Bruno:
- In Bruno, open the Collection Settings.
- Go to Auth and set the Bearer Token to your Personal Access Token.
Request:
POST
{{baseURL}}/api/deposits/project/
Headers:
content-type: application/vnd.project.partial+json
Parameters:
Name | Type | Location | Description | Required/Optional |
---|---|---|---|---|
$schema | string | body | Schema URL for the project creation. | Required |
category | string | body | Category of the project. | Required |
type | string | body | Type of the project. | Required |
_access | json | body | Access options for the project. | Optional |
contributors | array<object> | body | List of contributors, including their details. | Optional |
description | string | body | Description of the project. | Optional |
title | json | body | Title of the project. | Optional |
keywords | list<json> | body | Keywords related to the project. | Optional |
Body:
To restrict the project, add _access/read
:
{
"$schema": "https://localhost:5000/schemas/deposits/records/videos/project/project-v1.0.0.json",
"_access": {
"update": [
"[email protected]",
"[email protected]"
],
"read": [
"[email protected]"
]
},
"category": "ATLAS",
"type": "VIDEO",
"contributors": [
{
"name": "Surname, Name",
"ids": [
{
"value": "cern id",
"source": "cern"
}
],
"email": "[email protected]",
"role": "Co-Producer"
}
],
"title":
{
"title":"project title"
},
"keywords":[
{
"name": "keyword",
"value": {
"name": "keyword"
}
},
{
"name": "keyword2",
"value": {
"name": "keyword2"
}
}
],
"description": "Description"
}
Response:
Created project JSON.
Request:
POST
{{baseURL}}/api/deposits/video/
Headers:
content-type: application/vnd.video.partial+json
Parameters:
Name | Type | Location | Description | Required/Optional |
---|---|---|---|---|
$schema | string | body | Schema URL for video creation. | Required |
_project_id | string | body | ID of the project. | Required |
title | string | body | Title of the video. | Required |
_access | json | body | Access details for the video. | Optional |
vr | boolean | body | Indicates if the video is 360. | Optional |
contributors | array<object> | body | List of contributors, including their details. | Required |
description | string | body | Description of the video. | Required |
date | string (date) | body | Date in YYYY-MM-DD format. |
Required |
language | string | body | Language of the video. | Optional |
featured | boolean | body | Whether the video is featured. (Available for members of VIDEOS_EOS_PATH_EGROUPS) | Optional |
keywords | list<json> | body | Keywords related to the video. | Optional |
related_links | list<json> | body | Links related to the video. | Optional |
Body:
To restrict the video, add _access/read
. The _access/update
will be the same as the project:
{
"$schema":"https://localhost:5000/schemas/deposits/records/videos/video/video-v1.0.0.json",
"_project_id":"{{project_id}}",
"title":
{
"title":"217490_medium"
},
"_access": {
"read": [
"[email protected]"
]
},
"vr": false,
"featured": false,
"language": "en",
"contributors": [
{
"name": "Surname, Name",
"ids": [
{
"value": "cern id",
"source": "cern"
}
],
"email": "[email protected]",
"role": "Co-Producer"
}
],
"description": "Description",
"date": "2024-11-12",
"keywords":[
{
"name": "keyword",
"value": {
"name": "keyword"
}
},
{
"name": "keyword2",
"value": {
"name": "keyword2"
}
}
],
"related_links":[
{
"name": "related link",
"url": "https://relatedlink"
}
]
}
Response:
Created video JSON.
Request:
PUT
{{baseURL}}/api/files/{{bucket_id}}/{{video_name}}
Headers:
content-type: video/mp4
Accept: application/json, text/plain, */*
Accept-Encoding: gzip, deflate, br, zstd
Parameters:
Name | Type | Location | Description |
---|---|---|---|
bucket_id | string | path | Bucket ID. |
video_name | string | path | Name of the video file. |
file | object | body | Video file. |
Response:
Uploaded video JSON.
Request:
POST
/api/flows/
Headers:
content-type: application/vnd.project.partial+json
Parameters:
Name | Type | Location | Description |
---|---|---|---|
version_id | string | body | Version ID from the uploaded video response. |
key | string | body | Video key from the uploaded video response. |
bucket_id | string | body | Bucket ID from the Create Video response. |
deposit_id | string | body | Deposit ID from the Create Video response. |
Body:
{
"version_id": "{{main_file_version_id}}",
"key": "{{video_key}}",
"bucket_id": "{{bucket_id}}",
"deposit_id": "{{video_id}}"
}
Response:
Created flow JSON.
Request:
PUT
{{baseURL}}/api/files/{{bucket_id}}/{{additional_file}}
Parameters:
Name | Type | Location | Description |
---|---|---|---|
bucket_id | string | path | ID of the bucket to upload the file. |
file_name | string | path | Name of the file. |
file | file | body | The file to be uploaded. |
Response:
Uploaded additional file JSON.
Request:
PUT
{{baseURL}}/api/deposits/video/{{video_id}}
Headers:
content-type: application/vnd.video.partial+json
Parameters:
Name | Type | Location | Description |
---|---|---|---|
video_id | string | path | ID of the video. |
Body:
To restrict the video, add _access/read
. If you want to change the access/update permissions, replace the email addresses in the update
field accordingly.
{
"_access": {
"update": [
"[email protected]",
"[email protected]"
],
"read": [
"[email protected]"
]
}
}
Response:
Updated video JSON.
Request:
GET
{{baseURL}}/api/deposits/project/{{project_id}}
Headers:
content-type: application/vnd.project.partial+json
Parameters:
Name | Type | Location | Description |
---|---|---|---|
project_id | string | path | ID of the project. |
Response:
Updated project JSON with flow status as state
:
{
"id": "b320568fc1264dda90a8f459be42892e",
"_cds": {
"state": {
"file_transcode": "STARTED",
"file_video_extract_frames": "SUCCESS",
"file_video_metadata_extraction": "SUCCESS"
}
}
}
Before publishing the video, ensure that the workflow is complete.
Request:
POST
{{baseURL}}/api/deposits/video/{{video_id}}/actions/publish
Headers:
content-type: application/json
Parameters:
Name | Type | Location | Description |
---|---|---|---|
video_id | string | path | ID of the video to publish. |
Response:
Published video deposit JSON.
Copyright (C) 2013-2024 CERN.
CDS is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
CDS is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with CDS; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307, USA.
In applying this licence, CERN does not waive the privileges and immunities granted to it by virtue of its status as an Intergovernmental Organization or submit itself to any jurisdiction.