This repo has codes for runwayml video image generation API call
This code uses DALL-E-3 as base image generator model. User gives text prompt hardcoded into the code to generate an image and download it in local directory called "generated_content". The images are named as image_1.png, image_2.png and so on. Images are downlaoded as .png format with size 1024x1024. The format can be changed to .jpg also.
Then, the image (say image_1.png) is converted to base64 format. Such conerverted format + text prompt together used to call the gen3a_turbo model from runwayml API. The call generated a video of 5 seconds duration. Then the generated video is downloaded as _video_1.mp4 (in .mp4 format) in the same directory i.e. "generated_content". For aother image, nsame file named video is generted.
The code uses one generated image (from DALL-E-3) to generate one video (from runwayml).
Runwayml API doc is available https://pypi.org/project/runwayml/ and https://docs.dev.runwayml.com/
Buy API credits for runwayml by making an organization.
Buy ChatGPT4o pro subscription that gives DALL-E-3 service too.
-
Install dependencies
pip install -r requirements.txt
-
Make a .env file
The .env file where you replace ENTER-YOUR-DALL-E-API-KEY from chatgpt and ENTER-YOUR-RUNWAYML-API-KEY with your actual API key from runwayml
OPENAI_API_KEY=ENTER-YOUR-DALL-E-API-KEY
RUNWAYML_API_SECRET=ENTER-YOUR-RUNWAYML-API-KEY
AND
In a shell terminal (Linux and MAC):
export RUNWAYML_API_SECRET=**ENTER-YOUR-RUNWAYML-API-KEY**
Note: For windows, no key export support exists
-
Change the "num_generations"
If num_generations = 2, then 2 images and 2 videos will be generated and downloaded. If num_generations = 5, then 5 images and 5 videos will be generated and downloaded. So, change the value as per your requirement.
-
Run the code below
python runwayml_gen3a_turbo.py
This code calls gen3a_turbo image-to-video generation model API to generate 5 second video. This model needs both "Image" + "Text". "Image" is mandatory. The prompt is already given inside the code main_prompt. You can change the main_prompt accordingly.