video_to_events_instructions

#Intructions for running ESIM#

This mostly follows from esim wiki page https://github.com/uzh-rpg/rpg_esim/wiki/Simulating-events-from-a-video
GitHub tutorial shows you how you can convert a video from YouTube into events using youtube-dl - however the instructions below are for converting an already locally downloaded video.

#install ffmpeg
sudo apt update
sudo apt install ffmpeg

#pull docker image
docker pull pcoll98/video_to_events_1:esim_video_to_events

#run docker image and mount directory - this will allow you to be able to use local files from your computer in your container and vice versa
docker run -it -v /path/to/Documents:/root/Documents pcoll98/video_to_events_1:esim_video_to_events
source /opt/ros/kinetic/setup.bash
source ~/sim_ws/devel/setup.bash

#create directories 
cd ~/sim_ws
mkdir -p /tmp/workspace
cd /tmp/workspace

#copy video from local files to docker container
cp /path/to/file/video.mkv /tmp/workspace

#cut video to desired length (any more than 10 seconds will create very large / slow files to work with), resize video for faster processing, make frames.
#perlin noise files 1-4 (Perlin_Ramp_.mkv) each have a different starting 'seed' and were used in the final dataset I created (see https://docs.google.com/spreadsheets/d/1y52cJEPmonaaT4vr6QqdaC3b-Pwl7Rar76JvPLBSm5A/edit?usp=sharing). To make more, perlin_long.mkv has been provided, which is a long video that can be cut up into 10 second sections using below method (this saves on having to make a number of short videos with different seeds).
ffmpeg -i video.mkv -ss 00:00:00 -t 00:00:10 -async 1 -strict -2 video_cut.mkv
ffmpeg -i video_cut.mkv -vf scale=260:346 -crf 0 video_sd.mkv

#the scale parameter should be set to the desired width of your video. [height]:-1 keeps the original aspect ratio of the video. Otherwise scale=[height]:[width] (scale=260:346 should match the output of the DV camera).

mkdir frames #delete old frames directory if already exists
ffmpeg -i video_sd.mkv frames/frames_%010d.png
cd ./frames
touch images.csv

#generate time stamps 
cd ~/sim_ws/src/rpg_esim/event_camera_simulator/esim_ros
python scripts/generate_stamps_file.py -i /tmp/workspace/frames -r 1200.0

#open another terminal window and run roscore - whenever opening another terminal window, make sure to run the same docker container. You can find the container id by typing: docker ps
docker exec -it [container id] bash
source /opt/ros/kinetic/setup.bash
source ~/sim_ws/devel/setup.bash
roscore

#back to first terminal
rosrun esim_ros esim_node \
 --data_source=2 \
 --path_to_output_bag=/tmp/out.bag \
 --path_to_data_folder=/tmp/workspace/frames \
 --ros_publisher_frame_rate=60 \
 --exposure_time_ms=10.0 \
 --use_log_image=1 \
 --log_eps=0.1 \
 --contrast_threshold_pos=0.15 \
 --contrast_threshold_neg=0.15

#extract events from rosbag script. 
#change directory to the path to this script provided (extract_events_from_rosbag.py) in mounted local volume (I originally found the file at https://github.com/uzh-rpg/rpg_e2vid/blob/master/scripts)
cd ~/path/to/file/
python extract_events_from_rosbag.py /tmp/out.bag --output_folder=/tmp --event_topic=/cam0/events --no-zip
# file taken from https://github.com/uzh-rpg/rpg_e2vid/blob/master/scripts/extract_events_from_rosbag.py

#copy txt file to local documents
cp /tmp/out.txt /path/to/local/workspace

#Note: if you would like to commit container to new image, follow: https://www.scalyr.com/blog/create-docker-image/. This will allow you to save image with all above changes made.