Raw2RGB

The Raw2RGB project is a pipeline designed to process raw Bayer images (.tiff format) captured from Emergent HB-50000 cameras. The images used for development and testing were taken from multiple Emergent HB-50000 cameras, making this tool optimized for multi-camera setups, where each camera has its own directory for storing captured images. Instead of processing a single file, the program efficiently processes entire folders of images, making it ideal for setups where multiple cameras capture data simultaneously. While optimized for Emergent HB-50000 cameras, this tool can potentially work with raw Bayer images generated by other camera systems. The default Bayer mode for Emergent HB-50000 cameras is 48 (CV_BayerRG2RGB_EA), but the tool allows you to choose other Bayer modes supported by OpenCV. These modes can be passed as an argument when running the tool. You can find details about available Bayer modes and their respective codes on the OpenCV documentation site. With an Intel Core i9-10940X CPU @ 3.30GHz, the pipeline processes images at a rate of approximately 0.3 seconds per image, ensuring high performance and scalability for large datasets.

Features

• Loading and Inspecting Images:

  • Loads raw Bayer .tiff files.
  • Prints details about images, such as depth, channel count, and specific pixel values.

• Dead Pixel Correction:

  • Identifies and corrects dead pixels using SIMD optimization for performance.

• Vignetting Correction:

  • Estimates vignetting coefficients using entropy minimization.
  • Applies vignetting correction using the estimated vignette parameters.
  • Supports exporting and loading of vignette correction maps.

• Gamma Correction:

  • Applies gamma correction to images for adjusting brightness and contrast.

• Demosaicing:

  • Converts raw Bayer images into RGB images based on Bayer patterns using OpenCV's demosaicing functions.

• Batch Processing:

  • Automatically scans a directory for .tiff files, processes them, and saves the results in .png format to an output folder.

• Performance Optimization:

  • Uses AVX-512, AVX, and SSE instructions for SIMD-enhanced performance during pixel corrections and image processing.
  • Employs parallel processing with OpenMP for multi-threaded performance across large image datasets.

Requirements

Dependences

• OpenCV 4.x or above

• C++17 standard or higher

• Supported SIMD Features:

  • AVX-512, AVX, or SSE (depending on hardware)

• OpenMP (for parallelization)

Build Environment

• CMake (Minimum version 3.10)

• A compatible C++ compiler supporting C++17:

  • GCC 7+ / Clang 5+ / MSVC 2017+

• Ensure libopencv-dev is installed on the system or linked manually.

Installation Steps:

1. Clone the repository:

   git clone <repository_url>
   cd Raw2RGB

1. Run CMake to generate the build files:

   mkdir build
   cd build
   cmake ..

1. Build the project:

   cmake --build .

1. Execute the binary (details provided in the Usage section).

Usage

Processing a Directory of Raw Images

Run the executable with the following parameters:

Raw2RGB <input_directory> <output_directory> <demosaicMode> <gamma> <vignette_path>

Parameters:

• <input_directory> — Folder containing input .tiff files.
• <output_directory> — Folder to store the processed .png files.
• <demosaicMode> — OpenCV demosaicing mode (e.g., CV_BayerRG2RGB).
• <gamma> — Gamma correction factor (e.g., 2.2).
• <vignette_path> — Optional vignette correction .dat file. If not present, the correction is skipped.

Example:

Raw2RGB ./raw_images ./processed_images CV_BayerRG2RGB 2.2 vignette_map.dat

API Overview

1. Load Images

Function: loadImage(const std::string &imagePath) Loads the raw Bayer image from the specified file.

cv::Mat image = loadImage("image.tiff");

2. Save Processed Images

Function: saveImage(const std::string &imagePath, const cv::Mat &image) Saves an image to the specified path.

saveImage("output.png", image);

3. Correct Dead Pixels

Function: correctDeadPixels(cv::Mat& rawImage, float brightFactor, float darkFactor) Corrects defective pixel regions in raw Bayer images.

correctDeadPixels(image, 1.5, 0.75);

4. Apply Vignetting Correction

Function: applyVignetteCorrection(const cv::Mat& rawBayer, const VignetteModel& model) Applies vignette correction using a pre-computed correction map.

VignetteModel model = estimateVignetting(image, "vignette.dat", "RG", 3);
image = applyVignetteCorrection(image, model);

5. Process All Images in a Directory

Function: processImages(const std::vector<Image> &images, const int demosaicMode, const double gamma, const std::string &vignettePath) Processes multiple images by loading the input files, applying corrections, and saving the results.

std::vector<Image> images = getRawFiles("input_folder", "output_folder");
processImages(images, CV_BayerRG2RGB, 2.2, "vignette.dat");

6. Vignette Estimation

Function: estimateVignetting(const cv::Mat& rawBayer, const std::string &outPath, const std::string& bayerPattern, int polyDegree) Calculates vignette correction coefficients by minimizing entropy on a Bayer-patterned image.

7. Calculating Entropy

Function: calculateEntropy(const float a, const float b, const float c, const cv::Mat& img) Helps determine optimal vignette correction coefficients by calculating image entropy.

File Structure

.
├── include/
│   ├── image_process.h          # Header for processing functions
├── src/
│   ├── main.cpp                 # Entry point
│   ├── image_process.cpp        # Image processing implementation
├── CMakeLists.txt               # Build configuration
└── README.md                    # Documentation

Known Limitations

• SIMD optimizations may not work on hardware without AVX, SSE, or similar instruction sets.

Future Improvements

• Add support for additional raw formats (e.g., .raw, .dng).
• Implement undistort process with Metashape .xml data.

License

This project is licensed under the MIT license. See LICENSE for more details.

A special thanks to Peter Flaherty and Flaherty Pictures for their support and generosity in open-sourcing this project. Their commitment to open-source development makes this work accessible to the community.