Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update benchmark #14

Merged
merged 5 commits into from
Jan 31, 2025
Merged

Update benchmark #14

merged 5 commits into from
Jan 31, 2025

Conversation

ternaus
Copy link
Contributor

@ternaus ternaus commented Jan 31, 2025

Summary by Sourcery

Tests:

  • Updated benchmark parameters for consistency across libraries.

Copy link

sourcery-ai bot commented Jan 31, 2025

Reviewer's Guide by Sourcery

This pull request updates the benchmark suite to include more transforms and use the latest versions of the libraries. The implementations of the transforms were updated to use the latest APIs and to ensure that the transforms are applied with a probability of 1.0.

Class diagram showing the updated transform implementations

classDiagram
    class KorniaImpl {
        +ColorJitter(params)
        +AutoContrast(params)
        +Blur(params)
        +Brightness(params)
        +ChannelDropout(params)
        +ChannelShuffle(params)
        +CLAHE(params)
        +Contrast(params)
        +Equalize(params)
        +RandomGamma(params)
        +GaussianBlur(params)
        +GaussianNoise(params)
        +Invert(params)
        +Posterize(params)
        +Solarize(params)
        +Sharpen(params)
        +Normalize(params)
        +Erasing(params)
        +OpticalDistortion(params)
        +ThinPlateSpline(params)
    }

    class AlbumentationsImpl {
        +Resize(params)
        +RandomCrop128(params)
        +HorizontalFlip(params)
        +VerticalFlip(params)
        +Rotate(params)
        +Affine(params)
        +Perspective(params)
        +Elastic(params)
        +ColorJitter(params)
        +GaussianBlur(params)
        +GaussianNoise(params)
        +Normalize(params)
        +Erasing(params)
    }

    class TorchvisionImpl {
        +Resize(params)
        +RandomCrop128(params)
        +RandomResizedCrop(params)
        +CenterCrop128(params)
        +HorizontalFlip(params)
        +VerticalFlip(params)
        +Pad(params)
        +Rotate(params)
        +Affine(params)
        +ColorJitter(params)
        +GaussianBlur(params)
        +Normalize(params)
    }

    note for KorniaImpl "Updated to use p=1 for all transforms"
    note for AlbumentationsImpl "Updated to use latest APIs"
    note for TorchvisionImpl "Added new transforms and updated APIs"
Loading

File-Level Changes

Change Details Files
Updated Kornia transform implementations to use the latest API and ensure that the transforms are applied with a probability of 1.0.
  • Updated all transforms to use the latest Kornia API.
  • Removed the probability parameter from all transforms and set the probability to 1.0.
  • Updated the Rotate transform to use kornia.geometry.transform.Rotate instead of Kaug.RandomRotation.
  • Updated the Affine transform to use a tuple for scale instead of a list.
  • Updated the MedianBlur transform to ensure the kernel size is odd.
  • Updated the MotionBlur transform to use angle_range and direction_range instead of angle and direction.
  • Updated the RGBShift transform to divide the pixel shift by 255.0.
  • Updated the Solarize transform to use thresholds instead of threshold.
  • Updated the CenterCrop transform to use pad_if_needed=True.
  • Updated the RandomCrop transform to use pad_if_needed=True.
  • Updated the Elastic transform to use a tuple for alpha and sigma.
  • Updated the Perspective transform to use resample instead of interpolation.
benchmark/transforms/kornia_impl.py
Updated Albumentations transform implementations to use the latest API and ensure that the transforms are applied with a probability of 1.0.
  • Updated all transforms to use the latest Albumentations API.
  • Removed the probability parameter from all transforms and set the probability to 1.0.
  • Updated the Resize transform to use cv2.INTER_LINEAR or cv2.INTER_NEAREST for interpolation.
  • Updated the Rotate transform to use a limit for the angle and border_mode and fill.
  • Updated the Affine transform to use translate_px instead of translate_percent and border_mode and fill.
  • Updated the Perspective transform to use cv2.INTER_LINEAR or cv2.INTER_NEAREST for interpolation.
  • Updated the Elastic transform to use cv2.INTER_LINEAR or cv2.INTER_NEAREST for interpolation.
  • Updated the GaussianBlur transform to use a tuple for sigma_limit.
  • Updated the GaussianNoise transform to use std_range and mean_range.
  • Updated the Posterize transform to use num_bits.
  • Updated the Solarize transform to use threshold_range.
  • Updated the Sharpen transform to use alpha and lightness.
  • Updated the JpegCompression transform to use quality_range.
  • Updated the RandomGamma transform to use gamma_limit.
  • Updated the MedianBlur transform to use a tuple for blur_limit.
  • Updated the MotionBlur transform to use angle_range and direction_range.
  • Updated the CLAHE transform to use clip_limit and tile_grid_size.
  • Updated the Brightness transform to use brightness_limit and contrast_limit.
  • Updated the Contrast transform to use brightness_limit and contrast_limit.
  • Updated the Blur transform to use a tuple for blur_limit.
  • Updated the HSV transform to use hue_shift_limit, sat_shift_limit, and val_shift_limit.
  • Updated the Rain transform to use drop_width and brightness_coefficient.
  • Updated the SaltAndPepper transform to use amount and salt_vs_pepper.
  • Updated the Saturation transform to use sat_shift_limit.
  • Updated the Snow transform to use snow_point_range.
  • Updated the OpticalDistortion transform to use distort_limit.
  • Updated the Shear transform to use shear, interpolation, border_mode, and fill.
  • Updated the ThinPlateSpline transform to use num_control_points and scale_range.
benchmark/transforms/albumentations_impl.py
Updated transform specifications to match the latest implementations and include new transforms.
  • Updated transform specifications to match the latest implementations.
  • Added new transforms to the transform specifications.
  • Removed the probability parameter from all transform specifications.
  • Updated the Affine transform to use a tuple for shift and shear.
  • Updated the GaussianBlur transform to use a tuple for kernel_size.
  • Updated the MotionBlur transform to use a tuple for angle_range and direction_range.
  • Updated the CoarseDropout transform to use a tuple for hole_height_range, hole_width_range, and num_holes_range.
  • Updated the Blur transform to use a border_mode.
  • Updated the HSV transform to use standard HSV adjustment values.
  • Updated the Elastic transform to use approximate and same_dxdy.
  • Updated the RandomResizedCrop transform to use interpolation.
  • Updated the Rotate transform to use mode and fill.
  • Updated the Affine transform to use mode and fill.
  • Updated the Perspective transform to use fill.
benchmark/transforms/specs.py
Updated Torchvision transform implementations to use the latest API.
  • Updated all transforms to use the latest Torchvision API.
  • Updated the Resize transform to use antialias=True.
  • Updated the Rotate transform to use fill.
  • Updated the Perspective transform to use fill.
  • Updated the Elastic transform to use interpolation.
  • Updated the Normalize transform to use a list for mean and std.
  • Updated the Erasing transform to use value.
benchmark/transforms/torchvision_impl.py
Updated Imgaug transform implementations to use the latest API.
  • Updated all transforms to use the latest Imgaug API.
  • Removed the probability parameter from all transforms and set the probability to 1.0.
  • Updated the MotionBlur transform to use angle_range instead of angle.
  • Updated the GaussianNoise transform to use scale instead of var.
  • Updated the Clahe transform to use AllChannelsCLAHE.
  • Updated the Contrast transform to use AddToHueAndSaturation.
  • Updated the Solarize transform to use threshold * 255.
  • Updated the Sharpen transform to use alpha and lightness.
  • Updated the Shear transform to use ShearX.
benchmark/transforms/imgaug_impl.py
Updated Augly transform implementations to use the latest API.
  • Updated all transforms to use the latest Augly API.
  • Removed the probability parameter from all transforms and set the probability to 1.0.
  • Updated the Rotate transform to use min_degrees and max_degrees.
  • Updated the RandomCrop transform to use relative coordinates.
  • Updated the Resize transform to use resample.
  • Updated the GaussianBlur transform to use radius.
  • Updated the JpegCompression transform to use quality.
  • Updated the GaussianNoise transform to use mean and var.
  • Updated the Blur transform to use radius.
  • Updated the Brightness transform to use min_factor and max_factor.
  • Updated the Contrast transform to use factor.
benchmark/transforms/augly_impl.py
Updated the benchmark to use the latest versions of the libraries.
  • Updated the albumentations version to 2.0.2.
  • Updated the kornia version to 0.8.0.
  • Updated the torchvision version to 0.20.1.
  • Updated the benchmark parameters to use 2000 images and 5 runs per transform.
  • Updated the comparison table to include the new transforms.
  • Updated the comparison table to show the speedup between the fastest library and the second-fastest library.
  • Updated the run_all.sh script to use uv pip install --force-reinstall.
  • Updated the run_single.sh script to use uv pip install --force-reinstall.
  • Updated the pyproject.toml to use ruff v0.9.4 and codespell v2.4.1.
  • Updated the pyproject.toml to use mypy v1.14.1.
README.md
output/comparison.md
pyproject.toml
run_all.sh
run_single.sh
requirements-dev.txt
requirements/albumentations.txt
Added a new script to generate comparison tables.
  • Added a new script to generate comparison tables from the benchmark results.
  • Updated the compare_results.py script to generate a csv file.
tools/generate_tables.py
tools/compare_results.py

Tips and commands

Interacting with Sourcery

  • Trigger a new review: Comment @sourcery-ai review on the pull request.
  • Continue discussions: Reply directly to Sourcery's review comments.
  • Generate a GitHub issue from a review comment: Ask Sourcery to create an
    issue from a review comment by replying to it. You can also reply to a
    review comment with @sourcery-ai issue to create an issue from it.
  • Generate a pull request title: Write @sourcery-ai anywhere in the pull
    request title to generate a title at any time. You can also comment
    @sourcery-ai title on the pull request to (re-)generate the title at any time.
  • Generate a pull request summary: Write @sourcery-ai summary anywhere in
    the pull request body to generate a PR summary at any time exactly where you
    want it. You can also comment @sourcery-ai summary on the pull request to
    (re-)generate the summary at any time.
  • Generate reviewer's guide: Comment @sourcery-ai guide on the pull
    request to (re-)generate the reviewer's guide at any time.
  • Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
    pull request to resolve all Sourcery comments. Useful if you've already
    addressed all the comments and don't want to see them anymore.
  • Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
    request to dismiss all existing Sourcery reviews. Especially useful if you
    want to start fresh with a new review - don't forget to comment
    @sourcery-ai review to trigger a new review!
  • Generate a plan of action for an issue: Comment @sourcery-ai plan on
    an issue to generate a plan of action for it.

Customizing Your Experience

Access your dashboard to:

  • Enable or disable review features such as the Sourcery-generated pull request
    summary, the reviewer's guide, and others.
  • Change the review language.
  • Add, remove or edit custom review instructions.
  • Adjust other review settings.

Getting Help

@ternaus ternaus merged commit bd910b3 into main Jan 31, 2025
1 check passed
@ternaus ternaus deleted the update_benchmark branch January 31, 2025 01:33
Copy link

@sourcery-ai sourcery-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @ternaus - I've reviewed your changes and they look great!

Here's what I looked at during the review
  • 🟡 General issues: 1 issue found
  • 🟢 Security: all looks good
  • 🟢 Testing: all looks good
  • 🟡 Complexity: 2 issues found
  • 🟢 Documentation: all looks good

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

benchmark/transforms/kornia_impl.py Show resolved Hide resolved
benchmark/transforms/kornia_impl.py Show resolved Hide resolved
benchmark/transforms/albumentations_impl.py Show resolved Hide resolved
tools/generate_tables.py Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant