Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a pull of ggml-org/ggml#1384 into the llama.cpp repository for review/sync to ggml, since I'm mostly unfamiliar with the contribution process.
I noted a lack of Metal-accelerated ops in GGML and thought Conv2d would be a simple target for my first contribution.
The results for performance test on M3 Max (the only hw I have for testing) show a substantial boost from leveraging simdgroup:
Copilot-generated summary:
This pull request adds support for 2D convolution (
CONV_2D) operations in the Metal backend of GGML, enabling hardware-accelerated execution of this operation on supported Apple devices. The changes include the implementation of the Metal kernel, integration into the operation pipeline, and updates to device capability checks and argument structures.2D Convolution (CONV_2D) Support:
kernel_conv_2dinggml-metal.metalfor efficient 2D convolution, with template instantiations for bothfloatandhalf.ggml_metal_kargs_conv_2dargument struct inggml-metal-impl.hto pass necessary parameters to the Metal kernel.ggml_metal_op_conv_2dfunction inggml-metal-ops.cppto encode and dispatch the 2D convolution operation.ggml-metal-ops.cpp,ggml-metal-ops.h) [1] [2].CONV_2Dinggml-metal-device.cppand declared it in the header [1] [2].CONV_2Dsupport inggml-metal-device.m.Other Minor Changes:
concatoperation.These changes collectively allow GGML to offload 2D convolution operations to the GPU via Metal, improving performance for models that use this operation.