-
-
Notifications
You must be signed in to change notification settings - Fork 14.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
opencv: misc cleanups; fix CUDA build #339619
Merged
Merged
Changes from all commits
Commits
Show all changes
2 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
81 changes: 81 additions & 0 deletions
81
pkgs/development/cuda-modules/tests/opencv-and-torch/default.nix
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,81 @@ | ||
{ | ||
cudaPackages, | ||
lib, | ||
writeGpuTestPython, | ||
# Configuration flags | ||
openCVFirst, | ||
useOpenCVDefaultCuda, | ||
useTorchDefaultCuda, | ||
}: | ||
let | ||
inherit (lib.strings) optionalString; | ||
|
||
openCVBlock = '' | ||
|
||
import cv2 | ||
print("OpenCV version:", cv2.__version__) | ||
|
||
# Ensure OpenCV can access the GPU. | ||
assert cv2.cuda.getCudaEnabledDeviceCount() > 0, "No CUDA devices found for OpenCV" | ||
print("OpenCV CUDA device:", cv2.cuda.printCudaDeviceInfo(cv2.cuda.getDevice())) | ||
|
||
# Ensure OpenCV can access the GPU. | ||
print(cv2.getBuildInformation()) | ||
|
||
a = cv2.cuda.GpuMat(size=(256, 256), type=cv2.CV_32S, s=1) | ||
b = cv2.cuda.GpuMat(size=(256, 256), type=cv2.CV_32S, s=1) | ||
c = int(cv2.cuda.sum(cv2.cuda.add(a, b))[0]) # OpenCV returns a Scalar float object. | ||
|
||
assert c == 2 * 256 * 256, f"Expected {2 * 256 * 256} OpenCV, got {c}" | ||
|
||
''; | ||
|
||
torchBlock = '' | ||
|
||
import torch | ||
print("Torch version:", torch.__version__) | ||
|
||
# Set up the GPU. | ||
torch.cuda.init() | ||
# Ensure the GPU is available. | ||
assert torch.cuda.is_available(), "CUDA is not available to Torch" | ||
print("Torch CUDA device:", torch.cuda.get_device_properties(torch.cuda.current_device())) | ||
|
||
a = torch.ones(256, 256, dtype=torch.int32).cuda() | ||
b = torch.ones(256, 256, dtype=torch.int32).cuda() | ||
c = (a + b).sum().item() | ||
assert c == 2 * 256 * 256, f"Expected {2 * 256 * 256} for Torch, got {c}" | ||
|
||
''; | ||
|
||
content = if openCVFirst then openCVBlock + torchBlock else torchBlock + openCVBlock; | ||
|
||
torchName = "torch" + optionalString useTorchDefaultCuda "-with-default-cuda"; | ||
openCVName = "opencv4" + optionalString useOpenCVDefaultCuda "-with-default-cuda"; | ||
in | ||
# TODO: Ensure the expected CUDA libraries are loaded. | ||
# TODO: Ensure GPU access works as expected. | ||
writeGpuTestPython { | ||
name = if openCVFirst then "${openCVName}-then-${torchName}" else "${torchName}-then-${openCVName}"; | ||
libraries = | ||
# NOTE: These are purposefully in this order. | ||
pythonPackages: | ||
let | ||
effectiveOpenCV = pythonPackages.opencv4.override (prevAttrs: { | ||
cudaPackages = if useOpenCVDefaultCuda then prevAttrs.cudaPackages else cudaPackages; | ||
}); | ||
effectiveTorch = pythonPackages.torchWithCuda.override (prevAttrs: { | ||
cudaPackages = if useTorchDefaultCuda then prevAttrs.cudaPackages else cudaPackages; | ||
}); | ||
in | ||
if openCVFirst then | ||
[ | ||
effectiveOpenCV | ||
effectiveTorch | ||
] | ||
else | ||
[ | ||
effectiveTorch | ||
effectiveOpenCV | ||
]; | ||
} content |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why are we referring to
prevAttrs
here?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I suspect we actually want
s/prevAttrs.cudaPackages/finalAttrs/
here?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Also why do we need this branch, we
cudaPackages
passed from the caller aren't enough? What exactly are we testing hereThere was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The
override
attribute also accepts a function of the formprevAttrs: <attr set>
, whereprevAttrs
is the previous set of attributes passed to (or resolved by)callPackage
.The
cudaPackages
provided toopencv4
should be the one given by top-level (or the one automatically provided bycallPackage
) whenuseOpenCVDefaultCuda
is true, thus essentially passing through the previous value forcudaPackages
. WhenuseOpenCVDefaultCuda
is false,cudaPackages
should be the version ofcudaPackages
from the enclosing scope (e.g.,cudaPackages_11_8
if the derivation is fromcudaPackages_11_8.tests.<whatever>
).In short, it's a matrix of tests of packages against versions of the
cudaPackages
package set.Because the tests exist in each
cudaPackages*
package set, it makes sense to have a copy of the test which builds not with thecudaPackages
argument supplied at the top-level or bycallPackage
, but with thecudaPackages
from the enclosing scope. In this way, we're testing a single package (well, in the case of this PR, OpenCV and PyTorch) against multiple different versions of CUDA.If we want to hold the version of
cudaPackages
used by both packages fixed (meaning they are either set to some specific version at the top-level due to compatibility requirements or use the default version by way ofcudaPackages
being populated bycallPackage
), it doesn't make sense for the test to live incudaPackages.tests
because the tests are irrespective of the version ofcudaPackages
.Does that make sense?