Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

otb: init at 9.0.0 #325630

Open
wants to merge 4 commits into
base: master
Choose a base branch
from
Open

otb: init at 9.0.0 #325630

wants to merge 4 commits into from

Conversation

daspk04
Copy link

@daspk04 daspk04 commented Jul 8, 2024

Description of changes

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.11 Release Notes (or backporting 23.11 and 24.05 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@github-actions github-actions bot added the 8.has: maintainer-list (update) This PR changes `maintainers/maintainer-list.nix` label Jul 8, 2024
@NixOSInfra NixOSInfra added the 12. first-time contribution This PR is the author's first one; please be gentle! label Jul 8, 2024
pkgs/by-name/it/itk_4_13/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
@ofborg ofborg bot added 8.has: package (new) This PR adds a new package 11.by: package-maintainer This PR was created by the maintainer of the package it changes 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild 10.rebuild-linux: 1-10 labels Jul 10, 2024
pkgs/by-name/sh/shark/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/sh/shark/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/sh/shark/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/sh/shark/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
];


propagatedBuildInputs = [
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we want to propagate all of this?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @SuperSandro2000.

As OTB allows to build (remote) modules (officially not part of) on top of it, so we propagate the inputs to able to build remote modules based on OTB via nix as well. As I wasn't sure if all are needed so I tried to follow similar to ITK c.f

Example of remote modules for OTB:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how this will integrate with nix. First of all propagatedBuildInputs are merged into buildInputs, so we don't need anything in buildInputs that is already in propagatedBuildInputs.

The other things, we don't want to propagate things if possible. Are those modules build with nix? If so we should move things to passthru. If not, then we just keep this ugly workaround for now until we have something better.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @SuperSandro2000 !

Currently those modules are not built with nix.

pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/it/itk_4_13/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/it/itk_4_13/package.nix Outdated Show resolved Hide resolved
@daspk04 daspk04 force-pushed the add-otb-9.0.0 branch 2 times, most recently from e83a4d6 to dbdaf48 Compare July 13, 2024 17:39
@daspk04
Copy link
Author

daspk04 commented Jul 13, 2024

@jackyliu16 I did some more changes. Will it be possible if you could review one more time ?

@jackyliu16
Copy link
Contributor

@daspk04 To be honst, I not a specialist in packing.
I can't comment much on the more detailed parts of your submission, which I not familiar with these things.

But the things I could suggest is follow:

  1. You should try to use nixfmt-rfc-style to format your code.
    maybe you could run something like nix run .#nixfmt-rfc-style -- pkgs/by-name/otb/<specify file> to formatting your file base on [RFC 0166] Nix formatting, take two rfcs#166
  2. You may consider to use override to cover some Attr in pkgs/development/libraries/itk , rather than recreate a package . (maybe you could use it's generate.nix file ?, I not sure)

* an old version of itk with extra configuration required for OTB 9.0.0
@daspk04
Copy link
Author

daspk04 commented Jul 21, 2024

@daspk04 To be honst, I not a specialist in packing. I can't comment much on the more detailed parts of your submission, which I not familiar with these things.

But the things I could suggest is follow:

  1. You should try to use nixfmt-rfc-style to format your code.
    maybe you could run something like nix run .#nixfmt-rfc-style -- pkgs/by-name/otb/<specify file> to formatting your file base on [RFC 0166] Nix formatting, take two rfcs#166

Thanks @jackyliu16. I have formatted using the nixfmt-rfc-style.

  1. You may consider to use override to cover some Attr in pkgs/development/libraries/itk , rather than recreate a package . (maybe you could use it's generate.nix file ?, I not sure)

As suggested by @SuperSandro2000 c.f, either override or local to the package I followed the second approach as it seemed easier and more logical to me, possibly can be done using overwrite but I'm still not sure how to do it for now😅

@daspk04
Copy link
Author

daspk04 commented Jul 21, 2024

Hi @SuperSandro2000 ,

Will it possible if you could review it one more time ?

I have formatted using nixfmt-rfc-style as suggested c.f. There are few question (c.f, c.f ) if you could suggested what to be done then that would be helpful and I can try to make changes accordingly. 🙏

@daspk04
Copy link
Author

daspk04 commented Aug 28, 2024

Hi @SuperSandro2000 ,

I would like this to be merged so I would like to have your suggestion regarding the questions that I have asked earlier in the comments, if you could please suggest what changes are further needed for this to be merged.

pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
optionals enablePython (with python3.pkgs; [ numpy ]) ++ (extraPythonPackages python3.pkgs);

otb-itk = callPackage ./itk_4_13/package.nix { };
otb-shark = shark.override { enableOpenMP = enableOpenMP; };
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why not enable openmp by default and drop the option?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @SuperSandro2000 !

By default it is OFF c.f so I have kept it as it is.

pkgs/by-name/ot/otb/package.nix Outdated Show resolved Hide resolved
Comment on lines +22 to +32
enableFeatureExtraction ? true,
enableHyperspectral ? true,
enableLearning ? true,
enableMiscellaneous ? true,
enableOpenMP ? false,
enablePython ? true,
extraPythonPackages ? ps: with ps; [ ],
enableRemote ? true,
enableSAR ? true,
enableSegmentation ? true,
enableStereoProcessing ? true,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need those options? or can we just turn them on by default and make things simple?

Copy link
Author

@daspk04 daspk04 Aug 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actually by default OTB has only core modules and all these options are OFF. They have mentioned in doc that incase one needs some module then one can turn on, but then that requires compiling again from source. I would prefer to keep these option as this is similar to as mentioned in doc, here I kept all ON by default (opposite from OTB) as compiling all takes a bit of time (~30min) incase one doesn't need some some module they can override.

Ideally I would prefer something like this in Nixpkgs so that one don't have to build or compile (possibly can be done later):

  • otb (full)
  • otbMinimal (only core)
  • otbSAR (otbMinimal + SAR related modules)
  • otbLearning (otbMinimal + Learning related modules)
  • ...

@SuperSandro2000
Copy link
Member

2. You may consider to use override to cover some Attr in pkgs/development/libraries/itk , rather than recreate a package . (maybe you could use it's generate.nix file ?, I not sure)

That would be nice but the files are already pretty different and maintaining backwards compatibility might be a pain in the future.

* init at an unstable version based on commit at 2024-05-25 as no more official release from 4.0.1 onwards.
@daspk04
Copy link
Author

daspk04 commented Aug 29, 2024

Hi @SuperSandro2000 !

I have applied all the suggested changes.

@daspk04
Copy link
Author

daspk04 commented Oct 5, 2024

Hi @SuperSandro2000 !

Will it possible if you could review once more ? I have already applied all of your suggested changes.

Copy link
Member

@FliegendeWurst FliegendeWurst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM mostly

nativeBuildInputs = [ cmake ];

meta = {
description = "Shark is a fast, modular, general open-source C++ machine learning library";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description = "Shark is a fast, modular, general open-source C++ machine learning library";
description = "Fast, modular, general open-source C++ machine learning library";

'';

meta = {
description = "Orfeo ToolBox";
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
description = "Orfeo ToolBox";
description = "Open Source processing of remote sensing images";

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
8.has: maintainer-list (update) This PR changes `maintainers/maintainer-list.nix` 8.has: package (new) This PR adds a new package 10.rebuild-darwin: 0 This PR does not cause any packages to rebuild 10.rebuild-linux: 1-10 11.by: package-maintainer This PR was created by the maintainer of the package it changes 12. first-time contribution This PR is the author's first one; please be gentle!
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants