Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add parameters input for non-tensor model inputs #670

Open
FynnBe opened this issue Nov 29, 2024 · 2 comments
Open

Add parameters input for non-tensor model inputs #670

FynnBe opened this issue Nov 29, 2024 · 2 comments
Labels
enhancement New feature or request help wanted Extra attention is needed question Further information is requested

Comments

@FynnBe
Copy link
Member

FynnBe commented Nov 29, 2024

The most recent time this came up is bioimage-io/core-bioimage-io-python#439

Some models may accept additional parameters after initialization as part of the forward method. (Not all model key word arguments would be suitable as sample parameters!)

What I have in mind when thinking about this is roughly the following

inputs:
  - id: tensor_as_ususal
    description: `An input tensor as the spec allows today already`
    axes: ...
  - parameters:
      - name: diameter
        description: "expected diameter in pixels of objects to segment"
        default: 30     

Would be pretty straight forward to add to spec and core, but we should collect some more actual use-cases to discuss if (and how) we want to add this and how challenging it would be to support this in other partner software.
One approach to make adaption easier would be that all parameters have to be optional and some software just doesn't allow users to change those default values...

cc @bioimage-io/spec-dev

@FynnBe FynnBe added enhancement New feature or request help wanted Extra attention is needed question Further information is requested labels Nov 29, 2024
@mese79
Copy link
Contributor

mese79 commented Nov 30, 2024

If those parameters are related to the model itself, it's better to have them in the model section rather than inputs.
Also, some models accept arguments for initialization, and setting them after couldn't be effective. Right now, we have them as kwargs in the model section, but I'm wondering if it is possible to init the model after loading via core.

model = load_description(...)
model.init(a=1, b=2, c="3")

@FynnBe
Copy link
Member Author

FynnBe commented Dec 2, 2024

those lines are really doing different things. here is some pseudo code to clear things up:

model_descr = load_description()  # spec
assert isinstance(model_descr, ModelDescr)

# create adapter that initializes the model
adapter = create_model_adapter(model_descr)  # core  

adapter.forward(...)

nn_module = PytorchAdapter.get_network(model_descr.weights.pytorch_state_dict)  # or get the nn_module directly (only for pytorch state dict..)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request help wanted Extra attention is needed question Further information is requested
Projects
None yet
Development

No branches or pull requests

2 participants