Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENH] Add the option to turn off all type checking and conversion and go straight to _fit/_predict/_transform #1799

Closed
TonyBagnall opened this issue Jul 14, 2024 · 1 comment
Labels
enhancement New feature, improvement request or other non-bug code enhancement

Comments

@TonyBagnall
Copy link
Contributor

TonyBagnall commented Jul 14, 2024

Describe the feature or idea you want to propose

in estimator base classes (looking at BaseCollectionTransformer but same elsewhere) we do checks, get meta data and convert to inner type. All good, but it does introduce an overhead. For example, this is minirocket with and without the checks

the checks involve several scans of the data. For larger data it would be good to have an option to just go straight to inner methods and fail without informative error.

This is minirocket in safe and unsafe modes (time in secs, to transform length 500 series varying number of cases). Note the difference is linear in n

<style> </style>
  With Unsafe Diff
500 0.49 0.16 -0.33
1000 0.88 0.20 -0.68
1500 1.30 0.26 -1.04
2000 1.74 0.32 -1.43
2500 2.02 0.49 -1.53
3000 2.46 0.44 -2.02
3500 2.81 0.49 -2.32
4000 3.19 0.54 -2.65
4500 3.66 0.60 -3.06
5000 4.08 0.73 -3.35
5500 4.57 0.75 -3.82
6000 4.75 0.83 -3.92
6500 5.17 0.83 -4.33
7000 5.80 0.95 -4.85
7500 6.14 0.99 -5.15
8000 6.51 1.01 -5.50
8500 6.85 1.05 -5.80
9000 7.33 1.13 -6.21
9500 7.55 1.23 -6.32
10000 7.92 1.22 -6.70

Describe your proposed solution

I would have a parameter in constructor, say "unsafe", that defaults to False, then just test in fit etc.

    def fit(self, X, y=None):
          if self.unsafe:
                 self._fit(X,y)
                 return self
        if self.get_tag("requires_y"):
            if y is None:
                raise ValueError("Tag requires_y is true, but fit called with y=None")
        # skip the rest if fit_is_empty is True
        if self.get_tag("fit_is_empty"):
            self._is_fitted = True
            return self
        self.reset()

etc

Describe alternatives you've considered, if relevant

No response

Additional context

No response

@TonyBagnall TonyBagnall added the enhancement New feature, improvement request or other non-bug code enhancement label Jul 14, 2024
@TonyBagnall
Copy link
Contributor Author

ok can ignore this, it was in fact an internal bug in mini rocket :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature, improvement request or other non-bug code enhancement
Projects
None yet
Development

No branches or pull requests

1 participant