Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(udf): support both ibis and original types #9937

Closed
1 task done
NickCrews opened this issue Aug 27, 2024 · 2 comments
Closed
1 task done

feat(udf): support both ibis and original types #9937

NickCrews opened this issue Aug 27, 2024 · 2 comments
Labels
feature Features or general enhancements

Comments

@NickCrews
Copy link
Contributor

Is your feature request related to a problem?

consider

import ibis

@ibis.udf.scalar.python
def add_one(x: int) -> int:
    return x + 1

add_one(ibis.literal(3))  # ibis.expr.types.numeric.IntegerScalar
add_one(3) # ibis.expr.types.numeric.IntegerScalar

Can we make it so that add_one(3) returns a vanilla python int? This would really help make my UDFs polymorphic out of the box.

I have this sense of deja vu I asked about this before, but I just searched the issues and didn't find anything. Apologize if so!

What is the motivation behind your request?

No response

Describe the solution you'd like

Currently I have this workaround:

def flexiblize_udfs(udf):
    def inner(x):
        if isinstance(x, (ibis.Value, ibis.Deferred)):
            return udf(x)
        else:
            return udf.__wrapped__(x)

    return inner

flexiblize_udfs(add_one)(3)  # vanilla python 4 

I think the logic I have above should suffice? The semantics of the UDF decorator is now "add udf support to the given function" instead of "turn the supplied function into a udf".

What version of ibis are you running?

main

What backend(s) are you using, if any?

No response

Code of Conduct

  • I agree to follow this project's Code of Conduct
@NickCrews NickCrews added the feature Features or general enhancements label Aug 27, 2024
@cpcloud
Copy link
Member

cpcloud commented Aug 28, 2024

I think this opens a large can of worms where we now have to think about local execution even in the case of a UDF running on a remote backend.

Hypothetically, what if some expression is evaluated locally in Python 3.10 and then remotely in Python 3.12 and there are dependencies in the mix, which might differ, we have to at least be forced to think about that and probably document it, whereas we can simply not do it and we don't have to worry about the environment disparity.

@cpcloud cpcloud closed this as not planned Won't fix, can't repro, duplicate, stale Aug 28, 2024
@github-project-automation github-project-automation bot moved this from backlog to done in Ibis planning and roadmap Aug 28, 2024
@NickCrews
Copy link
Contributor Author

I didn't think about that, I only was thinking about my duckdb experience. That makes sense, I agree that potential inconsistency seems dangerous. My wrapper is a pretty easy workaround. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature Features or general enhancements
Projects
Archived in project
Development

No branches or pull requests

2 participants