-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implemented remaining Python builtins #268
Changes from 2 commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -54,6 +54,18 @@ Below is the list of every function/operator currently supported in PyDough as a | |
- [Window Functions](#window-functions) | ||
* [RANKING](#ranking) | ||
* [PERCENTILE](#percentile) | ||
- [Unsupported Magic Methods](#unsupported-magic-methods) | ||
* [FLOOR](#floor) | ||
* [CEIL](#ceil) | ||
* [TRUNC](#trunc) | ||
* [REVERSED](#reversed) | ||
* [INT](#int) | ||
* [FLOAT](#float) | ||
* [COMPLEX](#complex) | ||
* [INDEX](#index) | ||
* [LEN](#len) | ||
* [CONTAINS](#contains) | ||
* [SETITEM](#setitem) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Include any other unsupported ones ( There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Will we be supporting both .CALCULATE() and conventional syntax of just ()? In that case I'll mention that. |
||
|
||
<!-- TOC end --> | ||
|
||
|
@@ -401,19 +413,23 @@ Below is each numerical function currently supported in PyDough. | |
<!-- TOC --><a name="abs"></a> | ||
### ABS | ||
|
||
The `ABS` function returns the absolute value of its input. | ||
The `ABS` function returns the absolute value of its input. The `abs()` magic method is also evaluated to the same. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Change to "The Python builtin |
||
|
||
```py | ||
Customers(acct_magnitude = ABS(acctbal)) | ||
# The below statement is equivalent to above. | ||
Customers(acct_magnitude = abs(acctbal)) | ||
``` | ||
|
||
<!-- TOC --><a name="round"></a> | ||
### ROUND | ||
|
||
The `ROUND` function rounds its first argument to the precision of its second argument. The rounding rules used depend on the database's round function. | ||
The `ROUND` function rounds its first argument to the precision of its second argument. The rounding rules used depend on the database's round function. The `round()` magic method is also evaluated to the same. | ||
|
||
```py | ||
Parts(rounded_price = ROUND(retail_price, 1)) | ||
# The below statement is equivalent to above. | ||
Parts(rounded_price = round(retail_price, 1)) | ||
``` | ||
|
||
<!-- TOC --><a name="power"></a> | ||
|
@@ -580,3 +596,164 @@ Customers.WHERE(PERCENTILE(by=acctbal.ASC(), n_buckets=1000) == 1000) | |
# For every region, find the top 5% of customers with the highest account balances. | ||
Regions.nations.customers.WHERE(PERCENTILE(by=acctbal.ASC(), levels=2) > 95) | ||
``` | ||
## Unsupported Magic Methods | ||
|
||
Below is a list of magic methods that are not supported in PyDough. Calling these methods will result in an Exception. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Magic methods is not a terminology that our users will likely understand, since we are not targeting advanced Python developers. We should simplify this to just be about banned Python logic. |
||
|
||
<!-- TOC --><a name="floor"></a> | ||
### FLOOR | ||
|
||
The `math.floor` function calls the `__floor__` magic method, which is not supported in PyDough. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not CURRENTLY supported |
||
|
||
```py | ||
def bad_floor_1(): | ||
# Using `math.floor` (calls __floor__) | ||
return Customer(age=math.floor(order.total_price)) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Refactor all of these examples to match the format of other examples in the file (just a single line of code, not a function with a return statement) |
||
``` | ||
|
||
<!-- TOC --><a name="ceil"></a> | ||
### CEIL | ||
|
||
The `math.ceil` function calls the `__ceil__` magic method, which is not supported in PyDough. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not CURRENTLY supported |
||
|
||
```py | ||
def bad_ceil_1(): | ||
# Using `math.ceil` (calls __ceil__) | ||
return Customer(age=math.ceil(order.total_price)) | ||
``` | ||
|
||
<!-- TOC --><a name="trunc"></a> | ||
### TRUNC | ||
|
||
The `math.trunc` function calls the `__trunc__` magic method, which is not supported in PyDough. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not CURRENTLY supported |
||
|
||
```py | ||
def bad_trunc_1(): | ||
# Using `math.trunc` (calls __trunc__) | ||
return Customer(age=math.trunc(order.total_price)) | ||
``` | ||
|
||
<!-- TOC --><a name="reversed"></a> | ||
### REVERSED | ||
|
||
The `reversed` function calls the `__reversed__` magic method, which is not supported in PyDough. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not CURRENTLY supported |
||
|
||
```py | ||
def bad_reversed_1(): | ||
# Using `reversed` (calls __reversed__) | ||
return Orders(reversed(order_key)) | ||
``` | ||
|
||
<!-- TOC --><a name="int"></a> | ||
### INT | ||
|
||
Casting to `int` calls the `__int__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an integer instead of a PyDough object. | ||
|
||
```py | ||
def bad_int_1(): | ||
# Casting to int (calls __int__) | ||
return Orders(limit=int(order.total_price)) | ||
``` | ||
|
||
<!-- TOC --><a name="float"></a> | ||
### FLOAT | ||
|
||
Casting to `float` calls the `__float__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return a float instead of a PyDough object. | ||
|
||
```py | ||
def bad_float_1(): | ||
# Casting to float (calls __float__) | ||
return Orders(limit=float(order.quantity)) | ||
``` | ||
|
||
<!-- TOC --><a name="complex"></a> | ||
### COMPLEX | ||
|
||
Casting to `complex` calls the `__complex__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return a complex instead of a PyDough object. | ||
|
||
```py | ||
def bad_complex_1(): | ||
# Casting to complex (calls __complex__) | ||
return Orders(limit=complex(order.total_price)) | ||
``` | ||
|
||
<!-- TOC --><a name="index"></a> | ||
### INDEX | ||
|
||
Using an object as an index calls the `__index__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an integer instead of a PyDough object. | ||
|
||
```py | ||
def bad_index_1(): | ||
# Using as an index (calls __index__) | ||
return Customers(sliced = name[:order]) | ||
``` | ||
|
||
<!-- TOC --><a name="nonzero"></a> | ||
### NONZERO | ||
|
||
Using an object in a boolean context calls the `__nonzero__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an integer instead of a PyDough object. | ||
|
||
```py | ||
def bad_nonzero_1(): | ||
# Using in a boolean context (calls __nonzero__) | ||
return Orders(discount=not order.total_price) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
``` | ||
|
||
<!-- TOC --><a name="len"></a> | ||
### LEN | ||
|
||
The `len` function calls the `__len__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an integer instead of a PyDough object. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggest using |
||
|
||
```py | ||
def bad_len_1(): | ||
# Using `len` (calls __len__) | ||
return Customers(len(customer.name)) | ||
``` | ||
|
||
<!-- TOC --><a name="contains"></a> | ||
### CONTAINS | ||
|
||
Using the `in` operator calls the `__contains__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return a boolean instead of a PyDough object. | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggest using |
||
|
||
```py | ||
def bad_contains_1(): | ||
# Using `in` operator (calls __contains__) | ||
return Orders('discount' in order.details) | ||
``` | ||
|
||
<!-- TOC --><a name="setitem"></a> | ||
### SETITEM | ||
|
||
Assigning to an index calls the `__setitem__` magic method, which is not supported in PyDough. This operation is not allowed. | ||
|
||
```py | ||
def bad_setitem_1(): | ||
# Assigning to an index (calls __setitem__) | ||
order.details['discount'] = True | ||
return order | ||
``` | ||
|
||
<!-- TOC --><a name="iter"></a> | ||
### ITER | ||
|
||
Iterating over an object calls the `__iter__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an iterator instead of a PyDough object. | ||
|
||
```py | ||
def bad_iter_1(): | ||
# Iterating over an object (calls __iter__) | ||
for item in customer: | ||
print(item) | ||
return customer | ||
|
||
def bad_iter_2(): | ||
# Using list comprehension (calls __iter__) | ||
return [item for item in customer] | ||
|
||
def bad_iter_3(): | ||
# Using list() constructor (calls __iter__) | ||
return list(customer) | ||
|
||
def bad_iter_4(): | ||
# Using tuple() constructor (calls __iter__) | ||
return tuple(customer) | ||
``` |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -124,6 +124,10 @@ def __getitem__(self, key): | |
args: MutableSequence[UnqualifiedNode] = [self] | ||
for arg in (key.start, key.stop, key.step): | ||
coerced_elem = UnqualifiedNode.coerce_to_unqualified(arg) | ||
if not isinstance(coerced_elem, UnqualifiedLiteral): | ||
raise PyDoughUnqualifiedException( | ||
"PyDough objects cannot be used as indices in Python slices." | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. NIT: "are not currently supported" There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are we going to support something like this: Customers[:orders] ? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. At the moment, no. Eventually, 🤷? |
||
) | ||
args.append(coerced_elem) | ||
return UnqualifiedOperation("SLICE", args) | ||
else: | ||
|
@@ -260,6 +264,69 @@ def __call__(self, *args, **kwargs: dict[str, object]): | |
calc_args.append((name, self.coerce_to_unqualified(arg))) | ||
return UnqualifiedCalc(self, calc_args) | ||
|
||
def __abs__(self): | ||
return UnqualifiedOperation("ABS", [self]) | ||
|
||
def __round__(self, n): | ||
if n is None: | ||
raise PyDoughUnqualifiedException( | ||
"PyDough requires a specific number of decimal places for rounding." | ||
) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Can just change this so the default is 0, like the normal Python behavior. |
||
n_unqualified = self.coerce_to_unqualified(n) | ||
return UnqualifiedOperation("ROUND", [self, n_unqualified]) | ||
|
||
def __floor__(self): | ||
raise PyDoughUnqualifiedException( | ||
"PyDough does not support the math.floor function at this time." | ||
) | ||
|
||
def __ceil__(self): | ||
raise PyDoughUnqualifiedException( | ||
"PyDough does not support the math.ceil function at this time." | ||
) | ||
|
||
def __trunc__(self): | ||
raise PyDoughUnqualifiedException( | ||
"PyDough does not support the math.trunc function at this time." | ||
) | ||
|
||
def __reversed__(self): | ||
raise PyDoughUnqualifiedException( | ||
"PyDough does not support the reversed function at this time." | ||
) | ||
|
||
def __int__(self): | ||
raise PyDoughUnqualifiedException("PyDough objects cannot be cast to int.") | ||
|
||
def __float__(self): | ||
raise PyDoughUnqualifiedException("PyDough objects cannot be cast to float.") | ||
|
||
def __complex__(self): | ||
raise PyDoughUnqualifiedException("PyDough objects cannot be cast to complex.") | ||
|
||
def __index__(self): | ||
raise PyDoughUnqualifiedException( | ||
"PyDough objects cannot be used as indices in Python slices." | ||
) | ||
|
||
def __nonzero__(self): | ||
return self.__bool__() | ||
|
||
def __len__(self): | ||
raise PyDoughUnqualifiedException( | ||
"PyDough objects cannot be used with the len function." | ||
) | ||
|
||
def __contains__(self, item): | ||
raise PyDoughUnqualifiedException( | ||
"PyDough objects cannot be used with the 'in' operator." | ||
) | ||
|
||
def __setitem__(self, key, value): | ||
raise PyDoughUnqualifiedException( | ||
"PyDough objects cannot support item assignment." | ||
) | ||
|
||
def WHERE(self, cond: object) -> "UnqualifiedWhere": | ||
cond_unqualified: UnqualifiedNode = self.coerce_to_unqualified(cond) | ||
return UnqualifiedWhere(self, cond_unqualified) | ||
|
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,6 +2,8 @@ | |
# mypy: ignore-errors | ||
# ruff & mypy should not try to typecheck or verify any of this | ||
|
||
import math | ||
|
||
|
||
def bad_bool_1(): | ||
# Using `or` | ||
|
@@ -73,3 +75,71 @@ def bad_slice_3(): | |
def bad_slice_4(): | ||
# Unsupported slicing: reversed | ||
return Customers(name[::-1]) | ||
|
||
|
||
def bad_floor_1(): | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. NIT: don't use |
||
# Using `math.floor` (calls __floor__) | ||
return Customer(age=math.floor(order.total_price)) | ||
|
||
|
||
def bad_ceil_1(): | ||
# Using `math.ceil` (calls __ceil__) | ||
return Customer(age=math.ceil(order.total_price)) | ||
|
||
|
||
def bad_trunc_1(): | ||
# Using `math.trunc` (calls __trunc__) | ||
return Customer(age=math.trunc(order.total_price)) | ||
|
||
|
||
def bad_reversed_1(): | ||
# Using `reversed` (calls __reversed__) | ||
return Orders(reversed(order_key)) | ||
|
||
|
||
def bad_int_1(): | ||
# Casting to int (calls __int__) | ||
return Orders(limit=int(order.total_price)) | ||
|
||
|
||
def bad_float_1(): | ||
# Casting to float (calls __float__) | ||
return Orders(limit=float(order.quantity)) | ||
|
||
|
||
def bad_complex_1(): | ||
# Casting to complex (calls __complex__) | ||
return Orders(limit=complex(order.total_price)) | ||
|
||
|
||
def bad_index_1(): | ||
# Using as an index (calls __index__) | ||
return Customers(sliced=name[:order]) | ||
|
||
|
||
def bad_nonzero_1(): | ||
# Using in a boolean context (calls __nonzero__) | ||
return Orders(discount=not order.total_price) | ||
|
||
|
||
def bad_len_1(): | ||
# Using `len` (calls __len__) | ||
return Customers(len(customer.name)) | ||
|
||
|
||
def bad_contains_1(): | ||
# Using `in` operator (calls __contains__) | ||
return Orders("discount" in order.details) | ||
|
||
|
||
def bad_setitem_1(): | ||
# Assigning to an index (calls __setitem__) | ||
order.details["discount"] = True | ||
return order | ||
|
||
|
||
def bad_iter_1(): | ||
# Iterating over an object (calls __iter__) | ||
for item in customer: | ||
print(item) | ||
return customer |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -391,6 +391,10 @@ def annotated_assignment(): | |
return Nations.WHERE(region.name == chosen_region) | ||
|
||
|
||
def abs_round_magic_method(): | ||
return DailyPrices(abs_low=abs(low), round_low=round(low, 2)) | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Also test |
||
|
||
|
||
def years_months_days_hours_datediff(): | ||
y1_datetime = datetime.datetime(2025, 5, 2, 11, 00, 0) | ||
return Transactions.WHERE((YEAR(date_time) < 2025))( | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Change all of these to the same format as the magic methods:
FLOOR
->__floor__