Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implemented remaining Python builtins #268

Merged
merged 5 commits into from
Feb 21, 2025
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
181 changes: 179 additions & 2 deletions documentation/functions.md
Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,18 @@ Below is the list of every function/operator currently supported in PyDough as a
- [Window Functions](#window-functions)
* [RANKING](#ranking)
* [PERCENTILE](#percentile)
- [Unsupported Magic Methods](#unsupported-magic-methods)
* [FLOOR](#floor)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change all of these to the same format as the magic methods: FLOOR -> __floor__

* [CEIL](#ceil)
* [TRUNC](#trunc)
* [REVERSED](#reversed)
* [INT](#int)
* [FLOAT](#float)
* [COMPLEX](#complex)
* [INDEX](#index)
* [LEN](#len)
* [CONTAINS](#contains)
* [SETITEM](#setitem)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Include any other unsupported ones (__call__ is only for function calls, __bool__ is banned for x and y)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will we be supporting both .CALCULATE() and conventional syntax of just ()? In that case I'll mention that.


<!-- TOC end -->

Expand Down Expand Up @@ -401,19 +413,23 @@ Below is each numerical function currently supported in PyDough.
<!-- TOC --><a name="abs"></a>
### ABS

The `ABS` function returns the absolute value of its input.
The `ABS` function returns the absolute value of its input. The `abs()` magic method is also evaluated to the same.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to "The Python builtin abs() function can also be used to accomplish the same thing."


```py
Customers(acct_magnitude = ABS(acctbal))
# The below statement is equivalent to above.
Customers(acct_magnitude = abs(acctbal))
```

<!-- TOC --><a name="round"></a>
### ROUND

The `ROUND` function rounds its first argument to the precision of its second argument. The rounding rules used depend on the database's round function.
The `ROUND` function rounds its first argument to the precision of its second argument. The rounding rules used depend on the database's round function. The `round()` magic method is also evaluated to the same.

```py
Parts(rounded_price = ROUND(retail_price, 1))
# The below statement is equivalent to above.
Parts(rounded_price = round(retail_price, 1))
```

<!-- TOC --><a name="power"></a>
Expand Down Expand Up @@ -580,3 +596,164 @@ Customers.WHERE(PERCENTILE(by=acctbal.ASC(), n_buckets=1000) == 1000)
# For every region, find the top 5% of customers with the highest account balances.
Regions.nations.customers.WHERE(PERCENTILE(by=acctbal.ASC(), levels=2) > 95)
```
## Unsupported Magic Methods

Below is a list of magic methods that are not supported in PyDough. Calling these methods will result in an Exception.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Magic methods is not a terminology that our users will likely understand, since we are not targeting advanced Python developers. We should simplify this to just be about banned Python logic.


<!-- TOC --><a name="floor"></a>
### FLOOR

The `math.floor` function calls the `__floor__` magic method, which is not supported in PyDough.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not CURRENTLY supported


```py
def bad_floor_1():
# Using `math.floor` (calls __floor__)
return Customer(age=math.floor(order.total_price))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor all of these examples to match the format of other examples in the file (just a single line of code, not a function with a return statement)

```

<!-- TOC --><a name="ceil"></a>
### CEIL

The `math.ceil` function calls the `__ceil__` magic method, which is not supported in PyDough.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not CURRENTLY supported


```py
def bad_ceil_1():
# Using `math.ceil` (calls __ceil__)
return Customer(age=math.ceil(order.total_price))
```

<!-- TOC --><a name="trunc"></a>
### TRUNC

The `math.trunc` function calls the `__trunc__` magic method, which is not supported in PyDough.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not CURRENTLY supported


```py
def bad_trunc_1():
# Using `math.trunc` (calls __trunc__)
return Customer(age=math.trunc(order.total_price))
```

<!-- TOC --><a name="reversed"></a>
### REVERSED

The `reversed` function calls the `__reversed__` magic method, which is not supported in PyDough.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not CURRENTLY supported


```py
def bad_reversed_1():
# Using `reversed` (calls __reversed__)
return Orders(reversed(order_key))
```

<!-- TOC --><a name="int"></a>
### INT

Casting to `int` calls the `__int__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an integer instead of a PyDough object.

```py
def bad_int_1():
# Casting to int (calls __int__)
return Orders(limit=int(order.total_price))
```

<!-- TOC --><a name="float"></a>
### FLOAT

Casting to `float` calls the `__float__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return a float instead of a PyDough object.

```py
def bad_float_1():
# Casting to float (calls __float__)
return Orders(limit=float(order.quantity))
```

<!-- TOC --><a name="complex"></a>
### COMPLEX

Casting to `complex` calls the `__complex__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return a complex instead of a PyDough object.

```py
def bad_complex_1():
# Casting to complex (calls __complex__)
return Orders(limit=complex(order.total_price))
```

<!-- TOC --><a name="index"></a>
### INDEX

Using an object as an index calls the `__index__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an integer instead of a PyDough object.

```py
def bad_index_1():
# Using as an index (calls __index__)
return Customers(sliced = name[:order])
```

<!-- TOC --><a name="nonzero"></a>
### NONZERO

Using an object in a boolean context calls the `__nonzero__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an integer instead of a PyDough object.

```py
def bad_nonzero_1():
# Using in a boolean context (calls __nonzero__)
return Orders(discount=not order.total_price)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bool(order.total_price) would be a better example

```

<!-- TOC --><a name="len"></a>
### LEN

The `len` function calls the `__len__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an integer instead of a PyDough object.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest using LENGTH instead.


```py
def bad_len_1():
# Using `len` (calls __len__)
return Customers(len(customer.name))
```

<!-- TOC --><a name="contains"></a>
### CONTAINS

Using the `in` operator calls the `__contains__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return a boolean instead of a PyDough object.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest using ISIN instead


```py
def bad_contains_1():
# Using `in` operator (calls __contains__)
return Orders('discount' in order.details)
```

<!-- TOC --><a name="setitem"></a>
### SETITEM

Assigning to an index calls the `__setitem__` magic method, which is not supported in PyDough. This operation is not allowed.

```py
def bad_setitem_1():
# Assigning to an index (calls __setitem__)
order.details['discount'] = True
return order
```

<!-- TOC --><a name="iter"></a>
### ITER

Iterating over an object calls the `__iter__` magic method, which is not supported in PyDough. This operation is not allowed because the implementation has to return an iterator instead of a PyDough object.

```py
def bad_iter_1():
# Iterating over an object (calls __iter__)
for item in customer:
print(item)
return customer

def bad_iter_2():
# Using list comprehension (calls __iter__)
return [item for item in customer]

def bad_iter_3():
# Using list() constructor (calls __iter__)
return list(customer)

def bad_iter_4():
# Using tuple() constructor (calls __iter__)
return tuple(customer)
```
67 changes: 67 additions & 0 deletions pydough/unqualified/unqualified_node.py
Original file line number Diff line number Diff line change
Expand Up @@ -124,6 +124,10 @@ def __getitem__(self, key):
args: MutableSequence[UnqualifiedNode] = [self]
for arg in (key.start, key.stop, key.step):
coerced_elem = UnqualifiedNode.coerce_to_unqualified(arg)
if not isinstance(coerced_elem, UnqualifiedLiteral):
raise PyDoughUnqualifiedException(
"PyDough objects cannot be used as indices in Python slices."
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: "are not currently supported"

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we going to support something like this: Customers[:orders] ?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

At the moment, no. Eventually, 🤷?

)
args.append(coerced_elem)
return UnqualifiedOperation("SLICE", args)
else:
Expand Down Expand Up @@ -260,6 +264,69 @@ def __call__(self, *args, **kwargs: dict[str, object]):
calc_args.append((name, self.coerce_to_unqualified(arg)))
return UnqualifiedCalc(self, calc_args)

def __abs__(self):
return UnqualifiedOperation("ABS", [self])

def __round__(self, n):
if n is None:
raise PyDoughUnqualifiedException(
"PyDough requires a specific number of decimal places for rounding."
)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can just change this so the default is 0, like the normal Python behavior.

n_unqualified = self.coerce_to_unqualified(n)
return UnqualifiedOperation("ROUND", [self, n_unqualified])

def __floor__(self):
raise PyDoughUnqualifiedException(
"PyDough does not support the math.floor function at this time."
)

def __ceil__(self):
raise PyDoughUnqualifiedException(
"PyDough does not support the math.ceil function at this time."
)

def __trunc__(self):
raise PyDoughUnqualifiedException(
"PyDough does not support the math.trunc function at this time."
)

def __reversed__(self):
raise PyDoughUnqualifiedException(
"PyDough does not support the reversed function at this time."
)

def __int__(self):
raise PyDoughUnqualifiedException("PyDough objects cannot be cast to int.")

def __float__(self):
raise PyDoughUnqualifiedException("PyDough objects cannot be cast to float.")

def __complex__(self):
raise PyDoughUnqualifiedException("PyDough objects cannot be cast to complex.")

def __index__(self):
raise PyDoughUnqualifiedException(
"PyDough objects cannot be used as indices in Python slices."
)

def __nonzero__(self):
return self.__bool__()

def __len__(self):
raise PyDoughUnqualifiedException(
"PyDough objects cannot be used with the len function."
)

def __contains__(self, item):
raise PyDoughUnqualifiedException(
"PyDough objects cannot be used with the 'in' operator."
)

def __setitem__(self, key, value):
raise PyDoughUnqualifiedException(
"PyDough objects cannot support item assignment."
)

def WHERE(self, cond: object) -> "UnqualifiedWhere":
cond_unqualified: UnqualifiedNode = self.coerce_to_unqualified(cond)
return UnqualifiedWhere(self, cond_unqualified)
Expand Down
70 changes: 70 additions & 0 deletions tests/bad_pydough_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@
# mypy: ignore-errors
# ruff & mypy should not try to typecheck or verify any of this

import math


def bad_bool_1():
# Using `or`
Expand Down Expand Up @@ -73,3 +75,71 @@ def bad_slice_3():
def bad_slice_4():
# Unsupported slicing: reversed
return Customers(name[::-1])


def bad_floor_1():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIT: don't use _1 if it is the only one of its kind.

# Using `math.floor` (calls __floor__)
return Customer(age=math.floor(order.total_price))


def bad_ceil_1():
# Using `math.ceil` (calls __ceil__)
return Customer(age=math.ceil(order.total_price))


def bad_trunc_1():
# Using `math.trunc` (calls __trunc__)
return Customer(age=math.trunc(order.total_price))


def bad_reversed_1():
# Using `reversed` (calls __reversed__)
return Orders(reversed(order_key))


def bad_int_1():
# Casting to int (calls __int__)
return Orders(limit=int(order.total_price))


def bad_float_1():
# Casting to float (calls __float__)
return Orders(limit=float(order.quantity))


def bad_complex_1():
# Casting to complex (calls __complex__)
return Orders(limit=complex(order.total_price))


def bad_index_1():
# Using as an index (calls __index__)
return Customers(sliced=name[:order])


def bad_nonzero_1():
# Using in a boolean context (calls __nonzero__)
return Orders(discount=not order.total_price)


def bad_len_1():
# Using `len` (calls __len__)
return Customers(len(customer.name))


def bad_contains_1():
# Using `in` operator (calls __contains__)
return Orders("discount" in order.details)


def bad_setitem_1():
# Assigning to an index (calls __setitem__)
order.details["discount"] = True
return order


def bad_iter_1():
# Iterating over an object (calls __iter__)
for item in customer:
print(item)
return customer
4 changes: 4 additions & 0 deletions tests/simple_pydough_functions.py
Original file line number Diff line number Diff line change
Expand Up @@ -391,6 +391,10 @@ def annotated_assignment():
return Nations.WHERE(region.name == chosen_region)


def abs_round_magic_method():
return DailyPrices(abs_low=abs(low), round_low=round(low, 2))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also test round(low)



def years_months_days_hours_datediff():
y1_datetime = datetime.datetime(2025, 5, 2, 11, 00, 0)
return Transactions.WHERE((YEAR(date_time) < 2025))(
Expand Down
Loading