More flexible tuple creation functions (3555)

[GalacticEmperor1]

Issue №3555 opened by itzpr3d4t0r at 2022-11-11 13:16:18

# # Idea

Recently we've merged PRs about new functions for efficiently creating tuples from two or three ints, (see # 3552 # 3401 # 3424) so instead of using Py_Buildvalue which is about 40 % slower than necessary we are now using two new functions that are

• pg_tuple_couple_from_values_int
• pg_tuple_triple_from_values_int

The issue with these is that we can't apply them other than for Tuple[int, int] or a Tuple[int, int, int], so I've been thinking of an efficient way to generalize these functions, so that instead of having to build 300 functions for each case we could simply have one that covers all possibilities.

With this i immediately came up with the va_args path ( which Py_BuildValue utilizes already though with much more complex structure), basically the new function will accept a variable number of arguments, be as efficient as a dedicated one and still 40% faster than Py_BuildValue. I've already tested it and all of these statements are true.

This is the function, it takes the number of elements as first parameter and then a sequence of params:

static PG_INLINE PyObject *
pg_tuple_from_ints(Py_ssize_t len, ...)
{
    /* This function turns a variable number of input integers into a python
     * tuple object. Currently, 5th November 2022, this is faster than using
     * Py_BuildValue to do the same thing.
     */
    PyObject *tup = PyTuple_New(len);
    if (!tup) {
        return NULL;
    }

    va_list args;
    va_start(args, len);
    Py_ssize_t i;
    for (i = 0; i < len; i++) {
        PyObject *tmp = PyLong_FromLong(va_arg(args, int));
        if (!tmp) {
            Py_DECREF(tup);
            va_end(args);
            return NULL;
        }
        PyTuple_SET_ITEM(tup, i, tmp);
    }
    va_end(args);

    return tup;
}

And we could easily have one for doubles, PyObject* and much more. The one for PyObject* would be like this

static PG_INLINE PyObject *
pg_tuple_from_PyObjects(Py_ssize_t len, ...)
{
    /* This function turns a variable number of input PyObject pointers into a
     * python tuple object. Currently, 5th November 2022, this is faster than
     * using Py_BuildValue to do the same thing.
     */
    PyObject *tup = PyTuple_New(len);
    if (!tup) {
        return NULL;
    }

    va_list args;
    va_start(args, len);
    Py_ssize_t i;
    for (i = 0; i < len; i++) {
        PyObject *tmp = va_arg(args, PyObject *);
        Py_INCREF(tmp);
        PyTuple_SET_ITEM(tup, i, tmp);
    }
    va_end(args);

    return tup;
}

# # Concerns

1. The programmer would have to be careful with its use and be certain that each value passed to the function is of the correct type as the behaviour of va_arg() (the function that actually extracts the object from the sequence) has undefined behaviour if the types don't match (see https://en.cppreference.com/w/c/variadic/va_arg). I assume this is the same for Py_BuildValue so we should be safe for that.
2. We woudn't be able to apply these functions for heterogeneous sequences of parameters as va_arg requires a predefined type that cannot change so creating tuples with different types is not possible with a simple implementation and would require more research.

# # Positives

1. Code clarity and simpler to use
2. Less code to maintain
3. More flexible use

an example, this is from color slicing:

static PyObject *
_color_slice(register pgColorObject *a, register Py_ssize_t ilow,
             register Py_ssize_t ihigh) {
  ...
   
  if (len == 4) {
        return pg_tuple_from_ints(4, c1, c2, c3, c4);
    }
    else if (len == 3) {
        return pg_tuple_from_ints(3, c1, c2, c3);
    }
    else if (len == 2) {
        return pg_tuple_from_ints(2, c1, c2);
    }
    else if (len == 1) {
        return pg_tuple_from_ints(1, c1);
    }
    else {
        return PyTuple_New(0);
    }
  
}

---

Comments

# # itzpr3d4t0r commented at 2022-11-11 13:35:03

PS. this can be used for lists as well ofc.

---

# # PurityLake commented at 2022-12-25 01:43:19

Could be a little bit more of setup BUT could we not do something like this:

static PG_INLINE PyObject *
pg_tuple_from_ints(Py_ssize_t len, int *args)
{
    /* This function turns a variable number of input integers into a python
     * tuple object. Currently, 5th November 2022, this is faster than using
     * Py_BuildValue to do the same thing.
     */
    PyObject *tup = PyTuple_New(len);
    if (!tup) {
        return NULL;
    }

    Py_ssize_t i;
    for (i = 0; i < len; i++) {
        PyObject *tmp = PyLong_FromLong(args[i]);
        if (!tmp) {
            Py_DECREF(tup);
            return NULL;
        }
        PyTuple_SET_ITEM(tup, i, tmp);
    }

    return tup;
}

With an example usage of maybe:

static PyObject *
_color_slice(register pgColorObject *a, register Py_ssize_t ilow,
             register Py_ssize_t ihigh) {
  ...
   
  if (len == 4) {
        int *args = (int[]){c1, c2, c3, c4};
        return pg_tuple_from_ints(4, args);
    }
    else if (len == 3) {
        int *args = (int[]){c1, c2, c3};
        return pg_tuple_from_ints(3, args);
    }

It still provides a way to create these tuples and makes things type safe. Afaik this is valid C99 code