How to allow a package completely?

[wildsonc]

BUG/PROBLEM REPORT / FEATURE REQUEST

What I did:

Added package to safe_globals:

import dateutil

safe_globals = {
# Others
"dateutil": dateutil
}

Code

dateutil.parser.parse("2024-01-01").strftime("%d/%m/%Y")

What I expect to happen:

Result: "01/01/2024"

What actually happened:

Error: '__import__'

This code code works:

dateutil.parser.parse("2024-01-01").date()

If the method has any import, it is not allowed

[d-maurer]

Wil wrote at 2024-6-28 13:12 -0700:

... Error: '\_\_import\_\_' This code code works: ```python dateutil.parser.parse("2024-01-01").date() ```` If the method has any import, it is not allowed

This is surprising: If you make a module available to restricted code, then access to the its attributes (recursively) inside the restricted code is supervised by `RestrictedPython`. But any activity inside the module itself is not restricted. Especially, it should not matter whether the module imports anything. Your problem description is too terse. Please provide a complete piece of code which reproduces the problem. I.e. your code should show * your additions to `safe_globals` * which restricted source you compile in what way * how you execute the restricted code

[wildsonc]

import json
import re
from datetime import datetime, time, timedelta, timezone
from typing import Any

import dateutil
import dateutil.parser
import requests
from core.plugins.salesforce import Salesforce
from RestrictedPython import (
    Eval,
    Guards,
    compile_restricted_function,
    limited_builtins,
    safe_builtins,
    utility_builtins,
)

from .utils import CompileError

allowed_imports = {
    "datetime": datetime,
    "dateutil": dateutil,
    "enumerate": enumerate,
    "format_str": lambda x, y: x.format(y),
    "strftime": lambda x, y: x.strftime(y),
    "json": json,
    "dict": dict,
    "min": min,
    "max": max,
    "all": all,
    "any": any,
    "sum": sum,
    "re": re,
    "requests": requests,
    "Salesforce": Salesforce,
    "time": time,
    "timedelta": timedelta,
    "timezone": timezone,
}

ALLOWED_BUILTINS = {}
ALLOWED_BUILTINS.update(safe_builtins)
ALLOWED_BUILTINS.update(limited_builtins)
ALLOWED_BUILTINS.update(utility_builtins)

safe_globals: dict[str, Any] = dict(
    __builtins__=ALLOWED_BUILTINS,
    _write_=Guards.full_write_guard,
    _getiter_=Eval.default_guarded_getiter,
    _iter_unpack_sequence_=Guards.guarded_iter_unpack_sequence,
    getattr=Guards.safer_getattr,
    setattr=Guards.guarded_setattr,
    delattr=Guards.guarded_delattr,
    **allowed_imports,
)


class Script:
    def __init__(self, *args, **kwargs) -> None:
        pass

    def run(self, code: str, parameters: dict = {}):
        if not code:
            return None

        function_name = "script"
        function_parameters = ", ".join(parameters.keys())
        compiled = compile_restricted_function(
            function_parameters,
            code,
            function_name,
            filename="<inline code>",
        )
        if compiled.errors:
            raise CompileError(compiled.errors)

        safe_locals = dict()

        exec(compiled.code, safe_globals, safe_locals)
        function = safe_locals[function_name]

        result = function(**parameters)
        return result

[d-maurer]

Wil wrote at 2024-6-28 13:58 -0700:

...

The reason why I asked for complete code is that I would like to run your code (in an environment with `RestrictedPYthon`) to reproduce the problem: i.e. "python <your code>` should reproduce the problem. This means, your code should only reference standard modules and packages (and e.g. not `core.plugins.salesform`). You have in the latest message provided some infrastructure -- but the concrete problematic code is not contained. Keep in mind: your code should reproduce the problem when run in an environment with `RestrictedPython` (and otherwise only standard packages).

[wildsonc]

Hey @d-maurer, try this version now:

import dateutil
from RestrictedPython import (
    Eval,
    Guards,
    compile_restricted,
    safe_builtins,
)


allowed_imports = {
    "dateutil": dateutil,
}


safe_global = dict(
    __builtins__=safe_builtins,
    _write_=Guards.full_write_guard,
    _getiter_=Eval.default_guarded_getiter,
    _iter_unpack_sequence_=Guards.guarded_iter_unpack_sequence,
    getattr=Guards.safer_getattr,
    setattr=Guards.guarded_setattr,
    delattr=Guards.guarded_delattr,
    **allowed_imports,
)

source_code = """
dt = dateutil.parser.parse("2024-01-01").strftime("%d/%m/%Y")
"""

# Working code
# source_code = """
# dt = dateutil.parser.parse("2024-01-01").date()
# """

compiled = compile_restricted(source_code)

safe_locals = dict()
exec(compiled, safe_global, safe_locals)
print(safe_locals["dt"])

[d-maurer]

Wil wrote at 2024-7-1 12:47 -0700:

Hey @d-maurer, try this version now: import dateutil ...

Almost (but not yet completely) there: apparently, `dateutil` does not belong to the standard Python library. I looked on `PyPI` but apparently there are several candidates. Which `dateutil` distribution do you use?

[wildsonc]

Sorry for the delay, I use this package:

https://github.com/dateutil/dateutil

An example using only datetime instead dateutil:

from datetime import datetime

from RestrictedPython import Eval, Guards, compile_restricted, safe_builtins

allowed_imports = {"datetime": datetime}


safe_global = dict(
    __builtins__=safe_builtins,
    _write_=Guards.full_write_guard,
    _getiter_=Eval.default_guarded_getiter,
    _iter_unpack_sequence_=Guards.guarded_iter_unpack_sequence,
    getattr=Guards.safer_getattr,
    setattr=Guards.guarded_setattr,
    delattr=Guards.guarded_delattr,
    **allowed_imports,
)

source_code = """
dt = datetime.now().strftime("%d/%m/%Y")
"""

compiled = compile_restricted(source_code)

safe_locals = dict()
exec(compiled, safe_global, safe_locals)
print(safe_locals["dt"])

[d-maurer]

Wil wrote at 2024-7-16 08:20 -0700:

... An example using only datetime instead dateutil: ...

A minimal example reproducing the problem is ```python from datetime import datetime from RestrictedPython import compile_restricted, safe_builtins safe_globals = dict(__builtins__=safe_builtins, strftime=datetime.now().strftime) code = compile_restricted("strftime('%Y')\n") exec(code, safe_globals) ``` The reason: `strftime` is implemented in "C", i.e. not a "normal" Python function. A normal Python function accesses globals (and among them builtins) via its `__globals__` attribute; functions implemented in "C" lack this possibility. The implementation of `strftime` tries to import Python's `time` module via a call to `__import__`. Because it lacks its own `__globals__` it tries to find `__import__` in the calling frame -- which happens in your case to be restricted and lacks `__import__`. `RestrictedPython` cannot do anything for this special situation. I see the following possibilities to work around the problem: * put an `__import__` definition in `safe_builtins` which allows to import the `time` module (and maybe reject any other import). * wrap the "C" function `strftime` into a normal Python function (defined in an unrestricted context).

[wildsonc]

Could you show me an example of how to implement import? Allowing to import only the time module

[d-maurer]

Wil wrote at 2024-7-22 05:22 -0700:

Could you show me an example of how to implement __import__? Allowing to import only the time module

No, I am not ready to do your work. But it is not difficult: you read the Python documentation how `__import__` is called and what it is expected to do. You do what is expected of `__import__` for the modules you would like to be importable (at least "time" in your case).

[wildsonc]

This way it worked as expected:

from RestrictedPython import compile_restricted, safe_builtins

allowed_imports = ["datetime", "time", "dateutil"]


def guarded_import(name, globals=None, locals=None, fromlist=(), level=0):
    base_name = name.split(".")[0]
    if base_name in allowed_imports:
        module = __import__(name, globals, locals, fromlist, level)
        return module
    raise ImportError(f"Import not allowed: {name}")


safe_builtins["__import__"] = guarded_import
safe_globals = dict(__builtins__=safe_builtins)

code = compile_restricted(
    'from dateutil.parser import parse\ndt = parse("2024-01-01").strftime("%d/%m/%Y")'
)
safe_locals = dict()
exec(code, safe_globals, safe_locals)
print(safe_locals["dt"])