adaptation to Python 3.9 #109
Conversation
| def DICT_UPDATE(self, instr): | ||
| v = self.pop() | ||
| assert instr.argval > 0 | ||
| dict.update(self._stack[-instr.arg].value, v) |
There was a problem hiding this comment.
这里不应该直接操作 value，应该转发到 DictVariable 上的 update 方法（可参考 keys 等方法的实现）
而且该操作会有 SideEffects 产生，该机制尚未完成，可以在相应函数处记录 TODO
There was a problem hiding this comment.
我尝试加了一个override_method_update
但是他会有如下报错, 可能是我哪个类型没写对吗，还是啥
symbolic_trace.utils.exceptions.InnerError: DummyTracker has no instructionsThere was a problem hiding this comment.
这里是由于 Tracker 机制，需要 Tracker 的组网输入找不到来源导致的（DummyTracker 是没有来源的），可参考 BUILD_xxx_UNPACK 的处理方式，先 get_wrapped_items 以确保数据是有 Tracker 的
There was a problem hiding this comment.
可参考 https://github.com/2742195759/paddle-symbolic-trace/blob/add-contributing-md/CONTRIBUTING.md#guard-%E5%8F%8A%E7%BC%93%E5%AD%98%E6%9C%BA%E5%88%B6 配置下 Tracker 可视化来理解，分别在 python 3.8 和 python 3.9 可视化下，可以看到两者 Tracker 的差异
3.8:
3.9(本 PR):
修改后两者 Tracker 的图应该是一致的
gouzil
changed the title
| runs-on: ubuntu-latest | ||
| name: Run unit tests and examples | ||
| strategy: | ||
| fail-fast: false |
There was a problem hiding this comment.
如果任何矩阵作业失败，GitHub 将不会取消所有正在进行的作业, 说明链接
gouzil
changed the title
| @@ -24,6 +24,9 @@ def get_paddle_api(): | |||
| for fn_name in getattr(module, "__all__", []): | |||
| fn = getattr(module, fn_name) | |||
| if inspect.isfunction(fn): | |||
| # NOTE: Maybe add a whitelist list | |||
| if fn.__name__ == "in_dygraph_mode": | |||
| continue | |||
…o add_opcode_list_dict
| dict.update(retval, self._stack[-instr.arg].get_wrapped_items()) | ||
| self._stack[-instr.arg] = DictVariable( | ||
| retval, self._graph, DummyTracker(dict_value) | ||
| ) |
There was a problem hiding this comment.
返回的 DictVariable 与原来的是不共享数据的吗？这是符合预期的吗？
There was a problem hiding this comment.
还是不建议在 OpCodeExecutor 里直接操作 value，这也做不到可复用，比如用户调用了 d1.update(d2)，仍然需要重新写一遍相同的逻辑，list.extend 有同样的问题
| ) | ||
|
|
||
| # Like DICT_UPDATE but raises an exception for duplicate keys. | ||
| DICT_MERGE = DICT_UPDATE |
There was a problem hiding this comment.
but raises an exception for duplicate keys.
这一点是如何体现的呢？另外新增的字节码也应该增加单测来确保正确性
There was a problem hiding this comment.
DICT_MERGE文档, 这个test_10_build_unpack就会测试到，以及paddle/nn/layer/layers下的_build_once也会使用到
测试值为{"a": a, "b": b}, {"a": c, "b": d}
按照之前的想法如果key重复会直接被CALL_FUNCTION_EX拦截, 这样其实是不太合理的
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/opcode_executor.py", line 306, in run
is_stop = self.step(cur_instr)
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/opcode_executor.py", line 314, in step
return getattr(self, instr.opname)(instr) # run single step.
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/opcode_executor.py", line 669, in CALL_FUNCTION_EX
ret = fn(*args, **kwargs)
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/variables.py", line 718, in __call__
return self.call_function(*args, **kwargs)
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/variables.py", line 795, in call_function
inline_executor = OpcodeInlineExecutor(self, *args, **kwargs)
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/opcode_inline_executor.py", line 47, in __init__
self._prepare_locals(*args, **kwargs)
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/opcode_inline_executor.py", line 54, in _prepare_locals
bound_args = sig.bind(*args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/inspect.py", line 3045, in bind
return self._bind(args, kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.9/lib/python3.9/inspect.py", line 3015, in _bind
raise TypeError('missing a required argument: {arg!r}'. \
TypeError: missing a required argument: 'c'已修复为:
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/opcode_executor.py", line 306, in run
is_stop = self.step(cur_instr)
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/opcode_executor.py", line 314, in step
return getattr(self, instr.opname)(instr) # run single step.
File "/Users/gouzi/Documents/git/paddle-symbolic-trace/symbolic_trace/opcode_translator/executor/opcode_executor.py", line 948, in DICT_MERGE
assert result == None, f"Repeat key: {key}"
AssertionError: Repeat key: aThere was a problem hiding this comment.
这里报 InnerError 吧，我们之后会统一规范下报错体系书写方式 #114
另外，报错信息和运行时错误保持一致：
TypeError: __main__.foo() got multiple values for keyword argument 'a'
TypeError: __main__.foo() 这部分我们可以暂时先不考虑，后面的报错信息保持一致吧
| list_value = self.pop() | ||
| self.push( | ||
| TupleVariable( | ||
| list_value.value, self._graph, DummyTracker([instr.argval]) |
There was a problem hiding this comment.
这里同样可以 get_wrapped_items 一下确保 Tracker 保留
|
#110 已 merge，麻烦解决下冲突～ |
…o add_opcode_list_dict # Conflicts: # symbolic_trace/opcode_translator/executor/variables.py
Done |
| # It will only be 0 or 1 | ||
| assert instr.argval == 0 or instr.argval == 1 | ||
| left = self.pop() | ||
| right = self.pop() |
There was a problem hiding this comment.
这里pop的顺序是不是有点问题？先pop出来的应该是right元素吧
There was a problem hiding this comment.
@gouzil 编写字节码模拟执行代码时候可以参考 cpython 的 ceval.c，我们在实现的时候也是优先参考和对齐它的：
https://github.com/python/cpython/blob/3.9/Python/ceval.c#L3019-L3031
（虽然 IS_OP 左右应该没太大问题，但最好还是对齐比较好）
There was a problem hiding this comment.
是的是的，已修复，这个跟COMPARE_OP其实还是挺像的
There was a problem hiding this comment.
IS_OP 应该是 3.9 版本从 COMPARE_OP 里抽离出来的部分操作，3.8 的 is/is not 是在 COMPARE_OP 里的
增加了
[DICT_UPDATE, DICT_MERGE, LIST_EXTEND, LIST_TO_TUPLE, IS_OP]适配python3.9
closes #105