Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DRAFT] Support for TruffleRuby/GraalVM #125

Draft
wants to merge 82 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
82 commits
Select commit Hold shift + click to select a range
9248966
skip rake build for truffelruby
jouhlmann May 7, 2020
5f1fac9
structure idea?
jouhlmann May 7, 2020
ec543f3
code snippet from https://github.com/oracle/truffleruby/issues/1997
jouhlmann May 7, 2020
0f09592
Add empty pycall interface for filling
stefreschke May 14, 2020
996f72b
Merge branch 'master' into graalvm-ify
stefreschke May 14, 2020
18dda59
code fron 14th may
jouhlmann May 20, 2020
5ff8bee
fix pycall for Polyglot
jouhlmann May 26, 2020
04d78d8
Remove :const_missing in truffleruby pycall
stefreschke Jun 4, 2020
65545e2
pyobjectwrapper truffle
jouhlmann Jun 11, 2020
3caed6c
dont use spec_helper for truffleruby
jouhlmann Jun 11, 2020
2caef4b
Merge branch 'graalvm-ify' of https://github.com/stefreschke/pycall.r…
jouhlmann Jun 11, 2020
40cedc6
more object_wrapper
jouhlmann Jun 11, 2020
258a923
Add constructur to object wrapper
stefreschke Jun 18, 2020
0065a51
wrap the objects
jouhlmann Jun 18, 2020
8306602
Add Slice again, implement list wrapper
stefreschke Jun 18, 2020
0e8825b
Add initial implementation for dicts
stefreschke Jun 25, 2020
00d42d6
Trying to solve problems
stefreschke Jun 25, 2020
7021086
Remove questionable double declaration of List and Dict
stefreschke Jun 25, 2020
a275e71
Remove weird type mapping stuff
stefreschke Jun 25, 2020
6ab690b
Add Tuple Wrapper (in Ruby (-> in your face))
stefreschke Jun 25, 2020
bbe512c
You shall not ... delete constants
stefreschke Jun 25, 2020
da66c88
Add PyModule implementation
stefreschke Jun 25, 2020
09bfa1b
fix module wrapper
jouhlmann Jun 25, 2020
81b4cbd
Add a whole bunch of new and "vital" wrappers
stefreschke Jun 25, 2020
3d3ec13
Have all tests up and running, finally
stefreschke Jul 2, 2020
f968e9d
Introduce pyenum.rb, some pyerrors
stefreschke Jul 2, 2020
136f6d0
EVERYTHING
stefreschke Jul 2, 2020
65e17f2
Merge remote-tracking branch 'upstream/master'
stefreschke Jul 6, 2020
654f8a4
Merge remote-tracking branch 'origin/master' into graalvm-ify
stefreschke Jul 6, 2020
20c3aa3
Merge remote-tracking branch 'origin/graalvm-ify' into graalvmify-specs
stefreschke Jul 6, 2020
931b72e
fix some tests
jouhlmann Jul 6, 2020
e006bf9
Merge branch 'graalvmify-specs' of https://github.com/stefreschke/pyc…
jouhlmann Jul 6, 2020
523ac21
4 Errors less on dicts
stefreschke Jul 9, 2020
ef06e3d
Forgiving lookup that returns nil (now only 1 spec red)
stefreschke Jul 9, 2020
5fe1d0d
fix some tests around spec/pycall_spec.rb
jouhlmann Jul 9, 2020
5245c30
Merge branch 'graalvmify-specs' of https://github.com/stefreschke/pyc…
jouhlmann Jul 9, 2020
d217412
fix list (minus sort!/sort)
jouhlmann Jul 15, 2020
c96f2d8
fix slices
jouhlmann Jul 15, 2020
c430f63
fix set / dict / tuple
jouhlmann Jul 15, 2020
d4413b4
Trying to make sort not-inplace
stefreschke Jul 16, 2020
b3b9187
Merge branch 'graalvmify-specs' of https://github.com/stefreschke/pyc…
stefreschke Jul 16, 2020
ca065d4
Always wrap Python lists to PyCall::List
stefreschke Jul 16, 2020
7b7f9e2
Fix another tuple test
stefreschke Aug 14, 2020
8c8c291
Try to add Conversion as single instance of conversion truth
stefreschke Aug 14, 2020
b85712b
Reduce ifNil check
stefreschke Aug 15, 2020
14a418e
Add Converter class to enable wrapping of Complex and else...
stefreschke Aug 15, 2020
f00769d
fix wrapping/conversion
jouhlmann Aug 15, 2020
8ec5270
stuff
jouhlmann Aug 15, 2020
f325758
use second hash for converting to python
stefreschke Aug 15, 2020
31889c0
Fix tuple test
stefreschke Aug 15, 2020
a451e85
fix conversion bug
jouhlmann Aug 15, 2020
9684843
fix type mapping
jouhlmann Aug 15, 2020
95cf6ef
list/dict fixes
jouhlmann Aug 15, 2020
6a35d38
Wrap ruby object
stefreschke Aug 15, 2020
ab83615
Merge branch 'use-conversion-for-type-matches' of https://github.com/…
stefreschke Aug 15, 2020
e6f6412
Fix tests for unregistering python-type-mapping
stefreschke Aug 15, 2020
d5daef8
fixes slice
jouhlmann Aug 15, 2020
cc0b2b6
fix some tests
jouhlmann Aug 16, 2020
a9804b8
Wrapping #to_ruby results, #kind_of overrides
stefreschke Aug 17, 2020
1083120
Merge branch 'use-conversion-for-type-matches' of https://github.com/…
stefreschke Aug 17, 2020
024d9b9
Fix one more getattr test
stefreschke Aug 17, 2020
0bd3afa
Add #same? from upstream again
stefreschke Aug 17, 2020
f1a6c6c
some more fixes
jouhlmann Aug 17, 2020
635b99b
Merge branch 'use-conversion-for-type-matches' of https://github.com/…
jouhlmann Aug 17, 2020
f67f143
fix some more tests
jouhlmann Aug 18, 2020
e35cb23
Remove unnecessary comment
stefreschke Aug 18, 2020
79c4191
Remove german comment
stefreschke Aug 18, 2020
9e974dc
Another conversion.rb test supposedly done
stefreschke Aug 18, 2020
118bd6c
fix more tests
jouhlmann Aug 18, 2020
8fd3063
Remove breakpoint
stefreschke Aug 18, 2020
030194c
Merge branch 'use-conversion-for-type-matches' of https://github.com/…
stefreschke Aug 18, 2020
b5345b2
fixed more tests
jouhlmann Aug 19, 2020
7133047
Merge branch 'use-conversion-for-type-matches' of https://github.com/…
jouhlmann Aug 19, 2020
9c20d75
cleanup (nil..nil).step() workaround thx to @eregon
jouhlmann Aug 19, 2020
d649fe2
Revert test subject change (accident)
jouhlmann Aug 20, 2020
9b9e412
Begin fixing conversion.rb, removing lots of class variables
stefreschke Aug 21, 2020
bfbcdfa
small cleanup
jouhlmann Aug 21, 2020
0258fa1
Merge branch 'use-conversion-for-type-matches' of https://github.com/…
jouhlmann Aug 21, 2020
dd8201c
Remove some more class variables
stefreschke Aug 21, 2020
7381a46
Merge branch 'use-conversion-for-type-matches' of https://github.com/…
stefreschke Aug 21, 2020
d272e94
Fix wrong indentation on spec_helper.rb
stefreschke Aug 21, 2020
5efce23
Remove one set of quotation marks for string
stefreschke Aug 21, 2020
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,7 @@
/pkg/
/spec/reports/
/tmp/
vendor/

__pycache__/
.ipynb_checkpoints/
Expand Down
24 changes: 16 additions & 8 deletions Rakefile
Original file line number Diff line number Diff line change
Expand Up @@ -8,16 +8,21 @@ require "rspec/core/rake_task"
Dir[File.expand_path('../tasks/**/*.rake', __FILE__)].each {|f| load f }

gem_spec = eval(File.read('pycall.gemspec'))
Rake::ExtensionTask.new('pycall', gem_spec) do |ext|
ext.lib_dir = File.join(*['lib', ENV['FAT_DIR']].compact)
ext.cross_compile = true
ext.cross_platform = %w[x86-mingw32 x64-mingw32]
ext.cross_compiling do |s|
s.files.concat %w[lib/2.2/pycall.so lib/2.3/pycall.so lib/2.4/pycall.so]

if RUBY_ENGINE != "truffleruby"
Rake::ExtensionTask.new('pycall', gem_spec) do |ext|
ext.lib_dir = File.join(*['lib', ENV['FAT_DIR']].compact)
ext.cross_compile = true
ext.cross_platform = %w[x86-mingw32 x64-mingw32]
ext.cross_compiling do |s|
s.files.concat %w[lib/2.2/pycall.so lib/2.3/pycall.so lib/2.4/pycall.so]
end
end
end

Rake::ExtensionTask.new('pycall/spec_helper')
if RUBY_ENGINE != "truffleruby"
Rake::ExtensionTask.new('pycall/spec_helper')
end

desc "Compile binaries for mingw platform using rake-compiler-dock"
task 'build:mingw' do
Expand All @@ -28,4 +33,7 @@ end
RSpec::Core::RakeTask.new(:spec)

task :default => :spec
task spec: :compile
if RUBY_ENGINE != "truffleruby"
task spec: :compile
end

13 changes: 13 additions & 0 deletions ext/pycall/extconf.rb
Original file line number Diff line number Diff line change
@@ -1,3 +1,16 @@
# Building with TruffleRuby does not work / is not required
# Seems like putting out an empty Makefile is the best solution to skip build
# https://stackoverflow.com/questions/17406246/native-extensions-fallback-to-pure-ruby-if-not-supported-on-gem-install/50886432

if RUBY_ENGINE == "truffleruby"
mfile = open("Makefile", "wb")
mfile.puts '.PHONY: install'
mfile.puts 'install:'
mfile.puts "\t" + '@echo "You are using Truffleruby, therefore skipping build of native extension"'
mfile.close
exit 0
end

require 'mkmf'

create_makefile('pycall')
119 changes: 5 additions & 114 deletions lib/pycall.rb
Original file line number Diff line number Diff line change
@@ -1,114 +1,5 @@
module PyCall
require 'pycall/version'
require 'pycall/libpython'
require 'pycall/pyerror'
require 'pycall/pyobject_wrapper'
require 'pycall/pytypeobject_wrapper'
require 'pycall/pymodule_wrapper'
require 'pycall/init'

module_function

def builtins
@builtins ||= wrap_module(LibPython::API.builtins_module_ptr)
end

def callable?(obj)
case obj
when PyObjectWrapper
builtins.callable(obj.__pyptr__)
when PyPtr
builtins.callable(obj)
else
raise TypeError, "unexpected argument type #{obj.class} (expected PyCall::PyPtr or its wrapper)"
end
end

def dir(obj)
case obj
when PyObjectWrapper
builtins.dir(obj.__pyptr__)
when PyPtr
builtins.dir(obj)
else
raise TypeError, "unexpected argument type #{obj.class} (expected PyCall::PyPtr or its wrapper)"
end
end

def eval(expr, globals: nil, locals: nil)
globals ||= import_module(:__main__).__dict__
builtins.eval(expr, globals, locals)
end

def exec(code, globals: nil, locals: nil)
globals ||= import_module(:__main__).__dict__
if PYTHON_VERSION >= '3'
builtins.exec(code, globals, locals)
else
import_module('PyCall.six').exec_(code, globals, locals)
end
end

def getattr(*args)
obj, *rest = args
LibPython::Helpers.getattr(obj.__pyptr__, *rest)
end

def hasattr?(obj, name)
LibPython::Helpers.hasattr?(obj.__pyptr__, name)
end

def same?(left, right)
case left
when PyObjectWrapper
case right
when PyObjectWrapper
return left.__pyptr__ == right.__pyptr__
end
end
false
end

def import_module(name)
LibPython::Helpers.import_module(name)
end

def len(obj)
case obj
when PyObjectWrapper
builtins.len(obj.__pyptr__)
when PyPtr
builtins.len(obj)
else
raise TypeError, "unexpected argument type #{obj.class} (expected PyCall::PyPtr or its wrapper)"
end
end

def sys
@sys ||= import_module('sys')
end

def tuple(iterable=nil)
pyptr = if iterable
builtins.tuple.(iterable)
else
builtins.tuple.()
end
Tuple.wrap_pyptr(pyptr)
end

def with(ctx)
begin
yield ctx.__enter__
rescue PyError => err
raise err unless ctx.__exit__(err.type, err.value, err.traceback)
rescue Exception => err
# TODO: support telling what exception has been catched
raise err unless ctx.__exit__(err.class, err, err.backtrace_locations)
else
ctx.__exit__(nil, nil, nil)
end
end
end

require 'pycall/iruby_helper' if defined? IRuby
if RUBY_ENGINE == "truffleruby"
require 'pycall/truffleruby/pycall'
else
require 'pycall/pycall'
end
121 changes: 4 additions & 117 deletions lib/pycall/import.rb
Original file line number Diff line number Diff line change
@@ -1,120 +1,7 @@
require 'pycall'

module PyCall
module Import
class << self
attr_reader :main_object
end
end
end

PyCall::Import.instance_variable_set(:@main_object, self)

module PyCall
module Import
def pyimport(mod_name, as: nil)
as = mod_name unless as
check_valid_module_variable_name(mod_name, as)
mod = PyCall.import_module(mod_name)
define_singleton_method(as) { mod }
end

# This function is implemented as a mimic of `import_from` function defined in `Python/ceval.c`.
def pyfrom(mod_name, import: nil)
raise ArgumentError, "missing identifier(s) to be imported" unless import

mod_name = mod_name.to_str if mod_name.respond_to? :to_str
mod_name = mod_name.to_s if mod_name.is_a? Symbol

import = Array(import)
fromlist = import.map.with_index do |import_name, i|
case import_name
when assoc_array_matcher
import_name[0]
when Symbol, String
import_name
else
raise ArgumentError, "wrong type of import name #{import_name.class} (expected String or Symbol)"
end
end
from_list = PyCall.tuple(from_list)

main_dict_ptr = PyCall.import_module('__main__').__dict__.__pyptr__
globals = main_dict_ptr # FIXME: this should mimic to `import_name` function defined in `Python/ceval.c`.
locals = main_dict_ptr # FIXME: this should mimic to `import_name` function defined in `Python/ceval.c`.
level = 0 # TODO: support prefixed dots (#25)
mod = LibPython::Helpers.import_module(mod_name, globals, locals, fromlist, level)

import.each do |import_name|
case import_name
when assoc_array_matcher
name, asname = *import_name
when Symbol, String
name, asname = import_name, import_name
end

if PyCall::LibPython::Helpers.hasattr?(mod.__pyptr__, name)
pyobj = PyCall::LibPython::Helpers.getattr(mod.__pyptr__, name)
define_name(asname, pyobj)
next
end

if mod.respond_to? :__name__
pkgname = mod.__name__
fullname = "#{pkgname}.#{name}"
sys_modules = PyCall.import_module('sys').modules
if sys_modules.has_key?(fullname)
pyobj = module_dict[fullname]
define_name(asname, pyobj)
next
end
end

raise ArgumentError, "cannot import name #{fullname}" unless pyobj
end
end

private

def define_name(name, pyobj)
if callable?(pyobj) && !type_object?(pyobj)
define_singleton_method(name) do |*args|
LibPython::Helpers.call_object(pyobj.__pyptr__, *args)
end
else
if constant_name?(name)
context = self
context = (self == PyCall::Import.main_object) ? Object : self
context.module_eval { const_set(name, pyobj) }
else
define_singleton_method(name) { pyobj }
end
end
end

def constant_name?(name)
name =~ /\A[A-Z]/
end

def check_valid_module_variable_name(mod_name, var_name)
var_name = var_name.to_s if var_name.kind_of? Symbol
if var_name.include?('.')
raise ArgumentError, "#{var_name} is not a valid module variable name, use pyimport #{mod_name}, as: <name>"
end
end

def assoc_array_matcher
@assoc_array_matcher ||= ->(ary) do
ary.is_a?(Array) && ary.length == 2
end
end

def callable?(pyobj)
LibPython::Helpers.callable?(pyobj.__pyptr__)
end

def type_object?(pyobj)
pyobj.__pyptr__.kind_of? PyTypePtr
end
end
if RUBY_ENGINE == "truffleruby"
require 'pycall/truffleruby/import'
else
require 'pycall/import_pycall'
end
Loading