Problems with passing arguments to Python object

[m-atoms]

Revision 0

I'm able to import pyvisa by following along with the documentation but I'm running into errors when trying to build the object:

require 'pycall/import'
include PyCall::Import
pyimport :pyvisa

rm = pyvisa.ResourceManager('/lib64/libiovisa.so')

error output:

<class 'pyvisa.highlevel.ResourceManager'>2020/02/25 11:26:19.424 (py_test.rb:10): 
ERROR: PyError : <class 'TypeError'>: list_resources() missing 1 required positional 
argument: 'self'

#55 explains that this error is caused by using Python's syntax for instantiation instead of Ruby's.

Revision 1

require 'pycall/import'
include PyCall::Import
pyimport :pyvisa

rm = pyvisa.ResourceManager('/lib64/libiovisa.so').new()

error output:

ERROR: PyError : <class 'TypeError'>: a bytes-like object is required, not 'str'

Now using the Ruby syntax for instantiation seems to work but the argument for sourcing the library fails. I've tried a number of ways to pass the argument using both Python and Ruby syntax and none of them have worked. I'd appreciate any advice on working with Python objects in Ruby.

[m-atoms]

@mrkn I have some additional information that might be useful. I'm testing the same code shown in rev 1 but now the error message is different. This is almost definitely an issue with my test environment - my ruby script is running inside an open source tool that's sometimes unstable after changes to code/environment.

The code (same as rev 1):

require 'pycall/import'
include PyCall::Import
pyimport :pyvisa

rm = pyvisa.ResourceManager('/lib64/libiovisa.so').new()

error output:

ERROR: PyError : <class 'ValueError'>: Could not locate a VISA implementation. Install either 
the NI binary or pyvisa-py.

This error message is more relevant than the one shown in my original comment. I'm unsure why the path I'm providing is failing to find the library. I verified that the .so exists at that location and I ran the native Python version just to confirm that pyvisa is able to source the shared object:

>>> import pyvisa
>>> rm = pyvisa.ResourceManager('/lib64/libiovisa.so')
>>> rm.list_resources()
('GPIB0::4::INSTR',)

The "GPIB0::4::INSTR" print out means that it was able to source the library and therefore able to provide the resource listing method. Any idea why the path argument isn't working in Ruby?

[mrkn]

If ResourceManager is a class, you must write pyvisa.ResourceManager.new('/lib64/libiovisa.so').
Could you try this form?

[m-atoms]

@mrkn I have investigated this issue thoroughly and located the cause of these behaviors. Here are the results:

Syntax Option 1

rm = pyvisa.ResourceManager('/lib64/libiovisa.so')
# or 
rm = pyvisa.ResourceManager('/lib64/libiovisa.so').new()

Error Output:

ERROR: PyError : <class 'ValueError'>: Could not locate a VISA implementation. Install either 
the NI binary or pyvisa-py.

As previously mentioned in my earlier comments and in issue #55, this syntax doesn't work. The string argument passed to ResourceManager() is discarded and only string arguments passed to new() make it to python.

Syntax Option 2

rm = pyvisa.ResourceManager.new('/lib64/libiovisa.so')

Error Output:

ERROR: PyError : <class 'TypeError'>: a bytes-like object is required, not 'str'

This required a fair bit more debugging. This TypeError occurs for any string argument passed from Ruby to any Python 3.x method expecting a string.

Results
This problem turned out to be caused by the way Ruby, Python 2.x, and Python 3.x handle strings, and the way pycall converts strings from Ruby to Python:

/* in pycall.c */
PyObject *
pycall_pystring_from_ruby(VALUE obj)
{
    int is_binary, is_ascii_only;

    if (RB_TYPE_P(obj, T_SYMBOL)) {
        obj = rb_sym_to_s(obj);
    }

    StringValue(obj);

    is_binary = (rb_enc_get_index(obj) == rb_ascii8bit_encindex());
    is_ascii_only = (ENC_CODERANGE_7BIT == rb_enc_str_coderange(obj));

    if (is_binary || (!python_is_unicode_literals && is_ascii_only)) {
        return Py_API(PyString_FromStringAndSize)(RSTRING_PTR(obj), RSTRING_LEN(obj));
    }
    return Py_API(PyUnicode_DecodeUTF8)(RSTRING_PTR(obj), RSTRING_LEN(obj), NULL);
}

To understand this problem we need to understand how text is handled:

• Ruby string literals are ASCII-8BIT
• Python 2.x string literals are ASCII-7BIT
• Python 3.x string literals are Unicode code points and encoding defaults to UTF-8

With this in mind, it follows that pycall_pystring_from_ruby() will enter into the conditional return statement when normal Ruby string literals are used because is_binary will be true since Ruby string literals are ASCII-8BIT. The problem arises when PyString_FromStringAndSize is invoked. This is a python 2.x method and is aliased in Python 3.x to return a bytes-like object, not the Python 3.x concept of a string. When Python 3.x tries to operate on this bytes-like object with a string method (Unicode code points), it fails because there is no automatic coercion.

Workaround
The workaround for this is actually quite simple:

rm = pyvisa.ResourceManager.new('/lib64/libiovisa.so'.encode("utf-8"))

By converting the string argument to UTF-8 encoding (instead of the default ASCII-8BIT) before passing the argument, pycall_pystring_from_ruby() reaches the return statement with PyUnicode_DecodeUTF8 which cleanly decodes to Python 3.x's idea of a string, Unicode code points.

Solution
To resolve this issue permanently, pycall_pystring_from_ruby() should be more explicit about Python version checking (using python_is_unicode_literals) in all cases and ensure that the proper decoding methods are used. I'd be happy to work on a pull request if you think it's appropriate.