Python refactoring: ideas #189

farhi · 2019-02-18T15:19:14Z

iData and iFunc classes can inherit from multiple sub-classes, e.g.:

Master class:

base methods and properties: Data, Aliases, Axes, UserData, meta-data, ...
check, set, get, copy, ...

Sub classes (see https://github.com/McStasMcXtrace/iFit/blob/master/Docs/Methods.html):

I/O: populates the object by reading files. Methods: read/write
View: print-out, plot, subplot, edit, ...
Math Unary methods
Math Binary methods
Math statistics
Math integration, derivatives, projections
Math conv, corr, FFT
Math combine, cat, dog, slice, ...
Math interpolation

Then we assemble

iData(Master, I/O, Math, ...)

which builds all bits and allow each sub-class to be rather small.

farhi · 2019-04-04T12:53:06Z

A good alternative is to use imatlab which brings a Matlab into Jupiter:

https://github.com/imatlab/imatlab

which can probably be made into a Debian package.

farhi · 2019-04-11T05:10:10Z

Install for Ubuntu 18.04, python 3.6 (from system), matlab R2017a:

cd /opt/MATLAB/R2017a/extern/engines/python/
add '3.6' into setup.py / supported versions
optional: cp /opt/MATLAB/R2017a/extern/engines/python/dist/matlab/engine/glnxa64/matlabengineforpython3_5.so /opt/MATLAB/R2017a/extern/engines/python/dist/matlab/engine/glnxa64/matlabengineforpython3_6.so
optional: sudo ghex /opt/MATLAB/R2017a/extern/engines/python/dist/matlab/engine/glnxa64/matlabengineforpython3_6.so
optional: change all '3_5' occurences to '3_6', and '3.5' to '3.6'. Save.
sudo python3.6 setup.py install
modify /usr/local/lib/python3.6/dist-packages/matlab/engine/init.py to add '3_6' in supported versions.
modify /usr/local/lib/python3.6/dist-packages/matlab/engine/init.py to explicitely explicitely import_module("matlabengineforpython3_5") [2 locations]
sudo apt install python3-plotly python3-widgetsnbextension python3-nbconvert python3-ipykernel python3-testpath
sudo pip3 install imatlab
sudo python3 -mimatlab install

Installation is done in:

matlab engine: /opt/MATLAB/R2017a/extern/engines/python
imatlab: /usr/local/lib/python3.6/dist-packages/imatlab

farhi · 2019-04-11T05:13:16Z

Notes above from:

Works OK in Jupyter.

farhi · 2019-04-11T08:59:55Z

Matlab Engine R2017a Installation output:

# cd /opt/MATLAB/R2017a/extern/engines/python
# sudo python3.6 setup.py install
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/matlab
copying dist/matlab/__init__.py -> build/lib/matlab
copying dist/matlab/mlexceptions.py -> build/lib/matlab
copying dist/matlab/mlarray.py -> build/lib/matlab
creating build/lib/matlab/engine
copying dist/matlab/engine/engineerror.py -> build/lib/matlab/engine
copying dist/matlab/engine/__init__.py -> build/lib/matlab/engine
copying dist/matlab/engine/matlabengine.py -> build/lib/matlab/engine
copying dist/matlab/engine/basefuture.py -> build/lib/matlab/engine
copying dist/matlab/engine/matlabfuture.py -> build/lib/matlab/engine
copying dist/matlab/engine/futureresult.py -> build/lib/matlab/engine
copying dist/matlab/engine/enginesession.py -> build/lib/matlab/engine
copying dist/matlab/engine/fevalfuture.py -> build/lib/matlab/engine
creating build/lib/matlab/_internal
copying dist/matlab/_internal/__init__.py -> build/lib/matlab/_internal
copying dist/matlab/_internal/mlarray_utils.py -> build/lib/matlab/_internal
copying dist/matlab/_internal/mlarray_sequence.py -> build/lib/matlab/_internal
running install_lib
creating /usr/local/lib/python3.6/dist-packages/matlab
creating /usr/local/lib/python3.6/dist-packages/matlab/_internal
copying build/lib/matlab/_internal/__init__.py -> /usr/local/lib/python3.6/dist-packages/matlab/_internal
copying build/lib/matlab/_internal/mlarray_utils.py -> /usr/local/lib/python3.6/dist-packages/matlab/_internal
copying build/lib/matlab/_internal/mlarray_sequence.py -> /usr/local/lib/python3.6/dist-packages/matlab/_internal
copying build/lib/matlab/__init__.py -> /usr/local/lib/python3.6/dist-packages/matlab
copying build/lib/matlab/mlexceptions.py -> /usr/local/lib/python3.6/dist-packages/matlab
copying build/lib/matlab/mlarray.py -> /usr/local/lib/python3.6/dist-packages/matlab
creating /usr/local/lib/python3.6/dist-packages/matlab/engine
copying build/lib/matlab/engine/engineerror.py -> /usr/local/lib/python3.6/dist-packages/matlab/engine
copying build/lib/matlab/engine/__init__.py -> /usr/local/lib/python3.6/dist-packages/matlab/engine
copying build/lib/matlab/engine/matlabengine.py -> /usr/local/lib/python3.6/dist-packages/matlab/engine
copying build/lib/matlab/engine/basefuture.py -> /usr/local/lib/python3.6/dist-packages/matlab/engine
copying build/lib/matlab/engine/_arch.txt -> /usr/local/lib/python3.6/dist-packages/matlab/engine
copying build/lib/matlab/engine/matlabfuture.py -> /usr/local/lib/python3.6/dist-packages/matlab/engine
copying build/lib/matlab/engine/futureresult.py -> /usr/local/lib/python3.6/dist-packages/matlab/engine
copying build/lib/matlab/engine/enginesession.py -> /usr/local/lib/python3.6/dist-packages/matlab/engine
copying build/lib/matlab/engine/fevalfuture.py -> /usr/local/lib/python3.6/dist-packages/matlab/engine
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/_internal/__init__.py to __init__.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/_internal/mlarray_utils.py to mlarray_utils.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/_internal/mlarray_sequence.py to mlarray_sequence.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/__init__.py to __init__.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/mlexceptions.py to mlexceptions.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/mlarray.py to mlarray.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/engine/engineerror.py to engineerror.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/engine/__init__.py to __init__.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/engine/matlabengine.py to matlabengine.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/engine/basefuture.py to basefuture.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/engine/matlabfuture.py to matlabfuture.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/engine/futureresult.py to futureresult.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/engine/enginesession.py to enginesession.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/matlab/engine/fevalfuture.py to fevalfuture.cpython-36.pyc
running install_egg_info
Writing /usr/local/lib/python3.6/dist-packages/matlabengineforpython-R2017a.egg-info

Test with:

# ipython3
import matlab.engine
eng = matlab.engine.start_matlab()

farhi · 2019-04-11T09:05:43Z

imatlab installation output [https://github.com/imatlab/imatlab] :

# sudo pip3 install imatlab
Collecting imatlab
  Downloading https://files.pythonhosted.org/packages/92/4d/ddd1e0df607266826ab2c3fe6641e3daaa7ca917c906595e362413894fcb/imatlab-0.3-py3-none-any.whl
Requirement already satisfied: widgetsnbextension>=1.0 in /usr/lib/python3/dist-packages (from imatlab)
Requirement already satisfied: nbconvert>=4.2 in /usr/lib/python3/dist-packages (from imatlab)
Requirement already satisfied: plotly>=1.13.0 in /usr/lib/python3/dist-packages (from imatlab)
Requirement already satisfied: ipykernel>=4.1 in /usr/lib/python3/dist-packages (from imatlab)
Requirement already satisfied: matlabengineforpython>=R2016b in /usr/local/lib/python3.6/dist-packages (from imatlab)
Requirement already satisfied: testpath in /usr/lib/python3/dist-packages (from nbconvert>=4.2->imatlab)
Installing collected packages: imatlab
Successfully installed imatlab-0.3
# sudo python3 -mimatlab install
Installed kernelspec imatlab in /usr/local/share/jupyter/kernels/imatlab

Test by starting Jupyter and a Matlab kernel (pop-up 'New' menu, top-right). You can then type e.g.:

imatlab_export_fig('print-png')
plot(1:10)

will display the static graphics in the Jupyter notebook session.

farhi · 2019-04-11T09:24:42Z

Configure Jupyter and apache for distant connexion:

https://lib.smgr.io/remote-jupyter-notebook-server/

This includes a user/passwd login.

farhi · 2019-04-18T08:07:55Z

Installation of PlotLy (only once to set the credentials):

download and extract plotly from e.g. https://plot.ly/matlab/getting-started/
go there
in a browser, login on plotly (https://plot.ly e.g. with GitHub or Facebook) and get your username and api_key from the Settings page.
matlab> plotlysetup('UserName', 'api_key')

Setting offline plotting (at every Matlab startup):

if behind a firewall, set matlab proxy (I use 'webbrowser' and its Navigation/proxy menu item -> dialog)
matlab> getplotlyoffline('http://cdn.plot.ly/plotly-latest.min.js'); NOTE: http and not https if an SSL error occurs.
this last command should be used at Matab startup.

Generating plotly figures:

matlab> plot(1:10); fig2plotly(gcf, 'offline', true);
To preserve the original MATLAB color and styling, use: fig2plotly(gcf,'strip',false,'offline',true)
this last command plot within Matlab, and also open a local PlotLy interactive view in a browser
simple plot are OK, volume plots are not rendered.

farhi · 2019-08-22T10:18:49Z

Accessing Notebook from 'outside' (not localhost):

https://stackoverflow.com/questions/42848130/why-i-cant-access-remote-jupyter-notebook-server

indicates that the configuration file has to allow distant connection in file:

~/.jupyter/jupyter_notebook_config.py

then changes lines:

c.NotebookApp.allow_origin = '*' #allow all origins
c.NotebookApp.ip = '0.0.0.0' # listen on all IPs

Adding authentication (basic)

To add a password, use:

jupyter notebook password

will modify the above configuration to store the password as a hash (encoded).

farhi · 2019-10-29T16:02:35Z

An other solution is to write a Jupyter Kernel around iFit runtime executable:

https://jupyter-client.readthedocs.io/en/stable/wrapperkernels.html

seems rather simple to set-up.

farhi · 2019-10-30T14:54:28Z

The above example does not start the kernel.
An example that works properly:

https://github.com/jupyter/echo_kernel

Installation:

# sudo python3 setup.py install
/usr/lib/python3.6/distutils/dist.py:261: UserWarning: Unknown distribution option: 'install_requires'
  warnings.warn(msg)
running install
running build
running build_py
creating build
creating build/lib
creating build/lib/echo_kernel
copying echo_kernel/__init__.py -> build/lib/echo_kernel
copying echo_kernel/install.py -> build/lib/echo_kernel
copying echo_kernel/kernel.py -> build/lib/echo_kernel
copying echo_kernel/__main__.py -> build/lib/echo_kernel
running install_lib
creating /usr/local/lib/python3.6/dist-packages/echo_kernel
copying build/lib/echo_kernel/__init__.py -> /usr/local/lib/python3.6/dist-packages/echo_kernel
copying build/lib/echo_kernel/install.py -> /usr/local/lib/python3.6/dist-packages/echo_kernel
copying build/lib/echo_kernel/kernel.py -> /usr/local/lib/python3.6/dist-packages/echo_kernel
copying build/lib/echo_kernel/__main__.py -> /usr/local/lib/python3.6/dist-packages/echo_kernel
byte-compiling /usr/local/lib/python3.6/dist-packages/echo_kernel/__init__.py to __init__.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/echo_kernel/install.py to install.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/echo_kernel/kernel.py to kernel.cpython-36.pyc
byte-compiling /usr/local/lib/python3.6/dist-packages/echo_kernel/__main__.py to __main__.cpython-36.pyc
running install_egg_info
Writing /usr/local/lib/python3.6/dist-packages/echo_kernel-1.1.egg-info
# sudo python3 -m echo_kernel.install
Installing Jupyter kernel spec

Locations:

echo kernel in /usr/local/share/jupyter/kernels/echo
echo_kernel package in /usr/local/lib/python3.6/dist-packages/echo_kernel with a dummy __init.py__ and __main.py__. The actual kernel is in kernel.py.

Actually, there is also a simple plotter kernel:

https://ipython-books.github.io/16-creating-a-simple-kernel-for-jupyter/

which mentions the format of the messaging protocol:

farhi added the enhancement label Feb 18, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Python refactoring: ideas #189

Python refactoring: ideas #189

farhi commented Feb 18, 2019 •

edited

Loading

farhi commented Apr 4, 2019

farhi commented Apr 11, 2019 •

edited

Loading

farhi commented Apr 11, 2019

farhi commented Apr 11, 2019 •

edited

Loading

farhi commented Apr 11, 2019 •

edited

Loading

farhi commented Apr 11, 2019

farhi commented Apr 18, 2019

farhi commented Aug 22, 2019

farhi commented Oct 29, 2019

farhi commented Oct 30, 2019

Python refactoring: ideas #189

Python refactoring: ideas #189

Comments

farhi commented Feb 18, 2019 • edited Loading

farhi commented Apr 4, 2019

farhi commented Apr 11, 2019 • edited Loading

farhi commented Apr 11, 2019

farhi commented Apr 11, 2019 • edited Loading

farhi commented Apr 11, 2019 • edited Loading

farhi commented Apr 11, 2019

farhi commented Apr 18, 2019

farhi commented Aug 22, 2019

Accessing Notebook from 'outside' (not localhost):

Adding authentication (basic)

farhi commented Oct 29, 2019

farhi commented Oct 30, 2019

farhi commented Feb 18, 2019 •

edited

Loading

farhi commented Apr 11, 2019 •

edited

Loading

farhi commented Apr 11, 2019 •

edited

Loading

farhi commented Apr 11, 2019 •

edited

Loading