diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..7c3e7ad --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,69 @@ +# Contributing + +## JupyterLab Extension + +### Development install + +Note: You will need `NodeJS` to build the extension package. Recommend use `pnpm` + +```bash +# Clone the repo to your local environment +# Change directory to the current project directory +# Install package in development mode +pip install -e "." +# Link your development version of the extension with JupyterLab +jupyter labextension develop . --overwrite +# Rebuild extension Typescript source after making changes +pnpm build +``` + +You can watch the source directory and run JupyterLab at the same time in different terminals to watch for changes in the extension's source and automatically rebuild the extension. + +```bash +# Watch the source directory in one terminal, automatically rebuilding when needed +pnpm watch +# Run JupyterLab in another terminal +jupyter lab +``` + +With the watch command running, every saved change will immediately be built locally and available in your running JupyterLab. Refresh JupyterLab to load the change in your browser (you may need to wait several seconds for the extension to be rebuilt). + +By default, the `pnpm build` command generates the source maps for this extension to make it easier to debug using the browser dev tools. To also generate source maps for the JupyterLab core extensions, you can run the following command: + +```bash +jupyter lab build --minimize=False +``` + +### Development uninstall + +```bash +pip uninstall neopyter +``` + +In development mode, you will also need to remove the symlink created by `jupyter labextension develop` +command. To find its location, you can run `jupyter labextension list` to figure out where the `labextensions` +folder is located. Then you can remove the symlink named `neopyter` within that folder. + +### Testing the extension + +#### Frontend tests + +This extension is using [Jest](https://jestjs.io/) for JavaScript code testing. + +To execute them, execute: + +```sh +pnpm install +pnpm test +``` + +#### Integration tests + +This extension uses [Playwright](https://playwright.dev/docs/intro) for the integration tests (aka user level tests). +More precisely, the JupyterLab helper [Galata](https://github.com/jupyterlab/jupyterlab/tree/master/galata) is used to handle testing the extension in JupyterLab. + +More information are provided within the [ui-tests](./ui-tests/README.md) README. + +### Packaging the extension + +See [RELEASE](RELEASE.md) diff --git a/README.md b/README.md index 4364105..219faf6 100644 --- a/README.md +++ b/README.md @@ -5,11 +5,24 @@ ## How does it work? -![](./doc/communication.png) +This project includes two parts: a [`JupyterLab extension`](https://jupyterlab.readthedocs.io/en/stable/user/extensions.html) and a Neovim plugin -The `Jupyter lab`'s frontend plugin provides RPC service which expose functions of `Jupyter lab`, Jupyterlab server -forward these RPC service by TCP Server. `Neovim`s plugin connect to TCP Server and call RPC service when receive events -from `Neovim` via `autocmd`. In the end, `Neopyter` can control `Juppyter lab`. `Neopyter` can implement more ability like [jupynium.nvim](https://github.com/kiyoon/jupynium.nvim), but better performance. +- The `JupyterLab extension` exposes functions of `Jupyter lab`, and provides a remote procedure call(RPC) service +- The `Neovim plugin` calls the RPC service when it receives events from `Neovim` via `autocmd` + +| proxy | direct | +| :--------------------------------------------------------: | :----------------------------------------------------------: | +| proxy mode | direct mode | + +This project provides two working modes for different network environments. If the browser where your jupyiter lab is +located cannot directly access nvim, you must use `proxy` mode; If you need to collaborate and use the same Jupyter with +others, you must use direct mode + +- `proxy` mode: Jupyterlab server + proxies the RPC service as a TCP server which `Neovim`s plugin connects to +- `direct` mode: Neovim plugin accesses these RPC service directly + +Ultimately, `Neopyter` can control `Juppyter lab`. `Neopyter` can implement abilities like [jupynium.nvim](https://github.com/kiyoon/jupynium.nvim). ## Screenshots @@ -17,14 +30,14 @@ from `Neovim` via `autocmd`. In the end, `Neopyter` can control `Juppyter lab`. | :-----------------------------------------------: | :-----------------------------------------------: | :-----------------------------------------------: | | Completion | Cell Magic | Line Magic | -## Installation - -### Requirements +## Requirements - 📔JupyterLab >= 4.0.0 -- ✌️ Neovim >= 9.0 +- ✌️ Neovim nightly - 👍`nvim-lua/plenary.nvim` - - 🤏`AbaoFromCUG/websocket.nvim` (optional for `rpc_client="websocket_server"`) + - 🤏`AbaoFromCUG/websocket.nvim` (optional for `mode="proxy"`) + +## Installation ### JupyterLab Extension @@ -34,11 +47,12 @@ To install the jupyterlab extension, execute: pip install neopyter ``` -To remove the extension, execute: +Configure `JupyterLab` in menu `Settings`>`Settings Editor`>`Neopyter` -```bash -pip uninstall neopyter -``` +- `mode`: refer to the previous introduction about mode +- `IP`: if `mode=proxy`, set to the IP of the host where jupyter is located. If `proxy=direct`, set to the IP of the + host where neovim is located +- `port`: idle port ### Neovim plugin @@ -50,9 +64,10 @@ With 💤lazy.nvim: opts = { -- auto define autocmd auto_attach = true, - -- auto connect server + -- auto connect rpc service auto_connect = true, - rpc_client = "async", + mode="direct", + -- same with JupyterLab settings remote_address = "127.0.0.1:9001", file_pattern = { "*.ju.*" }, on_attach = function(bufnr) @@ -177,7 +192,6 @@ require'nvim-treesitter.configs'.setup { - Open JupyterLab `jupyter lab`, there is a sidebar named `Neopyter`, which display neopyter ip+port - Open a `*.ju.py` file in neovim -- [Optional] if `auto_attach` is `false`, you can connect jupyterlab manually via `:Neopyter connect 127.0.0.1:9001` - Now you can type `# %%` in Neovim to create a code cell. - You'll see everything you type below that will be synchronised in the browser @@ -269,73 +283,3 @@ require'nvim-treesitter.configs'.setup { ## Acknowledges - [jupynium.nvim](https://github.com/kiyoon/jupynium.nvim): Selenium-automated Jupyter Notebook that is synchronised with NeoVim in real-time. - -## Contributing - -### JupyterLab Extension - -#### Development install - -Note: You will need `NodeJS` to build the extension package. Recommend use `pnpm` - -```bash -# Clone the repo to your local environment -# Change directory to the current project directory -# Install package in development mode -pip install -e "." -# Link your development version of the extension with JupyterLab -jupyter labextension develop . --overwrite -# Rebuild extension Typescript source after making changes -pnpm build -``` - -You can watch the source directory and run JupyterLab at the same time in different terminals to watch for changes in the extension's source and automatically rebuild the extension. - -```bash -# Watch the source directory in one terminal, automatically rebuilding when needed -pnpm watch -# Run JupyterLab in another terminal -jupyter lab -``` - -With the watch command running, every saved change will immediately be built locally and available in your running JupyterLab. Refresh JupyterLab to load the change in your browser (you may need to wait several seconds for the extension to be rebuilt). - -By default, the `pnpm build` command generates the source maps for this extension to make it easier to debug using the browser dev tools. To also generate source maps for the JupyterLab core extensions, you can run the following command: - -```bash -jupyter lab build --minimize=False -``` - -#### Development uninstall - -```bash -pip uninstall neopyter -``` - -In development mode, you will also need to remove the symlink created by `jupyter labextension develop` -command. To find its location, you can run `jupyter labextension list` to figure out where the `labextensions` -folder is located. Then you can remove the symlink named `neopyter` within that folder. - -#### Testing the extension - -##### Frontend tests - -This extension is using [Jest](https://jestjs.io/) for JavaScript code testing. - -To execute them, execute: - -```sh -pnpm install -pnpm test -``` - -##### Integration tests - -This extension uses [Playwright](https://playwright.dev/docs/intro) for the integration tests (aka user level tests). -More precisely, the JupyterLab helper [Galata](https://github.com/jupyterlab/jupyterlab/tree/master/galata) is used to handle testing the extension in JupyterLab. - -More information are provided within the [ui-tests](./ui-tests/README.md) README. - -#### Packaging the extension - -See [RELEASE](RELEASE.md) diff --git a/doc/communication_direct.png b/doc/communication_direct.png new file mode 100644 index 0000000..8c4b8be Binary files /dev/null and b/doc/communication_direct.png differ diff --git a/doc/communication.png b/doc/communication_proxy.png similarity index 100% rename from doc/communication.png rename to doc/communication_proxy.png diff --git a/doc/neopyter.txt b/doc/neopyter.txt index 8b67af0..a11cc5c 100644 --- a/doc/neopyter.txt +++ b/doc/neopyter.txt @@ -16,16 +16,14 @@ The bridge between Neovim and jupyter lab, edit in Neovim and preview/run in jup ------------------------------------------------------------------------------ *neopyter-usage* -neopyter.Option +Class ~ +{neopyter.Option} Fields ~ {remote_address} `(string)` {file_pattern} `(string[])` {auto_attach} `(boolean)` Automatically attach to the Neopyter server when open file_pattern matched files {auto_connect} `(boolean)` # auto connect jupyter lab -{rpc_client} -"'async'" # AsyncRpcClient, default -"'block'" # BlockRpcClient -"'websocket_server'" # WSServerClient +{mode} "direct"|"proxy" {filename_mapper} `(fun(ju_path:string):string)` {on_attach} `(optional)` `(fun(bufnr:number))` {jupyter} neopyter.JupyterOption @@ -46,7 +44,7 @@ Fields ~ auto_attach = true, auto_connect = true, - rpc_client = "async", + mode = "proxy", jupyter = { auto_activate_file = true, -- Always scroll to the current cell. @@ -69,7 +67,6 @@ Fields ~ }, } - < Type ~ neopyter.Option @@ -86,21 +83,24 @@ Parameters ~ ============================================================================== ------------------------------------------------------------------------------ -neopyter.JupyterOption +Class ~ +{neopyter.JupyterOption} Fields ~ {auto_activate_file} `(boolean)` ------------------------------------------------------------------------------ *JupyterLab* `JupyterLab` -neopyter.JupyterLab +Class ~ +{neopyter.JupyterLab} Fields ~ {client} neopyter.RpcClient {private} augroup `(number)` {notebook_map} `({[string]:)` neopyter.Notebook} ------------------------------------------------------------------------------ -neopyter.NewJupyterLabOption +Class ~ +{neopyter.NewJupyterLabOption} Fields ~ {address} `(optional)` `(string)` @@ -167,7 +167,8 @@ Return ~ documents](https://jupyterlab.readthedocs.io/en/stable/user/commands.html#commands-list) ------------------------------------------------------------------------------ -neopyter.NewUntitledOption +Class ~ +{neopyter.NewUntitledOption} Fields ~ {path} `(optional)` `(string)` {type} `(optional)` `notebook`|`file` @@ -185,7 +186,8 @@ current notebook of jupyter lab ============================================================================== ------------------------------------------------------------------------------ -neopyter.Cell +Class ~ +{neopyter.Cell} Fields ~ {start_line} `(number)` include {lines} `(string[])` @@ -200,7 +202,8 @@ Fields ~ ------------------------------------------------------------------------------ *Notebook* `Notebook` -neopyter.Notebook +Class ~ +{neopyter.Notebook} Fields ~ {private} client neopyter.RpcClient {bufnr} `(number)` @@ -211,7 +214,8 @@ Fields ~ {private} augroup? `(number)` ------------------------------------------------------------------------------ -neopyter.NewNotebokOption +Class ~ +{neopyter.NewNotebokOption} Fields ~ {client} neopyter.RpcClient {bufnr} `(number)` diff --git a/lua/neopyter.lua b/lua/neopyter.lua index 83e3c06..b1caaf8 100644 --- a/lua/neopyter.lua +++ b/lua/neopyter.lua @@ -25,10 +25,7 @@ local neopyter = {} ---@field file_pattern string[] ---@field auto_attach boolean Automatically attach to the Neopyter server when open file_pattern matched files ---@field auto_connect boolean # auto connect jupyter lab ----@field rpc_client ----| "'async'" # AsyncRpcClient, default ----| "'block'" # BlockRpcClient ----| "'websocket_server'" # WSServerClient +---@field mode "direct"|"proxy" ---@field filename_mapper fun(ju_path:string):string ---@field on_attach? fun(bufnr:number) ---@field jupyter neopyter.JupyterOption @@ -47,7 +44,7 @@ neopyter.config = { auto_attach = true, auto_connect = true, - rpc_client = "async", + mode = "proxy", jupyter = { auto_activate_file = true, -- Always scroll to the current cell. diff --git a/lua/neopyter/health.lua b/lua/neopyter/health.lua index a46e8ee..92ce7ba 100644 --- a/lua/neopyter/health.lua +++ b/lua/neopyter/health.lua @@ -21,7 +21,6 @@ local function run_blocking(suspend_fn, ...) local resolved = false vim.schedule(function() a.run(suspend_fn, function(ee) - print(ee) resolved = true end) end) diff --git a/lua/neopyter/jupyter/jupyterlab.lua b/lua/neopyter/jupyter/jupyterlab.lua index 78235d5..c309c43 100644 --- a/lua/neopyter/jupyter/jupyterlab.lua +++ b/lua/neopyter/jupyter/jupyterlab.lua @@ -29,9 +29,14 @@ function JupyterLab:new(opts) local config = require("neopyter").config local RpcClient - if config.rpc_client == "block" then - RpcClient = require("neopyter.rpc.blockclient") - elseif config.rpc_client == "websocket_server" then + if config["rpc_client"] ~= nil then + vim.notify( + "`rpc_client` is deprecated, please reference to https://github.com/SUSTech-data/neopyter/issues/4", + vim.log.levels.ERROR, + { title = "Neopyter" } + ) + end + if config.mode == "direct" then RpcClient = require("neopyter.rpc.wsserverclient") else RpcClient = require("neopyter.rpc.asyncclient") diff --git a/schema/labplugin.json b/schema/labplugin.json index b38b5f8..11e3a70 100644 --- a/schema/labplugin.json +++ b/schema/labplugin.json @@ -20,11 +20,11 @@ "title": "Neopyter work mode", "description": "Different work mode determine different methods of communication arcitecture. The `proxy` mode will communicate through jupyter server, while the `direct` mode will communicate directly with nvim", "$ref": "#/definitions/mode", - "default": "proxy" + "default": "direct" }, "ip": { "title": "IP", - "description": "For `proxy` mode, this is the listening `IP:port` of the jupyter server's tcp server; for `direct` mode, this is `IP:port` of nvim's websocket server", + "description": "For `proxy` mode, this is the IP of the host where jupyter server is located; For `direct` mode, this is the IP of the host where nvim is located ", "$ref": "#/definitions/ip", "default": "127.0.0.1" },