diff --git a/.github/workflows/prod_release.yml b/.github/workflows/prod_release.yml index 082dded2..dd175e27 100644 --- a/.github/workflows/prod_release.yml +++ b/.github/workflows/prod_release.yml @@ -98,7 +98,7 @@ jobs: run: echo "Manually approved" publish: - needs: manual-approve + needs: manual-approval runs-on: ubuntu-latest steps: - name: Checkout source code diff --git a/README.md b/README.md index e0d2d849..8a3db2ec 100644 --- a/README.md +++ b/README.md @@ -1,16 +1,17 @@ # kdb Visual Studio Code extension -This is a companion extension for kdb developers to edit q files, connect to kdb processes, and run queries. This VS Code extension can be used alongside [kdb Insights Enterprise](https://code.kx.com/insights/enterprise/index.html) when using a shared kdb process. +[![KX VS Code CI Testing](https://github.com/KxSystems/kx-vscode/actions/workflows/main.yml/badge.svg)](https://github.com/KxSystems/kx-vscode/actions/workflows/main.yml) [![KX VS Code Release](https://github.com/KxSystems/kx-vscode/actions/workflows/release.yml/badge.svg)](https://github.com/KxSystems/kx-vscode/actions/workflows/release.yml) -Please use vscode-questions@kx.com to raise any questions or feedback. +The **kdb Visual Studio Code extension** provides developers with an extensive set of features that enable them to create and edit q files, connect to multiple kdb processes, and execute queries. -[![KX VS Code CI Testing](https://github.com/KxSystems/kx-vscode/actions/workflows/main.yml/badge.svg)](https://github.com/KxSystems/kx-vscode/actions/workflows/main.yml) +This extension can be used with [kdb Insights Enterprise](https://code.kx.com/insights/enterprise/index.html) when using a shared kdb process. -[![KX VS Code Release](https://github.com/KxSystems/kx-vscode/actions/workflows/release.yml/badge.svg)](https://github.com/KxSystems/kx-vscode/actions/workflows/release.yml) +> Please use vscode-questions@kx.com to raise any questions or feedback. ## Contents +This guide provides information on the following: -- [Why q for VS Code?](#why-q-for-vs-code) +- [Benefits of using kdb VS Code Extension](#benefits-of-kdb-vs-code-extension) - [Get Started](#get-started) - [Connections](#connections) - [kdb language server](#kdb-language-server) @@ -19,171 +20,208 @@ Please use vscode-questions@kx.com to raise any questions or feedback. - [Settings](#settings) - [Shortcuts](#shortcuts) -## Why q for VS Code? +## Benefits of kdb VS Code Extension With the kdb VS Code extension you can: - Install q. -- Write q syntax with support for predict and autocomplete. -- Execute q from a single line of code, code block or q file. -- Write and execute q code against kdb Insights Enterprise. +- Write q syntax with support for syntax highlighting, predict and autocomplete. +- Write and execute q from a single line of code, code block or q file. +- Write and execute q and Python code against kdb Insights Enterprise. +- Connect to one or more q process or kdb Insights Enterprise deployment. +- Use a KX data source to choose a connection, specify the parameters and run API requests, SQL or qSQL. +- Use a KX workbook to choose a connection and run q or Python code against any connection. - View results from your queries. -## Get started -If you have an existing q installation, you will see the message `q runtime installed` and can go directly to adding [connections](#connections). +## Getting Started -If you are new to q, you can install q with the kdb VS Code extension; see [install q](#install-q). +This is divided into three sections: -### Install q +- [Installing kdb VS Code Extension](#installing-kdb-vs-code-extension) +- [Installing q](#installing-q) +- [Integrating VS Code with q](#integrate-vs-code-with-q) -**Step 1**: Download the latest version of kdb+. Different versions of kdb+ are available, and the version you download will determine the supported features in VS Code: +### Installing kdb VS Code Extension -| item | write q | run q queries | explore results | shared kdb process with kdb Insights | -| ----------------------------------------------------------------------------------------------- | ------- | ------------- | --------------- | ------------------------------------ | -| [kdb+ Personal Edition](https://kx.com/kdb-personal-edition-download/) | yes | yes | yes | no | -| [kdb Insights Personal Edition](https://kx.com/kdb-insights-personal-edition-license-download/) | yes | yes | yes | no | -| kdb Insights Enterprise | yes | yes | yes | yes | +This section assumes you have [VS Code installed](https://code.visualstudio.com/download) -Contact licadmin@kx.com for commercial kdb licensing. +- [Install the kdb VS Code extension by clicking the Install button here](https://marketplace.visualstudio.com/items?itemName=KX.kdb) +- The [next step](#installing-kdb/q) depends on whether or not you have kdb/q installed. -**Step 2**: After registering, you will receive an email with a link to download an installation file. +Once installed KX appears in the Activity Bar on the left-hand side and once selected shows the following Views in the Primary Sidebar: -Extract the file to a directory; instructions are available for [Linux, macOS and Windows](https://code.kx.com/q/learn/install/#step-2-unzip-your-download). +- [Connections](#connections) +- [Datasources](#data-sources) +- [Workbooks](#workbooks) +- [Query History](#query-history) -- I want to learn more about [kdb+ installation](https://code.kx.com/q/learn/install/) +### Installing q -**Step 3**: The welcome email will also have your kdb, `k4.lic` or `kc.lic` license as an attachment. We recommend you add your kdb license to your q installation directory, or `QHOME`, and define this location in your environment variables. It is essential you define an environment variable for VS Code to recognize a valid license. +The next action depends on whether or not you already have q installed as it is required to use the extension. -_Note_: If your kdb license is stored outside of your q (`QHOME`) directory, create a `QLIC` environment variable instead. +- q is already installed: the message `q runtime installed` is displayed, after you [install the extension](#installing-kdb-vs-code-extension) and you can go directly to adding [connections](#connections). -- I want to learn [how to define a QHOME environment variable](https://code.kx.com/q/learn/install/#step-5-edit-your-profile). +- q is not installed: a notification gives you option to download and register, install [kdb Insights Personal Edition](https://kx.com/kdb-insights-personal-edition-license-download/) and add your license file. -### Install q with VS Code. + After install the extension provides a seamless integration with q, by displaying a notification with an option to download, register and install [kdb Insights Personal Edition](https://kx.com/kdb-insights-personal-edition-license-download/). For details on the other versions available see [here](#versions-available). -You can install q from the VS Code extension. You still need to register for [kdb Insights Personal Edition](https://kx.com/kdb-insights-personal-edition-license-download/) to obtain a license and VS Code will guide you through steps to install. The license will be incorporated as part of the kdb VS Code extension, although a `QHOME` environment variable is still required if you wish to utilise the q installation outside of VS Code. -**Step 1**: Click `Install new instance` from the prompt if no q installation is flagged. If this prompt is cancelled, then use the aforementioned [install q](#install-q). +#### Integrating q with VS Code -![installnewinstance](https://github.com/KxSystems/kx-vscode/blob/main/img/installnewinstance.jpg?raw=true) +To integrate VS Code with q take the following steps: -**Step 2**: If you have already registered for kdb+ or kdb Insights, then choose `Select/Enter a license`. If you haven't registered, choose `Acquire license`, this will open a dialog with a redirect link to register for [kdb Insights Personal Edition](https://kx.com/kdb-insights-personal-edition-license-download/). +1. Ensure the kdb VS Code extension is installed. -![findlicense](https://github.com/KxSystems/kx-vscode/blob/main/img/findlicense.jpg?raw=true) +1. Click `Install new instance` from the prompt. If the prompt is not visible ensure the kdb extension is selected in the Activity bar on the left, if that does not display the prompt, close and re-open VS Code. -**Step 3**: With the license secured, you can then link this to VS Code by either `pasting license string` or `selecting license file` from your PC; the latter method is recommended for new users. + ![installnewinstance](https://github.com/KxSystems/kx-vscode/blob/main/img/installnewinstance.jpg?raw=true) -![findlicense](https://github.com/KxSystems/kx-vscode/blob/main/img/pastelicense.jpg?raw=true) +1. An dropdown will be displayed with the two options: -The base64 encoded license string can be found in the welcome email received after registration, under the download link for the license file. + ![findlicense](https://github.com/KxSystems/kx-vscode/blob/main/img/findlicense.jpg?raw=true) -![welcomeemaillicense](https://github.com/KxSystems/kx-vscode/blob/main/img/weclomeemail.jpg?raw=true) + - If you haven't yet registered for q, choose `Acquire license`, this will open a dialog with a redirect link to register for [kdb Insights Personal Edition](https://kx.com/kdb-insights-personal-edition-license-download/). + + - If you have already registered for any of the [versions of q available](#versions-available) choose `Select/Enter a license`. -The `k4.lic` or `kc.lic` license file can be downloaded to your PC. +1. Once registered you will receive an email with you license details. -**Step 4**: Set a [`QHOME` environment variable](https://code.kx.com/q/learn/install/#step-5-edit-your-profile) to the location used by the kdb VS Code install. A notification dialog displays the location of q, as do the extension [settings](#settings). This will allows you to use q outside of VSCode. +1. With the license secured, you can link this to VS Code by either choosing `pasting license string` or `selecting license file` from your PC; the latter method is recommended for new users. -![qfound](https://github.com/KxSystems/kx-vscode/blob/main/img/installationofqfound.jpg?raw=true) + ![findlicense](https://github.com/KxSystems/kx-vscode/blob/main/img/pastelicense.jpg?raw=true) -If q is installed at `C:\q`, then `QHOME` is `C:\q`. + The base64 encoded license string can be found in the welcome email received after registration, under the download link for the license file. -To finish, a prompt is offered with an opt-in to receive a newsletter. + ![welcomeemaillicense](https://github.com/KxSystems/kx-vscode/blob/main/img/weclomeemail.jpg?raw=true) -## Connections + The `k4.lic` or `kc.lic` license file can be downloaded to your PC using the link also found in the welcome email. -The kdb VS Code extension allows you to connect VS Code to a q process; this can be remote, referred to as an [**unmanaged q session**](#my-q) or a [**managed q session**](#bundled-q), which uses the q installed as part of the kdb VS Code extension installation. It runs a child q process from within the extension and is fully managed by the extension. Only one connection can be active at any given time. +1. To finish, a prompt is offered with an opt-in to receive a newsletter. -### Bundled q +> **Using q outside of VS Code** -This runs a q session using the existing kdb installed as part of the kdb VS Code extension. +> If you want to use q outside of VS Code, set a [`QHOME` environment variable](https://code.kx.com/q/learn/install/#step-5-edit-your-profile) to the location used by the kdb VS Code install. A notification dialog displays the location of q, as do the extension [settings](#settings). T -**Step 1**: Click _connect to kdb server_ or _Add new connection_ from the _CONNECTIONS_ context menu. + ![qfound](https://github.com/KxSystems/kx-vscode/blob/main/img/installationofqfound.jpg?raw=true) -![connecttoakdbserver](https://github.com/KxSystems/kx-vscode/blob/main/img/connecttoakdbserver.png?raw=true) + If q is installed at `C:\q`, then `QHOME` is `C:\q`. -**Step 2**: A new window will open with the type of connection you desire to add, **Select Bundled q**. -![setendpoint](https://github.com/KxSystems/kx-vscode/blob/main/img/bundleqform.png?raw=true) +#### Versions available -**Step 3**: The _server name / alias_ will already be set as `local`. +There are commercial and non-commercial editions available. We recommend you start with the kdb+ Personal Edition or kdb Insights Personal Edition. The following table lists the editions with links to downloads and the functionality they support. + +| Edition | write q | run q queries | explore results | shared kdb process with kdb Insights | +| ----------------------------------------------------------------------------------------------- | ------- | ------------- | --------------- | ------------------------------------ | +| [kdb+ Personal Edition](https://kx.com/kdb-personal-edition-download/) | yes | yes | yes | no | +| [kdb Insights Personal Edition](https://kx.com/kdb-insights-personal-edition-license-download/) | yes | yes | yes | no | +| **kdb Insights Enterprise** | yes | yes | yes | yes | -**Step 4**: The _connection address_ will be already be set as `127.0.0.1` which corresponds to your _localhost_ +> **kdb Insights Enterprise** requires a commercial license. Please contact licadmin@kx.com for further information. -**Step 5**: Set the _port_ for the kdb server. Ensure the port used doesn't conflict with any other running q process; e.g. _5002_ +After registering for your chosen version, you will receive an email with a link to download an installation file and a `k4.lic` or `kc.lic` license file. Follow the instructions [here](https://code.kx.com/q/learn/install) for Linux, macOS and Windows to install q and a license file before proceeding. -- I want to learn more about [setting a q port](https://code.kx.com/q/basics/ipc/) +Once you have installed q and downloaded a license file the next step is to [Integrating q with VS Code](#integrating-q-with-vs-code). -**Step 6**: Click **Create connection** to confirm the _connection creation_ -**Step 7**: Right-click the q bundled process listed under _KX:CONNECTIONS_, and click _Start q process_. +## Connections -![setendpoint](https://github.com/KxSystems/kx-vscode/blob/main/img/managedqprocess.jpg?raw=true) +The kdb VS Code extension allows you to have multiple connections open at once, enabling development and testing across different q and kdb Insights Enterprise connections using both q and Python. -**Step 8**: From the same right-click menu, click _Connect kdb server_. This connects to the child q process running inside the kdb VS Code extension. +To add connections: -If you close the extension, the connection to the child q process also closes. +1. Select the extension from the Activity Bar and the **CONNECTIONS** View will be visible at the top of the Primary Sidebar. -### My q +1. When you first install the extension there are no connections so click **Add Connection** OR if there are existing connections click **+** for New Connection in the **Connections** menu. -**Step 1**: Identify the remote location of a running process. The hostname and port will be required along with any authentication information. + ![connecttoakdbserver](https://github.com/KxSystems/kx-vscode/blob/main/img/connecttoakdbserver.png?raw=true) -**Step 2**: Within the kdb VS Code extension, click _connect to kdb server_, or *Add new connection\*\* from the *CONNECTIONS\* context menu. + This opens the **Add a new connection** screen which has three tabs; one for each of the three connection types. + - [Bundled q](#bundled-q): This is a managed q session, which uses the q installed as part of the kdb VS Code extension installation. It runs a child q process from within the extension and is fully managed by the extension. + - [My q](#my-q): This is an unmanaged q session and is a connection to a remote q process. + - [Insights](#insights-connection): This accesses **kdb Insights Enterprise** API endpoints and a user-specific scratchpad process within a **kdb Insights Enterprise** deployment. -![connecttoakdbserver](https://github.com/KxSystems/kx-vscode/blob/main/img/connecttoakdbserver.png?raw=true) + ![setendpoint](https://github.com/KxSystems/kx-vscode/blob/main/img/bundleqform.png?raw=true) -**Step 3**: A new window will open with the type of connection you desire to add, **Select My q**. +1. Set the properties appropriate to the connection type. -![setendpoint](https://github.com/KxSystems/kx-vscode/blob/main/img/myq.png?raw=true) -**Step 4**: Assign a _server name / alias_. The server name selected **cannot be `local` or `insights`**, as these are reserved for use by [Bundled q connections](#bundled-q) and [Insights connections](#insights-connection), respectively; e.g. dev +### Bundled q -**Step 5**: Set the _connection address_ or ip address of the kdb server; e.g. _localhost_. +When you select **Bundled q** as the connection type and set the following properties: -**Step 6**: Set the _port_ used by the kdb server; e.g. _5001_. +|Property | Description | +|----------------------|--------------------------------------| +|Server Name | The name is already set as **local**.| +|The connection address| This is already be set as `127.0.0.1` which corresponds to your _localhost_ | +|Port | Port for the kdb server. Ensure the port used doesn't conflict with any other running q process; e.g. _5002_. [Read here for more about setting a q port](https://code.kx.com/q/basics/ipc/)| -- I want to learn more about [setting a q port](https://code.kx.com/q/basics/ipc/) +1. Click **Create Connection** and the connection appears under **CONNECTIONS** in the Primary Sidebar.. -**Step 7**: If authentication is needed, fill in the username and password fields, otherwise, leave these fields **blank** +1. Right-click the q bundled process listed under **CONNECTIONS**, and click _Start q process_. -**Step 8**: If TLS is enabled, check the checkbox. + ![setendpoint](https://github.com/KxSystems/kx-vscode/blob/main/img/managedqprocess.jpg?raw=true) -- I want to learn more [about TLS encryption](https://code.kx.com/q/kb/ssl/). +1. From the same right-click menu, click _Connect server_. This connects to the child q process running inside the kdb VS Code extension. -**Step 9**: Click **Create connection** to confirm the _connection creation_ +If you close the extension, the connection to the child q process also closes. -Upon completion, the localhost connection appears under _KX:CONNECTIONS_ in the left hand panel. +### My q -### Insights Connection +When you select **My q** as the connection type, identify the remote location of a running process. The hostname and port will be required along with any authentication information. -For kdb Insights Enterprise, the kdb VS Code extension is using a shared kdb process. Unlike for a **managed q session**, you must have [kdb Insights Enterprise Personal Edition](https://trykdb.kx.com/kx/signup) running before using these connections. +Set the following properties: -**Step 1**: Click _connect to kdb server_. +|Property | Description | +|-----------------------|--------------------------------------| +|Server Name | The server name / alias. The server name selected **cannot be `local` or `insights`**, as these are reserved for use by [Bundled q connections](#bundled-q) and [Insights connections](#insights-connection), respectively; e.g. dev| -![connecttoakdbserver](https://github.com/KxSystems/kx-vscode/blob/main/img/connecttoakdbserver.png?raw=true) +|The connection address | Set to the IP address of the kdb server; e.g. _localhost_. | +|Port | Set the _port_ used by the kdb server; e.g. _5001_. Learn more about [setting a q port](https://code.kx.com/q/basics/ipc/) . | +Username | If authentication is needed, fill in the username otherwise, leave **blank** | +Password | If authentication is needed, fill in the password otherwise, leave **blank** | +Enable TLS Encryption | Check the box is TLS is enabled. Learn more [about TLS encryption](https://code.kx.com/q/kb/ssl/). | -**Step 2**: A new window will open with the type of connection you desire to add, **Select Insights connection**. +![setendpoint](https://github.com/KxSystems/kx-vscode/blob/main/img/myq.png?raw=true). -![connecttoinsights](https://github.com/KxSystems/kx-vscode/blob/main/img/insightsconnection.png?raw=true) +1. Click **Create Connection** and the connection appears under **CONNECTIONS** in the Primary Sidebar. + +1. Right-click the my q process listed under **CONNECTIONS**, and click _Connect server_. This connects to the child q process running inside the kdb VS Code extension. -**Step 3**: Create a _server name / alias_; this can be any name, aside from `local`, which is used by [Bundled q connection](#bundled-q). +If you close the extension, the connection also closes. -**Step 4**: Set the _hostname_. This is the remote address of your kdb Insights Enterprise deployment: e.g `https://mykdbinsights.cloudapp.azure.com` -**Step 5**: Click **Create connection** to confirm the _connection creation_ +### Insights Connection -**Step 6**: The kdb Insights Enterprise connection is listed under _KX:Connections_, with its own icon. Right-click the connection and _Connect to Insights_ +For kdb Insights Enterprise, the kdb VS Code extension uses a shared kdb process. Unlike for a **managed q session**, you must have [kdb Insights Enterprise Personal Edition](https://trykdb.kx.com/kx/signup) running before using these connections. -![connecttoinsights](https://github.com/KxSystems/kx-vscode/blob/main/img/kdbinsightsconnection.jpg?raw=true) +When you select **Insights connection** as the connection type, the kdb VS Code extension uses a shared kdb process. Unlike for a **managed q session**, you must have [kdb Insights Enterprise Personal Edition](https://trykdb.kx.com/kx/signup) running before using connections of this type. -**Step 7**: The kdb VS Code extension runs an authentication step with the remote kdb Insights Enterprise process; sign-in to kdb Insights Enterprise. +Set the following properties: -![authenticateinsights](https://github.com/KxSystems/kx-vscode/blob/main/img/insightsauthenticate.jpg?raw=true) +|Property | Description | +|----------------------|--------------------------------------| +|Server Name | The server name / alias. This can be any name, aside from `local`, which is used by [Bundled q connection](#bundled-q)| +|The connection address|This is the remote address of your kdb Insights Enterprise deployment: e.g. `https://mykdbinsights.cloudapp.azure.com` | -After a successful connection to a kdb Insights Enterprise process, a new _DATA SOURCES_ panel will become available in the kdb VS Code extension. -![insightsdatasources](https://github.com/KxSystems/kx-vscode/blob/main/img/datasources.jpg?raw=true) +![connecttoinsights](https://github.com/KxSystems/kx-vscode/blob/main/img/insightsconnection.png?raw=true) + +1. Click **Create Connection** and the kdb Insights Enterprise connection appears under **CONNECTIONS** in the Primary Sidebar. + +1. Right-click the connection, and click _Connect server_. + + ![connecttoinsights](https://github.com/KxSystems/kx-vscode/blob/main/img/kdbinsightsconnection.jpg?raw=true) + +1. The kdb VS Code extension runs an authentication step with the remote kdb Insights Enterprise process; sign-in to kdb Insights Enterprise. + + ![authenticateinsights](https://github.com/KxSystems/kx-vscode/blob/main/img/insightsauthenticate.jpg?raw=true) + +If you close the extension, the connection also closes. + +Once connected, go to [execute code](#execute-code). -Once connected to a q process, go to [execute code](#execute-code). [//]: # "In what context is the reserved alias name `insights` used? - BMA - the context is used on build the connection tree; different icon; different connection process. - DF - Is this connection process currently supported in kdb VS Code extension; if so, do we need to document it here?" @@ -212,15 +250,22 @@ While developing q scripts, the kdb VS Code extension supports: - Go to definition - Navigate to the definition of a function + Navigate to the definition of a function - Find/go to all references - View references of a function both on the side view and inline with the editor + View references of a function both on the side view and inline with the editor + + ![Find all references](https://github.com/KxSystems/kx-vscode/blob/main/img/find-all-references.png?raw=true) + + ![Go to References](https://github.com/KxSystems/kx-vscode/blob/main/img/go-to-references.png?raw=true) + +- Outline View + + Use the Overview at the bottom of the Explorer view which shows the symbol tree of the currently active q file. - ![Find all references](https://github.com/KxSystems/kx-vscode/blob/main/img/find-all-references.png?raw=true) + ![Outline View](https://github.com/KxSystems/kx-vscode/blob/main/img/outline.png?raw=true) - ![Go to References](https://github.com/KxSystems/kx-vscode/blob/main/img/go-to-references.png?raw=true) ### Code Completion @@ -240,61 +285,171 @@ Supports renaming symbols in text editor. Right-click and select "Rename Symbol" Leaning on VS Code's extensive integrations with SCMs, all code is typically stored and loaded into a VS Code workspace. From there, the kdb VS Code extension allows you execute code against both kdb processes, and kdb Insights Enterprise endpoints. -### kdb process executing code +### kdb process executing q and Python code -There are three options available from the right-click menu for executing code: +For any file with a `.q` or `.py` extension there are additional options available from the right-click menu for executing code: -- Execute current selection +- Execute entire file - Takes the current selection (or current line if nothing is selected) and executes it against the connected q process. Results are displayed in the [output window and/or the kdb results window](#view-results). + Takes the current file and executes it against the active connection Results are displayed in the [output window](#view-results). Returned data is displayed in the [kdb results window](#view-results). -- Execute entire file +- Execute current selection - Takes the current file and executes it against the connected q process. Results are displayed in the [output window](#view-results). Returned data are displayed in the [kdb results window](#view-results). + Takes the current selection (or current line if nothing is selected) and executes it against the active connection. Results are displayed in the [output window and/or the kdb results window](#view-results). -- Run q file in new q instance +- Run q file in new q instance - for `.q` files only if q is installed and executable from the terminal - If q is installed and executable from the terminal, you can execute an entire script on a newly launched q instance. Executing a file on a new instance is done in the terminal, and allows interrogation of the active q process from the terminal window. + You can execute an entire q script on a newly launched q instance. Executing a file on a new instance is done in the terminal, and allows interrogation of the active q process from the terminal window. ### Insights query execution kdb Insights Enterprise offers enhanced connectivity and enterprise level API endpoints, providing additional means to query data and interact with kdb Insights Enterprise that are not available with standard kdb processes. You must have an instance of kdb Insights Enterprise running, and have created a [connection](#connections) within the kdb VS Code extension. -Similarly, you can execute arbitrary code against kdb Insights Enterprise. The code is executed on a user-specific sandbox process within the kdb Insights Enterprise deploy. The sandbox is instanciated upon the first request to execute code when connected to a kdb Insights Enterprise connection. It remains active until timed out or until you log out. +Similarly, you can execute arbitrary code against kdb Insights Enterprise. The code is executed on a user-specific scratchpad process within the kdb Insights Enterprise deploy. The scratchpad is instantiated upon the first request to execute code when connected to a kdb Insights Enterprise connection. It remains active until timed out or until you log out. #### Data sources -kdb Insights Enterprise supports the use of data sources, where you can build a query within VS Code and run it against the [kdb Insights Enterprise API endpoints](https://code.kx.com/insights/api/index.html). The UI helps you to build a query based on the available API on your instance of kdb Insights Enterprise, parameterize it and return the data results to the output or kdb results window. +KX Data source files allow you to build a query within VS Code, associate it with a connection and run it against the [kdb Insights Enterprise API endpoints](https://code.kx.com/insights/api/index.html). These are workspace specific files that have the following features: + +- Listed in the **DATASOURCES** View in the Primary Sidebar. +- Can be associated with a connection +- Have the **kdb.json** extension +- Are stored in a **.kx** folder at the root of your open folder + + The data source UI helps you to build a query, based on the available API on your instance of **kdb Insights Enterprise**, parameterize it and return the data results to the output or kdb results window. + +To create a data source and run it against a specific connection: + +1. Ensure you have at least one folder open in VS Code. +1. In the **DATASOURCES** View, click **+**. Specify the parameters defined in the following table: + + | Property | Description | + |-----------------------|-------------------------------------------------------------------------| + | Connection | Select a Connection from the Connections dropdown | + | Select API | Choose **getData** from the **Select API** dropdown. | + | Table | Choose the table you wish to query from the **Tables** dropdown. | + | Start Time/End Time | Select the Start and End Time for the query. | + | Additional Parameters | You can choose from the additional parameters as required. | + -To create a data source: +1. Click **Save** to store the settings you have chosen, for reuse later. When you save a data source; query parameters and the connection details are stored. The data source icon is green if it is associated with a connection and grey if there is no association. -1. In the Data Sources view, click the Options button and select 'Add Data Source'. -1. Click on the created data source where the name, API and parameterization can be applied. -1. Click Save to persist the data source to the VS Code workspace. + ![data Source](https://github.com/KxSystems/kx-vscode/blob/main/img/data-source.png?raw=true) -![data Source](https://github.com/KxSystems/kx-vscode/blob/main/img/data-source.png?raw=true) +1. Click **Run**. -To run a data source, click 'Run' and the results populate the output and kdb results windows. +1. The results populate: + + - The kdb results window, if it is active + + ![KDB Results](https://github.com/KxSystems/kx-vscode/blob/main/img/datasource-kdbresults.png) + + - Otherwise it will populate the output window + + ![Output](https://github.com/KxSystems/kx-vscode/blob/main/img/datasource-output.png) + + +In addition to [API queries](https://code.kx.com/insights/api/database/query/get-data.html), if the query environment is enabled on the deployed instance of kdb Insights Enterprise, qSQL and SQL queries can be used within a data source with the appropriate parameterization. If qSQL or SQL is required and issues occur trying to run these queries contact a kdb Insights Enterprise administrator for assistance. -In addition to [API queries](https://code.kx.com/insights/api/database/query/get-data.html), if the query environment is enabled on the deployed instance of kdb Insights Enterprise, qSQL and SQL queries can be used within a data source with the appropriate parameterization. #### Populate scratchpad -You can use a data source to populate a scratchpad with a dataset, allowing you to build complex APIs and pipelines within VS Code and kdb Insights Enterprise. +You can use a data source to populate a scratchpad process running in a **kdb Insights Enterprise** instance with a dataset. This allows you to then execute q or python code against the data stored in that variable in the scratchpad. This facilitates the generation of complex APIs and pipelines within VS Code and kdb Insights Enterprise. To do this: -1. Create a data source and execute it using the 'Populate Scratchpad' button. - The scratchpad is populated. -1. At the prompt, provide a variable to populate your own scratchpad instance with the data. +1. Create a data source and execute it by clicking **Populate Scratchpad**. + + ![Populate Scratchpad](https://github.com/KxSystems/kx-vscode/blob/main/img/populate-scratchpad-button.png?raw=true) + +1. At the prompt, provide a variable to populate your own scratchpad instance running in the connected kdb Insights Enterprise with the data. + + ![Populate Scratchpad Variable](https://github.com/KxSystems/kx-vscode/blob/main/img/populate-scratchpad.png?raw=true) + +1. The scratchpad process is populated. + +1. Use a [Workbook](#workbooks) to execute q or Python code against the data in your scratchpad using the variable you provided. + + + +#### Workbooks + +Workbooks provide a convenient way to prototype and execute q and python code against a q process and using the variables [populated into the scratchpad](#populate-scratchpad) of a **kdb Insights Enterprise** deployment by data sources. + +Standard **.q** and **.py** files only run on the active connection. Workbook files have the following features: + +- Listed in the **WORKBOOKS** View in the Primary Sidebar. +- Can be associated with a connection +- Have the **.kdb.q.** or **kdb.py** extension +- Are stored in a **.kx** folder at the root of your open folder + +This allows you to have multiple Workbooks running against different connections at the same time. + +To create a Workbook and run code against a specific connection: + +1. Ensure you have at least one folder open in VS Code. +1. In the **WORKBOOKS** View in the Primary Sidebar, click one of the '+' buttons to create a q or Python workbook. + + ![new workbook](https://github.com/KxSystems/kx-vscode/blob/main/img/addnewworkbook.png?raw=true) + +1. Write code you wish to execute. + +1. Run the code: + + 1. To run all the code in the file you can use one of the following methods: + + 1. Click 'Run' from above the first line of code in the workbook file. + ![workbook links](https://github.com/KxSystems/kx-vscode/blob/main/img/workbookrunlink.png) + + 1. Select 'Run' from the upper right of the editor. Using the dropdown next to the button you can choose to 'KX: Execute Entire File' or 'KX Execute Current Selection'. + ![play dropdown](https://github.com/KxSystems/kx-vscode/blob/main/img/wortkbookplaydropdown.png) + + 1. Click the 'Run' button on the right-hand side of the status bar. + ![status bar run ](https://github.com/KxSystems/kx-vscode/blob/main/img/workbookstatusbarrun.png) + + 1. Right click and choose 'KX: Execute Entire File' from the menu. + + 1. If you wish to only run the current selection (or current line if nothing is selected), right click and choose 'KX: Execute Current Selection' from the menu. + +1. If you have not yet chosen a connection to associate with the workbook you will be asked to choose a connection before the code is executed. + ![choose connection](https://github.com/KxSystems/kx-vscode/blob/main/img/workbookconnectionlink.png) + +1. The results populate the kdb results window if it is active, otherwise it will populate the output window. + +When you save a workbook file the code and the connection details are stored. The workbook icon will be green if it is associated with a connection and grey if there is no association. + +You can also change the connection associated with a workbook at any time by clicking on 'Choose Connection' from above the first line of code in the workbook file. + ![choose connection](https://github.com/KxSystems/kx-vscode/blob/main/img/wortkbookplaydropdown.png) + + + +#### Query History + +The Query History View in the Primary Sidebar captures each query execution and enables you to re-run any of the queries listed. Initially the query history view will be empty but once you run a query it will be captured and displayed in the window - with a separate row displayed for every execution. All information is stored in memory and not persisted upon application exit. + +![Query History](https://github.com/KxSystems/kx-vscode/blob/main/img/query-history.png) + +Rows consists of the following parts: + +- Status icon: Shows if the code or query executed successfully or an error occurred. +- Connection Name: The Server name associated with the connection on which the query was executed +- Connection Type: 'local', 'myq' or 'insights' +- Time: Time the query was execute +- Data Source/File/Workbook: Name of the file being executed +- Query: When Code is being execute the code is shown +- Data Source Type: When a datasource is being run 'API' is displayed + +![Query History Details](https://github.com/KxSystems/kx-vscode/blob/main/img/query-history-details.png) + + +Right-clicking on a history row will bring up a menu with the option to 'Rerun query'. -1. Return to VS Code and execute q code against the data in your scratchpad. +![Query History Clear](https://github.com/KxSystems/kx-vscode/blob/main/img/query-history-clear.png) -![Populate Scratchpad](https://github.com/KxSystems/kx-vscode/blob/main/img/populate-scratchpad.png?raw=true) ## View results -All query executions happen remotely from the kdb VS Code extension either against a running q process or against an instance of kdb Insights Enterprise. The results, successful or otherwise, are returned to VS Code as: +All query executions happen remotely from the **kdb VS Code extension** either against a running q process or against your user-specific scratchpad process in kdb Insights Enterprise. The results, successful or otherwise, are returned to VS Code as: - An output view @@ -306,7 +461,7 @@ All query executions happen remotely from the kdb VS Code extension either again ![Output autoscrolling](https://github.com/KxSystems/kx-vscode/blob/main/img/auto-scrolling.png?raw=true) - **Note** You can hide or show the full details in the console output. Go to settings of the vscode, search for kdb, check the option "Hide Detailed Console Query Output" (this option is checked by default) + **Note** You can hide or show the full details in the console output. Go to settings of the VS Code, search for kdb, check the option "Hide Detailed Console Query Output" (this option is checked by default) ![Hide Detailed Console Query Output](https://github.com/KxSystems/kx-vscode/blob/main/img/hide-detailed-console-query.png?raw=true) @@ -324,17 +479,18 @@ q REPL can be started from the command prompt by searching "q REPL". ## Settings -To update kdb VS Code settings, search for `kdb` from _Preferences_ > _Settings_, or right-click the settings icon in kdb VS Code marketplace panel and choose _Extension Settings_. +To update kdb VS Code settings, search for `kdb` from _Preferences_ > _Settings_, or right-click the settings icon in kdb VS Code marketplace panel and choose **Extension Settings**. | Item | Action | | ---------------------------------------------------------- | ------------------------------------------------------------------- | | Hide notification of installation path after first install | yes/no; default `no` | | Hide subscription to newsletter after first install | yes/no; default `no` | | Insights Enterprise Connections for Explorer | [edit JSON settings](#insights-enterprise-connections-for-explorer) | +| Linting | Enable linting for q and quke files | | QHOME directory for q runtime | Display location path of q installation | | Servers | [edit JSON settings](#servers) | -### Insights Enterprise Connections for Explorer +### kdb Insights Enterprise Connections for Explorer ```JSON { diff --git a/img/addnewworkbook.png b/img/addnewworkbook.png new file mode 100644 index 00000000..62bc34bc Binary files /dev/null and b/img/addnewworkbook.png differ diff --git a/img/data-source.png b/img/data-source.png index 38a4b721..b97e9103 100644 Binary files a/img/data-source.png and b/img/data-source.png differ diff --git a/img/datasource-kdbresults.png b/img/datasource-kdbresults.png new file mode 100644 index 00000000..6c47700d Binary files /dev/null and b/img/datasource-kdbresults.png differ diff --git a/img/datasource-output.png b/img/datasource-output.png new file mode 100644 index 00000000..9befb532 Binary files /dev/null and b/img/datasource-output.png differ diff --git a/img/image.png b/img/image.png new file mode 100644 index 00000000..a8fb7f1f Binary files /dev/null and b/img/image.png differ diff --git a/img/kdbinsightsconnection.jpg b/img/kdbinsightsconnection.jpg index 9bd55bfa..be28c404 100644 Binary files a/img/kdbinsightsconnection.jpg and b/img/kdbinsightsconnection.jpg differ diff --git a/img/outline.png b/img/outline.png new file mode 100644 index 00000000..dab2e7de Binary files /dev/null and b/img/outline.png differ diff --git a/img/populate-scratchpad-button.png b/img/populate-scratchpad-button.png new file mode 100644 index 00000000..3b265e5f Binary files /dev/null and b/img/populate-scratchpad-button.png differ diff --git a/img/populate-scratchpad.png b/img/populate-scratchpad.png index 5571e343..86774e0d 100644 Binary files a/img/populate-scratchpad.png and b/img/populate-scratchpad.png differ diff --git a/img/query-history-clear.png b/img/query-history-clear.png new file mode 100644 index 00000000..6ce4185a Binary files /dev/null and b/img/query-history-clear.png differ diff --git a/img/query-history-details.png b/img/query-history-details.png new file mode 100644 index 00000000..21488601 Binary files /dev/null and b/img/query-history-details.png differ diff --git a/img/query-history.png b/img/query-history.png new file mode 100644 index 00000000..7e593fd5 Binary files /dev/null and b/img/query-history.png differ diff --git a/img/workbookchooseconnection.png b/img/workbookchooseconnection.png new file mode 100644 index 00000000..848793b7 Binary files /dev/null and b/img/workbookchooseconnection.png differ diff --git a/img/workbookconnectionlink.png b/img/workbookconnectionlink.png new file mode 100644 index 00000000..2e994cd8 Binary files /dev/null and b/img/workbookconnectionlink.png differ diff --git a/img/workbookplaydropdown.png b/img/workbookplaydropdown.png new file mode 100644 index 00000000..cad847b0 Binary files /dev/null and b/img/workbookplaydropdown.png differ diff --git a/img/workbookrunlink.png b/img/workbookrunlink.png new file mode 100644 index 00000000..58e97d7e Binary files /dev/null and b/img/workbookrunlink.png differ diff --git a/img/workbookstatusbarrun.png b/img/workbookstatusbarrun.png new file mode 100644 index 00000000..5f368d9c Binary files /dev/null and b/img/workbookstatusbarrun.png differ