Review documentation

README.md

@@ -1,3 +1,28 @@
 # README
 
-See [Wiki](https://github.com/dataiku/lca/wiki).
+## Plugin documentation
+
+The documentation website for Visual Edit is at [https://dataiku.github.io/dss-visual-edit/](https://dataiku.github.io/dss-visual-edit/). Quick Links:
+
+* For Data Experts:
+  * [Installation](install-plugin)
+  * User guides:
+    * [Use Case #1: Correcting Source Data](get-started)
+    * [Use Case #2: Reviewing Machine-Generated Data](reviewing)
+    * [Deploying in Production](deploy)
+  * Advanced features:
+    * [Linked Records](linked-records)
+    * [Editschema](editschema)
+    * [Python API](get-started-crud-python-api)
+  * [FAQ](faq)
+* For Business Users:
+  * [How to use the data table](data-table-features)
+
+## Contents of this repository
+
+* `dash_tabulator`: source code for the Dash component powering the plugin's "data table", based on the [Tabulator](http://tabulator.info) Javascript library.
+* `plugin-dss-visual-edit`: actual plugin code; when installing the plugin in Dataiku, we would provide this directory only.
+* `docs/`: source code in Markdown for the documentation website
+  * `backend/`: technical documentation generated from docstrings by Mkdocs (following [this tutorial](https://realpython.com/python-project-documentation-with-mkdocs/)). Updates are manual. They require running `mkdocs build` from `plugin-dss-visual-edit/python-lib/` and moving the output (in `site/`) to the repo's `docs/backend/`.
+
+In addition to the documentation website, check out these directories' README files for additional explanations of how the plugin works.

docs/data-table-features.md

@@ -1,28 +1,14 @@
-# Data table features
-
-## Rich editing experience
-
-* **Automatic input validation**: Formatting and editing adapt to the detected data types:
-  * Numerical cells don’t allow characters
-  * Boolean cells are edited via a checkbox
-  * Date cells are edited via a date picker
-  * Linked records restrict selection to entries in a linked dataset.
-* **Airtable-like editing of [linked records](linked-records)** (aka foreign keys) via dropdown, with the ability to **search and display lookup columns** in real time, from a linked dataset of any size.
-
-## Powerful data browsing
-
-* **Filtering, sorting and grouping** in real time.
-* Re-ordering, re-sizing and hiding columns, so you can focus on content that's important to you.
-* Pagination of large datasets.
-* **Automatic detection of changes** in the source dataset.
-
-## How to use these features
+# How to use the data table
 
 * Right-clicking on the column name will show a menu with an option to hide the column, and an option to group rows by value.
-* Filtering:
-  * The default filter is a textual one.
-  * In the case of a display-only boolean column, the filter is a tri-state checkbox (or a simple toggle if you specified the column type to be "boolean_tick" via the advanced settings ([editschema](editschema)).
-  * Editable boolean columns have a textual filter that you can use by typing "true" or "false".
+* Sorting is done by clicking on the arrow next to column name.
+* Filtering widgets are located below each column name.
 * All of this can be reset by clicking on the "Reset View" button in the bottom-left corner.
 * Linked records are edited via dropdowns which include the ability to search through all options: when there are many of them, only options that start with the search term are presented.
-* Changes in the original dataset are automatically detected and signaled. The user needs to refresh the webpage in order to see them.
+* Changes in the original dataset are automatically detected and signaled. The user needs to refresh the webpage in order to see them.
+
+## Troubleshooting
+
+If the webapp isn't functioning as expected, clear the browser's cached images and files.
+
+If you get a "backend" error, get in touch with your admin.

docs/faq.md

@@ -1,19 +1,9 @@
 # FAQ
 
-## The webapp isn't functioning as expected. What should I do?
-
-If the webapp backend started successfully but the webapp itself isn't functioning as expected, clear the browser's cached images and files.
-
 ## Can the webapp be used by several users at the same time?
 
 Several users can view and edit data at the same time, but they won't see each other's edits in real-time; if 2 or more users try to edit the same cell at the same time, all their edits will be logged but only the last edit will make its way to the _editlog\_pivoted_ dataset.
 
-## What happens if my source dataset changes?
-
-The webapp automatically detects changes in the original dataset, by periodically checking the last build date of the dataset. As a consequence, it only works if this isn't a "source" dataset but the output of a recipe.
-
-![](refresh_data.png)
-
 ## What happens if I change primary keys or editable columns in the webapp settings?
 
 * Primary key:

docs/get-started.md

@@ -1,10 +1,10 @@
-# Getting started | Plugin: Visual Edit | Dataiku
+# Correcting Source Data | Plugin: Visual Edit | Dataiku
 
 ## Use case description
 
 There are two main types of use cases for the plugin's Visual Webapp:
 
-* Making corrections on "source" data (meaning data that's not machine-generated)
+* Correcting source data
 * Reviewing machine-generated data
 
 In this guide, we focus on the former. The latter is slightly more complex and will be covered in a separate guide.
@@ -117,5 +117,5 @@ The Scenario tile is displayed as a button to run a chosen scenario (typically t
 
 * If you need to customize the way the webapp displays data and create your own Visual Edit front-end, check out the guide to our [CRUD Python API](get-started-crud-python-api) and the examples it contains to learn how to use the plugin's data persistence layer for your webapp's backend.
 * Because we're building a project with an interface where users can enter data and this gets processed, we'll need to have two instances of the project leveraging the plugin: one for development, one for production; each will have its own set of edits. Once all your tests are successful, the next step is to [deploy your project](deploy) on an automation node, or as a duplicate project on your design node.
-* If you're interested in the use case of reviewing machine-generated data, check out the [dedicated guide](reviewing).
-* You can also check out the [FAQ](faq) to learn more about the plugin.
+* If you're interested in the use case of [reviewing machine-generated data](reviewing), check out the dedicated guide.
+* You can also check out the [FAQ](faq) to learn more about the plugin.

docs/index.md

@@ -1,39 +1,37 @@
 # Plugin: Visual Edit | Dataiku
 
-## Build apps to edit data and write it back to the Flow
+## Build apps to edit records of data within pipelines
 
 <iframe src="https://www.loom.com/embed/7b79e45e755544f8baf1ff3ed1bf60ee" frameborder="0" webkitallowfullscreen="" mozallowfullscreen="" allowfullscreen="" style="height: 400px; width: 600px"></iframe>
 
 [Book a demo!](https://calendar.google.com/calendar/u/0/appointments/schedules/AcZssZ1cgQ-IQ2k2eJMm6mUrZxabQgtPSSwaZ9TgNcHcnaUDvrqfweAkf-B7xzZbTSNyYeSRc2smgLbp){: .btn}
 
-## Use cases
+Use Cases: enable domain experts to...
 
-* Human-in-the-loop analytics and AI:
-  * **Entity Resolution** (aka record linkage): review and correct fuzzy matches between 2 or more datasets representing the same entities (e.g. customers, suppliers, products) but with differing IDs and names. [Demo project →](sample-project-company-resolution)
-  * **AI Validation and Improvement**: continuously provide feedback on AI outputs in production, and ingest back into the AI's training data. [Demo project →](sample-project-ai-feedback-app)
-  * **Forecast Override**: machine-made forecasts (e.g. sales) need to be adjusted by a human, to take into account contextual information that wasn't available to the machine, or to correct low-confidence predictions.
-  * **Automation Reviewing**: deal with exceptions by overriding values in the output of business rules, or filling in missing values.
-* **Data Quality Campaigns**: fix incorrect records and improve downstream analytics.
+* Make tactical corrections on source data, to ensure correctness of an analytics project.
+* Review the output of rules or AI used for
+  * Mass corrections/enrichment of source data: **entity resolution, deduplication, categorization**, etc.
+  * Operational processes: **log decisions on alerts, validate suggestions of amounts to pay or orders to place**, etc.
 
-## Features
+Features:
 
-* **No-code webapp**:
-  * Powerful data browsing (filtering, sorting, grouping in real-time)
-  * Rich editing experience (automatic input validation, linked records, lookup columns)
-  * [Learn more →](data-table-features)
-* **Dataset of edited rows**: use as input to a recipe that writes back to a source system, or for the implementation of a feedback loop (e.g. stacking with an AI training dataset).
-* **Governance**: built-in audit trail of all edits made by end-users.
-* **Reviewing**: when marking a row of generated data as reviewed, generated values are automatically persisted (even if they weren't edited).
-* **Continuous updates**: seamless integration of new edits into production data pipelines, and of new data into the webapp (changes are detected automatically and previous edits are applied to the new data).
-* **API to Create, Read, Update, Delete rows (CRUD)**, for coders building custom webapps.
-
-## [Get started right away!](get-started)
+* Powerful data table with rich editing experience and a special Review checkbox.
+* Easy webapp setup (no-code and low-code options) and integration of edits in your pipeline.
+* Ability to leverage human feedback for continuous AI improvement.
+* Audit trail of all edits.
 
 ## Quick links
-* [Plugin installation](install-plugin)
-* [Data table features](data-table-features)
-  * [Linked records](linked-records)
-  * [Editschema](editschema)
-* [Python library](get-started-crud-python-api)
-* [Deploying in production](deploy)
-* [FAQ](faq)
+
+* For Data Experts:
+  * [Installation](install-plugin)
+  * User guides:
+    * [Use Case #1: Correcting Source Data](get-started)
+    * [Use Case #2: Reviewing Machine-Generated Data](reviewing)
+    * [Deploying in Production](deploy)
+  * Advanced features:
+    * [Linked Records](linked-records)
+    * [Editschema](editschema)
+    * [Python API](get-started-crud-python-api)
+  * [FAQ](faq)
+* For Business Users:
+  * [How to use the data table](data-table-features)

docs/install-plugin.md

@@ -3,26 +3,22 @@
 ## Requirements
 
 * Dataiku 9 or more recent
-* Ability to install plugins on your Dataiku instance and to create an associated code env based on Python 3
-* For Dataiku Online:
+* Ability to install plugins on your Dataiku instance and to create an associated code env based on Python 3.9
+* For Dataiku Cloud:
   * Make sure the "code env" feature is activated; if it's not, use the contents of the requirements file to create your first environment
   * Enable maintenance access so that someone can install the plugin for you
 
 ## How to install
 
-* [Contact Louis Dorard at Dataiku to get access to the plugin (private beta)](https://calendar.google.com/calendar/u/0/appointments/schedules/AcZssZ1cgQ-IQ2k2eJMm6mUrZxabQgtPSSwaZ9TgNcHcnaUDvrqfweAkf-B7xzZbTSNyYeSRc2smgLbp)
-* Upload the ZIP file that contains the plugin, to your Dataiku instance
-  * ![](plugins.png)
-  * ![](add_plugin_upload.png)
-  * ![](add_plugin_select_zip.png)
+* TODO: instructions to install via Git
 * Create the code environment for this plugin
-  * Select the types of containers you plan to use as backend for Visual Edit webapps or for running recipes provided by the plugin. The easiest is to choose All. ![](add_plugin_creating_code_env.png)
+  * Select the types of containers you plan to use as backend for data editing webapps or for running recipes provided by the plugin. The easiest is to choose All. ![](add_plugin_creating_code_env.png)
   * ![](add_plugin_creating_code_env_2.png)
   * ![](add_plugin_creating_code_env_done.png)
   * ![](add_plugin_creating_code_env_done_2.png)
   * ![](add_plugin_done.png)
-* The plugin is installed! ![](plugin_installed.png)
+* The plugin is installed! ![](plugin_installed.png) We'll see how to use each of its components in the next section.
 
 ## Next
 
-Check out the guide to [Get started](get-started) for an introduction to the plugin's components and how to use them.
+Check out the guide to [Get started](get-started) for an introduction to the plugin's components and how to use them.