diff --git a/website/docs/guides/visual-editor-qs.md b/website/docs/guides/visual-editor-qs.md new file mode 100644 index 00000000000..2ed6e9633dc --- /dev/null +++ b/website/docs/guides/visual-editor-qs.md @@ -0,0 +1,288 @@ +--- +title: "Quickstart for the dbt Cloud Visual Editor" +id: "visual-editor" +level: 'Beginner' +icon: 'dbt' +hide_table_of_contents: true +tags: ['Visual Editor','Analyist', 'dbt Cloud','model'] +recently_updated: true +--- + +
+ +## Introduction + +:::note beta feature + +The Visual Editor is currently in a limited beta. [Contact us](https://www.getdbt.com/contact/) if you're interested in joining it. Features currently in the beta are subject to change or removal. + +::: + +The dbt Cloud Visual Editor offers a quick and straightforward way for anyone to build analytics models, no background in analytics engineering is required! In this guide, you will learn about: + +- Accessing the Visual Editor and creating a new model +- Navigating the interface +- Building a model using operators +- Committing your changes to Git +- Locating your Visual Editor model and data + +## Prerequisites + +To use the Visual Editor, you must meet the following prerequisites: + +- Be on an [Enterprise dbt Cloud account](/faqs/Accounts/cloud-upgrade-instructions#enterprise-plan) +- You have the following configured on your account: + - A data warehouse connection + - Integration with a Git provider +- Source models for the Visual Editor must have been run at least once in production +- You have a `developer` license +- You have credentials configured for your data warehouse and Git provider in the **Your profile** section of the **Account settings** + +:::note Prerequisite for using the Jaffle Shop + +The examples in this guide use the [Jaffle Shop](https://github.com/dbt-labs/jaffle-shop) GitHub repo sample project. You can use your own data, but the Jaffle Shop offers a full-featured project useful for testing dbt features. Ask your dbt Cloud administrator about importing it to a project in your environment. They must also execute `dbt run` on the Jaffle Shop project in your `Production` environment before you begin, or you will be unable to reference the source models. + +::: + +## Access the Visual Editor + +To access the Visual Editor: + +1. Click **Develop** from the main menu. If you do not see the **Develop** option, ensure you have selected a **Project** from the menu. +2. Click **Visual Editor**. + + + +3. From the right side, click **Get started** and then click **Create new model**. + + + +## Navigating the interface + +The Visual Editor comprises a series of menus activated by clicking icons surrounding the border of the larger canvas. With none of the menu items activated, the workspace looks like this: + + + +Click on an icon to expand its section or execute an action depending on its purpose. The options are as follows: + +1. The model's title. This defaults to "Untitled" but can be edited anytime by clicking on it. +2. The **Operators** toolbar contains the building blocks for creating a model with the editor. +3. The **SQL code** area displays the SQL that compiles your model. +4. The **Runs and previews** that displays run data and previews data for individual operators. +5. The **Commit history** display. +6. The navigation tab that has icons for (from top to bottom): + - Zoom in + - Zoom out + - Center the model to fit the screen + - Zoom to selection (the operator selected on or off screen will be zoomed and centered) + - Auto-layout option for the individual operator tiles +7. The [Copilot](/docs/cloud/dbt-copilot) icon (where available). Use natural language to build your dbt Visual Editor models. +8. The **Run** command executes `dbt run` for the model. +9. This button is initially a **Commit** command for your integrated Git provider. It will change to "Open pull request" once your first commit is made. The button will not initially appear until you begin working in the canvas area. + +## Create a model + +This section will walk you through creating a model with operators using sample data from the [Jaffle Shop](https://github.com/dbt-labs/jaffle-shop) project. With this guide, you will create a basic model that transforms two datasets to build a view of repeat customer purchases while you consider a loyalty program for your shop. + +The operators are the heart of your model. They determine what data will be transformed and how. Click the **+** icon to open the operator menu. + +Operators are divided into three types: +- **Input:** Input operators configure the source data. +- **Transform:** Transform operators change and shape your data. +- **Output:** Output operators define your model name and location. + + + +Read more about the [individual operators](/docs/cloud/visual-editor-interface#operators) to understand the basic purpose of each. The dbt model created by the Visual Editor builds off of existing models. In this guide, there will be input (source) models and an output model (what you are building) which will be _your model_. + +### Operator tiles + +The operators are drag-and-drop from their menu to the canvas, and when they are dropped they will create a tile. + +The tiles have the same basic setup with different fields depending on their function. All operators except for **Model** must be connected to another tile before configuring. Once configured, they’ll have the same basic layout. + + + +1. **The connectors:** Click-and-drag to the connector on another operator to link them. +2. **The title:** Click to change. The examples and images in this guide will use the default names. +3. **Play icon and menu:** Preview the data at any point in its transformation by clicking the tile's **play icon**. The dropdown menu contains the option to **Delete** a tile. +4. **Column icon:** The number next to it represents the number of columns in the data at that point in its transformation. + +:::tip + +Make operator tile titles unique compared to your column names to avoid confusion, and the same applies to any aliases you create. + +::: + +### Create your model from pre-existing models + +To get started: + +1. Expand the **Operators** menu and drag the **Input Model** operator over to the canvas. +2. Click **Choose a model** and then select the source `stg_models` from the dropdown. +3. Click the **Select model** option in the window that lists the columns. + + + +You now have your first input model in Visual Editor! + +4. Drag a new **Input Model** operator to the canvas below the first and repeat the previous steps, but this time set the source model to `stg_order_items`. + + + +Now, you have two input models and are ready to transform the data! + +:::tip + +Don't see a pre-existing model you're looking for? Ask your dbt admins to ensure it's been run in your Production environment recently and hasn't gone stale. + +::: + +### Create a join + +1. From the **Operators** menu, drag the **Join** operator onto the canvas to the right of the source models. + + + +2. Click and drag a line from the **+** connector below the `L` on the join border to the **+** on the `stg_orders` model. Do the same for the `R` connector to the `stg_order_items` model. + + + +3. In the **Join** tile, click **Configure inputs.** +4. Set the **Join type** to `Inner`. +5. In the pair of dropdowns, set both `stg_orders` and `stg_order_items` to `ORDER_ID`. +6. Click **Select and rename columns** and click **Configure columns** +select the following columns: + - From `stg_orders` click `ORDER_ID` and `CUSTOMER_ID`. + - From `stg_order_items` click `PRODUCT_ID`. + - Note: These will appear in the order they are clicked. +7. You've now built your join! Test it by clicking the **Play icon** in the top right corner of the join tile. Your data will populate in the **Runs and previews** pane. + + + +:::tip + +Your work in the Visual Editor is automatically saved as you progress, so if you need a break, you can always come back to a session later. Just be sure to give it a unique title! + +::: + +## Enhance your model + +You've got the basics going with your Visual Editor model! It has successfully joined two pre-existing input models, but you want to transform the data further to get what you need: a list of customers who buy repeat items as you consider a loyalty club rewards program. + +### Aggregate data + +Multiple options for transforming your data include custom formulas, filters, and unions. Keep it simple and add an aggregation operator to tell you which customers buy the most repeat products. + +1. Drag the **Aggregation** operator over to the right of the join. +2. Connect the aggregation operator to the join operator. +3. Click **Configure aggregation** in the **Aggregation tile**. +4. Click in the **Group by** field and first select `CUSTOMER_ID` then `PRODUCT_ID`. +5. Configure the next three fields with the following: + - **Function:** Count + - **Column:** PRODUCT_ID + - **Alias:** count_PRODUCT_ID + + + +6. Click the **Play icon** to preview the data. You're starting to see the results you're looking for, but the data is scattered. Let's clean it up a bit more. + +:::tip + +As your model grows, you can zoom in and out to view your needs. Click and hold in empty canvas space to drag your setup across the screen. Click the **Fit view** icon to see your entire model on the screen. Click the **Auto layout** icon to auto-arrange the tiles efficiently. + +::: + +### Add some order + +There's a lot of data there. Dozens of customers are buying hundreds of products. You will sort it so that the customers are listed ascending by their CUSTOMER_ID number, with the most purchased products listed in descending order. + +1. Drag the **Order** operator over to the right of the **Aggregation** tile and connec them. +2. Click the **pencil edit icon**. +3. In the **Sort order** field click **Select column** and click `Aggregation1.CUSTOMER_ID` from the dropdown. Set it to `Asc`. +4. Click **Add sorting** and in the new **Select column** field select `Aggregation1.count_PRODUCT_ID`. Set it to `Desc`. +5. Press the **Play icon** to preview the new data. + + + +:::tip + +Want to practice on your own? Try adding a **Filter** operator that removes items with less than 10 sales for any customer ID. Be sure to run the preview and verify the data is correct. + +::: + +## Configure your output model + +Now that you've built your model, you need to customize the output name and location: +1. Drag the **Output Model** operator to the right of your **Order** operator. +2. Connect the **Order** and **Output Model** operators. +3. The **Output Model** configuration will default to the name of your Visual Editor project and the default models directory. Click the **pencil edit icon** to configure the optional fields: + - Edit the **Model name** field if you want the name to be different than that of your project. + - Edit the **File path** if you have a custom directory for your Visual Editor models. + - Hover over a column name and click the **-** icon to remove it from the output model. +4. Click the **play icon** to preview your final model. + + + +:::tip Model locations + +You can customize the location for Visual Editor models to keep them separate from other dbt Cloud models. Check with your dbt admins for best practices and ideas for Visual Editor model locations and naming conventions. + +::: + +## Run and share your model + +Now that you've built a model that results in the data you want, it's time to run it and push it to your Git repo. Before you run your model, keep a few items in mind: + +- When you run previews (at any stage in the process), it does not affect the state of your warehouse. So, you can test and develop in the Visual Editor without impacting anything outside of the dbt Cloud Development environment. +- When you're ready to use this mode in a downstream tool, you can run it to materialize it in your data warehouse development schema. +- Once your model is ready for production and ready to be used by others or orchestrated, commit it and open a PR. + +### Run + +To run your model, you only need to click the big **Run** button. With the Visual Editor, there is no command line and no need to memorize a list of commands; there is only **Run**. Click it to see the results populate in the **Runs and previews** pane. + + + +This will [materialize](/docs/build/materializations) the data as a `view` in your developer schema in the database. Once the model has been merged with your project and `dbt run` is executed in your Staging or Production environments, it will be materialized as a view in their related schemas. + + + +:::tip + +Have dbt [Copilot](/docs/cloud/dbt-copilot) enabled for your dbt Cloud Enterprise account? Clear the board and try using natural language to build the model in this guide without manually configuring any operators. + +::: + +### Git commit + +The models built in the Visual Editor are a part of your larger dbt project. They are stored in the `visual_editor` folder of your `/models` directory. This is all done automatically; you don't have to configure any paths or directories. + + + +However, it won't be created in your Git repo until you commit your first model. So, back in the model's view: + +1. Click **Commit** in the top right. + - If you've already created a commit and wish to make more, click the arrow next to **Create a pull request** to see the **Commit** option. +2. Fill out the **Description** field with information about your model. If it's long, part of it will be included in the pull request title, and the rest will be in the body. That's okay! You can correct it during the PR creation process. +3. Click **Commit**. +4. The **Commit** button will change to **Create a pull request**. You can add more commits, but click the **Create a pull request** button for now. You will then be redirected to your Git provider in a new tab. + +The following examples use GitHub as the provider: + + + +5. Click **Create pull request** in the GitHub window. +6. Complete the **Add a title** and **Add a description** fields. If your description is split between both, copy all the contents to the description field and give it a shorter title. +7. Click **Create pull request**. + +You've just submitted your first model from the Visual Editor for review. Once approved and merged, the model will be included in your organization’s project and run whenever `dbt run` is executed in any environment your model is in. You're now on your way to becoming an expert in data transformation! + +:::tip + +Want to take your skills to the next level? Try taking the SQL output from your Visual Editor model and using it to create a model in the [IDE](/docs/cloud/dbt-cloud-ide/develop-in-the-cloud). + +::: + + +
\ No newline at end of file diff --git a/website/static/img/docs/dbt-cloud/visual-editor/aggregation.png b/website/static/img/docs/dbt-cloud/visual-editor/aggregation.png new file mode 100644 index 00000000000..4aabcb5638f Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/aggregation.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/demo-model-github.png b/website/static/img/docs/dbt-cloud/visual-editor/demo-model-github.png new file mode 100644 index 00000000000..3ed5f590d36 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/demo-model-github.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/join-connected.png b/website/static/img/docs/dbt-cloud/visual-editor/join-connected.png new file mode 100644 index 00000000000..9ffabd2c032 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/join-connected.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/join-not-connected.png b/website/static/img/docs/dbt-cloud/visual-editor/join-not-connected.png new file mode 100644 index 00000000000..b82bd0c1a85 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/join-not-connected.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/one-model-operator.png b/website/static/img/docs/dbt-cloud/visual-editor/one-model-operator.png new file mode 100644 index 00000000000..a6e95ee9c1e Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/one-model-operator.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/operator-tile.png b/website/static/img/docs/dbt-cloud/visual-editor/operator-tile.png new file mode 100644 index 00000000000..cd38fb94abe Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/operator-tile.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/operators.png b/website/static/img/docs/dbt-cloud/visual-editor/operators.png new file mode 100644 index 00000000000..13bf108e5e6 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/operators.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/order.png b/website/static/img/docs/dbt-cloud/visual-editor/order.png new file mode 100644 index 00000000000..19855b3731b Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/order.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/output-model.png b/website/static/img/docs/dbt-cloud/visual-editor/output-model.png new file mode 100644 index 00000000000..c7a480657c2 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/output-model.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/preview-data.png b/website/static/img/docs/dbt-cloud/visual-editor/preview-data.png new file mode 100644 index 00000000000..f84ae7d4839 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/preview-data.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/preview-join.png b/website/static/img/docs/dbt-cloud/visual-editor/preview-join.png new file mode 100644 index 00000000000..62493d14ad2 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/preview-join.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/run-results.png b/website/static/img/docs/dbt-cloud/visual-editor/run-results.png new file mode 100644 index 00000000000..82e8add4b92 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/run-results.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/two-model-operators.png b/website/static/img/docs/dbt-cloud/visual-editor/two-model-operators.png new file mode 100644 index 00000000000..38a2df282e9 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/two-model-operators.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/ve-create-new-model.png b/website/static/img/docs/dbt-cloud/visual-editor/ve-create-new-model.png new file mode 100644 index 00000000000..dc001aa2927 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/ve-create-new-model.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/ve-main-menu.png b/website/static/img/docs/dbt-cloud/visual-editor/ve-main-menu.png new file mode 100644 index 00000000000..4f537bbf74a Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/ve-main-menu.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/ve-model-folder.png b/website/static/img/docs/dbt-cloud/visual-editor/ve-model-folder.png new file mode 100644 index 00000000000..353ba2c361d Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/ve-model-folder.png differ diff --git a/website/static/img/docs/dbt-cloud/visual-editor/ve-screen.png b/website/static/img/docs/dbt-cloud/visual-editor/ve-screen.png new file mode 100644 index 00000000000..cc9685234f2 Binary files /dev/null and b/website/static/img/docs/dbt-cloud/visual-editor/ve-screen.png differ