Skip to content

Commit

Permalink
derive snippet type from extension
Browse files Browse the repository at this point in the history
  • Loading branch information
sh-rp committed Sep 11, 2023
1 parent 594b618 commit 42d54a3
Show file tree
Hide file tree
Showing 3 changed files with 14 additions and 13 deletions.
22 changes: 11 additions & 11 deletions docs/website/docs/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ Make sure that your `dlt` version is **0.3.15** or above. Check it in the termin
Let's load a list of Python objects (dictionaries) into `duckdb` and inspect the created dataset:

<!--@@@DLT_SNIPPET_START ./getting-started-snippets.py::start-->
```python
```py
import dlt

data = [
Expand Down Expand Up @@ -128,7 +128,7 @@ The library will create/update tables, infer data types and deal with nested dat
<TabItem value="json">

<!--@@@DLT_SNIPPET_START json-->
```python
```py
import dlt

from dlt.common import json
Expand Down Expand Up @@ -158,7 +158,7 @@ We import **json** from `dlt` namespace. It defaults to `orjson`(otherwise `simp
Pass anything that you can load with Pandas to `dlt`

<!--@@@DLT_SNIPPET_START csv-->
```python
```py
import dlt
import pandas as pd

Expand All @@ -181,7 +181,7 @@ print(load_info)
<TabItem value="api">

<!--@@@DLT_SNIPPET_START api-->
```python
```py
import dlt
from dlt.sources.helpers import requests

Expand Down Expand Up @@ -212,7 +212,7 @@ to sync your databases with warehouses, data lakes, or vector stores.
:::

<!--@@@DLT_SNIPPET_START db-->
```python
```py
import dlt
from sqlalchemy import create_engine

Expand Down Expand Up @@ -257,7 +257,7 @@ For example, if the CSV file is updated, how we can refresh it in the database?
One method is to tell `dlt` to replace the data in existing tables by using `write_disposition`:

<!--@@@DLT_SNIPPET_START replace-->
```python
```py
import dlt

data = [
Expand Down Expand Up @@ -301,7 +301,7 @@ Instead of using `replace` write_disposition and downloading all issues each tim


<!--@@@DLT_SNIPPET_START incremental-->
```python
```py
import dlt
from dlt.sources.helpers import requests

Expand Down Expand Up @@ -377,7 +377,7 @@ To get always fresh content of all the issues you combine incremental load with
like in the script below.

<!--@@@DLT_SNIPPET_START incremental_merge-->
```python
```py
import dlt
from dlt.sources.helpers import requests

Expand Down Expand Up @@ -433,7 +433,7 @@ This is a fun but practical example that reads GitHub events from **dlt** reposi
Each event type is sent to a different table in `duckdb`.

<!--@@@DLT_SNIPPET_START table_dispatch-->
```python
```py
import dlt
from dlt.sources.helpers import requests

Expand Down Expand Up @@ -511,7 +511,7 @@ Learn more:
Below we extract text from PDFs and load it to [Weaviate](dlt-ecosystem/destinations/weaviate) vector store.

<!--@@@DLT_SNIPPET_START pdf_to_weaviate-->
```python
```py
import os

import dlt
Expand Down Expand Up @@ -591,7 +591,7 @@ python pdf_to_weaviate.py

Now it is time to query our documents.
<!--@@@DLT_SNIPPET_START pdf_to_weaviate_read-->
```python
```py
import weaviate

client = weaviate.Client("http://localhost:8080")
Expand Down
2 changes: 1 addition & 1 deletion docs/website/docs/intro.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ There's no need to start any backends or containers. Import `dlt` in your Python
<!--AUTO-GENERATED-CONTENT:END-->

<!--@@@DLT_SNIPPET_START index-->
```python
```py
import dlt
from dlt.sources.helpers import requests
# Create a dlt pipeline that will load
Expand Down
3 changes: 2 additions & 1 deletion docs/website/tools/update_snippets.js
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,8 @@ function getSnippet(fileName, snippetName) {
let result = lines.slice((snippetMap[snippetName]["start"]+1), snippetMap[snippetName]["end"]);
// dedent works on strings, not on string arrays, so this is very ineffective unfortunately...
result = dedent(result.join("\n")).split(/\r?\n/);
result.unshift("```python");
const codeType = path.extname(snippetsFileName).replace(".", "");
result.unshift(`\`\`\`${codeType}`);
result.push("```");
return filterDirectives(result);
}
Expand Down

0 comments on commit 42d54a3

Please sign in to comment.