Skip to content

Commit

Permalink
feat: enhance meltano.yml example
Browse files Browse the repository at this point in the history
  • Loading branch information
Léo Grosjean committed Oct 18, 2023
1 parent 82cf8c7 commit 565613a
Show file tree
Hide file tree
Showing 3 changed files with 54 additions and 15 deletions.
55 changes: 43 additions & 12 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,51 @@ To get your API Token, follow the [official documentation](https://apidocs.getre

You can easily run `tap-getresponse` by itself or in a pipeline using [Meltano](https://meltano.com/).

### Executing the Tap Directly

### Execute in a Meltano pipeline

#### Install the project

```bash
meltano install
```

#### Select Entities and Attributes to Extract

```bash
meltano select tap-getresponse <entity> <attribute>
meltano select tap-getresponse --exclude <entity> <attribute>

# For example:
meltano select tap-getresponse webinars "*"
```
Verify that only the intended entities and attributes are now selected using `meltano select --list`:
```bash
meltano select tap-getresponse --list
```
#### Run the pipeline
Run your newly added GetResponse extractor and chosen loader in a pipeline using `meltano run`:
```bash
meltano run tap-getresponse <loader>
# For example:
meltano run tap-getresponse target-jsonl
```
There is also the `meltano elt` (or `meltano el`) command which is a more rigid command for running only EL pipelines.
Or directly using the `meltano invoke`, which only executes a single plugin at a time. This can be useful for debugging the extractor (`meltano invoke tap-getresponse`).
```bash
tap-getresponse --version
tap-getresponse --help
tap-getresponse --config CONFIG --discover > ./catalog.json
# Test invocation:
meltano invoke tap-getresponse --version
# OR run a test `elt` pipeline:
meltano elt tap-getresponse <loader>
# Example
meltano elt tap-getresponse target-jsonl
```
## Developer Resources
Expand Down Expand Up @@ -112,14 +151,6 @@ cd tap-getresponse
meltano install
```
Now you can test and orchestrate using Meltano:

```bash
# Test invocation:
meltano invoke tap-getresponse --version
# OR run a test `elt` pipeline:
meltano elt tap-getresponse target-jsonl
```
### SDK Dev Guide
Expand Down
12 changes: 10 additions & 2 deletions meltano.yml
Original file line number Diff line number Diff line change
@@ -1,9 +1,17 @@
version: 1
send_anonymous_usage_stats: true
project_id: tap-getresponse
default_environment: test
default_environment: dev
environments:
- name: test
- name: dev
config:
plugins:
extractors:
- name: tap-getresponse
# select:
# - contact_activities.*
- name: uat
- name: prod
plugins:
extractors:
- name: tap-getresponse
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[tool.poetry]
name = "tap-getresponse"
version = "0.0.1"
version = "0.0.2"
description = "`tap-getresponse` is a Singer tap for GetResponse, built with the Meltano Singer SDK."
readme = "README.md"
authors = ["Leo Grosjean <[email protected]>"]
Expand Down

0 comments on commit 565613a

Please sign in to comment.