Skip to content

A Singer.io tap for extracting data from the Autopilot API

License

Notifications You must be signed in to change notification settings

pedromachados/tap-autopilot

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

47 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

tap-autopilot

CircleCI

This is a Singer tap that produces JSON-formatted data following the Singer spec.

This tap:

Quick start

  1. Install

    > pip install tap-autopilot
  2. Get your Autopilot API Key

    Login to your Autopilot account, navigate to your account settings and then to the Autopilot API section. Generate a New API Key, you'll need it for the next step.

  3. Create the config file

    Create a JSON file called config.json containing the api key you just generated and a start date, the tap will only return contacts who have been updated after the date chosen. Start dates should conform to the RFC3339 specification.

    {
        "api_key": "your-autopilot-api-token",
        "start_date": "2017-01-01T00:00:00Z"
    }
  4. Discover and Catalog

    Use the discover flag to explore the schema for each of this tap's resources

    > tap-autopulot --config config.json --discover

    Pipe the output of this file to a file that will serve as the catalog, where you will select which streams and properties to sync

    > touch catalog.json
    > tap-autopilot --config config.json --discover >> catalog.json

    The catalog is an object with a key streams that has an array of the streams for this tap. For each stream you want to sync, add a "selected": true property on the stream object. Below is an example of how you would select to sync the contacts stream. This property is recursive so it will select all children. If you don't want to sync a property, you can add "selected": false on that property.

            {
            "schema": {
                "properties": {...},
                "type": "object",
                "selected": true
            },
            "stream": "contacts",
            "tap_stream_id": "contacts"
        }
  5. [Optional] Add additional optional config parameters

    You can include a user_agent key in your config.json to further customize the behavior of this Tap.

    • user_agent should be set to something that includes a contact email address should the API provider need to contact you for any reason.

    If you were to use the user_agent, your complete config.json should look something like this.

    {
      "api_key": "your-api-token",
      "start_date": "2017-01-01T00:00:00Z",
      "user_agent": "Stitch ([email protected])"
    }
  6. [Optional] Create the initial state file

    You can provide JSON file that contains a date for the API endpoints to force the application to only fetch data newer than those dates. If you omit the file it will fetch all Autopilot data. State Files will be created for you after a successful run and will keep the status of the previous run for the next time it is invoked.

    {
      "contacts": "2017-01-17T20:32:05Z"
    }
    ``
    
  7. Run the application

    tap-autopilot can be run with:

    tap-autopilot --config config.json --catalog catalog.json [--state state.json]

Copyright © 2017 Stitch

About

A Singer.io tap for extracting data from the Autopilot API

Resources

License

Stars

Watchers

Forks

Packages

No packages published

Languages

  • Python 100.0%