Skip to content

Commit

Permalink
Initial implementation of HTTP connector
Browse files Browse the repository at this point in the history
  • Loading branch information
edgarrmondragon committed May 23, 2023
1 parent 1459e91 commit 002c4f8
Show file tree
Hide file tree
Showing 17 changed files with 502 additions and 63 deletions.
8 changes: 8 additions & 0 deletions docs/classes/singer_sdk.connectors.BaseConnector.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
singer_sdk.connectors.BaseConnector
===================================

.. currentmodule:: singer_sdk.connectors

.. autoclass:: BaseConnector
:members:
:special-members: __init__, __call__
32 changes: 32 additions & 0 deletions docs/guides/custom-connector.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
# Using a custom connector class

The Singer SDK has a few built-in connector classes that are designed to work with a variety of sources:

* [`SQLConnector`](../../classes/singer_sdk.SQLConnector) for SQL databases

If you need to connect to a source that is not supported by one of these built-in connectors, you can create your own connector class. This guide will walk you through the process of creating a custom connector class.

## Subclass `BaseConnector`

The first step is to create a subclass of [`BaseConnector`](../../classes/singer_sdk.connectors.BaseConnector). This class is responsible for creating streams and handling the connection to the source.

```python
from singer_sdk.connectors import BaseConnector


class MyConnector(BaseConnector):
pass
```

## Implement `get_connection`

The [`get_connection`](http://127.0.0.1:5500/build/classes/singer_sdk.connectors.BaseConnector.html#singer_sdk.connectors.BaseConnector.get_connection) method is responsible for creating a connection to the source. It should return an object that implements the [context manager protocol](https://docs.python.org/3/reference/datamodel.html#with-statement-context-managers), e.g. it has `__enter__` and `__exit__` methods.

```python
from singer_sdk.connectors import BaseConnector


class MyConnector(BaseConnector):
def get_connection(self):
return MyConnection()
```
1 change: 1 addition & 0 deletions docs/guides/index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,4 +7,5 @@ The following pages contain useful information for developers building on top of
porting
pagination-classes
custom-connector
```
9 changes: 9 additions & 0 deletions docs/reference.rst
Original file line number Diff line number Diff line change
Expand Up @@ -140,3 +140,12 @@ Batch

batch.BaseBatcher
batch.JSONLinesBatcher

Abstract Connector Classes
--------------------------

.. autosummary::
:toctree: classes
:template: class.rst

connectors.BaseConnector
3 changes: 2 additions & 1 deletion noxfile.py
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,9 @@
test_dependencies = [
"coverage[toml]",
"pytest",
"pytest-snapshot",
"pytest-durations",
"pytest-httpserver",
"pytest-snapshot",
"freezegun",
"pandas",
"pyarrow",
Expand Down
103 changes: 68 additions & 35 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ types-simplejson = "^3.18.0"
types-PyYAML = "^6.0.12"
coverage = {extras = ["toml"], version = "^7.2"}
pyarrow = ">=11,<13"
pytest-httpserver = "^1.0.6"
pytest-snapshot = "^0.9.0"

# Cookiecutter tests
Expand Down
Loading

0 comments on commit 002c4f8

Please sign in to comment.