Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

source-api-ibge-municipios contribution from Breno-Teodomiro #50419

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
# API_IBGE_MUNICIPIOS
This directory contains the manifest-only connector for `source-api-ibge-municipios`.

Extrair dados de municípios do site do IBGE

## Usage
There are multiple ways to use this connector:
- You can use this connector as any other connector in Airbyte Marketplace.
- You can load this connector in `pyairbyte` using `get_source`!
- You can open this connector in Connector Builder, edit it, and publish to your workspaces.

Please refer to the manifest-only connector documentation for more details.

## Local Development
We recommend you use the Connector Builder to edit this connector.

But, if you want to develop this connector locally, you can use the following steps.

### Environment Setup
You will need `airbyte-ci` installed. You can find the documentation [here](airbyte-ci).

### Build
This will create a dev image (`source-api-ibge-municipios:dev`) that you can use to test the connector locally.
```bash
airbyte-ci connectors --name=source-api-ibge-municipios build
```

### Test
This will run the acceptance tests for the connector.
```bash
airbyte-ci connectors --name=source-api-ibge-municipios test
```

Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
# See [Connector Acceptance Tests](https://docs.airbyte.com/connector-development/testing-connectors/connector-acceptance-tests-reference)
# for more information about how to configure these tests
connector_image: airbyte/source-api-ibge-municipios:dev
acceptance_tests:
spec:
tests:
- spec_path: "manifest.yaml"
connection:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
discovery:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
basic_read:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
incremental:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
full_refresh:
bypass_reason: "This is a builder contribution, and we do not have secrets at this time"
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Original file line number Diff line number Diff line change
@@ -0,0 +1,213 @@
version: 6.5.2

type: DeclarativeSource

description: Extrair dados de municípios do site do IBGE

check:
type: CheckStream
stream_names:
- Municipios

definitions:
streams:
Municipios:
type: DeclarativeStream
name: Municipios
retriever:
type: SimpleRetriever
requester:
$ref: "#/definitions/base_requester"
path: /localidades/municipios
http_method: GET
request_body_json:
id: 1100015
nome: Alta Floresta D'Oeste
microrregiao:
id: 11006
nome: Cacoal
mesorregiao:
UF:
id: 11
nome: Rondônia
sigla: RO
regiao:
id: 1
nome: Norte
sigla: "N"
id: 1102
nome: Leste Rondoniense
record_selector:
type: RecordSelector
extractor:
type: DpathExtractor
field_path: []
primary_key:
- id
schema_loader:
type: InlineSchemaLoader
schema:
$ref: "#/schemas/Municipios"
base_requester:
type: HttpRequester
url_base: https://servicodados.ibge.gov.br/api/v1

streams:
- $ref: "#/definitions/streams/Municipios"

spec:
type: Spec
connection_specification:
type: object
$schema: http://json-schema.org/draft-07/schema#
required: []
properties: {}
additionalProperties: true

metadata:
assist: {}
testedStreams:
Municipios:
hasRecords: true
streamHash: 1a3283b838415d4d35393ffe01c0f9d3b6017bf1
hasResponse: true
primaryKeysAreUnique: true
primaryKeysArePresent: true
responsesAreSuccessful: true
autoImportSchema:
Municipios: true

schemas:
Municipios:
type: object
$schema: http://json-schema.org/schema#
required:
- id
properties:
id:
type: number
nome:
type:
- string
- "null"
microrregiao:
type:
- object
- "null"
properties:
id:
type:
- number
- "null"
nome:
type:
- string
- "null"
mesorregiao:
type:
- object
- "null"
properties:
UF:
type:
- object
- "null"
properties:
id:
type:
- number
- "null"
nome:
type:
- string
- "null"
sigla:
type:
- string
- "null"
regiao:
type:
- object
- "null"
properties:
id:
type:
- number
- "null"
nome:
type:
- string
- "null"
sigla:
type:
- string
- "null"
id:
type:
- number
- "null"
nome:
type:
- string
- "null"
regiao-imediata:
type:
- object
- "null"
properties:
id:
type:
- number
- "null"
nome:
type:
- string
- "null"
regiao-intermediaria:
type:
- object
- "null"
properties:
UF:
type:
- object
- "null"
properties:
id:
type:
- number
- "null"
nome:
type:
- string
- "null"
sigla:
type:
- string
- "null"
regiao:
type:
- object
- "null"
properties:
id:
type:
- number
- "null"
nome:
type:
- string
- "null"
sigla:
type:
- string
- "null"
id:
type:
- number
- "null"
nome:
type:
- string
- "null"
additionalProperties: true
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
metadataSpecVersion: "1.0"
data:
allowedHosts:
hosts:
- "servicodados.ibge.gov.br"
registryOverrides:
oss:
enabled: true
cloud:
enabled: true
remoteRegistries:
pypi:
enabled: false
packageName: airbyte-source-api-ibge-municipios
connectorBuildOptions:
baseImage: docker.io/airbyte/source-declarative-manifest:6.12.4@sha256:b73460101d4ff373826b56f00ef2012b635f5eab04c429892547dd055cd779dc
connectorSubtype: api
connectorType: source
definitionId: aa8c2b50-ac2b-4e65-86f3-1e26cc0178a4
dockerImageTag: 0.0.1
dockerRepository: airbyte/source-api-ibge-municipios
githubIssueLabel: source-api-ibge-municipios
icon: icon.svg
license: MIT
name: API_IBGE_MUNICIPIOS
releaseDate: 2024-12-24
releaseStage: alpha
supportLevel: community
documentationUrl: https://docs.airbyte.com/integrations/sources/api-ibge-municipios
tags:
- language:manifest-only
- cdk:low-code
ab_internal:
ql: 100
sl: 100
23 changes: 23 additions & 0 deletions docs/integrations/sources/api-ibge-municipios.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
# API_IBGE_MUNICIPIOS
Extrair dados de municípios do site do IBGE

## Configuration

| Input | Type | Description | Default Value |
|-------|------|-------------|---------------|

## Streams
| Stream Name | Primary Key | Pagination | Supports Full Sync | Supports Incremental |
|-------------|-------------|------------|---------------------|----------------------|
| Municipios | id | No pagination | ✅ | ❌ |

## Changelog

<details>
<summary>Expand to review</summary>

| Version | Date | Pull Request | Subject |
|------------------|-------------------|--------------|----------------|
| 0.0.1 | 2024-12-24 | | Initial release by [@Breno-Teodomiro](https://github.com/Breno-Teodomiro) via Connector Builder |

</details>
Loading