Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

HTTP Processor: change RawData into []byte #1497

Closed
maha-hajja opened this issue Apr 17, 2024 · 0 comments · Fixed by #1498
Closed

HTTP Processor: change RawData into []byte #1497

maha-hajja opened this issue Apr 17, 2024 · 0 comments · Fixed by #1498
Assignees
Labels
bug Something isn't working
Milestone

Comments

@maha-hajja
Copy link
Contributor

Bug description

the processor response is saved as RawData even if it's under a StructuredData field.
investigate the problem, and change RawData into []byte (should work for .Payload.After and other fields)

Steps to reproduce

example pipeline config file:

version: 2.2
pipelines:
  - id: sfdc-to-webhook
    status: running
    description: >
      TBD.
    connectors:
      - id: employees-source
        type: source
        plugin: builtin:generator
        settings:
          format.type: "structured"
          format.options: "id:int,name:string,company:string,trial:bool"
          recordCount: "15"
          readTime: "1s"
      - id: webhook-sink
        type: destination
        plugin: standalone:http
        settings:
          url: "https://webhook.site/4fd2add3-db9f-4fe2-bf4d-c1cc88e60dff"
    processors:
      - id: fetch-pokemon-ditto
        plugin: "webhook.http"
        settings:
          backoffRetry.count: "0"
          backoffRetry.factor: "2"
          backoffRetry.max: "5s"
          backoffRetry.min: "100ms"
          request.body: ""
          request.contentType: "application/json"
          request.method: "GET"
          request.url: "https://pokeapi.co/api/v2/pokemon/ditto"
          response.body: ".Payload.After.response"
          response.status: ".Metadata[\"http_status\"]"
#      - id: decode-json
#        plugin: json.decode
#        settings:
#          field: ".Payload.After.response"

doesn't work without the json decoder processor.

Version

latest

@maha-hajja maha-hajja added bug Something isn't working triage Needs to be triaged labels Apr 17, 2024
@maha-hajja maha-hajja self-assigned this Apr 17, 2024
@maha-hajja maha-hajja added this to the 0.10.0 milestone Apr 17, 2024
@maha-hajja maha-hajja removed the triage Needs to be triaged label Apr 17, 2024
@maha-hajja maha-hajja moved this from Triage to In Progress in Conduit Main Apr 17, 2024
@github-project-automation github-project-automation bot moved this from In Progress to Done in Conduit Main Apr 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
Archived in project
Development

Successfully merging a pull request may close this issue.

1 participant