Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Scale down on functionality to provide a "lighter" version of the library #16

Closed
4 tasks
ghost opened this issue Oct 26, 2022 · 2 comments
Closed
4 tasks
Labels
enhancement New feature or request

Comments

@ghost
Copy link

ghost commented Oct 26, 2022

In order to have a simpler, lighter library that really just focuses on chaining our Clojure operations we identified the following parts to be removed:

  • input-path
  • output-path
  • result-selector
  • result-path
@ghost ghost added the enhancement New feature or request label Oct 26, 2022
@Quantisan
Copy link
Member

I looked at this a bit more. Turns out, we actually do need a couple of those. e.g.

this is common to include the error exception into the output:

:catch [{:error-equals :States.ALL
                       :result-path  "$.error"
                       :next         :export-job-failed}]

to include a nested SFN defined by Stepwise:

:type     :task
              ;; This starts execution of the create-and-wait-for-eloqua-export-job
              ;; state machine. The .sync means this will wait for a response from
              ;; that SFN, and :2 returns the output as JSON object instead of a
              ;; string which allows us to use the output in the following steps.
              :resource "arn:aws:states:::states:startExecution.sync:2"
              :output-path "$.Output"
              :parameters (json/generate-string
                            {:StateMachineArn (state-machine-arn :eloqua-live/create-and-wait-for-eloqua-export-job)
                             :Input {"AWS_STEP_FUNCTIONS_STARTED_BY_EXECUTION_ID.$" "$$.Execution.Id"
                                     ;; adds the state input as field "StatePayload" to the SFN input
                                     "StatePayload.$" "$"}})
              :next :download-from-eloqua
              ;; This retry gurantees that the task will run even if SFN execution
              ;; limit is reached recommended in this documentation:
              ;; https://docs.aws.amazon.com/step-functions/latest/dg/concepts-nested-workflows.html
              :retry [{:error-equals "StepFunctions.ExecutionLimitExceeded"}]

@Quantisan
Copy link
Member

@ixffxi could you patch :result-selector to accept Clojure map too please?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant