Skip to content

Commit

Permalink
added a couple of points on event-driven and near-realtime processing…
Browse files Browse the repository at this point in the history
… as an additional examples
  • Loading branch information
annajgibson committed Mar 27, 2024
1 parent 02419d2 commit f19c02b
Showing 1 changed file with 2 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,8 @@ On the Data Analytics Platform, we typically use [AWS Lambda](https://docs.aws.a
* Making an _API call_ and writing the response to the landing zone, including a crawler execution
* _Reading_ in data from a _Google Sheet_
* _Moving data_ from one zone to another zone, with a simple refinement
* _Event-driven_ processing, for example once new data arrives in S3, a Lambda is triggered to perform a task immediately
* Near _Real-time processing_ for example, when a new person is added to a permissions group, a Lambda is triggered to grant this permission

If your process involves reading in very large datasets, applying complex algorithms using Spark or and undertaking data transformations with multiple datasets, then this might be better suited to a different approach such as AWS Glue.

Expand Down

0 comments on commit f19c02b

Please sign in to comment.