diff --git a/_partials/_integration-prereqs-cloud-only.md b/_partials/_integration-prereqs-cloud-only.md index e04a12ab26..7cadd3c9aa 100644 --- a/_partials/_integration-prereqs-cloud-only.md +++ b/_partials/_integration-prereqs-cloud-only.md @@ -1,5 +1,5 @@ Before integrating: -* Create a [target $SERVICE_LONG][create-service]. +* Create a target [$SERVICE_LONG][create-service]. [create-service]: /getting-started/:currentVersion:/services/ diff --git a/_partials/_integration-prereqs.md b/_partials/_integration-prereqs.md index f28c62bb2f..204bcaf87e 100644 --- a/_partials/_integration-prereqs.md +++ b/_partials/_integration-prereqs.md @@ -1,8 +1,9 @@ Before integrating: -* Create a [target $SERVICE_LONG][create-service]. You need [your connection details][connection-info] to follow this procedure. +* Create a target [$SERVICE_LONG][create-service] - This procedure also works for [self-hosted $TIMESCALE_DB][enable-timescaledb]. + You need [your connection details][connection-info] to follow the steps in this page. This procedure also + works for [self-hosted $TIMESCALE_DB][enable-timescaledb]. [create-service]: /getting-started/:currentVersion:/services/ [enable-timescaledb]: /self-hosted/:currentVersion:/install/ diff --git a/_partials/_migrate_import_prerequisites.md b/_partials/_migrate_import_prerequisites.md index 16e5b69e36..31f1a00185 100644 --- a/_partials/_migrate_import_prerequisites.md +++ b/_partials/_migrate_import_prerequisites.md @@ -5,7 +5,7 @@ data from your source database to your target $SERVICE_LONG. Before you migrate your data: -- [Create a target $SERVICE_LONG][created-a-database-service-in-timescale]. +- Create a target [$SERVICE_LONG][created-a-database-service-in-timescale]. Each $SERVICE_LONG has a single database that supports the [most popular extensions][all-available-extensions]. $SERVICE_LONGs do not support tablespaces, diff --git a/_partials/_migrate_prerequisites.md b/_partials/_migrate_prerequisites.md index 41ac59b1fa..dd21896185 100644 --- a/_partials/_migrate_prerequisites.md +++ b/_partials/_migrate_prerequisites.md @@ -5,7 +5,7 @@ data from your source database to your target Timescale Cloud service. Before you move your data: -- [Create a target Timescale Cloud service][created-a-database-service-in-timescale]. +- Create a target [$SERVICE_LONG][created-a-database-service-in-timescale]. Each Timescale Cloud service has a single database that supports the [most popular extensions][all-available-extensions]. $SERVICE_LONGs do not support tablespaces, diff --git a/tutorials/simulate-iot-sensor-data.md b/tutorials/simulate-iot-sensor-data.md index f6e6e66fea..5898087364 100644 --- a/tutorials/simulate-iot-sensor-data.md +++ b/tutorials/simulate-iot-sensor-data.md @@ -20,7 +20,7 @@ To simulate a more advanced dataset, see [Time-series Benchmarking Suite (TSBS)] To follow this tutorial, you need to: -- [Create a target Timescale Cloud service][create-a-service]. +- Create a target [Timescale Cloud service][create-a-service]. - [Connect to your service][connect-to-service]. ## Simulate a dataset diff --git a/use-timescale/integrations/aws-lambda.md b/use-timescale/integrations/aws-lambda.md new file mode 100644 index 0000000000..9ce2c86805 --- /dev/null +++ b/use-timescale/integrations/aws-lambda.md @@ -0,0 +1,210 @@ +--- +title: Integrate AWS Lambda with $CLOUD_LONG +excerpt: ADD +products: [cloud, mst, self_hosted] +keywords: [connect, integrate, aws, lambda] +--- + +import IntegrationPrereqs from "versionContent/_partials/_integration-prereqs.mdx"; + +# Integrate AWS Lambda with $CLOUD_LONG + +[AWS Lambda][AWS-Lambda] is a serverless computing service provided by Amazon Web Services (AWS) that allows you to run +code without provisioning or managing servers, scaling automatically as needed. + +This page shows you how to integrate AWS Lambda with $SERVICE_LONG to process and store time-series data efficiently. + +## Prerequisites + + + +* Set up an [AWS Account][aws-sign-up]. +* Install and configure [AWS CLI][install-aws-cli]. +* Install [NodeJS v18.x or later][install-nodejs]. + + +## Prepare your $SERVICE_LONG to ingest data from AWS Lambda + +Create a table in $SERVICE_LONG to store time-series data. + + + +1. **Connect to your $SERVICE_LONG** + + For $CLOUD_LONG, open an [SQL editor][run-queries] in [$CONSOLE][open-console]. For self-hosted, use [`psql`][psql]. + +1. **Create a table to store sensor data** + + ```sql + CREATE TABLE sensor_data ( + time TIMESTAMPTZ NOT NULL, + sensor_id TEXT NOT NULL, + value DOUBLE PRECISION NOT NULL + ); + ``` + +1. **For better performance and easier real-time analytics, convert the table to a hypertable** + + [Hypertables][about-hypertables] are PostgreSQL tables that automatically partition your data by time. You interact + with hypertables in the same way as regular PostgreSQL tables, but with extra features that makes managing your + time-series data much easier. + + ```sql + SELECT create_hypertable('sensor_data', 'time'); + ``` + + + +## Create the code to inject data into a $SERVICE_LONG + +Write an AWS Lambda function in a Node.js project that processes and inserts time-series data into a $SERVICE_LONG. + + + +1. **Initialize a new Node.js project to hold your Lambda function** + + ```shell + mkdir lambda-timescale && cd lambda-timescale + npm init -y + ``` + +1. **Install the PostgreSQL client library in your project** + + ```shell + npm install pg + ``` + +1. **Write a Lambda Function that inserts data into your $SERVICE_LONG** + + Create a file named `index.js`, then add the following code: + + ```javascript + const { + Client + } = require('pg'); + + exports.handler = async (event) => { + const client = new Client({ + host: process.env.TIMESCALE_HOST, + port: process.env.TIMESCALE_PORT, + user: process.env.TIMESCALE_USER, + password: process.env.TIMESCALE_PASSWORD, + database: process.env.TIMESCALE_DB, + }); + + try { + await client.connect(); + // + const query = ` + INSERT INTO sensor_data (time, sensor_id, value) + VALUES ($1, $2, $3); + `; + + const data = JSON.parse(event.body); + const values = [new Date(), data.sensor_id, data.value]; + + await client.query(query, values); + + return { + statusCode: 200, + body: JSON.stringify({ + message: 'Data inserted successfully!' + }), + }; + } catch (error) { + console.error('Error inserting data:', error); + return { + statusCode: 500, + body: JSON.stringify({ + error: 'Failed to insert data.' + }), + }; + } finally { + await client.end(); + } + + }; + ``` + + + +## Deploy your Node project to AWS Lambda + +To create an AWS Lambda function that injects data into your $SERVICE_LONG: + + + +1. **Compress your code into a `.zip`** + + ```shell + zip -r lambda-timescale.zip . + ``` + +1. **Deploy to AWS Lambda** + + In the following example, replace `` with your [AWS IAM credentials][aws-iam-role], then use + AWS CLI to create a Lambda function for your project: + + ```shell + aws lambda create-function \ + --function-name TimescaleIntegration \ + --runtime nodejs14.x \ + --role \ + --handler index.handler \ + --zip-file fileb://lambda-timescale.zip + ``` + +1. **Set up environment variables** + + In the following example, use your [connection details][connection-info] to add your $SERVICE_LONG connection settings to your Lambda function: + ```shell + aws lambda update-function-configuration \ + --function-name TimescaleIntegration \ + --environment "Variables={TIMESCALE_HOST=,TIMESCALE_PORT=, \ + TIMESCALE_USER=,TIMESCALE_PASSWORD=, \ + TIMESCALE_DB=}" + ``` + +1. **Test your AWS Lambda function** + + 1. Invoke the Lambda function and send some data to your $SERVICE_LONG: + + ```shell + aws lambda invoke \ + --function-name TimescaleIntegration \ + --payload '{"body": "{\"sensor_id\": \"sensor-123\", \"value\": 42.5}"}' \ + --cli-binary-format raw-in-base64-out \ + response.json + ``` + + 1. Verify that the data is in your $SERVICE_SHORT. + + Open an [SQL editor][run-queries] and check the `sensor_data` table: + + ```sql + SELECT * FROM sensor_data; + ``` + You see something like: + + | time | sensor_id | value | + |-- |-- |--------| + | 2025-02-10 10:58:45.134912+00 | sensor-123 | 42.5 | + + + + +You can now seamlessly ingest time-series data from AWS Lambda into $CLOUD_LONG. + +[AWS-Lambda]: https://docs.aws.amazon.com/lambda/latest/dg/welcome.html +[lambda-functions]: https://console.aws.amazon.com/lambda/home#/functions +[aws-sign-up]: https://signin.aws.amazon.com/signup?request_type=register +[install-aws-cli]: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2.html +[install-nodejs]: https://nodejs.org/en/download +[install-postgresql]: https://www.postgresql.org/download/ +[console]: https://console.cloud.timescale.com/ +[run-queries]: /getting-started/:currentVersion:/run-queries-from-console/ +[psql]: /use-timescale/:currentVersion:/integrations/psql/ +[about-hypertables]: /use-timescale/:currentVersion:/hypertables/about-hypertables/ +[aws-iam-role]: https://docs.aws.amazon.com/IAM/latest/UserGuide/access-keys-admin-managed.html#admin-list-access-key +[open-console]: https://console.cloud.timescale.com/dashboard/services +[connection-info]: /use-timescale/:currentVersion:/integrations/find-connection-details/ diff --git a/use-timescale/integrations/index.md b/use-timescale/integrations/index.md index 8d1ca0fda7..4bc9b34776 100644 --- a/use-timescale/integrations/index.md +++ b/use-timescale/integrations/index.md @@ -47,10 +47,10 @@ Some of the most in-demand integrations for $CLOUD_LONG are listed below, with l ## Data engineering and extract, transform, load -| Name | Description | -|:--------------------------------:|----------------------------------------------------------| -| [Apache Airflow][apache-airflow] | Programmatically author, schedule, and monitor workflows. | - +| Name | Description | +|:--------------------------------:|-------------------------------------------------------------------------------------| +| [Apache Airflow][apache-airflow] | Programmatically author, schedule, and monitor workflows. | +|[AWS Lambda][aws-lambda]| Run code without provisioning or managing servers, scaling automatically as needed. | [psql]: /use-timescale/:currentVersion:/integrations/psql/ @@ -64,5 +64,6 @@ Some of the most in-demand integrations for $CLOUD_LONG are listed below, with l [tableau]: /use-timescale/:currentVersion:/integrations/tableau/ [terraform]: /use-timescale/:currentVersion:/integrations/terraform [apache-airflow]: /use-timescale/:currentVersion:/integrations/apache-airflow +[aws-lambda]: /use-timescale/:currentVersion:/integrations/aws-lambda [postgresql-integrations]: https://slashdot.org/software/p/PostgreSQL/integrations/ [prometheus]: /use-timescale/:currentVersion:/integrations/prometheus diff --git a/use-timescale/page-index/page-index.js b/use-timescale/page-index/page-index.js index 83af6784b0..794d9fff96 100644 --- a/use-timescale/page-index/page-index.js +++ b/use-timescale/page-index/page-index.js @@ -788,6 +788,11 @@ module.exports = [ href: "cloudwatch", excerpt: "Integrate Amazon Cloudwatch with Timescale Cloud", }, + { + title: "AWS Lambda", + href: "aws-lambda", + excerpt: "Integrate AWS Lambda with Timescale Cloud", + }, { title: "Azure Data Studio", href: "azure-data-studio",