This API is in preview and subject to change without notice. We make a best effort to update this doc as we make changes to the API, but please raise an issue or PR in this repo if you notice something out-of-date.
Pipedream components are Node.js modules that export an object with the following properties:
module.exports = {
name: "name", // required
version: "0.0.1", // required
props,
methods,
run(event) {
console.log("Run any Node.js code here");
}
};The component API borrows concepts and structure from frontend components, like those in Vue.js or React.
Components run on Pipedream infrastructure.
Component can:
- Accept input via props
- Define methods.
requireany npm package- Spin up Pipedream infrastructure, like HTTP servers and scheduled jobs, by declaring the correct interfaces
- Store and retrieve state using the built-in key-value store
- Emit data you process within the component, allowing you to access it outside of Pipedream via API.
- name
- version
- run method
- Referencing this
- Interfaces
- props
- methods
this.$emit$.service.db- Using npm packages
- How components run
The name of the component, a string which identifies components deployed to users' accounts.
This name will show up in the Pipedream UI, in CLI output (for example, from pd list commands), etc.
Each time a user deploys a component to their account, a name slug is also generated, based on the name provided. Name slugs are composed of shell-safe characters, so you can reference the component by name slug in the CLI and other environments.
For example, a name of Hello, World! will generate a name slug of hello-world.
The name, as provided in the name property of the component, still appears in the Pipedream UI for the component. This lets you declare a human-readable name with any special characters you'd like. The name slug is displayed beside the name, and you use the name slug to interact with the component programmatically.
If the user deploys a component to their account, but a component with that name slug already exists, the component will be named using an incrementing integer suffix. For example, if you deploy a component with the name my-component twice, the first deployed component will be named my-component, and the second deployed component will be named my-component-1.
The version of a component, a string.
There are no constraints on the version, but consider using semantic versioning.
Each time a component is invoked (for example, via HTTP request), its run method is called.
The event that triggered the component is passed to run, so that you can access it within the method:
async run(event) {
console.log(event)
}You can reference this within the run method. this refers to the component, and provides access to props, methods, and Pipedream-provided functions like this.$emit.
You can view logs produced by the run method in the LOGS section of the Pipedream UI for the component, or using the pd logs CLI command:
pd logs <deployed-component-name>If the run method emits events using this.$emit, you can access the events in the EVENTS section of the Pipedream UI for the component, or using the pd events CLI command:
pd events <deployed-component-name>this refers to the component, and provides access to props, methods, and Pipedream-provided functions like this.$emit.
Props can be accessed using this.<prop-name>. For example, a secret prop:
props: {
secret: "string"
},can be referenced using this.secret.
Methods can be accessed using this.<method-name>. For example, a random method:
methods: {
random() {
return Math.random()
},
}can be run like so:
const randomNum = this.random();Interfaces are infrastructure abstractions provided by the Pipedream platform. They declare how a component is invoked — via HTTP request, run on a schedule, etc. — and therefore define the shape of the events it processes.
Interfaces make it possible to work with HTTP servers or scheduled tasks using nothing more than props. Once you declare the interface for your component, Pipedream creates the infrastructure for that component when it's deployed. For example, if you deploy a component that uses the HTTP interface, Pipedream creates a unique HTTP endpoint URL for that component. HTTP requests to that endpoint invoke the component, executing its run method.
Interfaces can also provide methods. For example, the HTTP interface exposes a respond method that lets your component issue HTTP responses:
// this.http references the HTTP interface for the component, as you'll see below
this.http.respond({
status: 200,
headers: {
"X-My-Custom-Header": "test"
},
body: event // This can be any string, object, or Buffer
});Interfaces are attached to components via props. To use the HTTP interface, declare a prop whose value is the string $.interface.http:
props: {
http: "$.interface.http"
},Since you control the name of props in your component, you can name the prop anything you'd like. The example just uses the name http for clarity.
With this prop declaration, a unique HTTP endpoint URL will be created for this component on deploy.
The HTTP interface exposes a respond method that lets your component issue HTTP responses. Assuming the prop for your HTTP interface is named http:
props: {
http: "$.interface.http"
},you can run this.http.respond to respond to the client:
this.http.respond({
status: 200,
headers: {
"X-My-Custom-Header": "test"
},
body: event // This can be any string, object, or Buffer
});Interfaces are attached to components via props. To use the timer interface, declare a prop whose value is the string $.interface.timer:
props: {
timer: {
type: "$.interface.timer"
}
},Since you control the name of props in your component, you can name the prop anything you'd like. The example just uses the name timer for clarity.
On component deploy, you'll be prompted to select one of two schedule types, and asked to provide a value:
cron, which accepts a cron expression (a string)intervalSeconds, which accepts the frequency of the job in seconds (an integer)
You can also specify the schedule type, and value, by explicitly including one of these params in the prop declaration:
props: {
timer: {
type: "$.interface.timer"
default: {
cron: "0 0 * * *" // Run job once a day
}
}
},props: {
timer: {
type: "$.interface.timer"
default: {
intervalSeconds: 60 // Run job once a minute
}
}
},Props allow components to accept input at deploy time. When deploying a component, users will be prompted to enter values for these props, setting the behavior of the component accordingly. Props make components reusable.
Props can be accessed within the run method, or within any methods defined by the component.
Props can be accessed using this.<prop-name>. For example, a secret prop:
props: {
secret: "string"
},can be referenced using this.secret.
See the http-require-secret component for an example of how to reference props. That component accepts a secret prop, which it uses in the run method to validate that HTTP requests contain that secret.
You can define helper functions within the methods property of your component. You have access to these functions within the run method, or within other methods.
Methods can be accessed using this.<method-name>. For example, a random method:
methods: {
random() {
return Math.random()
},
}can be run like so:
const randomNum = this.random();this.$emit is a Pipedream-provided function that allows you to emit data from your component. Emitted events appear in the EVENTS section of the UI for your source. You can also access these events programmatically, in your own app, using Pipedream APIs.
Within your run method, pass the data you'd like to emit to the this.$emit function:
this.$emit({
name: "Luke Skywalker"
});Each time you run this.$emit(), you emit the data as an event.
Events can be retrieved using the REST API, CLI, or SSE stream tied to your component. For example, you can use the CLI to retrieve the last 10 events:
λ pd events -n 10 <source-name>
{ name: "Luke Skywalker" }
{ name: "Leia Organa" }
{ name: "Han Solo" }This makes it easy to retrieve data processed by your component from another app. Typically, you'll want to use the REST API to retrieve events in batch, and connect to the SSE stream to process them in real time.
$.service.db provides access to a simple, component-specific key-value store. It implements two methods: set and get.
Like with interfaces, you attach $.service.db via props:
props: {
db: "$.service.db";
}Then, within the run handler and methods, you have access to the db prop using this.db.
Since you control the name of props in your component, you can name the prop anything you'd like. The example here and below use the name db for clarity.
The data you store in this DB is specific to your deployed component.
this.db.set("key", value);Sets the value of a specific key.
this.db.get("key");Gets the value of a key. Returns undefined if the key doesn't exist.
To use an npm package in a component, just require it:
const _ = require("lodash");When you deploy a component, Pipedream downloads these packages and bundles them with your deployment. There's no need to include a package.json file with your component.
Some packages — for example, packages like Puppeteer, which includes large dependencies like Chromium — may not work on Pipedream. Please reach out if you encounter a specific issue.
Components run on Pipedream infrastructure. You deploy components to Pipedream using the Pipedream CLI, API, or UI.
A component is triggered, or invoked, by events sent its interface.
For example, a component configured to accept HTTP requests using $.interface.http runs on each HTTP request sent to its endpoint URL.
Each time a component is invoked, its run method is called. You can view logs produced by this method in the LOGS section of the Pipedream UI for the component, or using the pd logs CLI command:
pd logs <deployed-component-name>If the run method emits events using this.$emit, you can access the events in the EVENTS section of the Pipedream UI for the component, or using the pd events CLI command:
pd events <deployed-component-name>