automations #1343

cpvalente · 2024-11-23T21:23:25Z

This is an attempt at re-writing the integrations service

For simplicity, I am renaming this to automation, this means that integrations becomes an internal name used for data into ontime

There are also some dependency upgrades which make the PR larger

The code parsing logic is the same as before, so hopefully there are not that many core changes to the outputs, however, the filtering logic is completely new

I am also proposing a new structure to our backend, it is annoying that I only did it in automations and - if we decide to go forwards with this - I will create a follow up PR to apply everywhere

in the same directory, I have collocated all the router and services. the service file, only has business logic. All crud logic is moved out of the way to a dao file
in the same way, all utilities like parsing are in that directory. this means the automation directory, contains all logic related to that domain concept
i also considered merging the controller and router files, but didnt see an obvious improvement. in a way, it is nice to be able to read all the available routes in a small file without worrying about what they do

In this PR

expose an integration service which allows filtering logic before sending a message
expose implementations for OSC, HTTP and Companion
expose testing function
create supporting UI
implement triggering of the new service
implement template parsing
implement clients
~~create migration code for old subscriptions~~ (decided to migrate OSC data and leave the rest)
persist data

TODO

OSC input port and toggle migrate to settings
consider UI for writing templates
update docs
double-check strict typing in new code
double-check deepsource

coderabbitai · 2024-11-23T21:23:32Z

Important

Review skipped

Auto reviews are disabled on this repository.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

This pull request introduces a comprehensive refactoring of the application's integration and automation capabilities. The changes replace the existing OSC and HTTP integration services with a new, more flexible automation system. The new architecture allows for creating and managing automation blueprints and rules that can trigger actions based on timer lifecycle events. The modifications span both client and server-side code, introducing new components for managing automation settings, creating blueprints, and handling output dispatching.

Changes

File/Folder	Change Summary
`apps/client/src/common/api/`	Added `automation.ts` for API interactions, removed `http.ts` and `osc.ts`
`apps/client/src/features/app-settings/`	Replaced integrations panel with automation panel, added new components for managing automations and blueprints
`apps/server/src/api-data/`	Introduced new automation-related controllers, routers, services, and validation
`apps/server/src/services/integration-service/`	Removed OSC and HTTP integration classes and services
`apps/server/src/models/`	Updated data models to replace OSC and HTTP settings with automation configuration

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Generate unit testing code for this file.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
- @coderabbitai generate unit testing code for this file.
- @coderabbitai modularize this function.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read src/utils.ts and generate unit testing code.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
- @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

@coderabbitai pause to pause the reviews on a PR.
@coderabbitai resume to resume the paused reviews.
@coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
@coderabbitai full review to do a full review from scratch and review all the files again.
@coderabbitai summary to regenerate the summary of the PR.
@coderabbitai generate docstrings to generate docstrings for this PR. (Beta)
@coderabbitai resolve resolve all the CodeRabbit review comments.
@coderabbitai configuration to show the current CodeRabbit configuration for the repository.
@coderabbitai help to get help.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

cpvalente · 2025-01-11T17:07:32Z

apps/client/src/common/models/AutomationSettings.ts

+
+export const automationPlaceholderSettings: AutomationSettings = {
+  enabledAutomations: false,
+  enabledOscIn: false,


@alex-Arc i am wondering if the OSC settings should be in the app settings instead of here

What do you think?

yes
at least it should be separate from the automations

cpvalente · 2025-01-11T17:28:11Z

apps/server/src/api-data/automation/automation.service.ts

+    const fieldValue = state[field];
+
+    // TODO: if value is empty string, the user could be meaning to check if the value does not exist
+    switch (operator) {


@alex-Arc i have been wondering if we could come up with a few examples and fine tune the operators?

cpvalente · 2025-01-11T17:36:41Z

apps/server/src/api-data/automation/clients/osc.client.ts

+ * Expose possibility to send a message using OSC protocol
+ */
+export function emitOSC(output: OSCOutput, state: RuntimeState) {
+  const message = preparePayload(output, state);


Skip sending in cloud

cpvalente · 2025-01-20T18:34:38Z

@alex-Arc , I believe this is ready to merge, would you mind sanity checking?

alex-Arc · 2025-01-21T12:08:29Z

apps/client/src/common/utils/__tests__/regex.test.ts

@@ -26,8 +26,8 @@ describe('simple tests for regex', () => {
  });

  test('startsWithHttp', () => {
-    const right = ['http://test'];
-    const wrong = ['https://test', 'testing', '123.0.1'];
+    const right = ['https://test', 'http://test'];


Did you get https working?
or did it always work we just did not allow it

i have been testing with ontime cloud and it seems fine. can you verify in one of your containers?

alex-Arc · 2025-01-21T12:16:48Z

apps/server/src/app.ts

@@ -283,7 +257,6 @@ export const shutdown = async (exitCode = 0) => {

  expressServer?.close();
  runtimeService.shutdown();


do we not still need the osc shutdown here?

yes we do, good catch

merge with automation service

rename automations > triggers rename blueprints > automations chore: add link to documentation

cpvalente force-pushed the integrations branch 3 times, most recently from e6ee477 to cd34149 Compare November 25, 2024 07:04

cpvalente changed the title ~~Integrations~~ wip: integrations Nov 29, 2024

cpvalente force-pushed the integrations branch 3 times, most recently from a20ed53 to 1264147 Compare December 29, 2024 11:22

cpvalente force-pushed the integrations branch 14 times, most recently from 6409c38 to 717a1ce Compare January 6, 2025 18:55

cpvalente force-pushed the integrations branch 6 times, most recently from 399fe0e to e3246f5 Compare January 11, 2025 15:55

cpvalente changed the title ~~wip: integrations~~ automations Jan 11, 2025

cpvalente commented Jan 11, 2025

View reviewed changes

cpvalente force-pushed the integrations branch 3 times, most recently from a5663da to 97d34bf Compare January 20, 2025 18:33

cpvalente force-pushed the integrations branch from 97d34bf to f9b9e64 Compare January 20, 2025 18:38

alex-Arc reviewed Jan 21, 2025

View reviewed changes

cpvalente force-pushed the integrations branch from f9b9e64 to b732178 Compare January 21, 2025 20:21

cpvalente and others added 13 commits January 22, 2025 20:51

refactor: allow both http and https

6dc3043

feat: automation service

d6525a0

feat: automation UI

d7fe927

merge with automation service

feat: send http

38df6e1

feat: send osc

c729ad6

refactor: migrate triggers to new service

4214907

refactor: remove legacy service

1148936

feat: filtering logic

53d41f1

refactor: migrate to OSC min

66f449e

chore: rename feature elements

aec8776

rename automations > triggers rename blueprints > automations chore: add link to documentation

refactor: handle non numeric comparisons

5636025

refactor: handle boolean comparisons

63b90ec

refactor: start with small runtimedata subset

3854534

cpvalente force-pushed the integrations branch 3 times, most recently from 4af6b14 to e4a4f78 Compare January 23, 2025 19:59

refactor: improve form ux

c4e2c81

cpvalente force-pushed the integrations branch from e4a4f78 to c4e2c81 Compare January 23, 2025 20:00

cpvalente added 2 commits January 23, 2025 21:07

refactor: simplify string comparisons

2ea3be7

refactor: remove > and < comparisons

60f6770

cpvalente merged commit a1d34c8 into master Jan 24, 2025
4 checks passed

cpvalente deleted the integrations branch January 24, 2025 12:53

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

automations #1343

automations #1343

cpvalente commented Nov 23, 2024 •

edited

Loading

coderabbitai bot commented Nov 23, 2024 •

edited

Loading

Review skipped

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Documentation and Community

cpvalente Jan 11, 2025

alex-Arc Jan 11, 2025

cpvalente Jan 11, 2025

cpvalente Jan 11, 2025

cpvalente commented Jan 20, 2025

alex-Arc Jan 21, 2025

cpvalente Jan 21, 2025

alex-Arc Jan 21, 2025

cpvalente Jan 21, 2025

		@@ -283,7 +257,6 @@ export const shutdown = async (exitCode = 0) => {

		expressServer?.close();
		runtimeService.shutdown();

automations #1343

automations #1343

Conversation

cpvalente commented Nov 23, 2024 • edited Loading

coderabbitai bot commented Nov 23, 2024 • edited Loading

Review skipped

Walkthrough

Changes

Chat

CodeRabbit Commands (Invoked using PR comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Documentation and Community

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cpvalente commented Jan 20, 2025

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

cpvalente commented Nov 23, 2024 •

edited

Loading

coderabbitai bot commented Nov 23, 2024 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)