Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor + new account sync metrics + isolating health status inside folder admin-panel > health-status #10314

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

ehconitin
Copy link
Contributor

@ehconitin ehconitin commented Feb 18, 2025

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

This PR introduces significant changes to the health status monitoring system, focusing on account synchronization and component organization.

  • Replaces message sync with comprehensive account sync monitoring in /packages/twenty-server/src/engine/core-modules/health/indicators/account-sync.health.ts, adding calendar sync metrics
  • Reorganizes health status components into dedicated /health-status folder structure for better code organization
  • Simplifies SystemHealth DTO in /packages/twenty-server/src/engine/core-modules/admin-panel/dtos/system-health.dto.ts to use direct status enums
  • Adds calendar sync health monitoring with 20% failure rate threshold and timeout handling
  • Integrates health monitoring into calendar event import system via HealthModule in related modules

Note: The PR description references issues about keyboard shortcuts and soft focus mode, but the actual changes are unrelated to those issues.

39 file(s) reviewed, 14 comment(s)
Edit PR Review Bot Settings | Greptile

the service is down
</StyledErrorMessage>
) : (
<StyledDetailsContainer>{formattedDetails}</StyledDetailsContainer>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: formattedDetails could be null here. Consider adding a fallback message when no details are available.

details,
title,
}: {
details: Record<string, any> | null;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Using Record<string, any> is too permissive. Consider creating a specific type for the details object structure

Comment on lines +151 to 153
accountSyncHealth.isHealthy.mockRejectedValue(
new Error(HEALTH_ERROR_MESSAGES.MESSAGE_SYNC_CHECK_FAILED),
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Using MESSAGE_SYNC_CHECK_FAILED error for accountSync is inconsistent with the service name change. Should use a new ACCOUNT_SYNC_CHECK_FAILED error message.

Comment on lines +141 to +147
const currentCounter =
await this.cacheStorage.get<AccountSyncJobByStatusCounter>(cacheKey);

const updatedCounter = {
...(currentCounter || {}),
[status]: (currentCounter?.[status] || 0) + increment,
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Duplicate code pattern between message and calendar sync counters. Consider extracting to a shared method to reduce duplication.

Comment on lines +279 to +281
expect(result.accountSync.status).toBe('up');
expect(result.accountSync.details.messageSync.status).toBe('up');
expect(result.accountSync.details.calendarSync.status).toBe('down');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Test case may be misleading - overall status is 'up' despite calendar sync being 'down'. Consider verifying if this is the intended behavior

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes I agree

Comment on lines +110 to +123
async isHealthy(): Promise<HealthIndicatorResult> {
const [messageResult, calendarResult] = await Promise.all([
this.checkMessageSyncHealth(),
this.checkCalendarSyncHealth(),
]);

const indicator = this.healthIndicatorService.check('accountSync');
const details = {
messageSync: messageResult.messageSync,
calendarSync: calendarResult.calendarSync,
};

return indicator.up({ details });
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: isHealthy() always returns 'up' status regardless of message/calendar sync health states. Should return 'down' if either sync check fails.

Suggested change
async isHealthy(): Promise<HealthIndicatorResult> {
const [messageResult, calendarResult] = await Promise.all([
this.checkMessageSyncHealth(),
this.checkCalendarSyncHealth(),
]);
const indicator = this.healthIndicatorService.check('accountSync');
const details = {
messageSync: messageResult.messageSync,
calendarSync: calendarResult.calendarSync,
};
return indicator.up({ details });
}
async isHealthy(): Promise<HealthIndicatorResult> {
const [messageResult, calendarResult] = await Promise.all([
this.checkMessageSyncHealth(),
this.checkCalendarSyncHealth(),
]);
const indicator = this.healthIndicatorService.check('accountSync');
const details = {
messageSync: messageResult.messageSync,
calendarSync: calendarResult.calendarSync,
};
if (messageResult.status === 'down' || calendarResult.status === 'down') {
return indicator.down({ details });
}
return indicator.up({ details });
}

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these are just metrics though

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with Greptile's comment. If we use an indicator to wrap thing then it should be a real indicator. Also true for the UI. What is the point of setting a failure rate threshold otherwise?

failureRate,
};

if (totalJobs === 0 || failureRate < 20) {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extract failure rate threshold as a const

@@ -4,7 +4,7 @@ export enum AdminPanelIndicatorHealthStatusInputEnum {
DATABASE = 'database',
REDIS = 'redis',
WORKER = 'worker',
MESSAGE_SYNC = 'messageSync',
ACCOUNT_SYNC = 'accountSync',
Copy link
Member

@FelixMalfait FelixMalfait Feb 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we used "connectedAccount" in the codebase so I probably would have stuck to that, "Connected Account Status" in the admin panel , etc

isLoading={loading}
RowRightComponent={({ item: service }) => (
<>
{service.id === 'ACCOUNT_SYNC' ? (
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make it a status like others

const services = [
{
id: 'DATABASE',
name: 'Database Status',
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also use this in breadcrumb?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Optimize Health Status Queries and Remove Redundant Data Fetching
2 participants