Implement PostHog telemetry across server and web OpenChat #43
Conversation
- Add base super properties and sanitize event properties for client and server PostHog usage - Track fallback storage usage in server router for chats and messages - Add detailed client event tracking in web chat handler and components - Introduce PosthogBootstrap component for consistent client identification and property registration - Enhance event tracking with user workspace IDs, auth states, theme info, and OpenRouter API key status - Track marketing CTA clicks and landing visits with detailed metadata - Improve rate limit and error event captures for chat and OpenRouter APIs - Add client property registration on model selection and OpenRouter key management - Add sync connection state events to PostHog This update improves analytics granularity and consistency across client and server, enabling better user behavior insights and troubleshooting. Co-authored-by: terragon-labs[bot] <terragon-labs[bot]@users.noreply.github.com>
|
@greptile what custom events were added? |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
Greptile Overview
Greptile Summary
This PR implements comprehensive PostHog telemetry across the OpenChat application, adding observability to both server and web components. The implementation introduces centralized telemetry infrastructure with environment-aware super properties, property sanitization, and workspace-scoped tracking. Key additions include a PostHogBootstrap component for initialization, enhanced user identification with workspace grouping, and detailed event tracking across UI flows (chat interactions, model selection, file attachments) and server operations (chat creation, rate limiting, storage fallback scenarios). The telemetry system captures user behavior patterns, system performance metrics, and error conditions while maintaining data quality through undefined value sanitization and privacy through IP hashing.
Important Files Changed
Changed Files
| Filename | Score | Overview |
|---|---|---|
| apps/web/src/lib/sync.ts | 4/5 | Added telemetry for WebSocket connection state tracking with retry patterns |
| apps/server/src/lib/posthog.ts | 4/5 | Implemented comprehensive server-side PostHog infrastructure with sanitization |
| apps/web/src/components/openrouter-link-modal.tsx | 5/5 | Added telemetry for API key prompt events with deduplication logic |
| apps/web/src/lib/posthog.ts | 4/5 | Enhanced client telemetry with workspace grouping and property utilities |
| apps/web/src/components/posthog-bootstrap.tsx | 4/5 | New centralized bootstrap component for telemetry initialization |
| apps/server/src/routers/index.ts | 4/5 | Added comprehensive telemetry across chat operations and fallback scenarios |
| apps/web/src/lib/posthog-server.ts | 4/5 | Server-side telemetry setup with potential naming inconsistency |
| apps/web/src/components/chat-composer.tsx | 4/5 | Added file attachment event tracking with metadata |
| apps/web/src/components/account-settings-modal.tsx | 5/5 | Implemented telemetry for OpenRouter key management events |
| apps/web/src/app/api/chat/chat-handler.ts | 4/5 | Added comprehensive chat API telemetry with IP hashing for privacy |
| apps/web/src/components/app-sidebar.tsx | 4/5 | Enhanced with user identification and dashboard engagement tracking |
| apps/web/src/components/providers.tsx | 4/5 | Improved pageview tracking with referrer analysis and bootstrap integration |
| apps/web/src/components/model-selector.tsx | 4/5 | Added model selection tracking for user preference analytics |
| apps/web/src/components/chat-room.tsx | 4/5 | Comprehensive telemetry for chat interactions and error conditions |
| apps/web/src/components/hero-section.tsx | 4/5 | Converted to client component with marketing analytics tracking |
Confidence score: 4/5
- This PR is safe to merge with minimal risk as it primarily adds observability features without changing core application logic
- Score reflects comprehensive telemetry implementation with good error handling patterns, though some files have minor concerns like potential naming inconsistencies and the conversion of server components to client components
- Pay close attention to apps/web/src/lib/posthog-server.ts for the "openchat-server" naming in web context and apps/web/src/components/hero-section.tsx for the server-to-client component conversion impact
Sequence Diagram
sequenceDiagram
participant User
participant Browser
participant PosthogBootstrap
participant WebApp
participant ServerAPI
participant PosthogCloud
User->>Browser: "Visit OpenChat"
Browser->>PosthogBootstrap: "Initialize on page load"
PosthogBootstrap->>PosthogCloud: "capture('$pageview')"
PosthogBootstrap->>PosthogBootstrap: "identifyClient(userId, workspaceId)"
PosthogBootstrap->>PosthogBootstrap: "registerClientProperties(auth_state, theme)"
User->>WebApp: "Click 'New Chat'"
WebApp->>ServerAPI: "client.chats.create()"
ServerAPI->>PosthogCloud: "capturePosthogEvent('chat.created')"
WebApp->>PosthogCloud: "captureClientEvent('chat.created')"
User->>WebApp: "Send chat message"
WebApp->>ServerAPI: "POST /api/chat"
ServerAPI->>ServerAPI: "isRateLimited(request)"
alt Rate limited
ServerAPI->>PosthogCloud: "captureServerEvent('chat.rate_limited')"
ServerAPI->>WebApp: "429 Too Many Requests"
else Not rate limited
ServerAPI->>ServerAPI: "streamText(model, messages)"
ServerAPI->>PosthogCloud: "captureServerEvent('chat_message_stream')"
end
User->>WebApp: "Save OpenRouter key"
WebApp->>WebApp: "saveOpenRouterKey(key)"
WebApp->>PosthogCloud: "captureClientEvent('openrouter.key_saved')"
WebApp->>PosthogCloud: "registerClientProperties({has_openrouter_key: true})"
User->>WebApp: "Attach file to chat"
alt File too large
WebApp->>PosthogCloud: "captureClientEvent('chat.attachment_event', {result: 'rejected'})"
else File accepted
WebApp->>PosthogCloud: "captureClientEvent('chat.attachment_event', {result: 'accepted'})"
end
User->>WebApp: "Select model"
WebApp->>PosthogCloud: "registerClientProperties({model_id: selectedModel})"
alt OpenRouter models fetch fails
WebApp->>PosthogCloud: "captureClientEvent('openrouter.models_fetch_failed')"
end
User->>WebApp: "Click marketing CTA"
WebApp->>PosthogCloud: "captureClientEvent('marketing.cta_clicked')"
Context used:
- Context from
dashboard- AGENTS.md (source)
15 files reviewed, 10 comments
|
|
||
| useEffect(() => { | ||
| if (!resolvedWorkspaceId) return; | ||
| if (identifyRef.current === resolvedWorkspaceId && session?.user) return; |
There was a problem hiding this comment.
logic: The deduplication logic may miss re-identification when session state changes from guest to member with the same workspace ID
| process.env.POSTHOG_DEPLOYMENT_REGION ?? process.env.VERCEL_REGION ?? "local"; | ||
|
|
||
| const BASE_SUPER_PROPERTIES = Object.freeze({ | ||
| app: "openchat-server", |
There was a problem hiding this comment.
logic: The app property is set to 'openchat-server' but this file is in apps/web. Consider using 'openchat-web' to match the web context.
| identifyClient(currentUserId, { | ||
| workspaceId: currentUserId, | ||
| properties: { auth_state: session?.user ? "member" : "guest" }, | ||
| }); | ||
| registerClientProperties({ | ||
| auth_state: session?.user ? "member" : "guest", | ||
| workspace_id: currentUserId, | ||
| }); | ||
| }, [currentUserId, session?.user]); |
There was a problem hiding this comment.
style: User identification runs on every session change, potentially causing redundant PostHog calls. Consider adding a check to only identify when user ID or auth state actually changes.
| try { | ||
| const now = new Date(); | ||
| const { id } = await client.chats.create({ title: "New Chat" }); | ||
| const { id, storageBackend = "postgres" } = await client.chats.create({ title: "New Chat" }); |
There was a problem hiding this comment.
logic: The storageBackend destructuring assumes the server response includes this field, but there's no type safety or validation. If the server doesn't return storageBackend, it will default to 'postgres' which may not be accurate.
| const dashboardTrackedRef = useRef(false); | ||
| useEffect(() => { | ||
| if (dashboardTrackedRef.current) return; | ||
| if (isLoading) return; | ||
| dashboardTrackedRef.current = true; | ||
| void (async () => { | ||
| let hasKey = false; | ||
| try { | ||
| const key = await loadOpenRouterKey(); | ||
| hasKey = Boolean(key); | ||
| registerClientProperties({ has_openrouter_key: hasKey }); | ||
| } catch { | ||
| hasKey = false; | ||
| } | ||
| const entryPath = typeof window !== "undefined" ? window.location.pathname || "/dashboard" : "/dashboard"; | ||
| captureClientEvent("dashboard.entered", { | ||
| chat_total: baseChats.length, | ||
| has_api_key: hasKey, | ||
| entry_path: entryPath, | ||
| brand_theme: brandTheme, | ||
| }); | ||
| })(); | ||
| }, [baseChats.length, brandTheme, isLoading]); |
There was a problem hiding this comment.
style: Dashboard tracking logic is complex and could be extracted into a custom hook for better testability and reusability across components.
| if (client) { | ||
| setPosthogClient(client); | ||
| client.capture("$pageview"); | ||
| const referrerUrl = document.referrer && document.referrer.length > 0 ? document.referrer : "direct"; |
There was a problem hiding this comment.
style: Consider using optional chaining: document.referrer?.length > 0 for cleaner syntax
| React.useEffect(() => { | ||
| if (!selectedValue) return | ||
| registerClientProperties({ model_id: selectedValue }) | ||
| }, [selectedValue]) |
There was a problem hiding this comment.
style: registering client properties on every selectedValue change, including initial load, could create noise in analytics if models are frequently switched or reset programmatically
| setInternalValue(currentValue) | ||
| } | ||
| onChange?.(currentValue) | ||
| registerClientProperties({ model_id: currentValue }) |
There was a problem hiding this comment.
logic: redundant registerClientProperties call since the useEffect on line 96 will trigger when selectedValue changes after this selection
| registerClientProperties({ has_openrouter_key: true }); | ||
| captureClientEvent("openrouter.key_saved", { | ||
| source: "modal", | ||
| masked_tail: key.slice(-4), | ||
| scope: "workspace", | ||
| }); |
There was a problem hiding this comment.
style: Logging masked API key tail could be a privacy risk - consider if this level of detail is necessary for analytics
|
|
||
| useEffect(() => { | ||
| if (visitTrackedRef.current) return | ||
| if (typeof session === 'undefined') return |
There was a problem hiding this comment.
logic: Session loading state check may cause telemetry to miss some visits if session loads after component mount
|
Superseded by #57 (feat: telemetry and security hardening) now on main. |
Summary
Changes
Core Telemetry
Backend Telemetry
Client Telemetry (UI)
OpenRouter / Key Storage Telemetry
Misc / Utilities
Observability & Data Quality
Verification / Test Plan
If any telemetry appears missing in dev, confirm that POSTHOG_API_KEY and host are configured, and that the new bootstrap and client helpers are loaded on the respective pages.
🌿 Generated by Terry
ℹ️ Tag @terragon-labs to ask questions and address PR feedback
📎 Task: https://www.terragonlabs.com/task/f9f3d737-4316-4784-b610-0c0cd5c6b386