Improve performance: batch updates, configurable flush, dates

[leoisadev8]

Summary

• Improves performance and reliability by batching chat index updates, making metrics flush configurable, and tightening persistence logic across server, web UI, and guest handling.

Changes

Server

• PostHog integration
  • Add configurable flush behavior via POSTHOG_FLUSH_AT and POSTHOG_FLUSH_AT_RAW (default 10) and use this value for flushAt with a fixed flushInterval of 5s.
• Chat router (index.ts)
  • Reworked update logic to batch updates into a single set operation when possible.
  • Compute chatUpdates and only write to the DB when there are actual updates, then publish a concise sidebarPayload with updatedAt and/or lastMessageAt when present.
  • Avoids multiple, separate writes and minimizes unnecessary publishes.

Web UI

• AppSidebar (app-sidebar.tsx)
  • Introduced time-tracking fields: updatedAtMs, lastMessageAtMs, lastActivityMs to normalize and compare dates reliably.
  • Refined merge/dedup logic to use large-scale timestamp comparisons (ms) and derive lastActivityMs from the max of updatedAt and lastMessageAt.
  • Sorting now relies on lastActivityMs to reflect the latest overall activity, with stable secondary ordering.
  • Optimistic chat creation and updates now leverage normalization for consistent state shapes.

Web Chat API

• Chat streaming persistence (chat-handler.ts)
  • Strengthened streaming flush behavior:
    • Compute delta since last persist and only flush when delta is positive and exceeds STREAM_MIN_CHARS_PER_FLUSH.
    • Respect STREAM_MIN_CHARS_PER_FLUSH and STREAM_FLUSH_INTERVAL_MS environment-based configuration.
  • Update lastPersistedLength accordingly to drive correct flush cadence.

Guest Client

• Guest ID persistence (guest.client.ts)
  • Hardened localStorage writes to avoid unnecessary rewrites and guard against private mode/storage errors.
  • Ensure cookie/localStorage consistency by only persisting when values actually change.
  • In ensureGuestIdClient, avoid redundant persistence and validate existing values before writing.

Why

• Reduces database churn by batching writes and avoiding unnecessary publishes.
• Aligns UI with reliable, time-based ordering for chat lists, improving UX.
• Improves resilience of guest ID persistence in privacy modes and across environments.
• Provides configurable, predictable behavior for analytics (PostHog) and streaming UI feedback.

Test plan

• Server
  • Set POSTHOG_API_KEY and POSTHOG_HOST, adjust POSTHOG_FLUSH_AT to various values (e.g., 5, 10, 20) and verify flushAt is set accordingly while flushInterval remains 5s.
• Chat router
  • Create new chats and multiple messages; verify that updates to chat index publish only once with updatedAt and/or lastMessageAt when appropriate.
• Web UI
  • Open app sidebar with existing chats; verify sorting uses lastActivityMs and reflects recent activity changes.
  • Simulate optimistic chat creation and confirm normalized payloads in UI state.
• Chat streaming
  • Feed streaming text and ensure persist is triggered only when delta exceeds STREAM_MIN_CHARS_PER_FLUSH and not on every keystroke.
• Guest ID persistence
  • In normal mode, ensure localStorage and cookies are written only when value changes; in private mode, ensure no crashes or writes occur.

Notes

• No breaking API changes; behavior enhancements are additive and guarded by existing environment variables where applicable.

🌿 Generated by Terry

---

ℹ️ Tag @terragon-labs to ask questions and address PR feedback

📎 Task: https://www.terragonlabs.com/task/10474417-4170-4315-a83a-c227695b1218

[leoisadev8]

Superseded by #60 (perf/chat-batching) after rebasing on main.