[feat(ai)]: Production-ready AI agent with RAG, streaming responses, and comprehensive tool ecosystem

[stringl1l1l1l]

This PR aggregate a series of development, transforming the Dubbo Admin AI from an initial prototype into a production-ready, intelligent troubleshooting platform. It merges the foundational work of the initial agent, the architectural refactoring for a robust ReAct pattern, and the final enhancements for advanced capabilities, production-grade serving, and a rich user experience.

Major Changes

Architecture Redesign

• Enhanced ReAct Agent Pipeline: The architecture has evolved from a preliminary prototype to a robust Think-Act-Observe cycle. The core logic was refactored into distinct agent and orchestrator components for improved modularity and is now enhanced with advanced memory management and context awareness.
• Modular Tool System: A flexible tool registry has been implemented, supporting multiple tool managers (Mock, Internal, MCP) to allow for extensible and manageable toolsets.
• Advanced Memory Management: To support complex conversations, we've added session-based conversation history featuring sliding window memory and turn-based organization.
• Streaming Response System: Building on the initial streaming support, this PR introduces a comprehensive Server-Sent Events (SSE) system for fully interactive, real-time AI responses.

AI Capabilities Enhancement

• Multi-Model Support: Expanded beyond initial LLM integration to support a variety of models, including Qwen3, Qwen-max, DeepSeek, and various embedding models.
• RAG Integration: Introduced a complete Retrieval-Augmented Generation (RAG) pipeline using the Pinecone vector database and Cohere's reranking model to provide accurate, context-aware answers from a knowledge base.
• Knowledge Base Tools: Integrated tools for retrieving information from Kubernetes documentation and other domain-specific knowledge sources.
• Intent Recognition: Implemented improved user intent classification, leveraging memory search and configuration guidance to better understand user queries.

Tool & Integration Ecosystem

• MCP (Model Context Protocol) Support: Added integration with an MCP server for performing real-time cluster operations and diagnostics.
• Memory Tools: Built a comprehensive set of tools for retrieving and managing session memory, allowing the agent to have long-term context.
• Document Processing: Added advanced Markdown and PDF processing capabilities for building and maintaining the knowledge base.

Production-Ready API Server

• RESTful API Design: A complete HTTP server was implemented using the Gin framework, featuring robust routing, middleware, and proper error handling.
• Session Management: Full session lifecycle management is now included, with automatic creation, tracking, and cleanup of user sessions.
• OpenAPI Documentation: Generated comprehensive API documentation with examples and schemas to facilitate integration and testing.
• CORS Support: Implemented Cross-Origin Resource Sharing (CORS) to allow for seamless integration with web-based frontends.

Enhanced User Experience

• Real-time Streaming: Users now receive live response streaming with structured content blocks and progress indicators for a more dynamic experience.
• Context Preservation: The system intelligently manages conversation context across multiple turns within a session.
• Error Recovery: Added robust error handling mechanisms for graceful degradation and clearer error reporting.
• Usage Tracking: Integrated token usage monitoring for tracking costs and managing API usage.

Technical Improvements

• Configuration Management: The configuration system has been centralized and enhanced to support environment-based settings with validation.
• Logging System: A structured logging system with configurable levels has been integrated for better observability.
• Testing Suite: A comprehensive suite of unit and integration tests has been added to ensure system stability and reliability.
• Build System: The build process and dependency management have been updated to streamline development and deployment.

Key Features Added

1. Advanced RAG Pipeline: Semantic search with vector embeddings and reranking for high-quality, knowledge-based answers.
2. Multi-Domain Knowledge: On-demand retrieval of Kubernetes and Dubbo-specific documentation.
3. Real-time Tool Execution: Live system monitoring and diagnostic capabilities through integrated tools.
4. Session Persistence: Long-term conversation history and context management for coherent interactions.
5. Production Monitoring: Includes health checks, metrics, and observability features for a production environment.

Breaking Changes

• The API has been completely redesigned; old endpoints are no longer compatible.
• A new session-based interaction model replaces the previous stateless requests.
• The configuration format and required environment variables have been updated.
• Response schemas have been modified to support the new streaming and content block format.
  To help us figure out who should review this PR, please put an X in all the areas that this PR affects.

• Docs
• Installation
• User Experience
• Dubboctl
• Console
• Core Component