Support for writing to ClickHouse local tables to improve write performance

[jiaozi07]

Use Cases

Hello Vector team,

I am currently using Vector for log processing and data ingestion, and I have been utilizing ClickHouse as my data storage solution. However, I have noticed that writing to ClickHouse distributed tables can sometimes introduce performance bottlenecks due to the additional overhead involved.

I would like to inquire if there are any plans to support writing directly to ClickHouse local tables. Writing to local tables could potentially improve write performance by reducing the overhead associated with distributed tables, especially in high-throughput scenarios.

Feature Request:

Support for writing to ClickHouse local tables: This feature would allow Vector to write directly to ClickHouse local tables, bypassing the additional overhead of distributed tables.
Configuration options: Provide configuration options to specify whether data should be written to local or distributed tables.
Use Case:

The ability to write to ClickHouse local tables would be particularly beneficial in scenarios where high write performance is critical. For example, in environments with high log ingestion rates, reducing the write overhead can lead to significant performance improvements.

Additional Context:

If there are any existing workarounds or best practices for optimizing write performance to ClickHouse using Vector, I would appreciate any guidance you can provide. Additionally, any insights into the feasibility and potential timeline for supporting this feature would be helpful.

Thank you for your attention to this request, and I look forward to your response.

Best regards