-
|
My scenario is: I have a large amount of data in this subject and I want to have JetStream stored on multiple machines, using 3 backups and 10 shard (such as partition in kafka) to make my data more disaster resistant |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
|
Hey Doslin. You don't need partitions (or shards) to improve HA and be more disaster resilient. You can set up your JetStream stream to have 3 replicas (R3) distributed across the cluster. If the above is not enough, then you can use deterministic subject token partitioning described here: |
Beta Was this translation helpful? Give feedback.
Hey Doslin.
You don't need partitions (or shards) to improve HA and be more disaster resilient.
Those are needed for performance scalability.
You can set up your JetStream stream to have 3 replicas (R3) distributed across the cluster.
If you find the performance limits on it, you can split that stream into two or more smaller ones (but still R3) based on subject tokens.
Usually, it's good to think about scalability when designing subjects architecture.
If the above is not enough, then you can use deterministic subject token partitioning described here:
https://docs.nats.io/nats-concepts/subject_mapping#deterministic-subject-token-partitioning to split data from one subject to many R3 stre…