[feat, multicast] Add IP pool multicast support #9084

zeeshanlakhani · 2025-09-25T01:23:42Z

This work introduces multicast IP pool capabilities to support external multicast traffic routing through the rack's switching infrastructure.

Closes #8217.

Includes:

Add IpPoolType enum (unicast/multicast) with unicast as default
Add database migration (multicast-support/up01.sql) with new columns and indexes
Add ASM/SSM range validation for multicast pools to prevent mixing
Add pool type-aware resolution for IP allocation
Add custom deserializer for switch port uplinks with deduplication
Update external API params/views for multicast pool configuration
Add SSM constants (IPV4_SSM_SUBNET, IPV6_SSM_FLAG_FIELD) for validation

Database schema updates:

ip_pool table: pool_type
Index on pool_type for efficient filtering
Migration preserves existing pools as unicast type by default

This provides the foundation for multicast group functionality while maintaining full backward compatibility with existing unicast pools.

References (for review):

RFD 488: https://rfd.shared.oxide.computer/rfd/488
Dendrite PRs (based on recency):

TODOs:

Multicast Group Support ([feat, multicast] Multicast Group Support #9091)

This work introduces multicast IP pool capabilities to support external multicast traffic routing through the rack's switching infrastructure. Includes: - Add IpPoolType enum (unicast/multicast) with unicast as default - Add multicast pool fields: switch_port_uplinks (UUID[]), mvlan (VLAN ID) - Add database migration (multicast-support/up01.sql) with new columns and indexes - Add ASM/SSM range validation for multicast pools to prevent mixing - Add pool type-aware resolution for IP allocation - Add custom deserializer for switch port uplinks with deduplication - Update external API params/views for multicast pool configuration - Add SSM constants (IPV4_SSM_SUBNET, IPV6_SSM_FLAG_FIELD) for validation Database schema updates: - ip_pool table: pool_type, switch_port_uplinks, mvlan columns - Index on pool_type for efficient filtering - Migration preserves existing pools as unicast type by default This provides the foundation for multicast group functionality while maintaining full backward compatibility with existing unicast pools. References (for review): - RFD 488: https://rfd.shared.oxide.computer/rfd/488 - Dendrite PRs (based on recency): * oxidecomputer/dendrite#132 * oxidecomputer/dendrite#109 * oxidecomputer/dendrite#14

Cargo.toml

Introduces end-to-end multicast group support across control plane and sled-agent, integrated with IP pool extensions required for supporting multicast workflows. This work enables project-scoped multicast groups with lifecycle-driven dataplane programming and exposes an API for operating multicast groups over instances. Highlights: - DB: new multicast_group tables; member lifecycle management - API: multicast group/member CRUD; source IP validation; VPC/project hierarchy integration with default VNI fallback - Control plane: RPW reconcilers for groups/members; sagas for dataplane updates atomically at the group level; instance lifecycle hooks and piggybacking - Dataplane: Dendrite DPD switch programming via trait abstraction; DPD client used in tests - Sled agent: multicast-aware instance management; network interface configuration for multicast traffic; cross-version testing; OPTE stubs present - Tests: comprehensive integration suites under nexus/tests/integration_tests/multicast/ Components: - Database schema: external and underlay multicast groups; member/instance association tables - Control plane modules: multicast group management, member lifecycle, dataplane abstraction; RPW reconcilers to ensure convergence - API layer: endpoints and validation; default-VNI semantics when VPC not provided - Sled agent: OPTE stubs and compatibility shims for older agents Workflows Implemented: 1. Instance lifecycle integration: - "Create" -> resolve VPC/VNI (or default), validate source IPs, create memberships, enqueue group ensure RPW - "Start" -> program dataplane via ensure/update sagas; activate member flows after switch ack - "Stop" -> deactivate dataplane membership; retain DB membership for fast restart - "Delete" -> remove instance memberships; group deletion is explicit - "Migrate" -> deactivate on source sled; activate on target; idempotent with ordering guarantees - Restart/recovery -> RPWs reconcile desired state; compensations clean up partial programming 2. RPW reconciliation: - ensure dataplane switches match database state - handle sled migrations and state transitions - Eventual consistency with retry logic Migrations: - Apply schema changes in schema/crdb/multicast-support/up01.sql (and update dbinit.sql) - Bump schema versions accordingly API/Compatibility: - OpenAPI updated: openapi/nexus.json, openapi/sled-agent/sled-agent-5.0.0-89f1f7.json - Regenerate clients where applicable References: - RFD 488: https://rfd.shared.oxide.computer/rfd/488 - IP Pool extensions: #9084 - Dendrite PRs (based on recency): * oxidecomputer/dendrite#132 * oxidecomputer/dendrite#109 * oxidecomputer/dendrite#14 Follow-ups include: - OPTE integration - commtest extension - omdb commands are tracked in issues - pool and group stats

Introduces end-to-end multicast group support across control plane and sled-agent, integrated with IP pool extensions required for supporting multicast workflows. This work enables project-scoped multicast groups with lifecycle-driven dataplane programming and exposes an API for operating multicast groups over instances. Highlights: - DB: new multicast_group tables; member lifecycle management - API: multicast group/member CRUD; source IP validation; VPC/project hierarchy integration with default VNI fallback - Control plane: RPW reconcilers for groups/members; sagas for dataplane updates atomically at the group level; instance lifecycle hooks and piggybacking - Dataplane: Dendrite DPD switch programming via trait abstraction; DPD client used in tests - Sled agent: multicast-aware instance management; network interface configuration for multicast traffic; cross-version testing; OPTE stubs present - Tests: comprehensive integration suites under nexus/tests/integration_tests/multicast/ Components: - Database schema: external and underlay multicast groups; member/instance association tables - Control plane modules: multicast group management, member lifecycle, dataplane abstraction; RPW reconcilers to ensure convergence - API layer: endpoints and validation; default-VNI semantics when VPC not provided - Sled agent: OPTE stubs and compatibility shims for older agents Workflows Implemented: 1. Instance lifecycle integration: - "Create" -> resolve VPC/VNI (or default), validate source IPs, create memberships, enqueue group ensure RPW - "Start" -> program dataplane via ensure/update sagas; activate member flows after switch ack - "Stop" -> deactivate dataplane membership; retain DB membership for fast restart - "Delete" -> remove instance memberships; group deletion is explicit - "Migrate" -> deactivate on source sled; activate on target; idempotent with ordering guarantees - Restart/recovery -> RPWs reconcile desired state; compensations clean up partial programming 2. RPW reconciliation: - ensure dataplane switches match database state - handle sled migrations and state transitions - Eventual consistency with retry logic Migrations: - Apply schema changes in schema/crdb/multicast-group-support/up01.sql (and update dbinit.sql) - Bump schema versions accordingly API/Compatibility: - OpenAPI updated: openapi/nexus.json, openapi/sled-agent/sled-agent-5.0.0-89f1f7.json - Contains a version change (to v5) as InstanceEnsureBody has been modified to include multicast_groups associated with an instance in the underlying sled config - Regenerate clients where applicable References: - RFD 488: https://rfd.shared.oxide.computer/rfd/488 - IP Pool extensions: #9084 - Dendrite PRs (based on recency): * oxidecomputer/dendrite#132 * oxidecomputer/dendrite#109 * oxidecomputer/dendrite#14 Follow-ups include: - OPTE integration - commtest extension - omdb commands are tracked in issues - pool and group stats

…support

nexus/types/src/external_api/params.rs

…support

… pools

zeeshanlakhani · 2025-10-09T15:13:11Z

~~New changes pushed onto this branch, but the PR is not picking it up: bcb4fc6#diff-6e5a0e9d23e313fccb92594537052a498855670462b49a8748d4ed3611ba4b37 and a merge commit to bring in the schema updates.~~ (it now works)

rcgoodfellow

Thanks for the updates @zeeshanlakhani. Comments follow from a full read through.

nexus/types/src/external_api/params.rs

common/src/address.rs

nexus/db-queries/src/db/datastore/ip_pool.rs

nexus/db-queries/src/db/datastore/switch_port.rs

nexus/src/app/ip_pool.rs

rcgoodfellow

LGTM. @internet-diglett can you take a look in terms of overall omicron integration?

internet-diglett

Great work! There were a few things that stuck out to me where I know you are probably just working around existing code, but it might be worth updating when we / if can do so.

common/src/vlan.rs

nexus/db-model/src/ip_pool.rs

Also, we add DB a constraint and specialize the range error.

…support

zeeshanlakhani · 2025-10-15T05:35:15Z

@internet-diglett made that change for ranges if you wanna 👀. Updating from main as well.

zeeshanlakhani force-pushed the zl/ip-pool-multicast-support branch from b692148 to 98bf71e Compare September 25, 2025 04:18

zeeshanlakhani requested review from FelixMcFelix, internet-diglett and rcgoodfellow September 25, 2025 06:37

zeeshanlakhani commented Sep 25, 2025

View reviewed changes

Cargo.toml Outdated Show resolved Hide resolved

zeeshanlakhani mentioned this pull request Sep 25, 2025

[feat, multicast] Multicast Group Support #9091

Open

zeeshanlakhani self-assigned this Sep 25, 2025

zeeshanlakhani requested a review from bnaecker September 25, 2025 16:13

zeeshanlakhani changed the title ~~[feat] Add IP pool multicast support~~ [feat, multicast] Add IP pool multicast support Sep 25, 2025

zeeshanlakhani requested a review from taspelund September 26, 2025 02:19

zeeshanlakhani added 2 commits September 30, 2025 08:13

Merge remote-tracking branch 'origin/main' into zl/ip-pool-multicast-…

6bd8e19

…support

Merge remote-tracking branch 'origin/main' into zl/ip-pool-multicast-…

20db268

…support

rcgoodfellow reviewed Oct 9, 2025

View reviewed changes

nexus/types/src/external_api/params.rs Outdated Show resolved Hide resolved

zeeshanlakhani added 2 commits October 9, 2025 11:16

Merge remote-tracking branch 'origin/main' into zl/ip-pool-multicast-…

6fe0c6a

…support

[review] move mvlan and switch port uplinks (for mcast egress) out of…

bcb4fc6

… pools

zeeshanlakhani force-pushed the zl/ip-pool-multicast-support branch from 0e7ad02 to bcb4fc6 Compare October 9, 2025 15:29

[fmt] fixes

4433066

zeeshanlakhani requested a review from rcgoodfellow October 9, 2025 15:39

rcgoodfellow reviewed Oct 11, 2025

View reviewed changes

zeeshanlakhani added 2 commits October 12, 2025 06:11

[review] comments, validation, & cleanup

f7991c3

[api] run generate post-API changes

1b24936

rcgoodfellow approved these changes Oct 12, 2025

View reviewed changes

internet-diglett approved these changes Oct 13, 2025

View reviewed changes

common/src/vlan.rs Show resolved Hide resolved

nexus/db-model/src/ip_pool.rs Outdated Show resolved Hide resolved

[review] have IP range conversions use tryfrom

666446f

Also, we add DB a constraint and specialize the range error.

zeeshanlakhani requested a review from internet-diglett October 14, 2025 04:23

Merge remote-tracking branch 'origin/main' into zl/ip-pool-multicast-…

c636f66

…support

[fix] db ordering

f52ce73

zeeshanlakhani merged commit 773799e into main Oct 15, 2025
17 checks passed

zeeshanlakhani deleted the zl/ip-pool-multicast-support branch October 15, 2025 13:28

Uh oh!

[feat, multicast] Add IP pool multicast support #9084

[feat, multicast] Add IP pool multicast support #9084

Uh oh!

Conversation

zeeshanlakhani commented Sep 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zeeshanlakhani commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rcgoodfellow left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rcgoodfellow left a comment

Choose a reason for hiding this comment

Uh oh!

internet-diglett left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

zeeshanlakhani commented Oct 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

zeeshanlakhani commented Sep 25, 2025 •

edited

Loading

zeeshanlakhani commented Oct 9, 2025 •

edited

Loading