Skip to content

Commit

Permalink
VTP migration docs
Browse files Browse the repository at this point in the history
  • Loading branch information
@jracusin
jracusin committed Oct 28, 2024
1 parent 4bc4368 commit c3c5afd
Show file tree
Hide file tree
Showing 2 changed files with 229 additions and 0 deletions.
3 changes: 3 additions & 0 deletions app/routes/docs.tsx
View file
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,9 @@ export default function () {
<NavLink key="schema" to="schema">
Schema Browser
</NavLink>,
<NavLink key="vtp" to="vtp">
VOEvent Transport Protocol Migration
</NavLink>,
<NavLink key="admin" to="admin">
Administration
</NavLink>,
Expand Down
226 changes: 226 additions & 0 deletions app/routes/docs.vtp.mdx
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,226 @@
---
handle:
breadcrumb: VOEvent Transport Protocol Migration
---

import {
Card,
CardBody,
CardFooter,
CardGroup,
CardHeader,
GridContainer,
Icon,
IconList,
IconListContent,
IconListIcon,
IconListItem,
} from '@trussworks/react-uswds'

# VOEvent Transport Protocol Migration

VOEvent Transport Protocol (VTP) is a product of the [International Virtual Observatory Alliance (IVOA)](https://www.ivoa.net) that provides a mechanism for distribution of VOEvents to astronomical communities. The [Comet VOEvent broker for Python](https://comet.transientskp.org/en/latest/) and the [GCN Classic VOEvent distribution system](https://gcn.gsfc.nasa.gov/voevent.html) have been operational for many years, but are not actively developed or maintained.

This migration plan addresses only the transport protocol, not the XML data serialization of VOEvents or the [VOEvent-specific schema](https://wiki.ivoa.net/internal/IVOA/IvoaVOEvent/voevent_v2.html). GCN continues to distribute VOEvent XML from all GCN Classic notice types and new VOEvent messages via Kafka for missions that provide it to GCN.

## The Future of VTP

Organizations including the new GCN, Rubin, SCiMMA, have moved away from VTP in favor of more modern, general-purpose data distribution protocols like [Kafka](client). Kafka is widely used throughout industry, is well supported, and is growing in its use in astronomy. As GCN services move from GCN Classic to the new GCN, we are planning to retire the existing GCN Classic VOEvent system as it is currently implemented in several cloud-based servers that do not meet NASA standards. The GCN team is gradually migrating to or replacing GCN Classic services in the new GCN, and is not planning any new development in GCN Classic.

We intend for this transition to be planned, announced, and implemented with ample warning for the user community. We propose 3 alternatives for how to complete this transition, and are soliciting [user feedback](vtp#feedback-survey). We know that many pipelines, robotic telescopes, and users depend on this service and we aim to be minimally distriptive to their operations.

## GCN VTP Migration Options

The GCN team is exploring the following options to meet the VOEvent needs of the GCN user community. The current GCN VOEvent broker must be updated to meet NASA standards, with a minimum change for VOEvent users to update their server names.

<br />
<CardGroup>
<Card gridLayout={{ tablet: { col: 4 } }} headerFirst>
<CardHeader className="bg-base-lighter">
<p className="margin-0 text-base-dark">Option 1:</p>
<h4 className="usa-card__heading">
Replace VTP with VOEvents over Kafka
</h4>
</CardHeader>
<CardBody>
<IconList>
<IconListItem>
<IconListIcon className="text-green">
<Icon.Check aria-hidden="true" />
</IconListIcon>
<IconListContent>
Service already exists and is available for use
</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-green">
<Icon.Check aria-hidden="true" />
</IconListIcon>
<IconListContent>Self service subscription</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-green">
<Icon.Check aria-hidden="true" />
</IconListIcon>
<IconListContent>Open source</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-green">
<Icon.Check aria-hidden="true" />
</IconListIcon>
<IconListContent>Transport security layer</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-green">
<Icon.Check aria-hidden="true" />
</IconListIcon>
<IconListContent>
Broker is the server, minimal firewall configuration{' '}
</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-green">
<Icon.Check aria-hidden="true" />
</IconListIcon>
<IconListContent>Client-side filtering</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-orange">
<Icon.ErrorOutline aria-hidden="true" />
</IconListIcon>
<IconListContent>
Requires users to modify their client code to stream notices via
Kafka using [GCN Kafka Client](client)
</IconListContent>
</IconListItem>
</IconList>
</CardBody>
</Card>
<Card gridLayout={{ tablet: { col: 4 } }} headerFirst>
<CardHeader className="bg-base-lighter">
<p className="margin-0 text-base-dark">Option 2:</p>
<h4 className="usa-card__heading">
Replace VTP Broker with Python Comet Client
</h4>
</CardHeader>
<CardBody>
<IconList>
<IconListItem>
<IconListIcon className="text-orange">
<Icon.ErrorOutline aria-hidden="true" />
</IconListIcon>
<IconListContent>Requires new development from GCN</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-green">
<Icon.Check aria-hidden="true" />
</IconListIcon>
<IconListContent>Self service subscription</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-green">
<Icon.Check aria-hidden="true" />
</IconListIcon>
<IconListContent>Open source</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-red">
<Icon.Close aria-hidden="true" />
</IconListIcon>
<IconListContent>No transport security layer</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-green">
<Icon.Check aria-hidden="true" />
</IconListIcon>
<IconListContent>
Broker is the server, minimal firewall configuration
</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-orange">
<Icon.ErrorOutline aria-hidden="true" />
</IconListIcon>
<IconListContent>
Subscriber becomes client and utilizes [client-side
filtering](https://github.com/lpsinger/pygcn/tree/filter?tab=readme-ov-file#server-side-filtering)
</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-orange">
<Icon.ErrorOutline aria-hidden="true" />
</IconListIcon>
<IconListContent>
Requires update to broker hostname and client
</IconListContent>
</IconListItem>
</IconList>
</CardBody>
</Card>
<Card gridLayout={{ tablet: { col: 4 } }} headerFirst>
<CardHeader className="bg-base-lighter">
<p className="margin-0 text-base-dark">Option 3:</p>
<h4 className="usa-card__heading">
Migrate Legacy VTP Brokers to GCN AWS
</h4>
</CardHeader>
<CardBody>
<IconList>
<IconListItem>
<IconListIcon className="text-red">
<Icon.Close aria-hidden="true" />
</IconListIcon>
<IconListContent>
Requires significant development effort by GCN team
</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-red">
<Icon.Close aria-hidden="true" />
</IconListIcon>
<IconListContent>
Subscriptions manually maintained by GCN team or anonymous
connections
</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-red">
<Icon.Close aria-hidden="true" />
</IconListIcon>
<IconListContent>Closed source</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-red">
<Icon.Close aria-hidden="true" />
</IconListIcon>
<IconListContent>No transport security layer</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-orange">
<Icon.ErrorOutline aria-hidden="true" />
</IconListIcon>
<IconListContent>
Requires firewall configuration to point inbound connection from new
broker IP address
</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-orange">
<Icon.ErrorOutline aria-hidden="true" />
</IconListIcon>
<IconListContent>Requires update to broker hostname</IconListContent>
</IconListItem>
<IconListItem>
<IconListIcon className="text-orange">
<Icon.ErrorOutline aria-hidden="true" />
</IconListIcon>
<IconListContent>Server-side filtering</IconListContent>
</IconListItem>
</IconList>
</CardBody>
</Card>
</CardGroup>

## Feedback Survey

To collect your feedback, we request that you fill out [this survey](https://forms.gle/8tVRkmQd2v99FqYYA) by November 1, 2024.

0 comments on commit c3c5afd

Please sign in to comment.