-
Notifications
You must be signed in to change notification settings - Fork 44
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
1 parent
2ad4607
commit 299a6fb
Showing
3 changed files
with
141 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
--- | ||
handle: | ||
breadcrumb: New Notice Producers | ||
--- | ||
|
||
import { | ||
ProcessList, | ||
ProcessListHeading, | ||
ProcessListItem, | ||
} from '@trussworks/react-uswds' | ||
|
||
# New Notice Producers | ||
|
||
The following steps guide new instrument, mission, or observatory producers | ||
into setting up new notices streams that are distributed to the user community | ||
via [Kafka](faq#what-is-kafka). This process requires interaction | ||
with the [GCN Team](https://heasarc.gsfc.nasa.gov/cgi-bin/Feedback?selected=kafkagcn) | ||
to enable accounts and Kafka topics creation on the GCN Kafka broker. The GCN Team is | ||
also happy to work with the mission teams to help construct your alerts. | ||
|
||
## Start Producing Alerts | ||
|
||
<ProcessList> | ||
<ProcessListItem> | ||
<ProcessListHeading type="h3"> | ||
Sign in / Sign up | ||
</ProcessListHeading> | ||
Decide which of your team members will have programmatic access to produce your alerts. | ||
Make sure that they have all signed in at least once to the [GCN website](https://gcn.nasa.gov/login) | ||
and the [GCN test website](https://test.gcn.nasa.gov/login). | ||
</ProcessListItem> | ||
<ProcessListItem> | ||
<ProcessListHeading type="h3"> | ||
Name Your Kafka Topics | ||
</ProcessListHeading> | ||
Names of Kafka topics follow the format <code>gcn.notices.<i>mission</i>.<i>notice_type</i></code>. | ||
Pick a prefix for your Kafka topic names, <code><i>mission</i>.\*</code>. | ||
</ProcessListItem> | ||
<ProcessListItem> | ||
<ProcessListHeading type="h3"> | ||
Contact the GCN Team | ||
</ProcessListHeading> | ||
Send the [GCN Team](/contact) | ||
your list of team members from Step 1 and your chosen Kafka topic prefix from Step 2. | ||
The GCN Team will reply after they have configured producer permissions for your team. | ||
</ProcessListItem> | ||
<ProcessListItem> | ||
<ProcessListHeading type="h3"> | ||
Build Producer Code | ||
</ProcessListHeading> | ||
- Log out and log back in. | ||
- Go through the [Start Streaming GCN Notices](/quickstart) process. | ||
- On Step 2, choose the scope <code>gcn.nasa.gov/kafka-<i>mission</i>-producer</code>. | ||
- Your producer code will look very similar to the [client example code](client) and | ||
Step 4 of [Start Streaming GCN Notices](/quickstart). `client_id` and `client_secret` | ||
can be found in Step 4 client example code. | ||
- Start from this and adjust the `client_id`, `client_secret`, `topic` and `data` content: | ||
|
||
```python | ||
from gcn_kafka import Producer | ||
# Connect as a producer. | ||
# Warning: don't share the client secret with others. | ||
producer = Producer(client_id='fill me in', client_secret='fill me in') | ||
# any topic starting with 'mission.' | ||
topic = 'gcn.notices.mission.example' | ||
data = b'...' # any bytes | ||
producer.produce(topic, data) | ||
``` | ||
|
||
</ProcessListItem> | ||
<ProcessListItem> | ||
<ProcessListHeading type="h3"> | ||
Create or Update the Mission Page | ||
</ProcessListHeading> | ||
Create a new mission page by submitting a [pull request](https://github.com/nasa-gcn/gcn.nasa.gov/pulls) | ||
or by sending text to the [GCN Team](https://heasarc.gsfc.nasa.gov/cgi-bin/Feedback?selected=kafkagcn). | ||
</ProcessListItem> | ||
|
||
<ProcessListItem> | ||
<ProcessListHeading type="h3"> | ||
Announce New Notice Types | ||
</ProcessListHeading> | ||
Work with the | ||
[GCN Team](/contact) | ||
to draft a community announcement, which the GCN Team will circulate. | ||
</ProcessListItem> | ||
</ProcessList> |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,51 @@ | ||
--- | ||
handle: | ||
breadcrumb: Advanced Documentation | ||
--- | ||
|
||
import { Highlight } from '~/components/Highlight' | ||
|
||
# Attachments | ||
|
||
## Parsing HEALPix maps | ||
|
||
TODO: Leo/Israel?:Instructions on how to create multiresolution healpix maps. | ||
|
||
## File Encoding and Decoding | ||
|
||
The following code sample shows how to encode/decode a file in python. The `base64` package includes the methods `b64decode` and `b64encode` to make this task simple. | ||
|
||
```python | ||
import base64 | ||
|
||
# Parse the content of your file to a base64 encoded string: | ||
with open("path/to/your/file", 'rb') as file: | ||
encoded_string = base64.b64encode(file.read()) | ||
|
||
print(encoded_string) | ||
# b'a1512dabc1b6adb3cd1b6dcb6d4c6......' | ||
|
||
# Decode and write the content to a local file: | ||
with open("path/to/destination/file", 'wb') as file: | ||
file.write(base64.b64decode(encoded_string)) | ||
|
||
``` | ||
|
||
For example, if we wanted to include a FITS file in a notice, you would need to add a property to your schema definition and define the value to be the following: | ||
|
||
<Highlight | ||
language="json" | ||
code={JSON.stringify( | ||
{ | ||
type: 'string', | ||
contentEncoding: 'base64', | ||
contentMediaType: 'image/fits', | ||
}, | ||
null, | ||
2 | ||
)} | ||
/> | ||
|
||
Then in your data production pipeline, you can use the encoding steps to convert your file to a bytestring, and set the value of the property to said bytestring. See [non json data](https://json-schema.org/understanding-json-schema/reference/non_json_data.html) for more information | ||
|
||
The following is a rough example of what that would look like in your producer code. We will assume for this example that the schema is defined such that the encoded data property is named `fits_data`: |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters