Skip to content

Commit

Permalink
HEAT-230 link correct component via cluster hostname (#21)
Browse files Browse the repository at this point in the history
  • Loading branch information
andrewrlee authored May 28, 2024
1 parent b079496 commit 5a203af
Show file tree
Hide file tree
Showing 11 changed files with 382 additions and 2,656 deletions.
98 changes: 3 additions & 95 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,10 @@

# hmpps-component-dependencies

POC to look into surfacing app insight dependencies for the developer portal
A recurring job that makes component runtime dependency information available for the developer portal

## why?

This is a proof of concept to show how we can expose dependency information about components by combining data from the service catalogue and app insights.

This supports the following use cases:

- It allows teams to easily see which components they would impact by changing their service
Expand All @@ -16,91 +14,9 @@ The benefit of using app insights for this is that it provides a live view of wh

## what?

This builds up a graph of bidirectionally interconnected component nodes.

This graph can be navigated in both directions, to view a components dependencies and also the things that rely on it.

Components can be retrieved by name and then the following 4 functions are available:

### `getDependencies`

This returns a list of all of the components that this component relies on.
So for example: `create-and-vary-a-licence-api` relies on the following components:

```js
[
"hmpps-auth",
"hmpps-external-users-api",
"hmpps-manage-users-api",
"hmpps-prisoner-search",
"nomis-user-roles-api",
"prison-api",
"probation-offender-search",
];
```

Note: there is no direct link between CVL API and `nomis-user-roles` but the function works recursively to provide info about transitive dependencies.

### `getDependencyPaths`

This returns a list of all of the possible dependencies and how they are related.
For CVL API example:

```js
[
'create-and-vary-a-licence-api => prison-api => hmpps-auth => hmpps-manage-users-api => hmpps-external-users-api',
...
'create-and-vary-a-licence-api => hmpps-prisoner-search => hmpps-auth => nomis-user-roles-api => *INF* (hmpps-auth)',
...
]
```

This shows two paths, the first example shows the chain of dependencies to demonstrate how `create-and-vary-a-licence-api` relies on `hmpps-external-users-api`.

The second path demonstrates how it handles the bidrectional dependency between `hmpps-auth` and `nomis-user-roles-api` using the arbitrary syntax: `*INF* (component-name)`.

### `getDependents`
This builds up a graph of bidirectionally interconnected component nodes, performs some categorisation of the types of dependencies that are seen and then publises the information into a shared REDIS instance.

This returns a list of all of the components that rely on this component (either directly or transitively).
For CVL, the list looks like this:

```js
[
"create-and-vary-a-licence",
"create-and-vary-a-licence-activate-licences-job",
"create-and-vary-a-licence-email-probation-practioner-job",
"create-and-vary-a-licence-remove-ap-conditions-job",
"hmpps-resettlement-passport-api",
"hmpps-resettlement-passport-ui",
"make-recall-decision-api",
"make-recall-decision-ui",
"make-recall-decisions-and-delius",
];
```

So, the frontend, some jobs and a few external clients including some integrating services.

### `getDependentPaths`

This returns the path information for the component above:

For CVL API:

```js
[
"create-and-vary-a-licence-api => make-recall-decision-api => make-recall-decision-ui",
"create-and-vary-a-licence-api => make-recall-decision-api => make-recall-decisions-and-delius",
"create-and-vary-a-licence-api => hmpps-resettlement-passport-api => hmpps-resettlement-passport-ui",
"create-and-vary-a-licence-api => create-and-vary-a-licence",
"create-and-vary-a-licence-api => create-and-vary-a-licence-activate-licences-job",
"create-and-vary-a-licence-api => create-and-vary-a-licence-email-probation-practioner-job",
"create-and-vary-a-licence-api => create-and-vary-a-licence-remove-ap-conditions-job",
];
```

We can see that if CVL went down it could transitively impact the `make-recall-decisions-and-delius` component.

NB: For components with lots of dependencies, this list can be very large as there are possible multiple reasons (paths) why it being down could affect a component. For instance, as well as CVL directly relying on `hmpps-auth`, CVL could go down because it relies on `prison-api` which also relies on `hmpps-auth`. This creates a bit of a combinatorial explosion: >900 paths.
The developer portal then accesses this information and displays aggregated information against the component page.

## how?

Expand All @@ -109,30 +25,22 @@ It requires two sets of information:
1. Component information with environments
2. An App insights query to retrieve all unique dependencies for a given time range

(example curl/AppInsights queries can be found in `data-sources.ts`)

The data is joined by linking:

- AppInsights `cloud_RoleName` to the `name` of the component
- AppInsights `target` to an environments `hostname`


This allows us to build a graph of each component to both it's dependencies and the things it depends on.

This graph can be navigated in both directions.

How this would this would be integrated into the component dashboard would require a bit of a conversation.

Potentially a node service/job that would semi regularly to gather the data and push the dependency info into redis or the service catalogue. Or we could push in the raw data and build up the dependency info on the fly on the frontend.

## and...

We could also use this linked dependency information to support:

- surfacing where people are missing health checks for their dependencies
- surfacing components that are not in the catalogue
- (this includes a lot of probation integration components, some of which are used by prison services)
- surfacing AWS dependencies - use of queues, topics, RDS, redis

## Limitations

Expand Down
Loading

0 comments on commit 5a203af

Please sign in to comment.