Skip to content

Commit

Permalink
Add events docs
Browse files Browse the repository at this point in the history 
Signed-off-by: Sunny <[email protected]>
  • Loading branch information
@darkowlzz
darkowlzz committed Aug 23, 2023
1 parent 33cd27a commit 03afcd3
Showing 1 changed file with 126 additions and 7 deletions.
133 changes: 126 additions & 7 deletions content/en/flux/monitoring/events.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,19 @@ description: "How to monitor the Flux events"
weight: 4
---

The Flux controllers emit Kubernetes events for every reconciliation operation.
The Flux controllers emit [Kubernetes events][kubernetes-events] during the
reconciliation operation to provide information about the object being
reconciled. Unlike logs, events are always associated with an object, which is a
Flux resource in this case. Events are supplemental data that can be used along
with logs to provide a complete picture of controllers' operations. Some of
the events emitted by Flux controllers are also used to send notifications.
See the [Alerts docs](/flux/monitoring/alerts/) to learn more about the Flux
Alerts based on events from controllers. In the following sections, we will go
through the Flux events and how to interpret them.

## Kubernetes events

The Flux controllers events contain the following fields:
The Flux controller events about a resource contain the following fields:

- `type` can be `Normal` or `Warning`
- `firstTimestamp` timestamp in the ISO 8601 format
Expand All @@ -18,11 +26,11 @@ The Flux controllers events contain the following fields:
- `reason` short machine understandable string
- `involvedObject` the API version, kind, name and namespace of the Flux object
- `metadata.annotations` the Flux specific metadata e.g. source revision
- `source.component` the Flux controller name
- `source.component` the Flux controller name where the event originated from.

### Samples
### Examples

Sample of a `Normal` event produced by kustomize-controller:
Example of a `Normal` event produced by kustomize-controller:

```json
{
Expand Down Expand Up @@ -52,8 +60,119 @@ Sample of a `Normal` event produced by kustomize-controller:
}
```

In the above example:
- The event is about a `Kustomization` named `flux-system` in the `flux-system`
namespace, indicated by the `involvedObject` field.
- The event originates from `kustomize-controller`, indicated by the
`source.component` field.
- The event is a `Normal` type event about a successful reconciliation,
indicated by the `reason` and `message` fields.
- The `metadata.annotations` field `kustomize.toolkit.fluxcd.io/revision`
contains information about the source revision that was successfully applied
as a result of successful reconciliation of the Kustomization.

Example of a `Warning` event produced by source-controller:

```json
{
"apiVersion": "v1",
"count": 4,
"eventTime": null,
"firstTimestamp": "2023-08-22T20:24:06Z",
"involvedObject": {
"apiVersion": "source.toolkit.fluxcd.io/v1",
"kind": "GitRepository",
"name": "podinfo",
"namespace": "default",
"resourceVersion": "1284973",
"uid": "2c2ed1da-556f-4793-863d-7d96e8bab3f5"
},
"kind": "Event",
"lastTimestamp": "2023-08-22T20:24:18Z",
"message": "failed to checkout and determine revision: unable to clone 'https://github.com/stefanprodan/podinfo': couldn't find remote ref \"refs/tags/v1.8.9\"",
"metadata": {
"creationTimestamp": "2023-08-22T20:24:06Z",
"name": "podinfo.177dce48bc7db3a4",
"namespace": "default",
"resourceVersion": "1285016",
"uid": "3c8f568a-c99b-4279-8093-6ef08fae325b"
},
"reason": "GitOperationFailed",
"reportingComponent": "",
"reportingInstance": "",
"source": {
"component": "source-controller"
},
"type": "Warning"
}
```

In the above example:
- The event is about a `GitRepository` named `podinfo` in the `default`
namespace, indicated by the `involvedObject` field.
- The event originates from `source-controller`, indicated by the
`source.component` field.
- The event is a `Warning` type event about a failed Git operation, indicated by
the `reason` and `message` fields.

## Events inspection with kubectl

```shell
kubectl events -n monitoring --for helmreleaase/kube-prom-stack
The events associated with a Flux resource can be queried using `kubectl events`
command:

```console
$ kubectl events -n flux-system --for kustomization/flux-system
LAST SEEN TYPE REASON OBJECT MESSAGE
58m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 448.00332ms, next run in 10m0s
48m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 486.826649ms, next run in 10m0s
38m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 502.282127ms, next run in 10m0s
28m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 543.745587ms, next run in 10m0s
18m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 465.177441ms, next run in 10m0s
8m27s Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 494.543068ms, next run in 10m0s
```

This shows all the events associated with the queried resource in an hour.

## Events inspection with flux CLI

The events associated with a Flux resource can be queried using the `flux
events` CLI command:

```console
$ flux events --for Kustomization/flux-system
LAST SEEN TYPE REASON OBJECT MESSAGE
52m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 506.467ms, next run in 10m0s
42m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 531.072726ms, next run in 10m0
32m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 506.673992ms, next run in 10m0
22m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 512.255817ms, next run in 10m0
12m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 507.521248ms, next run in 10m0
2m31s Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 448.00332ms, next run in 10m0s
```

This can also be used to watch all the events issues by the Flux controllers
across all the namespaces:

```console
$ flux events --all-namespaces --watch
NAMESPACE LAST SEEN TYPE REASON OBJECT MESSAGE
flux-system 34m (x3 over 154m) Normal GitOperationSucceeded GitRepository/flux-system no changes since last reconcilation: observed revision 'main@sha1:4d768edba5d409feb60870dd3b0ac0d307299898'
flux-system 54m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 486.814878ms, next run in 10m0s
flux-system 44m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 486.203813ms, next run in 10m0s
flux-system 34m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 512.160373ms, next run in 10m0s
flux-system 24m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 543.806383ms, next run in 10m0s
flux-system 14m Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 524.293527ms, next run in 10m0s
flux-system 4m5s Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 522.671955ms, next run in 10m0s
flux-system 47s Normal ReconciliationSucceeded Kustomization/flux-system Reconciliation finished in 523.892245ms, next run in 10m0s
flux-system 34m Normal ReconciliationSucceeded Kustomization/monitoring-configs Reconciliation finished in 104.609707ms, next run in 1h0m0s
flux-system 42s Normal ReconciliationSucceeded Kustomization/monitoring-configs Reconciliation finished in 90.70521ms, next run in 1h0m0s
flux-system 34m Normal ReconciliationSucceeded Kustomization/monitoring-controllers Reconciliation finished in 118.651968ms, next run in 1h0m0s
flux-system 39s Normal ReconciliationSucceeded Kustomization/monitoring-controllers Reconciliation finished in 132.34839ms, next run in 1h0m0s
monitoring 34m (x3 over 154m) Normal ArtifactUpToDate HelmChart/monitoring-kube-prometheus-stack artifact up-to-date with remote revision: '48.3.3'
monitoring 34m (x3 over 154m) Normal ArtifactUpToDate HelmChart/monitoring-loki-stack artifact up-to-date with remote revision: '2.9.11'
```

Refer to the [`flux events`](/flux/cmd/flux_events/) CLI docs to learn more
about it.


[kubernetes-events]: https://kubernetes.io/docs/reference/kubernetes-api/cluster-resources/event-v1/

0 comments on commit 03afcd3

Please sign in to comment.