Include failure events from Deployment/Replicaset/Pod in case of timeout being 'ready'

[gregjones]

We've seen users having problems debugging deploy errors in a couple of cases:

• errors with liveness/readiness checks
• errors deploying with resource requests/limits higher than cluster allows

My idea is to capture these events when fiaas decides the deployment is failed due to a timeout, and attach them to the ApplicationStatus object, in a similar way to what is done for logs, so they can be displayed in the mast UI.

[mortenlj]

While I like the idea, I'm wondering if this should be outside the responsibilities of fiaas-deploy-daemon. It already has too many responsibilities in my mind, I'm not sure it's a good idea to give it more.

On the other hand, the ApplicationStatus object is an object that can be said to be owned by FDD, so in some ways you could say that FDD should be the one adding additional information.

There are also some open questions:

• For how long should FDD be watching for liveness/readiness check failures?
• Should these events be added as a log-line, or do we need a new field on the ApplicationStatus object for them?

Could it instead be some other component that watches for these types of errors and attaches k8s Events to the ApplicationStatus object?

[gregjones]

For how long should FDD be watching for liveness/readiness check failures?

I think the existing behaviour of ReadyCheck is fine for this, it's just an extra thing for it to do just before it sets the status as failed.

Should these events be added as a log-line, or do we need a new field on the ApplicationStatus object for them?

I think as a separate property gives more flexility for how we display them to users, but extra log-lines could work.

Could it instead be some other component that watches for these types of errors and attaches k8s Events to the ApplicationStatus object?

It's an option, though I'm not sure using ApplicationStatus would make so much sense if it's another component, it seems like mixing ownership could become confusing. But something that watches ApplicationStatus objects and stores the events somewhere else that can be referenced somehow from a UI could work