Flagd no longer watches for changes to flagd configmap

[sjkaris]

Bug Report

Which version of the demo you are using?
1.12.0

Symptom

After launch of the flagd ui, the flagd container no longer watches or updates based on changes made to its configmap.

What is the expected behavior?
What do you expect to see?

I expect flagd to continue to respect changes made to the configmap. So that if a flag is adjusted in the flagd configmap, that change is propagated to the flagd container and the rest of the demo services.
Ideally, changes made in the flagdui are also persisted back to the configmap.

What is the actual behavior?

Changes to the configmap only get applied if the flagd pod is redeployed / restarted.

Reproduce

1. Make a change to the flagd configmap.
2. Notice that the flag does not flip in the flagdui and the behavior of the demo does not change.
3. Restart the flagd deployment
4. Notice that the flag has flipped

Additional Context

I believe this is somewhat due to the way flagdui interacts with the configmap.
The flagdui only loads the configmap on init, and copies it to a new rw location. code
The flagd container only has access to the rw file. code
This means that changes to the configmap are ignored, and only changes applied through the UI are respected once flagd is running.

[puckpuck]

Live updates from flagd-ui back into the ConfigMap would require leveraging the K8s API which is beyond the scope of what the UI should do.

When we added the flagd-ui, we did so intentionally, knowing that setting default flags would no longer be possible unless someone forks the Helm Chart. I think a valid enhancement for the Helm chart would be to add the ability to set default flag values.

Given the current architecture with the flagd-ui however, it would not be possible to manually update the ConfigMap and have it be live reloaded. If changing the ConfigMap manually, you will need to restart the flagd Pod for the changes to take effect.

[sjkaris]

Hey @puckpuck, thanks for getting back to me.

I understand that leveraging the k8s API to back-update might be out-of-scope.

Would it be possible to have updates to the configmap respected in flagdui though? This would just require watching the configmap and updating the local flagd file upon update to the configmap. Updates to the UI would not need to necessarily be back updated to the configmap (so no use of the k8s API would be needed).

This preserves backwards compatibility of flagd manipulation via configmap, while also letting users use the new flagd ui.

[julianocosta89]

@sjkaris I'd vote against not having the values streamlined across configmap and flagd-ui.

But I do agree we should think in a solution that can be configured via Helm or kubectl.

[puckpuck]

To explain the architecture of how this works and the limitations we are faced with.

Volumes mounted as config maps are read-only. This means that flagd-ui can't update this file.
To overcome the read limitation, we use an init-container to copy the contents of the ConfigMap to a shared temp volume, which is also mounted to the flagd and the flagd-ui sidecar containers.
After the pod is started, the contents of the ConfigMap are not mounted to the running containers for the flagd service.

Given these limitations, the only real solution would be a complete re-architecture of how the services interact and the shared flag configuration. As a workaround, I would suggest doing a Deployment restart for the flagd service as part of manually updating the ConfigMap.

It does still make sense that we add the ability to specify defaults when installing the Helm chart, but overcoming a way to update after the demo is running would be a much larger project, and I'm not sure I could see the value benefit from it.