Unleash as a configuration service provider?

[samuelneff]

We're using Unleash for feature flags throughout our company and some are looking to use it as a full configuration management solution. For this we need not only boolean flags but other scalar data types, strings, numbers, dates. It seems we might be able to use variants for this and target specific groups of users by a variety of custom fields.

Examples of configuration are timeouts that can vary by client organization, dates that can vary by specific site (within the context of a client), or numeric thresholds based on user role (across clients).

Can anyone share experiences using Unleash in this way? Any concerns we should watch out for?

Thanks,

Sam

[thomasheartman]

Hi, @samuelneff 👋🏼 Great question!

As you've identified: using variants, you should be able to add any data that you can represent as a string. And yes, custom context fields should let you target any user group you want.

For clarity: were you thinking of using different variants for one feature, and target the users with custom fields for the variant rollout? Unleash's strategy constraints would allow you to target users very well on the strategy-level, but the same control isn't yet available on the variant-level. However, this is something we are looking into and will likely support in the future.

Can anyone share experiences using Unleash in this way? Any concerns we should watch out for?

I don't think I've heard much about people using it for this purpose, so I don't have any experiences to share, but there are some limitations you should be aware of at least.

First: Unleash is focused on availability over consistency. So if you have a very distributed setup, such as multiple Edge or proxy instances, then you could get different results between different instances while any changes from the Unleash server propagates to the Edge/proxy instances. If you have only one Edge/proxy instance, then that won't be an issue, but there can be a small delay during which the changes propagate from the Unleash server to that instance.

If you only connect directly to the Unleash instance itself, then that shouldn't be an issue, though.

Also, depending on the kind of configuration you're after, Unleash may be more or less suited. Oftentimes, configuration values will be pretty static: you read them on startup and then they can't change until you restart the app. Unleash is best suited for dynamic configuration values that can change during runtime. However, most of our SDKs can block execution until they have synced with the server, and most also support reading a configuration from a file, so there's certainly ways around that.

So it comes down to that classic software answer: it depends. Depending on your applications and your general setup, it may be more or less suited. Of course, we'd be happy to try and help you out and provide suggestions if you want to share more about how you're planning to use it and your systems ☺️

Hope that helps 😄

[samuelneff]

Thanks for that. All spot on.

We spent several days working through a single specific example using variants and then using strategies. It appeared to work well at first but in both situations we ran into problems with more complex usages where Unleash would return the wrong variant or would return multiple strategies when only one was desired.

I have a huge write-up including two sets of 14-16 steps to repro/demo with per-step screenshots and notes. I can share later on after sanitizing company info if you're interested.

Very brief example of what does not work:

Scenario: Given a desired configuration value allocation-strategy with possible values of round-robin, lowest-load, and never, configure different values for various combinations of offices, roles, and users.

Variant setup. Not actual values but same scenarios.

Name                    Overrides                           Value 
------------------      ------------------------------      ----------------
default                 *none*                              never
corp                    office=corp                         lowest-load
boston                  office=boston                       round-robin
boston-managers         office=boston, role=manager         never
atlanta                 office=atlanta                      never
atlanta-managers        office=atlanta, role=manager        round-robin

[thomasheartman]

Great! And yeah, we would be very interested in seeing your findings, especially if they point to what appears to be issues on our end!

It appeared to work well at first but in both situations we ran into problems with more complex usages where Unleash would return the wrong variant or would return multiple strategies when only one was desired.

Regarding the above statement (emphasis mine), could you elaborate on what you mean "return multiple strategies"? Features in Unleash always get all their strategies (in that specific environment) evaluated, so when you request it from the API, you'll always get all strategies. Did you mean that it appeared to approve strategies that you didn't expect?

If that's the case and you're ever wondering why some strategies evaluate to true and others don't, you can use the Unleash playground to see exactly why certain strategies (or variants) were chosen.

Scenario: Given a desired configuration value allocation-strategy with possible values of round-robin, lowest-load, and never, configure different values for various combinations of offices, roles, and users.

Variant setup. Not actual values but same scenarios.

Appreciate the detail here! If I understand correctly, the "name" column is the name of the variant. The "overrides" column is how it should map to Unleash's context and overrides, and the "value" column is the expected value (which you would presumably add as a payload).

The reason that won't work is explained in the overrides section of the variants doc (emphasis added):

When using multiple overrides, each user only has to match one of them. In other words, if you use the following overrides, the users with IDs aa599 and aa65 and any users who use the client with ID 123abc will receive the specified variant:

• userId: aa599, aa65
• clientId: 123abc

Note that if multiple variants use overrides that affect the same user, then Unleash can not guarantee which override will take effect. We recommend that you do not use multiple overrides that can conflict in this way, as it will probably not do what you expect.

That said, I think this is a very valid use case you have here, and I think the ability to constrain on multiple context fields in the same way that constraints work for strategies would be useful. I know we're looking at expanding the variant override capabilities, so this will be useful input for that.

[samuelneff]

Thanks for the detailed response. I'll share the full write-up as soon as I can sanitize it.

Unleash is not doing anything wrong.

Unleash is behaving exactly as it's designed and documented. Our use-case and desired response is inconsistent with the intended behavior of Unleash. Unleash is not really the right product for our use case, but since we are using it for feature flags throughout our organization (8,000+ people), our hope was it could serve this need as well.

We would like to find a workaround, but as of right now I don't see one that doesn't involve moving a lot of the functionality and logic we are looking for to custom code wrapped around what Unleash provides.

We'll likely look to other options that do not include Unleash for this use case. Perhaps it is something that can be supported in the future if there is interest from others.

Thanks for all the attention and responses.

[thomasheartman]

Thanks for the detailed response. I'll share the full write-up as soon as I can sanitize it.

Looking forward to it 🤓

Unleash is not doing anything wrong. [...]

Unleash is behaving exactly as it's designed and documented. Our use-case and desired response is inconsistent with the intended behavior of Unleash. Unleash is not really the right product for our use case, but since we are using it for feature flags throughout our organization (8,000+ people), our hope was it could serve this need as well.

We'll likely look to other options that do not include Unleash for this use case. Perhaps it is something that can be supported in the future if there is interest from others.

Yeah, that makes total sense. I think you might be better off exploring other options for this at the moment, but I'll make sure we make a note of this. It's not the first time we've heard about it and I'm sure it won't be the last. But as with all things it's a matter of priorities. Oh, and making it work with the rest of the system in an intuitive manner.

Appreciate your input and looking forward to seeing the write-up 💯

[samuelneff]

Unleash should work perfectly for our use case if we could choose between

• Overrides match ALL
  vs
• Overrides match ANY

If we submit a PR to add this functionality, what is the likelihood it would be merged?

This change would affect the UI, variant selection within the "engine" (whatever it's called internally), hopefully some automated tests, and some of the documentation.

Initial implementation thoughts:

• Add a field to the "variant" object (or an "overrides" container if that exists as a distinct object) such as overrideMatchBehavior with possible values of ANY or ALL. Default to ANY, current behavior.
• Within the UI add an an option to the "Add/Edit variant" pop-up Overrides section whenever multiple overrides are added.
  • "Apply variant when user matches (*) ANY or ( ) ALL overrides."
  • Show multi-override matching selection only when there are multiple overrides, right above the Add override button.
• Update documentation here: https://docs.getunleash.io/reference/feature-toggle-variants
• Search docs for any others affected.
• Identify automated tests affected and adjust as needed.
• Add new automated tests for newly added scenarios and edge cases.

From my perspective this would be a useful addition with not effect on current behavior and would be invisible to the vast majority of users who never use multiple overrides anyways.

Full write-up still in-progress. I'm updating based on more recent discussions and this proposal; it's not just sanitizing the content right now.

[thomasheartman]

I'm gonna respond to both your previous message in this thread and your other message about the Java SDK in one; hope that's alright.

First off, it looks like you've done your research 🤓 There's a few things to consider, but let's start at the top:

If we submit a PR to add this functionality, what is the likelihood it would be merged?

That's a good question. While I think the idea is sound, this is a pretty big change that would have to be discussed internally. I know we've thought about improving the variants functionality, and while I imagine we would want to do something like this, I'm not entirely sure whether it would be conflicting or not.

That said, we also haven't prioritized it yet, so I can't say for sure about anything.

On the other hand:

it looks like this is actually controlled client-side

Yup; that's correct. All server-side SDKs do their evaluation themselves. So the good news is that if you know that you'll always want that, then you're free to go ahead and fork the SDK and use it for that.

However, if it's something that we wanted to work in as an update, it would also require us to update all server-side SDKs in preparation for it. We've done that with previous features (most recently with the strategy constraint update in 4.16). Of course, we're not averse to doing the work, but it would require a significant amount of coordination and planning.

But! We are working on Yggdrasil, which will unify the evaluation logic for all SDKs. That should make these kinds of changes much easier. But we don't have a clear path to get that done just yet. However, @sighphyre, the primary driver of Yggdrasil, says that:

If [you] want features that should be implemented across all our SDKs or a way to expose the underlying rule grammar to Unleash, I would love to see that become an Issue on the GitHub repo - that would give me ammo to drive the project forward

---

So to sum it up, here's what I suggest:
I'll try and get a discussion going about this internally and see how we feel about accepting a PR for it. As I said, I think it's a sound idea, and I'd like to see if we can make it happen.

For right now, though, forking the Java SDK should provide you with what you need, so you could start there?

I'll try and get some input on this over the past week. Feel free to ping me if you think I've forgotten about you!

Does that sound alright?

[samuelneff]

Maybe this is wishful thinking... it looks like this is actually controlled client-side and would be a one-word change in the Java SDK?

https://github.com/Unleash/unleash-client-java/blob/main/src/main/java/io/getunleash/variant/VariantUtil.java#L52

    private static Optional<VariantDefinition> getOverride(
            List<VariantDefinition> variants, UnleashContext context) {
        return variants.stream()
                .filter(
                        variant ->
                                variant.getOverrides().stream()
                                        .anyMatch(overrideMatchesContext(context))) // Add support for optional allMatch here?
                .findFirst();
    }

We wouldn't be selecting server side if a variant is "match any" vs "match all" but would do it in client instead, which could make it inconsistent if different clients use it differently, but does what we're looking for. Right?

[samuelneff]

Great. I created a feature request for yggdrasil: Unleash/yggdrasil#32.