Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Duplicated hosts in custom HostSupplier implementation #264

Open
chrisbendel opened this issue Apr 12, 2019 · 0 comments
Open

Duplicated hosts in custom HostSupplier implementation #264

chrisbendel opened this issue Apr 12, 2019 · 0 comments

Comments

@chrisbendel
Copy link

chrisbendel commented Apr 12, 2019

Observed weird/confusing behavior when implementing a dyno client into our Spring application.
Dynoclient is being initiated as a singleton service (using spring @service annotation). This could all very well be my misunderstanding of how spring's service annotation works under the hood, or just bad code in general. Looking to get a bit more information and understanding of what might be going on. I am suspicious that getHosts() is being called twice during the initialization of the client, and the hosts variable inside customHostSupplier is having the two hosts added twice. This all happened upon building the application and initializing the client.

Basic information:

  • Two hosts on two different racks, same datacenter
  • Before clearing the hosts (see the // hack comment in code below), we were hitting unexpected NullPointerExceptions around here (hostFromTokenMapSupplier was null)
  • We essentially had 4 hosts (one dupe of each actual host), so clearing out the hosts in my customHostSupplier forced it to have the correct number of hosts

Hoping to maybe gain some more knowledge on things, for all I know I could be doing something wrong. Let me know if you need any more information.

Disclaimer: Configurations seem to be all working and set up properly, things work with the hack in place

@Service
public class Dynosaur {
    private DynoJedisClient client;
    private static final Gson gson = new Gson();
    private Integer ttl;
    private Boolean enabled = true;

    public Dynosaur(
        // Params are coming from spring application.properties
        @Value("${dyno.servers}") List<String> servers,
        @Value("${dyno.racks}") List<String> racks,
        @Value("${dyno.port}") Integer port,
        @Value("${dyno.ttl}") Integer ttl,
        @Value("${dyno.token}") Long token,
        @Value("${dyno.enabled}") Boolean enabled
    ) {
        this.enabled = enabled;
        this.ttl = ttl;

        HostSupplier customHostSupplier = new HostSupplier() {
            private final List<Host> hosts = new ArrayList<>();

            @Override
            public List<Host> getHosts() {
                // The current hack to work around this
                hosts.clear();
                Integer i = 0;
                for (String server : servers) {
                    hosts.add(new Host(server, port, racks.get(i), Host.Status.Up));
                    i++;
                }
                return hosts;
            }
        };

        TokenMapSupplier tokenMapSupplier = new TokenMapSupplier() {
            Map<Host, HostToken> tokenMap = new HashMap<>();

            @Override
            public List<HostToken> getTokens(Set<Host> activeHosts) {
                activeHosts.forEach(activeHost -> tokenMap.put(activeHost, new HostToken(token, activeHost)));
                return new ArrayList<>(tokenMap.values());
            }

            @Override
            public HostToken getTokenForHost(Host host, Set<Host> activeHosts) {
                Set<Map.Entry<Host, HostToken>> set = tokenMap.entrySet();
                for (Object aSet : set) {
                    Map.Entry mentry = (Map.Entry) aSet;
                    if (((Host) mentry.getKey()).getHostName().equals(host.getHostName())) {
                        return (HostToken) mentry.getValue();
                    }
                }
                return null;
            }
        };

        String localRack = System.getProperty("LOCAL_RACK");

        ConnectionPoolConfigurationImpl cp = new ConnectionPoolConfigurationImpl("appname")
                .setFailOnStartupIfNoHosts(true)
                .setRetryPolicyFactory(new RetryNTimes.RetryFactory(2, true))
                .setLocalRack(localRack)
                .withTokenSupplier(tokenMapSupplier);

        this.client = new DynoJedisClient.Builder()
                .withApplicationName("appname")
                .withHostSupplier(customHostSupplier)
                .withCPConfig(cp)
                .build();
    }
}

Application properties that are being injected look something like this

dyno.servers=host1.abc.com,host2.abc.com
dyno.racks=rack1,rack2
dyno.port=8102
dyno.token=0
dyno.ttl=24
dyno.enabled=true

Invoking the dynosaur client was done through autowiring in spring in a service/controller, and would be called like

    @Autowired
    Dynosaur dynosaur;

    dynosaur.get("key");
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant