Transport client best practices

[ESamir]

The documentation only explain how to add a single transport address corresponding to one of the IP addresses returned when resolving the Elasticsearch cluster host name. but if the node IP changes, the application won't be able to contact the node anymore because Elastic Search client will never attempt to resolve again the host name to find a new IP address.

The ELB for a cluster have multiple IPs associted with it,
via a DNS resolution we can fetch all the IP addresses associated with the cluster ELB. add all the IP addresses as transport addresses in the ES transport client. The ES client should then automatically use a round-robin logic when contacting the different ips.

To make sure the client node pool is refreshed, we would have a thread periodically polling the client for the list of IPs and comparing it against the list of addresses returned via a DNS resolution on the cloud cluster hostname. Any IP that isn't part of the DNS response would be removed from the pool while new IPs would be added.

Can we add an example in the documentation

[nrichers]

@ESamir Do you have an example of what we should add, please?

[henrikno]

Duplicate of #8?

[davidklebanoff]

@henrikno. I believe so, but I think @ESamir described it quite well. So I closed my issue (#8) to let this one remain.

[davidklebanoff]

As explained in an older forum post, it seems the Transport Client often encounters NoNodeAvailableException when used with Found/Elastic Cloud for the reasons in this issue.

The linked forum post has an example of a possible fix that involves periodically polling the DNS.

It's also worth noting that since the code in the forum was written, Elastic has added the HostFailureListener. It may play a role in a modern solution to the issue.

[geekpete]

Hey team, I've created another issue around Java API docs in general for more visibility and to link all the various tickets about the one issue to a single place

elastic/elasticsearch#24604