Can't use Railway private networking between ENSNode and ENSRainbow service

[tk-o]

Setting ENSRAINBOW_URL env var for ENSNode deployment to http://${{ENSRainbow.RAILWAY_PRIVATE_DOMAIN}} results in connections errors, for example:

TypeError: fetch failed
    at Module.labelByHash (/app/apps/ensnode/src/lib/graphnode-helpers.ts:29:24)
    at /app/apps/ensnode/src/handlers/Registry.ts:133:31
  27 |  **/
  28 | export async function labelByHash(labelhash: Labelhash): Promise<string | null> {
> 29 |   const healResponse = await ensRainbowApiClient.heal(labelhash);
     |                        ^
  30 |
  31 |   if (healResponse.status === StatusCode.Success) {
  32 |     // original label found for the labelhash
caused by: Error: connect ECONNREFUSED fd12:11c:7a35:0:8000:4:d9f1:6886:80
    at TCPConnectWrap.afterConnect [as oncomplete] (node:net:1610:16)

However, setting the same ENSRAINBOW_URL env var to https://${{ENSRainbow.RAILWAY_PUBLIC_DOMAIN}} allows network communication. It happens through a public network, so adds lots of connection overhead.

That's why we need to make the private connection to work.

[lightwalker-eth]

@tk-o Would it help if we used these? https://docs.railway.com/guides/variables

[BanaSeba]

@tk-o I was able to use a private endpoint in the beta environment. We should double-check whether it's still an issue

[tk-o]

@BanaSeba and I caught up and figured why my previous attempt to connect ENSNode and ENSRainbow over private network did not work.

Railway networking

Railway Private Domain (RAILWAY_PRIVATE_DOMAIN) is an IPv6 address, followed by port number, for example:

fd12:11c:7a35:0:8000:4:d9f1:6886:80

Railway Public Domain (RAILWAY_PUBLIC_DOMAIN) is... well a public domain, for example:

api.ensrainbow.io

and it can be mapped to any internal service port, for example map 80 to 8080:

ENSRainbow ports

ENSRainbow runs on Railway infrastructure, which sets deployment port to 8080 by default. Connection attempts over port 80 get refused

Error: connect ECONNREFUSED fd12:11c:7a35:0:8000:4:d9f1:6886:80

On the other hand, the public URL reference worked as the public domain that ENSRainbow service uses is mapped from external port 80 to internal port 8080.

[tk-o]

We could fix the issue with the following steps:

1. Set an explicit port number for ENSRainbow runtime by setting the ENSRAINBOW_INTERNAL_PORT env var in Shared Variables panel
   

2. Use the ENSRAINBOW_INTERNAL_PORT reference and set the PORT env var for ENSRailway service.
   

3. Use the ENSRAINBOW_INTERNAL_PORT reference and set the ENSRAINBOW_URL env var for ENSNode Alpha/Subgraph services.
   

[tk-o]

@BanaSeba plans to test the recommended solution out on the beta environment.

Rolling out any updates to alpha environment requires approval from @lightwalker-eth.

[tk-o]

Solving this issue should improve I/O time needed for communication between ENSNode and ENSRainbow services.