Website is missing documentation because 2.10 seems to produce a white start page after a few hours when deployed

[natankeddem]

Description

Hello,
I noticed that the documentation website is not showing the new event example (https://nicegui.io/documentation/dialog#events). Is it missing the latest updates or is there some weird caching I am experiencing?

[falkoschindler]

Hi @natankeddem,

We're currently experiencing an issue with nicegui.io, which sometimes shows up as a blank page. To narrow down the problem, and to keep the documentation up and running, we reverted it back to version 2.9.1. But we hope to find the root cause as soon as possible so that we can serve the latest version of the documentation.

A few more details:

• The "blank page" reminds of issue White screen after socket reconnect #4253, but we're not sure if it is really related.
• It appears immediately when loading a page, but only after the server has been running for a while.
• It seems like only the index page is affected. Other pages keep working.
• We tried reverting individual commits, but couldn't fix the website reliably without going all the way back to 2.9.1. Maybe it's a combination of code changes?

[natankeddem]

Hi @falkoschindler,
I don't know if there is some low effort (non-manual) way of making the website display the version of NiceGUI running? That would ensure users are aware of what version of documentation they are looking at. Obviously, this would only help going forward once you can continue to release new versions to the production servers.

Is this blank page issue only reproducible when you run the website in the production servers or if you run it locally as well? Is it mentioned in any previous issues with more details? I could take a look and see if I can find anything...

[rodja]

I don't know if there is some low effort (non-manual) way of making the website display the version of NiceGUI running?

Thats worth a thought. But it's not straight forward. The version is written into the pyproject.toml. But let's stay focused on the issue here.

Is this blank page issue only reproducible when you run the website in the production servers or if you run it locally as well?

We have not found a way to reproduce it locally. And it only happens in the production environment after a few hours. There are no logs shown on the server. The html also seems to load fine. Only there is no elements when client.build_response is called.

[rodja]

I just deployed 2.10.0 with almost all code reverted to https://nicegui-preview.fly.dev/. It's only a single instance in Frankfurt, Germany. But the same code produced the "white start page" after a few hours on https://nicegui.io yesterday. Feel free to test and try to break it.

[natankeddem]

I am brainstorming how we can get more data on this issue. Can you host the site using a different method? Then for a day or so you could implement a redirect mechanism to see if the problem can be recreated in a different environment. I can volunteer to host something if you want via docker or even as a debug instance...

[rodja]

@natankeddem https://nicegui.io is currently running 32 instances around the globe with fly.io. I do not see an easy way to change that. I hope that we can reproduce it with a single instance on https://nicegui-preview.fly.dev/. But so far it is not broken. Maybe we need to put it under stress? Maybe you can setup a selenium monkey test?

[natankeddem]

You can't temporarily change the DNS entry to the alternative host? That seems like the most straightforward thing although I userstand DNS propagation doesn't exactly turn on a dime so things might be in flux for a few days.

I could try to build something to exercise the site. It is rather expansive though so unless i get lucky or it is something generic going on it might be hard to trigger. Could you somehow narrow down the scope of the problem? Maybe, temporarily disable the 'See more' links as a start.

[rodja]

You can't temporarily change the DNS entry to the alternative host?

Of course we can do that. But as I said, the alternative "host" must be a global swarm of instances which can take the load. And I am not sure what we would gain from it. The experiments so far show that the issue appears only if 2.10 is deployed. So I do not think it's a matter of the server hosting.

Could you somehow narrow down the scope of the problem? Maybe, temporarily disable the 'See more' links as a start.

We have tried a lot. Almost all changes from 2.10.0 are reverted and still the error appears after a few hours. We are reluctant to deploy to production again because everytime the main page is white which is a quite bad experience for users looking into NiceGUI.

[natankeddem]

Can you get more parity with production on the 'nicegui-preview' site? Maybe spin up 4 total globally diverse instances? Since you haven't seen the problem there yet perhaps that is part of the equation somehow.

[rodja]

Can you get more parity with production on the 'nicegui-preview' site?

Sure. Desperate times ;-)
Now https://nicegui-preview.fly.dev/ has two instances in Frankfurt, two in Amsterdam and two in Miami. But I guess we somehow need traffic to reproduce it... not sure how we could do that.

[natankeddem]

Perhaps a docker based utility that would interact with the site that a few people around the world could run concurrently?

[rodja]

Perhaps a docker based utility that would interact with the site that a few people around the world could run concurrently?

Yes. That would be great.

But I have another idea. I just added another health-check to "preview":

  [[services.http_checks]]
    interval = "20s"
    timeout = "10s"
    grace_period = "1m0s"
    method = "get"
    path = "/"
    protocol = "http"
    tls_skip_verify = false
    headers = {}
    body = "With a gentle learning curve, NiceGUI is user-friendly for beginners"

This seems to pass. All six instances are running and showing the main page -- as they should.
I would be comfortable to deploy 2.10.0 with this change to production. Then we can observe the failure by looking into the logs -- without presenting a white page for a long time to users; the health check will fail in such cases and hence restart the broken instance.

[natankeddem]

The health check is a good idea to get things working for production in the short term. Can you disable the health check on the on the preview site for now? Maybe we could get a few people to run locust for a day and see if we can trigger the failure on preview? It is pretty simple to get setup just install:

pip3 install locust

Create a simple locustfile.py file

from locust import HttpUser, task


class User(HttpUser):
    @task
    def root(self):
        self.client.get("/")

they just run locust and a webui should be accessible on http://localhost:8089/
you can also run it via docker https://docs.locust.io/en/stable/running-in-docker.html

I got a bit overzealous on my user count and seemed to have DDOSed https://nicegui-preview.fly.dev/ site, it is throwing a 502/503 currently...

[rodja]

Ok. I deployed a stripped down variant of 2.10.0 to nicegui.io. It includes the new health check. The preview is also updated to not resurrect when the error appears.

[rodja]

Oh wow. That was fast. https://nicegui.io already shows a white page for me:

const app = createApp(parseElements(String.raw`{"0":{"tag":"q-layout","class":["nicegui-layout"],"props":{"view":"hhh lpr fff"},"children":[1]},"1":{"tag":"q-page-container","children":[2]},"2":{"tag":"q-page","children":[3]},"3":{"tag":"div","class":["nicegui-content"]}}`), {...

But it should start like this:

const app = createApp(parseElements(String.raw`{"0":{"tag":"q-layout","class":["nicegui-layout"],"props":{"view":"hHh lpr fff"},"children":[5,1]},"1":{"tag":"q-page-container","children":[2]},"2":{"tag":"q-page","children":[3]},"3":{"tag":"div","class":["nicegui-content","p-0","gap-0"],"children":[4,65,71,94,153,182,286,292,438,448]},"4":{"tag":"nicegui-dark_mode",...

And later in the same line having the string With a gentle learning curve, NiceGUI is user-friendly for beginners.
But the health check did not catch it. From the status it looks like it is getting the content.

[rodja]

Super strange. The health check passes on a broken server

fly machines status e825e25a6d1378
Machine ID: e825e25a6d1378
Instance ID: 01JJFCXGZSSVBDJBH18Z6KD9XJ
State: started
HostStatus: ok

VM
  ID            = e825e25a6d1378
  Instance ID   = 01JJFCXGZSSVBDJBH18Z6KD9XJ
  State         = started
  Image         = nicegui:deployment-01JJFCR3XFQMF293F1E9HKWHED
  Name          = lively-snow-9023
  Private IP    = fdaa:0:fa27:a7b:73:6fe3:a7c1:2
  Region        = fra
  Process Group = app
  CPU Kind      = shared
  vCPUs         = 1
  Memory        = 512
  Created       = 2025-01-21T16:27:34Z
  Updated       = 2025-01-25T19:11:40Z
  Entrypoint    =
  Command       =

Checks [3/3]
NAME                     	STATUS 	LAST UPDATED	OUTPUT
servicecheck-01-http-8080	passing	30m2s ago   	Ok
servicecheck-02-http-8080	passing	10m52s ago  	s dark_mode } from "/_nicegui/2.10.1/components/b0b17893a51343979e2090deee730538/dark_mode.js";
                         	       	            	app.component("nicegui-dark_mode", dark_mode);
                         	       	            	import { default as link } from "/_nicegui/2.10.1/components/b0b17893a51343979e2090deee730538/link.js";
                         	       	            	app.component("nicegui-link", link);
                         	       	            	import { default as input } from "/_nicegui/2.10.1/components/b0b17893a51343979e2090deee730538/input.js";
                         	       	            	app.component("nicegui-input", input);
                         	       	            	import { default as keyboard } from "/_nicegui/2.10.1/components/b0b17893a51343979e2090deee730538/keyboard.js";
                         	       	            	app.component("nicegui-keyboard", keyboard);
                         	       	            	import { default as markdown } from "/_nicegui/2.10.1/components/b0b17893a51343979e2090deee730538/markdown.js";
                         	       	            	app.component("nicegui-markdown", markdown);
                         	       	            	import { default as select } from "/_nicegui/2.10.1/components/b0b17893a51343979e2090deee730538/select.js";
                         	       	            	app.component("nicegui-select", select);
                         	       	            	import { default as intersection_observer } from "/_nicegui/2.10.1/components/85399eb40199d450158f9cadc41dea61/intersection_observer.js";
                         	       	            	app.component("nicegui-intersection_observer", intersection_observer);
                         	       	            	      var joystick = app.component('nicegui-joystick', {template:'#tpl-joystick',
                         	       	            	  async mounted() {
                         	       	            	    await import("nipplejs");
                         	       	            	    const joystick = nipplejs.create({
                         	       	            	      zone: this.$el.children[0],
                         	       	            	      position: { left: "50%", top: "50%" },
                         	       	            	      dynamicPage: true,
                         	       	            	      ...this.options,
                         	       	            	    });
                         	       	            	    joystick.on("start", (e) => this.$emit("start", e));
                         	       	            	    joystick.on("move", (_, data) => this.$emit("move", { data }));
                         	       	            	    joystick.on("end", (e) => this.$emit("end", e));
                         	       	            	  },
                         	       	            	  props: {
                         	       	            	    options: Object,
                         	       	            	  },
                         	       	            	});
                         	       	            	var plotly = app.component('nicegui-plotly', {template:'#tpl-plotly',
                         	       	            	  async mounted() {
                         	       	            	    await import("plotly");
                         	       	            	    this.update();
                         	       	            	  },
                         	       	            	  updated() {
                         	       	            	    this.update();
                         	       	            	  },
                         	       	            	  methods: {
                         	       	            	    update() {
                         	       	            	      // wait for plotly to be loaded
                         	       	            	      if (typeof Plotly === "undefined") {
                         	       	            	        setTimeout(this.update, 10);
                         	       	            	        return;
                         	       	            	      }

                         	       	            	      // default responsive to true
                         	       	            	      const options = this.options;
                         	       	            	      if (options.config === undefined) options.config = { responsive: true };
                         	       	            	      if (options.config.responsive === undefined) options.config.responsive = true;

                         	       	            	      // re-use plotly instance if config is the same
                         	       	            	      if (JSON.stringify(options.config) == JSON.stringify(this.last_options.config)) {
                         	       	            	        Plotly.react(this.$el.id, this.options.data, this.options.layout);
                         	       	            	      } else {
                         	       	            	        Plotly.newPlot(this.$el.id, this.options.data, this.options.layout, options.config);
                         	       	            	        this.set_handlers();
                         	       	            	      }

                         	       	            	      // store last options
                         	       	            	      this.last_options = options;
                         	       	            	    },
                         	       	            	    set_handlers() {
                         	       	            	      // forward events
                         	       	            	      for (const name of [
                         	       	            	        // source: https://plotly.com/javascript/plotlyjs-events/
                         	       	            	        "plotly_click",
                         	       	            	        "plotly_legendclick",
                         	       	            	        "plotly_selecting",
                         	       	            	        "plotly_selected",
                         	       	            	        "plotly_hover",
                         	       	            	        "plotly_unhover",
                         	       	            	        "plotly_legenddoubleclick",
                         	       	            	        "plotly_restyle",
                         	       	            	        "plotly_relayout",
                         	       	            	        "plotly_webglcontextlost",
                         	       	            	        "plotly_afterplot",
                         	       	            	        "plotly_autosize",
                         	       	            	        "plotly_deselect",
                         	       	            	        "plotly_doubleclick",
                         	       	            	        "plotly_redraw",
                         	       	            	        "plotly_animated",
                         	       	            	      ]) {
                         	       	            	        this.$el.on(name, (event) => {
                         	       	            	          const args = {
                         	       	            	            ...event,
                         	       	            	            points: event?.points?.map((p) => ({
                         	       	            	              ...p,
                         	       	            	              fullData: undefined,
                         	       	            	              xaxis: undefined,
                         	       	            	              yaxis: undefined,
                         	       	            	            })),
                         	       	            	            xaxes: undefined,
                         	       	            	            yaxes: undefined,
                         	       	            	          };
                         	       	            	          this.$emit(name, args);
                         	       	            	        });
                         	       	            	      }
                         	       	            	    },
                         	       	            	  },
                         	       	            	  data() {
                         	       	            	    return {
                         	       	            	      last_options: {},
                         	       	            	    };
                         	       	            	  },
                         	       	            	  props: {
                         	       	            	    options: Object,
                         	       	            	  },
                         	       	            	});

                         	       	            	      const dark = False;
                         	       	            	      Quasar.lang.set(Quasar.lang["en-US".replace('-', '')]);
                         	       	            	      Quasar.Dark.set(dark === None ? "auto" : dark);

                         	       	            	      if (dark !== None) tailwind.config.darkMode = "class";
                         	       	            	      if (dark === True) document.body.classList.add("dark");


                         	       	            	      app.mount("#app");
                         	       	            	    </script>
                         	       	            	  </body>
                         	       	            	</html>
servicecheck-00-tcp-8080 	passing	11m2s ago   	Success

Event Logs
STATE   	EVENT  	SOURCE	TIMESTAMP                    	INFO
started 	start  	flyd  	2025-01-25T20:11:40.41+01:00
starting	restart	flyd  	2025-01-25T20:11:38.989+01:00
stopped 	exit   	flyd  	2025-01-25T20:11:38.462+01:00	exit_code=130,oom_killed=false,requested_stop=true
stopping	restart	user  	2025-01-25T20:11:29.603+01:00
started 	start  	flyd  	2025-01-25T19:52:31.775+01:00

While curl on the same machine says there is not such a text:

 fly console --machine e825e25a6d1378
Connecting to fdaa:0:fa27:a7b:73:6fe3:a7c1:2... complete
root@e825e25a6d1378:/app# curl localhost:8080 |grep "gentle learning curve" && echo "yes" || echo "no"
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 20241  100 20241    0     0  3596k      0 --:--:-- --:--:-- --:--:-- 3953k
no

[rodja]

I just deployed the same version as is running on preview and was failing before to https://nicegui.io with only one small alternation: I added @chriswi93's suggested fix #4253 (comment) for #4253. Just a feeling 🤷‍♂️
And it seems to work. No white page showing on https://nicegui.io yet. We still need it to run a day or two before we can be sure, but I guess something in 2.10. was introduced which brings #4253 to the surface not only after a long disconnect but also when directly serving pages ...

[rodja]

No, the applying of #4253 had not fixed the issue.

In hindsight I feel stupid. The reproduction is very simple:

1. start the main.py in the NiceGUI root dir
2. browse to http://127.0.0.1:8080 and verify you can see the main page
3. browse to http://127.0.0.1:8080/documentation/storage
4. go back to http://127.0.0.1:8080 -> the main page is blank

A new documentation example demonstrates how to alter the maximum age of tab storage. But it also re-defined the @ui.page('/') route.

[natankeddem]

I apologize for posting a closed issue but I thought I would throw out one idea. Does it make sense to make a test to hit each endpoint on the nicegui website and then verify some basic functionality on the root and or other metric? Maybe this is just testing for the sake of testing and couldn't be built to do actual fault finding now that you know the fault.

[rodja]

Thanks for the idea @natankeddem! The test would be quite long in execution and it's not clear to me for what kind of errors we are looking. Spending the resources (mainly electricity) is not worth the small safety net it would provide.