Discord bot account offline but xud-docker-bot service running

[reliveyy]

No description provided.

[reliveyy]

Experience this issue again

[kilrau]

I don't understand how the discord but account can be offline in the first place, but ... will the discord account come "online" by itself or is there anything you actively need to do?

Plus since this os only affecting discord notification messages and manual builds, but not auto build functionality this shouldn't be that serious.

[reliveyy]

A short explaination about the possible reason why this issue happens:

The bot application is running in a single thread which runs an event loop in it. The Discord client, HTTP server and modified images analyzing routine are all running in the same thead. That means when the analyzing routine took minutes to finish its job the Discord client may disconnect (but this still cannot explain why it does not reconnect back). So in order to make the program fully async 2 major parts need to be refactored:

1. The clients for dockerhub and travis
2. The modified images analyzing routine

After 2 days trying I found that it couldn't be done in a short time (like one day). So here are the steps I suggest we should take to resolve this issue incrementally:

• Write an endpoint to check if the Discord client is working well in the bot application (resolved by feat: add health endpoint #21)
• Create a cron job to query the endpoint. If it shows the Discord client is not working then sudo systemctl restart xud-docker-bot
• Log more details in xud-docker-bot (may introduing structlog) try to find if there are some concurrency issues
• Introducing guppy3 and objgraph to diagnose if there are some memory leaks in the application

[kilrau]

• Write an endpoint to check if the Discord client is working well in the bot application
• Create a cron job to query the endpoint. If it shows the Discord client is not working then sudo systemctl restart xud-docker-bot

As discussed, I think these two are enough for now.