HTTP 500 Error when updating library

[Toomblr]

Describe the bug

When I try to update the library, I get the error below:

Steps To Reproduce

Steps to reproduce the behavior:

1. Go to library
2. Click on Update library
3. See error

Expected behavior

An updated library

Environment (please complete the following information where applicable):

• OS: TrueNAS SCALE Electric Eel 24.10
• Browser: Firefox 133.0.3
• Retrom: v0.4.8

Logs

Logs are not displaying any apparent errors

retrom-db  | 2024-12-16 08:32:33.439 UTC [1] LOG:  starting PostgreSQL 16.6 (Debian 16.6-1.pgdg120+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 12.2.0-14) 12.2.0, 64-bit
retrom-db  | 2024-12-16 08:32:33.439 UTC [1] LOG:  listening on IPv4 address "0.0.0.0", port 5432
retrom-db  | 2024-12-16 08:32:33.439 UTC [1] LOG:  listening on IPv6 address "::", port 5432
retrom-db  | 2024-12-16 08:32:33.442 UTC [1] LOG:  listening on Unix socket "/var/run/postgresql/.s.PGSQL.5432"
retrom-db  | 2024-12-16 08:32:33.448 UTC [29] LOG:  database system was shut down at 2024-12-16 08:32:26 UTC
retrom-db  | 2024-12-16 08:32:33.457 UTC [1] LOG:  database system is ready to accept connections
retrom     | 2024-12-16T08:32:33.779227Z  INFO retrom_service: Starting Retrom 0.4.8 service at: 0.0.0.0:5101
retrom     | ! Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-9.5.0.tgz
retrom     | 
retrom     | > web@ preview /app/web
retrom     | > vite preview
retrom     | 
retrom     | 
retrom     | ♻️  Generating routes...
retrom     | ✅ Processed routes in 140ms
retrom     |   ➜  Local:   http://localhost:3000/
retrom     |   ➜  Network: http://172.16.2.45:3000/

Additional context

docker-compose

services:
  retrom:
    container_name: retrom
    image: ghcr.io/jmberesford/retrom-service:latest
    #    ports:
    #      - 5101:5101
    environment:
      PUID: 568
      PGID: 568
    volumes:
      - /mnt/vol2/media/retro/roms:/app/library
      - /mnt/ssd0/docker/data/retrom/config:/config/
    restart: unless-stopped
    networks:
      - npm
    depends_on:
      - retrom-db
  retrom-db:
    container_name: retrom-db
    hostname: retrom-db
    image: postgres:16
    restart: unless-stopped
    volumes:
      - /mnt/ssd0/docker/data/retromdb/config:/var/lib/postgresql/data
    environment:
      PUID: 568
      PGID: 568
      POSTGRES_USER: ${POSTGRES_USER}
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
      POSTGRES_DB: retrom
    networks:
      - npm
networks:
  npm:
    external: true

config.json

{
    "connection": {
      "port": 5101,
      "db_url": "postgres://mypostgresusername:mypostgrespassword@retrom-db/retrom"
    },
    "content_directories": [
      {
        "path": "/app/library",
        "storage_type": "MultiFileGame"
      }
    ],
    "igdb": {
      "client_secret": "myigdbsecret",
      "client_id": "myigdbid"
    }
  }

[SimHat]

Same issue here. I was looking at the api documentation for IGDB and they reference 2 different URLs https://api.igdb.com/v4/games/ and https://id.twitch.tv/oauth2/token. I'm wondering if they changed the URL.

[aureliensoria]

Hello ! I have the same issue when updating my library.
Here is the full backtrace related to this issue:

stack backtrace:
   0:     0x56020ee3e27a - <std::sys::backtrace::BacktraceLock::print::DisplayBacktrace as core::fmt::Display>::fmt::h5b6bd5631a6d1f6b
   1:     0x56020ee664f3 - core::fmt::write::h7550c97b06c86515
   2:     0x56020ee3a613 - std::io::Write::write_fmt::h7b09c64fe0be9c84
   3:     0x56020ee3e0c2 - std::sys::backtrace::BacktraceLock::print::h2395ccd2c84ba3aa
   4:     0x56020ee3f1ac - std::panicking::default_hook::{{closure}}::he19d4c7230e07961
   5:     0x56020ee3eff2 - std::panicking::default_hook::hf614597d3c67bbdb
   6:     0x56020ee3f787 - std::panicking::rust_panic_with_hook::h8942133a8b252070
   7:     0x56020ee3f61a - std::panicking::begin_panic_handler::{{closure}}::hb5f5963570096b29
   8:     0x56020ee3e759 - std::sys::backtrace::__rust_end_short_backtrace::h6208cedc1922feda
   9:     0x56020ee3f2ac - rust_begin_unwind
  10:     0x56020e4b1b80 - core::panicking::panic_fmt::h0c3082644d1bf418
  11:     0x56020e4b1b5b - core::option::expect_failed::h95d2432053ef5ebb
  12:     0x56020e5a047d - retrom_service::providers::steam::provider::SteamWebApiProvider::get_owned_games::{{closure}}::{{closure}}::ha6e95efc126530e7
  13:     0x56020e5652ef - <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::h8a295e1bce475cf8
  14:     0x56020e59f5ee - retrom_service::providers::steam::provider::SteamWebApiProvider::get_owned_games::{{closure}}::h447e07c5e7bf8fb9
  15:     0x56020e58da1a - retrom_service::grpc::library::update_handlers::update_library::{{closure}}::{{closure}}::hb6427dac009cd7ce
  16:     0x56020e55578f - <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::h1d9db0fa6efe79bf
  17:     0x56020e558325 - <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll::h30024eac25489af2
  18:     0x56020e880733 - <retrom_service::grpc::library::LibraryServiceHandlers as retrom_codegen::retrom::library_service_server::LibraryService>::update_library::{{closure}}::hd0396bcf4c3a3de6
  19:     0x56020e913096 - <<retrom_codegen::retrom::library_service_server::LibraryServiceServer<T> as tower_service::Service<http::request::Request<B>>>::call::UpdateLibrarySvc<T> as tonic::server::service::UnaryService<retrom_codegen::retrom::UpdateLibraryRequest>>::call::{{closure}}::h43beb7169225686a
  20:     0x56020e9038a0 - <retrom_codegen::retrom::library_service_server::LibraryServiceServer<T> as tower_service::Service<http::request::Request<B>>>::call::{{closure}}::hfb4e56dc6a66a358
  21:     0x56020e5d7de1 - <tower_http::cors::ResponseFuture<F> as core::future::future::Future>::poll::h573baec943ad2f16
  22:     0x56020e693f86 - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h3a47c648e3bf87cc
  23:     0x56020e694246 - <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll::h47d79de3e3b71c24
  24:     0x56020e5a0839 - <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll::hac7c6e328bebacb9
  25:     0x56020eb8e12b - <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll::hcf3e011fbb6341d9
  26:     0x56020eb90f08 - <tonic::transport::service::router::RoutesFuture as core::future::future::Future>::poll::h2148275af62f65a5
  27:     0x56020e5d7ad0 - <tower_http::cors::ResponseFuture<F> as core::future::future::Future>::poll::h4a03c977c160bb43
  28:     0x56020e61ba08 - <core::pin::Pin<P> as core::future::future::Future>::poll::h48e3d0811bbc7ff0
  29:     0x56020e74b8b1 - hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_inner::he6559cba0efef7ef
  30:     0x56020e749dbc - hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_catch::h709ab810f0b84f9d
  31:     0x56020e8fb3ea - <hyper::server::server::new_svc::NewSvcTask<I,N,S,E,W> as core::future::future::Future>::poll::h70901b7c4c30fe06
  32:     0x56020e7cf6a7 - tokio::runtime::task::core::Core<T,S>::poll::h88b5c09546b46e5d
  33:     0x56020e4d00d7 - tokio::runtime::task::harness::Harness<T,S>::poll::h4223d0a7ca25b2c5
  34:     0x56020ede2079 - tokio::runtime::scheduler::multi_thread::worker::Context::run_task::he292ba0e304c1f5d
  35:     0x56020ede1161 - tokio::runtime::scheduler::multi_thread::worker::Context::run::h86f99e80c7078c44
  36:     0x56020ede5006 - tokio::runtime::context::scoped::Scoped<T>::set::hcc4198274b525f0a
  37:     0x56020edf528e - tokio::runtime::context::runtime::enter_runtime::h9a86228fedaeb11d
  38:     0x56020ede0fca - tokio::runtime::scheduler::multi_thread::worker::run::h0bd14c0a667602e2
  39:     0x56020edde377 - <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll::h96e86121baf14b6c
  40:     0x56020edf3243 - tokio::runtime::task::core::Core<T,S>::poll::hcf7589f2a5e1b45f
  41:     0x56020edd1ca4 - tokio::runtime::task::harness::Harness<T,S>::poll::hc577832071493a58
  42:     0x56020edebc82 - tokio::runtime::blocking::pool::Inner::run::hb3f276ae89feef52
  43:     0x56020edd498e - std::sys::backtrace::__rust_begin_short_backtrace::h05b15654737a441b
  44:     0x56020edd3a29 - core::ops::function::FnOnce::call_once{{vtable.shim}}::heb14499f4640c23d
  45:     0x56020ee42c4b - std::sys::pal::unix::thread::Thread::new::thread_start::hcc78f3943333fa94
  46:     0x7f12114111c4 - <unknown>
  47:     0x7f121149185c - <unknown>
  48:                0x0 - <unknown>
9:02:06 AM [vite] http proxy error: /retrom.LibraryService/UpdateLibrary
Error: socket hang up
    at Socket.socketOnEnd (node:_http_client:528:25)
    at Socket.emit (node:events:530:35)
    at endReadableNT (node:internal/streams/readable:1698:12)
    at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

[DenowRammy]

same issue here

[JMBeresford]

@Toomblr Is your port binding really commented out, or is that an accident:

services:
  retrom:
    container_name: retrom
    image: ghcr.io/jmberesford/retrom-service:latest
    #    ports:
    #      - 5101:5101

This is what I expect is causing your 500 error. If this is not the case, then please provide more information re: your networking situation. What IP is your retrom service running under, are you certain your client is pointed at the correct hostname + port to access the server, etc.

@SimHat The update library job does not interact w/ IGDB in any way, that is the Download Library job. IGDB is also optional, so being unable to download metadata would not lead to the inability to update the library entries and use them.

[Toomblr]

Yes is it commented out, cause I'm using Nginx Proxy Manager combined with a docker network to reach Retrom. The client is just the webinterface of the container. So it is pointed at the right IP. This worked so far, but after the latest update it stopped working.

[JMBeresford]

Ah I see, I did not know this was a regression. I believe I found the issue and will look to fix soon.

[DenowRammy]

With RUST_BACKTRACE=1, I get

2024-12-24T12:28:27.272941977Z 2024-12-24T12:28:27.272788Z  INFO retrom_service: Starting Retrom 0.4.8 service at: 0.0.0.0:5101
2024-12-24T12:28:27.453517842Z ! Corepack is about to download https://registry.npmjs.org/pnpm/-/pnpm-9.5.0.tgz
2024-12-24T12:28:28.226072015Z 
2024-12-24T12:28:28.226087427Z > web@ preview /app/web
2024-12-24T12:28:28.226089602Z > vite preview
2024-12-24T12:28:28.226091227Z 
2024-12-24T12:28:28.581585567Z 
2024-12-24T12:28:28.581610424Z ♻️  Generating routes...
2024-12-24T12:28:28.681121878Z ✅ Processed routes in 100ms
2024-12-24T12:28:28.685151204Z   ➜  Local:   http://localhost:3000/
2024-12-24T12:28:28.685167099Z   ➜  Network: http://10.0.0.33:3000/
2024-12-24T12:28:28.685282657Z   ➜  Network: http://172.18.0.6:3000/
2024-12-24T12:28:28.685289115Z   ➜  Network: http://10.0.10.7:3000/
2024-12-24T12:28:56.353369316Z thread 'tokio-runtime-worker' panicked at packages/service/src/providers/steam/provider.rs:222:33:
2024-12-24T12:28:56.353391982Z No Steam Config
2024-12-24T12:28:56.353394918Z stack backtrace:
2024-12-24T12:28:56.362689472Z    0: rust_begin_unwind
2024-12-24T12:28:56.363226369Z    1: core::panicking::panic_fmt
2024-12-24T12:28:56.363238480Z    2: core::option::expect_failed
2024-12-24T12:28:56.363677820Z    3: retrom_service::providers::steam::provider::SteamWebApiProvider::get_owned_games::{{closure}}::{{closure}}
2024-12-24T12:28:56.364338123Z    4: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-12-24T12:28:56.364954102Z    5: retrom_service::providers::steam::provider::SteamWebApiProvider::get_owned_games::{{closure}}
2024-12-24T12:28:56.365568287Z    6: retrom_service::grpc::library::update_handlers::update_library::{{closure}}::{{closure}}
2024-12-24T12:28:56.366158222Z    7: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-12-24T12:28:56.366197388Z    8: <tracing::instrument::Instrumented<T> as core::future::future::Future>::poll
2024-12-24T12:28:56.366275732Z    9: <retrom_service::grpc::library::LibraryServiceHandlers as retrom_codegen::retrom::library_service_server::LibraryService>::update_library::{{closure}}
2024-12-24T12:28:56.366932126Z   10: <<retrom_codegen::retrom::library_service_server::LibraryServiceServer<T> as tower_service::Service<http::request::Request<B>>>::call::UpdateLibrarySvc<T> as tonic::server::service::UnaryService<retrom_codegen::retrom::UpdateLibraryRequest>>::call::{{closure}}
2024-12-24T12:28:56.366979044Z   11: <retrom_codegen::retrom::library_service_server::LibraryServiceServer<T> as tower_service::Service<http::request::Request<B>>>::call::{{closure}}
2024-12-24T12:28:56.367016373Z   12: <tower_http::cors::ResponseFuture<F> as core::future::future::Future>::poll
2024-12-24T12:28:56.367635259Z   13: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
2024-12-24T12:28:56.368164448Z   14: <futures_util::future::future::map::Map<Fut,F> as core::future::future::Future>::poll
2024-12-24T12:28:56.368207456Z   15: <tower::util::map_response::MapResponseFuture<F,N> as core::future::future::Future>::poll
2024-12-24T12:28:56.368262041Z   16: <tower::util::oneshot::Oneshot<S,Req> as core::future::future::Future>::poll
2024-12-24T12:28:56.368289565Z   17: <tonic::transport::service::router::RoutesFuture as core::future::future::Future>::poll
2024-12-24T12:28:56.368852288Z   18: <tower_http::cors::ResponseFuture<F> as core::future::future::Future>::poll
2024-12-24T12:28:56.368879144Z   19: <core::pin::Pin<P> as core::future::future::Future>::poll
2024-12-24T12:28:56.368919878Z   20: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_inner
2024-12-24T12:28:56.369304363Z   21: hyper::proto::h1::dispatch::Dispatcher<D,Bs,I,T>::poll_catch
2024-12-24T12:28:56.369358824Z   22: <hyper::server::server::new_svc::NewSvcTask<I,N,S,E,W> as core::future::future::Future>::poll
2024-12-24T12:28:56.369392809Z   23: tokio::runtime::task::core::Core<T,S>::poll
2024-12-24T12:28:56.369426253Z   24: tokio::runtime::task::harness::Harness<T,S>::poll
2024-12-24T12:28:56.369451283Z   25: tokio::runtime::scheduler::multi_thread::worker::Context::run_task
2024-12-24T12:28:56.369926688Z   26: tokio::runtime::scheduler::multi_thread::worker::Context::run
2024-12-24T12:28:56.369933531Z   27: tokio::runtime::context::scoped::Scoped<T>::set
2024-12-24T12:28:56.370356454Z   28: tokio::runtime::context::runtime::enter_runtime
2024-12-24T12:28:56.370365964Z   29: tokio::runtime::scheduler::multi_thread::worker::run
2024-12-24T12:28:56.370381405Z   30: <tokio::runtime::blocking::task::BlockingTask<T> as core::future::future::Future>::poll
2024-12-24T12:28:56.370407137Z   31: tokio::runtime::task::core::Core<T,S>::poll
2024-12-24T12:28:56.370434900Z   32: tokio::runtime::task::harness::Harness<T,S>::poll
2024-12-24T12:28:56.370456340Z   33: tokio::runtime::blocking::pool::Inner::run
2024-12-24T12:28:56.371368894Z note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
2024-12-24T12:28:56.386118996Z 12:28:56 PM [vite] http proxy error: /retrom.LibraryService/UpdateLibrary
2024-12-24T12:28:56.386138299Z Error: socket hang up
2024-12-24T12:28:56.386149181Z     at Socket.socketOnEnd (node:_http_client:528:25)
2024-12-24T12:28:56.386151974Z     at Socket.emit (node:events:530:35)
2024-12-24T12:28:56.386153702Z     at endReadableNT (node:internal/streams/readable:1698:12)
2024-12-24T12:28:56.386155423Z     at process.processTicksAndRejections (node:internal/process/task_queues:82:21)

My compose file is

services:
  retrom:
    depends_on:
      - retrom-db
    image: ghcr.io/jmberesford/retrom-service:latest
    ports:
      - 5101:5101
      - 3000:3000 # to access the web client
    volumes:
      - config:/config/ # directory containing your config file
      - /mnt/nvme/Emulator/GBA:/gba/
      - /mnt/nvme/Emulator/GCN:/gcn/
      - /mnt/nvme/Emulator/NDS:/nds/
      - /mnt/nvme/Emulator/NSW:/nsw/
      - /mnt/nvme/Emulator/PS2:/ps2/
      - /mnt/nvme/Emulator/SNES:/snes/
      - /mnt/nvme/Emulator/3DS:/3ds/
      - /mnt/nvme/Emulator/GBC:/gbc/
      - /mnt/nvme/Emulator/N64:/n64/
      - /mnt/nvme/Emulator/NES:/nes/
      - /mnt/nvme/Emulator/PS1:/ps1/
      - /mnt/nvme/Emulator/PSP:/psp/
      - /mnt/nvme/Emulator/Wii:/wii/
    environment:
      - UID=${PUID}
      - GID=${PGID}
      - USER=${USER}
      - RUST_BACKTRACE=1


  # OPTIONAL: spin up a postgres container to use as the database, if you
  # don't have one already.
  #
  # read the docs here: https://hub.docker.com/_/postgres
  retrom-db:
    container_name: retrom-db
    hostname: retrom-db # this should match the db_url in your config file
    image: postgres:16
    restart: unless-stopped
    volumes:
      # to store the DB data on the host, change this path to any directory you like
      - data:/var/lib/postgresql/data
    environment:
      - UID=${PUID}
      - GID=${PGID}
      - POSTGRES_USER=${POSTGRES_USER}
      - POSTGRES_PASSWORD=${POSTGRES_PASSWORD}
      - POSTGRES_DB=${POSTGRES_DB}
      - USER=${USER}

volumes:
  config:
  data:

And my config file is:

{
  "connection": {
    "port": 5101,
    "dbUrl": "postgres://[redacted]/retrom"
  },
  "contentDirectories": [
    {
      "path": "/gba",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/gcn",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/nds",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/nsw",
      "storageType": "MultiFileGame"
    },
    {
      "path": "/ps2",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/snes",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/3ds",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/gbc",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/n64",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/nes",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/ps1",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/psp",
      "storageType": "SingleFileGame"
    },
    {
      "path": "/wii",
      "storageType": "SingleFileGame"
    }
  ],
  "igdb": {
    "clientSecret": "[redacted]",
    "clientId": "[redacted]"
  }
}

[DenowRammy]

It looks to me like it's failing because it tries to get my steam information, but as you can see I never added my steam info to the config, nor am I interested in using steam with retrom.

[JMBeresford]

Yes, I can reproduce this. Steam config is intended to be optional, will shoot out a fix soon. Thanks much!

[JMBeresford]

@DenowRammy @Toomblr Tracked in #205, fixed in version 0.4.9 which should be published within the hour.