Input file contains unsupported image format

[breakingflower]

The bug

What happened?
Thumbnail generator reports err because the image file format is not supported when manually triggering jobs in the jobs page.

What did you expect to happen?
To not have passed the initial ingress, or to have a cleaning ability

[Nest] 7  - 02/10/2024, 12:35:59 PM   ERROR [JobService] Unable to run job handler (thumbnailGeneration/generate-jpeg-thumbnail): Error: Input file contains unsupported image format
[Nest] 7  - 02/10/2024, 12:35:59 PM   ERROR [JobService] Error: Input file contains unsupported image format
    at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
    at MediaRepository.resize (/usr/src/app/dist/infra/repositories/media.repository.js:41:14)
    at MediaService.generateThumbnail (/usr/src/app/dist/domain/media/media.service.js:121:44)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async MediaService.handleGenerateJpegThumbnail (/usr/src/app/dist/domain/media/media.service.js:108:28)
    at async /usr/src/app/dist/domain/job/job.service.js:116:37
    at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
    at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
[Nest] 7  - 02/10/2024, 12:35:59 PM   ERROR [JobService] Object:
{
  "id": "<redacted>"
}

The OS that Immich Server is running on

ubuntu22.04 docker

Version of Immich Server

v1.94.1

Version of Immich Mobile App

v1.94.1

Platform with the issue

• Server
• Web
• Mobile

Your docker-compose.yml content

version: "3.8"

networks:
  proxy:
    external: true

services:
  immich-server:
    container_name: immich_server
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    command: ["start.sh", "immich"]
    volumes:
      - ${UPLOAD_LOCATION}/photos:/usr/src/app/upload
      - /etc/localtime:/etc/localtime:ro
    ports:
      - 2283:3001
    env_file:
      - .env
    depends_on:
      - redis
      - database
    networks: 
      - proxy
      - default
    restart: always

  immich-microservices:
    container_name: immich_microservices
    image: ghcr.io/immich-app/immich-server:${IMMICH_VERSION:-release}
    extends:
      file: hwaccel.transcoding.yml
      service: nvenc
    command: ["start.sh", "microservices"]
    volumes:
      - ${UPLOAD_LOCATION}/photos:/usr/src/app/upload
      # - ./config/geocoding:./.reverse-geocoding-dump/
      - /etc/localtime:/etc/localtime:ro
    env_file:
      - .env
    depends_on:
      - redis
      - database
    restart: always

    
  immich-machine-learning:
    container_name: immich_machine_learning
    # For hardware acceleration, add one of -[armnn, cuda, openvino] to the image tag.
    # Example tag: ${IMMICH_VERSION:-release}-cuda
    image: ghcr.io/immich-app/immich-machine-learning:${IMMICH_VERSION:-release}-cuda
    extends: # uncomment this section for hardware acceleration - see https://immich.app/docs/features/ml-hardware-acceleration
      file: hwaccel.ml.yml
      service: cuda # set to one of [armnn, cuda, openvino, openvino-wsl] for accelerated inference - use the `-wsl` version for WSL2 where applicable
    volumes:
      - ${UPLOAD_LOCATION}/model_cache:/cache
    env_file:
      - .env
    restart: always

  redis:
    container_name: immich_redis
    image: redis:6.2-alpine@sha256:70a7a5b641117670beae0d80658430853896b5ef269ccf00d1827427e3263fa3
    restart: always

  database:
    container_name: immich_postgres
    image: tensorchord/pgvecto-rs:pg14-v0.1.11
    env_file:
      - .env
    environment:
      POSTGRES_PASSWORD: ${DB_PASSWORD}
      POSTGRES_USER: ${DB_USERNAME}
      POSTGRES_DB: ${DB_DATABASE_NAME}
    volumes:
      - ${UPLOAD_LOCATION}/db:/var/lib/postgresql/data
    restart: always

Your .env content

# You can find documentation for all the supported env variables at https://immich.app/docs/install/environment-variables

# The location where your uploaded files are stored
UPLOAD_LOCATION=./storage

# The Immich version to use. You can pin this to a specific version like "v1.71.0"
IMMICH_VERSION=release
TZ=Europe/Brussels

# Connection secrets for postgres and typesense. You should change these to random passwords
DB_PASSWORD=<redacted>

# The values below this line do not need to be changed
###################################################################################
DB_HOSTNAME=immich_postgres
DB_USERNAME=postgres
DB_DATABASE_NAME=immich

REDIS_HOSTNAME=immich_redis

Reproduction steps

1. Upload files to server
2. Trigger job generate thumbnail
3. Error in logs, nothing reported in frontend
4. Re-triggering job triggers same output
...

Additional information

No response

[aviv926]

Can you add a sample file? Or provide the file extension?

[breakingflower]

Hi @aviv926 , unfortunately I have no more info than what is shown in the logs. Can you advise on how I can figure out what the corrupted file is, and if I can remove it manually without breaking anything?

A copy of the corresponding logs with LOG_LEVEL=verbose:

immich_server            | [Nest] 7  - 02/12/2024, 1:21:22 PM   DEBUG [JobService] Handling command: queue=thumbnailGeneration,force=false
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM   ERROR [JobService] Unable to run job handler (thumbnailGeneration/generate-jpeg-thumbnail): Error: Input file contains unsupported image format
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM   ERROR [JobService] Error: Input file contains unsupported image format
immich_microservices     |     at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
immich_microservices     |     at MediaRepository.resize (/usr/src/app/dist/infra/repositories/media.repository.js:41:14)
immich_microservices     |     at MediaService.generateThumbnail (/usr/src/app/dist/domain/media/media.service.js:121:44)
immich_microservices     |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices     |     at async MediaService.handleGenerateJpegThumbnail (/usr/src/app/dist/domain/media/media.service.js:108:28)
immich_microservices     |     at async /usr/src/app/dist/domain/job/job.service.js:116:37
immich_microservices     |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_microservices     |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM   ERROR [JobService] Object:
immich_microservices     | {
immich_microservices     |   "id": "8da64e75-200e-4b91-a1b3-3f0b107a2232"
immich_microservices     | }
immich_microservices     | 
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM    WARN [MediaService] Skipped thumbnail generation for asset 97d5458e-f938-43d9-b5a0-a0e22e269ea6: no video streams found
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM    WARN [MediaService] Skipped thumbnail generation for asset 97d5458e-f938-43d9-b5a0-a0e22e269ea6: no video streams found

[aviv926]

Hi @aviv926 , unfortunately I have no more info than what is shown in the logs. Can you advise on how I can figure out what the corrupted file is, and if I can remove it manually without breaking anything?

A copy of the corresponding logs with LOG_LEVEL=verbose:

immich_server            | [Nest] 7  - 02/12/2024, 1:21:22 PM   DEBUG [JobService] Handling command: queue=thumbnailGeneration,force=false
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM   ERROR [JobService] Unable to run job handler (thumbnailGeneration/generate-jpeg-thumbnail): Error: Input file contains unsupported image format
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM   ERROR [JobService] Error: Input file contains unsupported image format
immich_microservices     |     at Sharp.toFile (/usr/src/app/node_modules/sharp/lib/output.js:89:19)
immich_microservices     |     at MediaRepository.resize (/usr/src/app/dist/infra/repositories/media.repository.js:41:14)
immich_microservices     |     at MediaService.generateThumbnail (/usr/src/app/dist/domain/media/media.service.js:121:44)
immich_microservices     |     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
immich_microservices     |     at async MediaService.handleGenerateJpegThumbnail (/usr/src/app/dist/domain/media/media.service.js:108:28)
immich_microservices     |     at async /usr/src/app/dist/domain/job/job.service.js:116:37
immich_microservices     |     at async Worker.processJob (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:394:28)
immich_microservices     |     at async Worker.retryIfFailed (/usr/src/app/node_modules/bullmq/dist/cjs/classes/worker.js:581:24)
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM   ERROR [JobService] Object:
immich_microservices     | {
immich_microservices     |   "id": "8da64e75-200e-4b91-a1b3-3f0b107a2232"
immich_microservices     | }
immich_microservices     | 
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM    WARN [MediaService] Skipped thumbnail generation for asset 97d5458e-f938-43d9-b5a0-a0e22e269ea6: no video streams found
immich_microservices     | [Nest] 7  - 02/12/2024, 1:21:23 PM    WARN [MediaService] Skipped thumbnail generation for asset 97d5458e-f938-43d9-b5a0-a0e22e269ea6: no video streams found

You can search for this entry on your uplode folder and check what files it shows:

8da64e7-200e-4b91-a1b3-3f0b107a2232

[breakingflower]

Hi @aviv926 , can you share how I can find this? The GUI shows 20+ photos and using find ./uploads -name "8da64e75-200e-4b91-a1b3-3f0b107a2232" does not return results.

[aviv926]

Hi @aviv926 , can you share how I can find this? The GUI shows 20+ photos and using find ./uploads -name "8da64e75-200e-4b91-a1b3-3f0b107a2232" does not return results.

My bad, try this:
97d5458e-f938-43d9-b5a0-a0e22e269ea6

[breakingflower]

Neither returns results, although the gui does show many (10+) results for both.

[aviv926]

Neither returns results, although the gui does show many (10+) results for both.

Can you send a screenshot of what GUI shows?

[breakingflower]

[aviv926]

When you type something like this in the search without adding
m: It searches by using the CLIP model, not by file name.
The properties you see here are not related to
97d5458e-f938-43d9-b5a0-a0e22e269ea6
Unless you search for:
m: 97d5458e-f938-43d9-b5a0-a0e22e269ea6

In any case, try what Jrasm suggested.

[breakingflower]

It seems like many dark videos / images. Upon hover all of the videos play without issues.

[jrasm91]

For now you'll have to use https://immich.app/docs/guides/database-queries to go from id to file location.

[breakingflower]

I've found the culprits:

select 
	"id",
	"deviceAssetId", 
	"originalPath"
from "assets" 
where 
	"id" in ('97d5458e-f938-43d9-b5a0-a0e22e269ea6', '8da64e75-200e-4b91-a1b3-3f0b107a2232')

There are 2 files, one mp4 that is actually an audio file that plays but has no video data:

It makes sense that you can't extract frames, there are none. Perhaps a test on ingest can be to verify if you can do the frame extraction?

The second is a jpeg file with size 0kb and no exif or other details. Its just a corrupted file.

What is the correct action to remove corrupted files?

[breakingflower]

I manually deleted the files but in hindsight that was not a good idea. The database is still in the state that the files exist, and now have errors that they in fact do not exist.

I tried the following after manually deleting the files, but getting a 400: "Not found or no asset.delete access":

curl --location --request DELETE 'http://${SERVER}/api/asset' --header 'Content-Type: application/json' --header 'x-api-key: ${API_KEY}' --data '{ "force" : true, "ids": ["'8da64e75-200e-4b91-a1b3-3f0b107a2232'"]}'

Can you please share what the correct steps are to remove the files?

[midnextlevel]

anyone that can help also, having the same problem, and want to delete those files with errors

[DonRichie]

In my case it was a permission problem. Please check if the immich user can access the file and directory.

Maybe the devs can seek for a way to make the error message more meaningful in this case.

[JanisV]

In my case it was a *.hif files that not is image

[Quba1]

I had noticed similar errors in logs for thumbnail generation and managed to find this solution:

1. (optional) Look up corrupted files in the database following this method to review the files or download them as needed. For example, I was able to recover thumbnails from corrupted files.
2. Do not remove the files from filesystem!
3. Use API to delete assets as described in docs. For me this command worked:

curl -L -X DELETE 'https://<your-server-url>/api/assets' \
-H 'Content-Type: application/json' \
-H 'x-api-key: <your-api-key>' \
-d '{
  "force": true,
  "ids": [
    "<corrupted-asset-id>"
  ]
}'

@breakingflower @midnextlevel Hope that helps you

[midnextlevel]

I will get back to you @Quba1 when i got the chance to access my immich instance, Thanks for this