libc++abi: terminating due to uncaught exception of type std::runtime_error: Attempting to allocate too much memory

[jrp2014]

vlm % python mytest.py
Model:  mlx-community/Qwen2-VL-72B-Instruct-8bit
Fetching 25 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 16513.01it/s]
Fetching 25 files: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 25/25 [00:00<00:00, 10979.85it/s]
Image:  20241116-164155_L1010474-Edit.jpg
==========
Image: /Users/xxx/Pictures/Processed/20241116-164155_L1010474-Edit.jpg 

Prompt: <|im_start|>system
You are a helpful assistant.<|im_end|>
<|im_start|>user
Provide a factual caption, description and comma-separated keywords or tags for this image so that it can be searched for easily<|vision_start|><|image_pad|><|vision_end|><|im_end|>
<|im_start|>assistant

libc++abi: terminating due to uncaught exception of type std::runtime_error: Attempting to allocate 135383101952 bytes which is greater than the maximum allowed buffer size of 77309411328 bytes.
zsh: abort      python mytest.py
/opt/homebrew/Caskroom/miniconda/base/envs/mlx/lib/python3.12/multiprocessing/resource_tracker.py:254: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

I got a similar error with Qwen/Qwen2-VL-7B-Instruct

[Blaizzy]

Could you share the specs of your machine?

[Blaizzy]

And the dimension of the image.

[jrp2014]

It's a 128Gb M4 Max. The image is 5825 × 8753 pixels.

[Blaizzy]

That's a really large image my friend.

Please use resize-shape to at least 1K to 2k

[jrp2014]

Well, it is the size that comes out of cameras these days!

Where does resize-shape come from? (ie, how do I use it? Is it a Pillow thing, or a vlm thing, or ...?)

I don't really want to edit the original jpeg., and generate only takes the file path. And a good few of the other models accept this size of picture. (I think that I submitted some transcripts last week.)

[Blaizzy]

Where does resize-shape come from? (ie, how do I use it? Is it a Pillow thing, or a vlm thing, or ...?)

It's a mlx-vlm thing that uses pillow to resize your image while maintaining the aspect ratio.

The problem here is that the image processing operation is very taxing on your resources and some models are built in a way that makes things worst for on-device inference.

If you want to maintain original image size you either will run out memory or will have slow inference.

Nevertheless, I'm working on improving this as we speak.