fix!: deal with huge incoming image pixelsizes #6825 #6935

ivan-cx · 2025-06-24T22:06:35Z

This pull request attempts to fix #6825

BREAKING CHANGE: messages with invalid images, images of unknown size, huge images, will have Viewtype::File

After changing the logic of Viewtype selection, I had to fix 3 old tests that used invalid Base64 image data.

- test_huge_image_becomes_file: message with huge image should have Viewtype::File - test_4k_image_stays_image: message with 4k image should have Viewtype::Image Refs: chatmail#6825

BREAKING CHANGE: messages with invalid images, images of unknown size, huge images, will have Viewtype::File Refs: chatmail#6825

- parse_inline_image - parse_thunderbird_html_embedded_image - parse_outlook_html_embedded_image The tests were failing because messages with images of unknown size (invalid images) now have Viewtype::File and not Viewtype::Image Refs: chatmail#6825

ivan-cx · 2025-06-25T09:22:50Z

The CI tests are failing, I assume, due to unset CHATMAIL_DOMAIN repository variable in my forked repo. I've added the variable now, let's see if it works?

ivan-cx · 2025-06-25T10:54:14Z

Ok, I do not know how to fix
"CAN NOT RUN COVERAGE correctly: Missing CHATMAIL_DOMAIN environment variable!"
and similar issues in tests. My forked repo has CHATMAIL_DOMAIN repository variable AND environment variable, but it does not affect the runs in this PR.
Local Python tests work fine for me on Ubuntu, I set CHATMAIL_DOMAIN manually when I run them.

ivan-cx · 2025-06-25T14:12:17Z

src/mimeparser.rs

+                }
+                // image is too big or size is unknown, display as file:
+                _ => Viewtype::File,
+            }


This "else if" block is the main change of the PR.
For Viewtype::Image and Viewtype::Gif we check image size and set width and height for valid images.
Otherwise, we change msg_type to Viewtype::File - for all images/gifs of unknown size, or too big.
Setting of width and height previously was done not only for Viewtype::Image and Viewtype::Gif, but for any mime::IMAGE, which is not the same (at least, SVG is mime::IMAGE, but Viewtype::File)

iequidoo

JSON-RPC tests failing with Missing CHATMAIL_DOMAIN environment variable! is a known problem for PRs from other repos.

src/mimeparser.rs

src/mimeparser/mimeparser_tests.rs

- treat Viewtype::Sticker the same way as Image and Gif - use safe `set_i64()` and `.into()` instead of unsafe `as i32` Reviewed-by: iequidoo

src/mimeparser/mimeparser_tests.rs

iequidoo · 2025-06-26T21:59:30Z

Everything else looks good. I created a pre-main branch so that the PR can be merged (probably, squash-merged) there and then pre-main can be merged to main additionally checking that JSON-RPC tests are working. Meanwhile other contributors may take a look at the PR (particularly at this 8.3 Mpx limit which is nearly optimal for me personally). Many thanks!

iequidoo · 2025-06-28T22:23:47Z

src/mimeparser/mimeparser_tests.rs

+async fn test_huge_image_becomes_file() {
+    let t = TestContext::new_alice().await;
+    receive_imf(
+        &t.ctx,
+        include_bytes!("../../test-data/message/image_huge_36M.eml"),
+        false,
+    )
+    .await
+    .unwrap();
+    let chats = Chatlist::try_load(&t.ctx, 0, None, None).await.unwrap();
+    let msg_id = chats.get_msg_id(0).unwrap().unwrap();
+    let msg = Message::load_from_db(&t.ctx, msg_id).await.unwrap();
+    // Huge image should be treated as file:
+    assert_eq!(msg.viewtype, Viewtype::File);
+    assert!(msg.get_file(&t).is_some());
+    assert_eq!(msg.get_filename().unwrap(), "huge_image.png");
+    assert_eq!(msg.get_filemime().unwrap(), "image/png");
+    // File has no width or height
+    assert_eq!(msg.param.get_int(Param::Width).unwrap_or_default(), 0);
+    assert_eq!(msg.param.get_int(Param::Height).unwrap_or_default(), 0);
+}
+
+#[tokio::test(flavor = "multi_thread", worker_threads = 2)]
+async fn test_4k_image_stays_image() {
+    let t = TestContext::new_alice().await;
+    receive_imf(
+        &t.ctx,
+        include_bytes!("../../test-data/message/image_4k.eml"),
+        false,
+    )
+    .await
+    .unwrap();
+    let chats = Chatlist::try_load(&t.ctx, 0, None, None).await.unwrap();
+    let msg_id = chats.get_msg_id(0).unwrap().unwrap();
+    let msg = Message::load_from_db(&t.ctx, msg_id).await.unwrap();
+    // 4K image should be treated as image:
+    assert_eq!(msg.viewtype, Viewtype::Image);
+    assert!(msg.get_file(&t).is_some());
+    assert_eq!(msg.get_filename().unwrap(), "4k_image.png");
+    assert_eq!(msg.get_filemime().unwrap(), "image/png");
+    assert_eq!(msg.param.get_int(Param::Width).unwrap_or_default(), 3840);
+    assert_eq!(msg.param.get_int(Param::Height).unwrap_or_default(), 2160);


Suggested change

async fn test_huge_image_becomes_file() {

let t = TestContext::new_alice().await;

receive_imf(

&t.ctx,

include_bytes!("../../test-data/message/image_huge_36M.eml"),

false,

)

.await

.unwrap();

let chats = Chatlist::try_load(&t.ctx, 0, None, None).await.unwrap();

let msg_id = chats.get_msg_id(0).unwrap().unwrap();

let msg = Message::load_from_db(&t.ctx, msg_id).await.unwrap();

// Huge image should be treated as file:

assert_eq!(msg.viewtype, Viewtype::File);

assert!(msg.get_file(&t).is_some());

assert_eq!(msg.get_filename().unwrap(), "huge_image.png");

assert_eq!(msg.get_filemime().unwrap(), "image/png");

// File has no width or height

assert_eq!(msg.param.get_int(Param::Width).unwrap_or_default(), 0);

assert_eq!(msg.param.get_int(Param::Height).unwrap_or_default(), 0);

}

#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

async fn test_4k_image_stays_image() {

let t = TestContext::new_alice().await;

receive_imf(

&t.ctx,

include_bytes!("../../test-data/message/image_4k.eml"),

false,

)

.await

.unwrap();

let chats = Chatlist::try_load(&t.ctx, 0, None, None).await.unwrap();

let msg_id = chats.get_msg_id(0).unwrap().unwrap();

let msg = Message::load_from_db(&t.ctx, msg_id).await.unwrap();

// 4K image should be treated as image:

assert_eq!(msg.viewtype, Viewtype::Image);

assert!(msg.get_file(&t).is_some());

assert_eq!(msg.get_filename().unwrap(), "4k_image.png");

assert_eq!(msg.get_filemime().unwrap(), "image/png");

assert_eq!(msg.param.get_int(Param::Width).unwrap_or_default(), 3840);

assert_eq!(msg.param.get_int(Param::Height).unwrap_or_default(), 2160);

async fn test_huge_image_becomes_file() -> Result<()> {

let t = TestContext::new_alice().await;

let msg_id = receive_imf(

&t,

include_bytes!("../../test-data/message/image_huge_36M.eml"),

false,

)

.await?

.unwrap()

.msg_ids[0];

let msg = Message::load_from_db(&t.ctx, msg_id).await.unwrap();

// Huge image should be treated as file:

assert_eq!(msg.viewtype, Viewtype::File);

assert!(msg.get_file(&t).is_some());

assert_eq!(msg.get_filename().unwrap(), "huge_image.png");

assert_eq!(msg.get_filemime().unwrap(), "image/png");

// File has no width or height

assert_eq!(msg.param.get_int(Param::Width).unwrap_or_default(), 0);

assert_eq!(msg.param.get_int(Param::Height).unwrap_or_default(), 0);

Ok(())

}

#[tokio::test(flavor = "multi_thread", worker_threads = 2)]

async fn test_4k_image_stays_image() -> Result<()> {

let t = TestContext::new_alice().await;

let msg_id = receive_imf(

&t,

include_bytes!("../../test-data/message/image_4k.eml"),

false,

)

.await?

.unwrap()

.msg_ids[0];

let msg = Message::load_from_db(&t.ctx, msg_id).await.unwrap();

// 4K image should be treated as image:

assert_eq!(msg.viewtype, Viewtype::Image);

assert!(msg.get_file(&t).is_some());

assert_eq!(msg.get_filename().unwrap(), "4k_image.png");

assert_eq!(msg.get_filemime().unwrap(), "image/png");

assert_eq!(msg.param.get_int(Param::Width).unwrap_or_default(), 3840);

assert_eq!(msg.param.get_int(Param::Height).unwrap_or_default(), 2160);

Ok(())

ivan-cx added 3 commits June 24, 2025 23:58

test: add tests for huge images

409a013

- test_huge_image_becomes_file: message with huge image should have Viewtype::File - test_4k_image_stays_image: message with 4k image should have Viewtype::Image Refs: chatmail#6825

fix!: chatmail#6825 deal with huge incoming image pixelsizes

ea22973

BREAKING CHANGE: messages with invalid images, images of unknown size, huge images, will have Viewtype::File Refs: chatmail#6825

ivan-cx changed the title ~~deal with huge incoming image pixelsizes #6825~~ fix!: deal with huge incoming image pixelsizes #6825 Jun 25, 2025

fix: do not set Param::Width and Param::Height for Viewtype::File

ef13c94

ivan-cx force-pushed the main branch from 32e3bec to ef13c94 Compare June 25, 2025 10:33

ivan-cx mentioned this pull request Jun 25, 2025

deal with huge incoming image pixelsizes #6825

Open

ivan-cx commented Jun 25, 2025

View reviewed changes

Merge branch 'chatmail:main' into main

aacab1e

iequidoo self-requested a review June 25, 2025 18:31

iequidoo reviewed Jun 25, 2025

View reviewed changes

src/mimeparser.rs Outdated Show resolved Hide resolved

src/mimeparser.rs Outdated Show resolved Hide resolved

src/mimeparser/mimeparser_tests.rs Show resolved Hide resolved

ivan-cx added 2 commits June 25, 2025 23:54

refactor: code review

dae4915

- treat Viewtype::Sticker the same way as Image and Gif - use safe `set_i64()` and `.into()` instead of unsafe `as i32` Reviewed-by: iequidoo

docs: improve comment after code review

c79e59b

ivan-cx requested a review from iequidoo June 26, 2025 11:42

link2xt force-pushed the main branch from 285d80a to 416131b Compare June 26, 2025 14:07

iequidoo reviewed Jun 26, 2025

View reviewed changes

src/mimeparser/mimeparser_tests.rs Outdated Show resolved Hide resolved

test: keep only specific checks in huge image tests, fix 4k_image.png

ac9d7b4

iequidoo reviewed Jun 26, 2025

View reviewed changes

src/mimeparser/mimeparser_tests.rs Show resolved Hide resolved

iequidoo changed the base branch from main to pre-main June 26, 2025 21:54

iequidoo requested a review from r10s June 26, 2025 22:00

iequidoo approved these changes Jun 26, 2025

View reviewed changes

iequidoo reviewed Jun 28, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Uh oh!

fix!: deal with huge incoming image pixelsizes #6825 #6935

fix!: deal with huge incoming image pixelsizes #6825 #6935

Uh oh!

ivan-cx commented Jun 24, 2025 •

edited

Loading

Uh oh!

ivan-cx commented Jun 25, 2025 •

edited

Loading

Uh oh!

ivan-cx commented Jun 25, 2025 •

edited

Loading

Uh oh!

ivan-cx Jun 25, 2025 •

edited

Loading

Uh oh!

iequidoo left a comment •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iequidoo commented Jun 26, 2025

Uh oh!

iequidoo Jun 28, 2025

Uh oh!

Uh oh!

Uh oh!

fix!: deal with huge incoming image pixelsizes #6825 #6935

Are you sure you want to change the base?

fix!: deal with huge incoming image pixelsizes #6825 #6935

Uh oh!

Conversation

ivan-cx commented Jun 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ivan-cx commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ivan-cx commented Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ivan-cx Jun 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

iequidoo left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

iequidoo commented Jun 26, 2025

Uh oh!

iequidoo Jun 28, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

ivan-cx commented Jun 24, 2025 •

edited

Loading

ivan-cx commented Jun 25, 2025 •

edited

Loading

ivan-cx commented Jun 25, 2025 •

edited

Loading

ivan-cx Jun 25, 2025 •

edited

Loading

iequidoo left a comment •

edited

Loading