Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The program will fail when copying images larger than 32MB. #1878

Closed
sunxiang0918 opened this issue Sep 12, 2024 · 6 comments · Fixed by #1879
Closed

The program will fail when copying images larger than 32MB. #1878

sunxiang0918 opened this issue Sep 12, 2024 · 6 comments · Fixed by #1879
Assignees
Labels
Bug Something isn't working High priority High priority

Comments

@sunxiang0918
Copy link
Contributor

How frequently does the bug occur?

Always

Description

当复制比较大的图片的时候,程序后台会报错,导致记录复制项失败.

分析源码:

当复制文件的时候,会进入到 FilesTypePlugin 这个文件类型的插件中去尝试进行文件的拷贝. 但是,由于文件大小超过了 maxBackupFileSize设置的大小(默认32mb). 会导致 useRefCopyFiles变量为true (104行).
在后续拷贝文件的时候,会跳过该文件,只记录该文件的相对路径(在这里实际就是文件名)(110-113行).
image
这些操作在对于非图片的文件上是没有问题的.

但是当复制的文件是图片的时候, 由于需要更改PasteItem的类型, 所以会进入到 FilesToImagesPlugin中.
在这个类中,pasteItems里面记录的实际上 仅仅是一个文件名. 那么 在从files文件夹移动到images文件夹的逻辑中的时候.
srcPath其实是错误的,文件并不存在于.user/files/ 中.(28-30行) .
从而导致了 移动文件失败,抛出IllegalStateException 异常.(30-33行)
image

修复的话,可能对于这种并没有真正拷贝的图片文件, 要参考pasteAppearItem的basePath是否存在来决定是否真正的做移动的操作. 如果不移动,构造的ImagesPasteItem的 relativePathList与basePath 可能需要设置成和fileItem一样的值.
但是这样处理后, 在 FilesPreviewView 去生成缩略图的时候可能还是会出现找不到文件的情况.可能ImagesPasteItem的getFilePaths也需要改造.

Stacktrace & log output

No response

Can you reproduce the bug?

Always

Reproduction Steps

复制大于 maxBackupFileSize配置的 图片文件.

Version

5194fbc

OS

MacOS

@CompileFuture2024 CompileFuture2024 changed the title 当复制大于32MB的图片的时候,程序会失败. The program will fail when copying images larger than 32MB. Sep 12, 2024
@guiyanakuang
Copy link
Member

@sunxiang0918. 👍🏻 Thank you very much for providing such a detailed bug description. This is extremely helpful for fixing the bug.

@guiyanakuang
Copy link
Member

@sunxiang0918. I have merged the fix into the main branch. Thank you very much for your help.
Also, here's a small tip: GitHub supports sharing code snippets, which is lighter than images and more specific. You just need to click on the code line, then hold shift to select the code snippet, and copy permalink.

image

// If the file size exceeds the limit
// or the file is copied from the cross-paste directory
// the file will not be copied
// Instead, record the file path
val useRefCopyFiles = copySizeExceeding || copyFromCrossPaste
for (file in files) {
val path = file.toOkioPath(normalize = true)
val fileName = file.name
if (useRefCopyFiles) {
val relativePath = path.name
relativePathList.add(relativePath)
fileInfoTrees[file.name] = DesktopFileUtils.getFileInfoTree(path)
} else {

@sunxiang0918
Copy link
Contributor Author

What a high efficiency!

I know this function of github, but I have the development environment of the project. When a problem occurs, I am more accustomed to using IDE break points to confirm the cause of the problem, so I took a screenshot~

@sunxiang0918
Copy link
Contributor Author

sunxiang0918 commented Sep 13, 2024

另外我还发现了一个生成缩略图相关的BUG.
在使用 skia 生成图片(缩略图) 的时候, 如果图片的尺寸过大,生成会失败,变成一个大小很小的空文件.
在CrossPaste的界面上,就是一个空白块.

image

我大概找了一下原因,网上说的可能和CPU与内存相关:
https://groups.google.com/g/skia-discuss/c/A47-OtBOnKU
https://groups.google.com/g/skia-discuss/c/886WuVcQUGo
https://stackoverflow.com/questions/52206463/large-image-resizing-shows-blank

我自己稍微试了一下.
在我的macmini( intel-i5 + 32g)的环境中, 最大尺寸是23170 * 23170 ( sqrt(uint32/8)),超过程序报错.
在我的mbp( apple m2 + 16g)的环境中, 测试到了 128k * 128k 也还能行...
所以,我也没具体测试出来边界和影响的因素到底是什么.

`
@test
fun testCreateImage() {
val width = 23170
val height = 23170

    val surface = Surface.makeRasterN32Premul(width, height)
    val canvas = surface.canvas

    // 填充灰色背景
    val paint = Paint().apply { color = Color.makeRGB(128, 128, 128) }
    canvas.drawRect(Rect.makeXYWH(0f, 0f, width.toFloat(), height.toFloat()), paint)

    val img = surface.makeImageSnapshot()

    val result = File("/Users/sun/Downloads/create1111.png").toOkioPath(true)
    
    img.encodeToData()?.bytes?.let {
        result.toNioPath().writeBytes(it)
    }
}

`

@CompileFuture2024
Copy link

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


In addition, I also found a BUG related to generating thumbnails.
When using skia to generate pictures (thumbnails), if the size of the picture is too large, the generation will fail and become an empty file with a small size.
On the CrossPaste interface, it is just a blank block.

image

I roughly looked for the reason, and what was said on the Internet may be related to CPU and memory:
https://groups.google.com/g/skia-discuss/c/A47-OtBOnKU
https://groups.google.com/g/skia-discuss/c/886WuVcQUGo
https://stackoverflow.com/questions/52206463/large-image-resizing-shows-blank

I tried it a little bit myself.
In the environment of my macmini (intel-i5 + 32g), the maximum size is 2317023170 (sqrt(uint32/8)), and the program reports an error if it exceeds the size.
In my mbp (apple m2 + 16g) environment, I tested 128k
128k and it still works...
Therefore, I have not specifically tested what the boundaries and influencing factors are.

`
@test
fun testCreateImage() {
val width = 23170
val height = 23170

    val surface = Surface.makeRasterN32Premul(width, height)
    val canvas = surface.canvas

    // fill gray background
    val paint = Paint().apply { color = Color.makeRGB(128, 128, 128) }
    canvas.drawRect(Rect.makeXYWH(0f, 0f, width.toFloat(), height.toFloat()), paint)

    val img = surface.makeImageSnapshot()

    val result = File("/Users/sun/Downloads/create1111.png").toOkioPath(true)
    
    img.encodeToData()?.bytes?.let {
        result.toNioPath().writeBytes(it)
    }
}

`

@guiyanakuang
Copy link
Member

I have created a new issue. Let's discuss how to solve this bug in the new issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working High priority High priority
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants