Run Error: assert not best_model is None

[pmj110119]

Hello, I tried to stitch on my images and had some problems. Do you know under what circumstances will the following error occur?

  File ".\xxx.py", line 41, in <module>
    result_df, _ = m2stitch.stitch_images(images[:44,:,:], rows, cols)
  File "E:\anaconda3\envs\torch\lib\site-packages\m2stitch\stitching.py", line 110, in stitch_images
    prob_uniform_w, mu_w, sigma_w = compute_image_overlap(grid, "left", W, H)
  File "E:\anaconda3\envs\torch\lib\site-packages\m2stitch\_stage_model.py", line 128, in compute_image_overlap
    assert not best_model is None
AssertionError

Thanks!

[yfukai]

Hello @pmj110119, thanks for your report! The original algorithm (4.1.2 Compute Stage Repeatability in the original paper) assumes that the displacements between tiles follow a Gaussian distribution, and I think this error means this assumption is not working well with your data. I'll make the parameter relevant to this assumption adjustable and then you might be able to solve the problem by easing the assumption. Please wait for an update!

[yfukai]

Hi @pmj110119, I updated the program (version 0.2.0, you can update it with pip) to allow the change of the parameter overlap_prob_uniform_threshold (default: 80). Could you try your stitching with increasing its value (for example, 85 or 90, even 95 to 100)? It might solve your problem.
The best way is to revise and rewrite the optimization process, but unfortunately, I don't have sufficient time for now. Hopefully, that will be done in the near future (by me or by someone kind)!

[pmj110119]

Thank you for your reply and update! But my data will still report errors :

100%|█████████████████████████████████████████████████████████████████████████████████████████████| 44/44 [00:02<00:00, 16.57it/s]
100%|█████████████████████████████████████████████████████████████████████████████████████████████| 44/44 [00:05<00:00,  8.49it/s] 
E:\anaconda3\envs\torch\lib\site-packages\scipy\optimize\_hessian_update_strategy.py:182: UserWarning: delta_grad == 0.0. Check if 
the approximated function is linear. If the function is linear better results can be obtained by defining the Hessian as zero instead of using quasi-Newton approximations.
  warn('delta_grad == 0.0. Check if the approximated '
 50%|█████████████████████████████████████████████                                             | 22/44 [00:00<00:00, 11028.41it/s]
Traceback (most recent call last):
  File ".\m2.py", line 20, in <module>
    result_df, _ = m2stitch.stitch_images(images, rows, cols)
  File "E:\anaconda3\envs\torch\lib\site-packages\m2stitch\stitching.py", line 131, in stitch_images
    grid = refine_translations(images, grid, r)
  File "E:\anaconda3\envs\torch\lib\site-packages\m2stitch\_constrained_refinement.py", line 101, in refine_translations
    int(g[f"{direction}_x_second"]),
ValueError: cannot convert float NaN to integer

My code

I tried many times but still failed, maybe my understanding of Example Code is wrong?

My test data is a 2x22 grid image(row snake scan) with an overlap of 10%, so I generated two lists to represent rows and cols:

rows = [1]*22+[2]*22
cols = list(range(1,23))+list(range(22,0,-1))

And my image is loaded as ndarray, which shape is (44, 512, 512), dtype is uint16

Then I used the interface and reported the initial error:

result_df, _ = m2stitch.stitch_images(images, rows, cols)

[yfukai]

Hi, @pmj110119, thank you for your investigation! What you're doing looks quite correct as far as I understand, and the error ValueError: cannot convert float NaN to integer is something unexpected and can be a bug.
Before proceeding, what we could try (maybe you already did) is to make sure that the dimension order is correct. This module (unfortunately with no good reason) assumes the dimension of the image is (position, x, y), which is a bit unintuitive, and changing col would change the position in the y direction. Are you assuming the same for your images?
(I now understand this is just the opposite of the usual view, but I'm hesitating to alter this as changing this order would break backward compatibility. I might flip this at some major revision with an appropriate warning.)

[yfukai]

If it would be the case and if you wouldn't mind, the contents of result_df, _ with full_output=True, (and if possible, the content of your image) would be helpful to check the source of this problem. If you're fine to send them privately, please contact me and I'll dig into the code when I have time ;-)

[pmj110119]

Thank you for your suggestions and helpfulness, your reply is so fast! But I tried to reverse row and col, and the result did not change.
Unfortunately, I can't run m2stitch.stitch_images() successfully now therefore there's no contents of result_df with full_output=True. Maybe you can have a try directly on my image if you have free time, I hope this won't disturb your life and work. 😢
( I don’t know how to contact you privately through github, so I create a temporary repository to store my data :-) )

[yfukai]

I'm sorry to be late @pmj110119, but I finally found the reason! the source of the problem was as follows:

• in the original implementation, the program uses the "normalized cross-correlation" (ncc) measure between the overlapping images and discards the estimates where ncc < 0.5, which was too high for your images.

I'll make this threshold adjustable (in a few days), and hopefully, let the user select the similarity measure in the future.
If you wouldn't mind, could I use your image in the repo for the automatic test, so that I can make sure that m2stitch always runs with it without errors?

[pmj110119]

If you wouldn't mind, could I use your image in the repo for the automatic test, so that I can make sure that m2stitch always runs with it without errors?

Thanks for your reply.
Of course, you can use my image do any thing.

[yfukai]

Unfortunately, the problem seems to be more complicated than I thought, and this week I might not be able to take time to solve it as I think I also should rewrite a part of the algorithm and refactor variables for clarity. Your image looks fine and I think it's important to make the program work for that case, so surely continue to work on it. Please be patient for a while...

[pmj110119]

That's fine. For my own project, I used alternative sub-optimal solutions for the time being. So you don't need to be too urgent because of me, just follow your rhythm. I will continue to pay attention and look forward to seeing this repository being improved in the future ;-)
Thank you for your contribution!

[HoukJangBNL]

Hi there,
I am having the same trouble.
Any chance to have a solution for this?

[yfukai]

Sorry, @HoukJangBNL, I started to work to improve the algorithm for more stability and easy-to-interpret errors, but unfortunately, my time to spend on this is limited, and it might take up to a month or so. Please be patient! You might also find ASHLAR useful.

[yfukai]

Hi, @HoukJangBNL and @pmj110119 I somehow updated the algorithm for more robust parameter findings.
Also now you can give an initial guess to the function to make use of the recorded stage position.
Still, I cannot take time for good documentation, but things may be clear from API or scripts in the tests/ folder. Please try them if you would feel like!

Note: I changed the input/output dimension for the sake of clarity. Still backward-compatible I believe but please check the example if you get stuck.

[yfukai]

I found some bugs, please use >=0.5.2 for getting correct results!