Fit segments

[Bill-Ren]

Thanks for your great work! I downloaded the test result "Webis-WebSeg-20-Algorithm-Segmentations" you posted on the website and tried to verify it. For the results of mmdetection, the indicators I got according to the "segmentations-mmdetection-unfitted.json" and "segmentations-mmdetection.json" tests are all right , which is consistent with the data in the "evaluation.csv". But when I try to use the fit segments method and try to change the "segmentations-mmdetection-unfitted.json" into data after fit,

# Fit segments Rscript cikm20/src/main/r/fit-segmentations-to-dom-nodes.R \ --input segmentations/mmdetection/segmentations-mmdetection-unfitted.json \ --segmentations mmdetection_segms \ --nodes webis-webseg-20/000000/nodes.csv \ --output segmentations/mmdetection_fitted.json

I find that the "mmdetection-fitted.json" data I get is inconsistent with the "segmentations-mmdetection.json" you gave. I have tried several times This problem is found in each sample, what is the reason for this? I didn't make any changes to the code of fit-segments, and I used the default values.

[johanneskiesel]

Thank you for your interest!

Can you be more specific and provide an example (the input files, output file you get, and output file you expect)? In which way is the data "inconsistent"?

Have you already plotted the output files to see whether the output files indeed differ?

Thanks again!

[Bill-Ren]

Thank you for answering my question so quickly. Sorry my previous description might have been a bit unclear. I downloaded the experimental results you provided from the dataset website you(https://zenodo.org/record/4337578) gave, the name is "Webis-WebSeg-20-Algorithm-Segmentations".

After I unzip this folder, I verify the picture numbered 000000, and I focus on the result of mmdetection.

I use the code you provided to fit segments on this "segmentations-mmdetection-unfitted.json".

Fit segments Rscript cikm20/src/main/r/fit-segmentations-to-dom-nodes.R \ --input segmentations/mmdetection/segmentations-mmdetection-unfitted.json \ --segmentations mmdetection_segms \ --nodes webis-webseg-20/000000/nodes.csv \ --output segmentations/mmdetection_fitted.json

The "mmdetection-fitted.json" data I got is different from the "segmentations-mmdetection.json" you provided. But I didn't do any other operations on the code, so the results of the "mmdetection-fitted.json" should be the same as "segmentations-mmdetection.json". And I evaluate the two results, for pixels as atomic elements.

For "mmdetection-fitted.json" ，which I got from "segmentations-mmdetection-unfitted.json", the "bcubed.precision","bcubed.recall","bcubed.f1" are 0.539,0.136,0.217. Compared with unfit “segmentations-mmdetection-unfitted.json”, it even dropped(the "bcubed.precision","bcubed.recall","bcubed.f1" are 0.577,0.185,0.281.).

For "segmentations-mmdetection.json" you provide, the "bcubed.precision","bcubed.recall","bcubed.f1" are 0.709,0.276,0.397.

I also verified the data for other numbers and found the same problem. So I'm curious, did you adjust other parameters in the fit segments step, because you mentioned in the paper: “we fit the resulting pixel mask segmentation to DOM nodes, which results in performance increases up to 0.12 in FB3∗”.

Follow your suggestion,I plotted the output files. Compared with the fitted json you gave, the segmentation result I obtained according to the algorithm ignores the image area.

Finally, can you provide your email address, it will be more convenient to contact by email. Thanks again for your patience

[Bill-Ren]

Sorry, I uploaded the first image wrongly, the result I got should be as below, under-segmented in the area where the image appears.

For "mmdetection-fitted.json" ，which I got from "segmentations-mmdetection-unfitted.json", the "bcubed.precision","bcubed.recall","bcubed.f1" are 0.539,0.136,0.217.