Regarding GraphBEV, my results are off by 20 points compared to the paper's reported results.

[fdy61]

Without adding noise, the reported results in the GraphBEV paper are mAP and NDS of 70.1 and 72.9, respectively. However, my results are 45 and 52. I use the config file of bevfusion_graph_deformable.yaml as you said.

[modaxiansheng]

I doubt the noise problem. Next you can train and test bevfusion
_graph.yaml

[fdy61]

是GlobalAlign的问题吗？bevfusion_graph_deformable.yaml和bevfusion_graph.yaml仅仅在Fuser有区别，分别是GlobalAlign和bevfusion的Confuser。GraphBEV的GlobalAlign.py里面我看到你定义了一个calculate_loss，但是并没有用来计算deformed_feature和mm_bev的loss，流程似乎也和文章里的有点出入，是还没有更新代码吗？

[modaxiansheng]

I haven't updated the latest code yet.

…

---Original--- From: ***@***.***> Date: Wed, Aug 7, 2024 10:38 AM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [adept-thu/RoboFusion] Regarding GraphBEV, my results are off by20 points compared to the paper's reported results. (Issue #4) 是GlobalAlign的问题吗？bevfusion_graph_deformable.yaml和bevfusion_graph.yaml仅仅在Fuser有区别，分别是GlobalAlign和bevfusion的Confuser。GraphBEV的GlobalAlign.py里面我看到你定义了一个calculate_loss，但是并没有用来计算deformed_feature和mm_bev的loss，流程似乎也和文章里的有点出入，是还没有更新代码吗？ — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[fdy61]

So I just use the bevfusion_graph.yaml or bevfusion.yaml and it will be OK?

[modaxiansheng]

You can try bevfusion_graph first, test clean and noisy setting.

[modaxiansheng]

Yes, if you have any questions, I will reply in time.

[fdy61]

OK, I'll try. Thanks you very much.

[fdy61]

Hello, the mAP and NDS of training bevfusion_graph on the clean set for 10 epochs are 62.97 and 66.4, respectively, whereas in the paper, they are reported as 69.7 and 72.4.

[modaxiansheng]

Please add a pre-trained model，bevfuaion.

[modaxiansheng]

Please train openpcdet's bevfusion to ensure that the config is correct and see what the indicators are.

[modaxiansheng]

Take a look at the clean and noise metrics change for bevfusion.yaml, then add bevfusion_graph.yaml to see the clean and noise changes. Remember to add pre-training bevfusion.

[fdy61]

Please add a pre-trained model，bevfuaion.

Oh, I forget to train the lidar-branch pretrain model, do you mean it?

[modaxiansheng]

Yes, you can also use https://github.com/open-mmlab/OpenPCDet/blob/master/docs/guidelines_of_approaches/bevfusion.md#performance

[zw-cell]

是的，您也可以使用https://github.com/open-mmlab/OpenPCDet/blob/master/docs/guidelines_of_approaches/bevfusion.md#performance

Do you have a training log over there for GraphBEV

[zw-cell]

Please add a pre-trained model，bevfuaion.

Oh, I forget to train the lidar-branch pretrain model, do you mean it?

Did you succeed？

[fdy61]

Please add a pre-trained model，bevfuaion.

Oh, I forget to train the lidar-branch pretrain model, do you mean it?

Did you succeed？

I'm working on it.

[zw-cell]

Please add a pre-trained model，bevfuaion.

Oh, I forget to train the lidar-branch pretrain model, do you mean it?

Did you succeed？

I'm working on it.

If your results are correct, can you send your checkpoints？

[modaxiansheng]

https://pan.baidu.com/s/1vibPImyEsM4wKy5OlpozJg?pwd=m1pi
We provided logs trained on the 1/4 Nuscenes dataset to compare baseline(bevfusion ) and bevfusion_graph.

[zw-cell]

https://pan.baidu.com/s/1vibPImyEsM4wKy5OlpozJg?pwd=m1pi We provided logs trained on the 1/4 Nuscenes dataset to compare baseline(bevfusion ) and bevfusion_graph.

Are there logs trained on all datasets？

[zw-cell]

Please add a pre-trained model，bevfuaion.

Oh, I forget to train the lidar-branch pretrain model, do you mean it?

Did you succeed？

I'm working on it.

can you send your checkpoints？

[fdy61]

Please add a pre-trained model，bevfuaion.

Oh, I forget to train the lidar-branch pretrain model, do you mean it?

Did you succeed？

I'm working on it.

can you send your checkpoints？
I haven't run the bevfusion_graph_deformable part.

[zw-cell]

Please add a pre-trained model，bevfuaion.

Oh, I forget to train the lidar-branch pretrain model, do you mean it?

Did you succeed？

I'm working on it.

can you send your checkpoints？
I haven't run the bevfusion_graph_deformable part.

Is bevfusion_graph OK？

[fdy61]

I just verify the bevfusion and it's OK, because I have met problems in bevfusion_graph_deformable. It seems that the bevfusion_graph_deformable haven't been fully upgraded. You can try the bevfusion_graph and make sure you add the LiDAR-branch pretrain model.

[fdy61]

I have try the bevfusion_graph earlier but the results are few points lower than the paper, because I forget to add the LiDAR-branch pretrained model. If you add it on, I think it will be OK.

[zw-cell]

https://pan.baidu.com/s/1vibPImyEsM4wKy5OlpozJg?pwd=m1pi 我们提供了在 1/4 Nuscenes 数据集上训练的日志，以比较基线（bevfusion）和 bevfusion_graph。

Do you have logs and checkpoints for training on the full Nuscenes dataset

[zw-cell]

https://pan.baidu.com/s/1vibPImyEsM4wKy5OlpozJg?pwd=m1pi We provided logs trained on the 1/4 Nuscenes dataset to compare baseline(bevfusion ) and bevfusion_graph.

used the entire dataset to train 10 epochs with bevfusion_graph.yaml and added pre-training weights for the Lidar branch, but evaluated the result as MAP: 64.7 NDS: 69.49，the reported results in the GraphBEV paper are mAP and NDS of 70.1 and 72.9,

[zw-cell]

我刚刚验证了 bevfusion，没问题，因为我在 bevfusion_graph_deformable 中遇到了问题。看来 bevfusion_graph_deformable 还没有完全升级。你可以试试 bevfusion_graph，并确保添加了 LiDAR-branch 预训练模型。

Hi, can I have a look at the bevfusion log? I repeat that there is a problem with bevfusion

[zw-cell]

I just verify the bevfusion and it's OK, because I have met problems in bevfusion_graph_deformable. It seems that the bevfusion_graph_deformable haven't been fully upgraded. You can try the bevfusion_graph and make sure you add the LiDAR-branch pretrain model.

Did you add LiDAR-branch when regenerating bevfusion?