How to set the evaluation mode for traffic prediction? single or average

[aptx1231]

Traffic prediction is a multi-step prediction problem, and a common scheme is to predict the future 12 steps based on the historical 12 steps.

There are 2 evaluation methods for multistep forecasting, (1) reporting the average results of multiple steps and (2) reporting the results of each step individually. These two methods have been used extensively in past articles.

In LibCity you can choose between these two methods by setting the evaluation mode.
For (1) you need to set "mode": "average", assuming that there are 12 steps of prediction, in this mode, the 12 results obtained are average results, that is, the result in line 12 is the average result of the first 12 steps, line 11 is the average result of the first 11 steps, and so on, and line 1 is the average result of the first 1 step.
For (2) you need to set "mode": "single", assuming that there are 12 steps in the prediction, in this mode the 12 results obtained are the results of each step individually, i.e., the result in line 12 is the result of step 12, line 11 is the result of step 11, and so on, and line 1 is the result of step 1.

This parameter is configured in the file libcity/config/evaluator/TrafficStateEvaluator.json.

Results in average mode are usually better than in single mode because multi-step predictions usually have worse results at more distant time steps, so multi-step averaging is better than results for each step individually.

[aptx1231]

交通预测是一个多步预测问题，常见的方案是基于历史12步对未来12步进行预测。

对于多步预测有2种评价方法，（1）汇报多步的平均结果，（2）汇报单独每一步的结果。这两个方法在过去的文章中广泛的使用。

在LibCity中可以通过设置评价模式来选择这两种方法，
对于（1）你需要设置"mode": "average"，假设有12步的预测，这个模式下，得到的12个结果是平均结果，也就是第12行的结果是前12步的平均结果，第11行是前11步的平均结果，以此类推，第1行是前1步的平均结果。
对于（2）你需要设置"mode": "single"，假设有12步的预测，这个模式下，得到的12个结果是单独每一步的结果，也就是第12行的结果是第12步的结果，第11行是第11步的结果，以此类推，第1行是第1步的结果。

这个参数配置在文件libcity/config/evaluator/TrafficStateEvaluator.json中。

average模式下的结果通常比single模式下好，因为多步预测通常在更远的时间步的结果更差，因此多步平均要好于单独的每一步的结果。

[aptx1231]

similar problem:

BUAABIGSCity/PDFormer#7
BUAABIGSCity/PDFormer#13