What is the split strategy when working with S3DIS data set?

[jerry3chen]

Hi.
According to paper, S3DIS data set is evaluate by 2 methods: eval on area 5 and 6-fold cross validation.
May I know how did you obtain the train/validate/test splits?
I assume test split is simply area 5 or whichever the area kept out in cross validation. But in which way did you split train/validate sets?

Thanks,
Jerry

[thangvubk]

S3DIS datasets consists of 6 Areas. eval on area 5 means training on Area 1, 2, 3, 4, 6 and eval on eval 5 (see here and here). Meanwhile, 6-fold cross validation training 6 independent model where each area is in turn being eval.

[jerry3chen]

Thanks for the explanation. So I guess that there is not explicit eval/test set split?

[thangvubk]

Yes. You are right.

[jerry3chen]

Last question: how exactly do you determine when to stop training? since there is not actual validation set, do you only stop when you reach the best result on test set(area5)?