About Training Epochs or Iterations

[Rhysess]

Dear author,

Your work aims to find a smaller dataset. So I wonder how many epochs or iterations are needed compared to original training process with one dataset or all datasets?

[yorkeyao]

Thank you for your question. In our paper, for a fair comparison, we use the same network and the same hypsometer for two datasets.

But we also believe that for different datasets, the network should be adjusted accordingly like NAS to get the highest accuracy it can get using such a training set.

[Rhysess]

I'm sorry to bother you again, I just want to gain a better understanding of your experimental setup and related papers.

1. For pruned datasets, if you use the same number of epochs compared with original training, both the number of iterations and the overall training time will decrease; if you use the same number of iterations, the number of epochs needs to correspondingly increase, with the overall training time remaining unchanged. Which approach did you use?

2. For the second condition: if you use the same iterations, can a pruned dataset lead to overfitting?

[yorkeyao]

We use the same number of epochs compared with the original training. This is because we think generally we take the number of epochs rather than the number of iterations as a hyperparameter for a NN.

[Rhysess]

It's so kind of you to reply.

I have one more question:
which layer of IncepetionV3 did you use to extract features?
Which do you think is better, deep or shallow features extracted from pre-trained models?