Speed up initial set up

[TalhaAsmal]

While running through a structured data test on an average sized dataset (560000 x 310) I noticed the initial set up (before training begins) was taking quite long. I traced it down to line 259 in auto_model.py, self._analyze_data(dataset).

I noticed basically each data point is checked for consistency and whether it's a categorical or numeric feature, and I couldn't help but feel there must be a better way than to sequentially iterate through each value in each row. Are there any plans to investigate speeding it up, perhaps making use of parallel processing or batch analysis?

[haifeng-jin]

It is not in our plan yet.
I think it is already running in parallel.
You are welcome to discuss it with us in the meeting and we can figure out a solution.