LSQ Rows info

[cortner]

For train-test splits on pre-assembled lsq systems it is useful to have access to which rows in the design matrix correspond to individual training structures. At the moment I do this:

   # copy-pasted from the assembly routine
   lsqrows = Array{UnitRange}(undef, length(_data))
   lsqrows[1] = 1:count_observations(_data[1])
   for i in 2:length(_data)
      lsqrows[i] = lsqrows[i - 1][end] .+ (1:count_observations(_data[i]))
   end

This is obviously a hack. Is it safe for now? Could something like that become part of the interface? Maybe it could be returned with the assembly routine?

[wcwitt]

This is a good idea in general. Would it be sufficient to build routines into a revamped LSQ database that provide this information?

[cortner]

Yes I would very much prefer that. It just turns out this was a pretty convenient thing to have after all.

What I would really like though is something a little broader - something like a "lazy learning task" that knows the model, the dataset, will assemble things on demand - store them for re-use if allowed, etc ... Can manage many fits, parameter scanning etc...

[cortner]

But as a quick fix just for now, can we maybe spin out that part of the assembly routine into a separate function that I can call so I can be certain the ordering will be the same?