TheBigPicture

DlangScience Design Considerations

A page for collecting ideas about the overall setup of the DlangScience project.

A bit of background: John Colvin's presentation "D is for Science" at DConf 2015:
https://www.youtube.com/watch?v=edjrSDjkfko.

The big picture (by @John-Colvin):

Ideas and discussion items

Notes by @John-Colvin's (2015-08-14):

There are many C++ libraries out there that provide very generic solutions to some problems. E.g. eigen can do advanced matrix manipulations and calculations for custom element types, see http://eigen.tuxfamily.org/dox/TopicCustomizingEigen.html#CustomScalarType it's more than just operator overloading, you tell it how to take a cosine etc.It's great that we have these very fast C and fortran libraries to call in to, but we don't want to be limited to only supporting the data types they support.
Lazy computations are great, we should support them as much as possible. Eager computation and allocation is the bane of numpy, matlab and even a lot of neat-looking C++.
Sparse matrices. To what extent do they need separate support? To what extent can they be an implementation detail with only a couple of enums exposed (e.g. sparse=true, storage=CSC?)
How big is too big? What are the boundaries of the project? E.g. do we include graph and network algorithms?
cerealD might be great for expressing file formats.
Dataframes: the weird midpoint between associative arrays and linear arrays. Lots of database operations. See pandas. Very convenient, often quite fast considering their relative complexity.
Dependencies: external dependencies (particularly non-dub) should probably be optional as