Removing some heavier dependencies

[sethaxen]

On Julia 1.8-beta3, even after the refactor into sub-packages, it takes ~1.8s to load GalacticOptim on my machine, which is quite heavy for an interface package:

julia> @time_imports using GalacticOptim
     11.0 ms    ┌ MacroTools
     18.6 ms  ┌ ZygoteRules
      0.1 ms    ┌ IteratorInterfaceExtensions
      0.5 ms  ┌ TableTraits
    502.0 ms    ┌ StaticArrays
    504.4 ms  ┌ DiffResults
      1.7 ms  ┌ Compat
      6.5 ms  ┌ OrderedCollections
      0.5 ms  ┌ Requires
    156.8 ms  ┌ FillArrays
      0.3 ms  ┌ DataValueInterfaces
      3.1 ms    ┌ DocStringExtensions
      0.2 ms    ┌ IfElse
     22.1 ms    ┌ RecipesBase
     38.6 ms      ┌ Static
    598.0 ms    ┌ ArrayInterface
      0.5 ms    ┌ Adapt
     61.9 ms    ┌ ChainRulesCore
    755.3 ms  ┌ RecursiveArrayTools
      0.3 ms  ┌ Reexport
      5.9 ms    ┌ AbstractTrees
      2.8 ms    ┌ ProgressLogging
      5.4 ms    ┌ LeftChildRightSiblingTrees
     15.4 ms  ┌ TerminalLoggers
      1.1 ms      ┌ DataAPI
     13.5 ms    ┌ Tables
      0.3 ms    ┌ CommonSolve
      0.8 ms    ┌ ConstructionBase
      0.2 ms    ┌ TreeViews
    256.1 ms  ┌ SciMLBase
      4.9 ms  ┌ ProgressMeter
      2.1 ms  ┌ LoggingExtras
      0.8 ms  ┌ ConsoleProgressMonitor
   1750.9 ms  GalacticOptim

The main culprits are:

• DiffResults (0.5s): this seems to only be used after conditionally loading packages that themselves depend on DiffResults, so this could be removed.
• FillArrays (0.2s): I have no idea why this is being loaded
• RecursiveArrayTools (0.8s): I also don't know why this is being loaded
  • ArrayInterface (0.6s): This is only being loaded to call ArrayInterface.zeromatrix. Is that functionality simple enough to duplicate here and drop the dependency?
• SciMLBase (0.3s): kind of necessary

[Vaibhavdixit02]

This is great, and indeed some of these could be removed, I'll look into this and clean it up. Thank you for looking into it

[ChrisRackauckas]

DiffResults (0.5s): this seems to only be used after conditionally loading packages that themselves depend on DiffResults, so this could be removed.

Yes this can be moved into the ForwardDiff stuff. I think we still really need to look into making the AD packages extension libraries too, measuring the startup time of Requires.

FillArrays (0.2s): I have no idea why this is being loaded

Same. Probably can be straight deleted.

RecursiveArrayTools (0.8s): I also don't know why this is being loaded

Same, looks like it's stale.

ArrayInterface (0.6s): This is only being loaded to call ArrayInterface.zeromatrix. Is that functionality simple enough to duplicate here and drop the dependency?

Note quite, since that is overloaded by some arrays as well. Though in a pinch we can use u .* u' .* false to get something similar in "most" cases. But we should just finish ArrayInterfaceCore ASAP instead.

[sethaxen]

FillArrays (0.2s): I have no idea why this is being loaded

Same. Probably can be straight deleted.

RecursiveArrayTools (0.8s): I also don't know why this is being loaded

Same, looks like it's stale.

Neither of these are direct dependencies of this package, but RecursiveArrayTools is a SciMLBase dependency, and FillArrays is a RecursiveArrayTools dependency.

But we should just finish ArrayInterfaceCore ASAP instead.

Ah, cool, I didn't know this was happening!

[ChrisRackauckas]

JuliaArrays/ArrayInterface.jl#211
JuliaArrays/ArrayInterface.jl#263

[Tokazama]

I can't make any promises yet, but it looks like we could get ArrayInterfaceCore below 0.03 seconds if we can get help from Base julia on invalidations related to StaticInt.

[sethaxen]

Now that ArrayInterfaceCore is finished, I just re-ran @time_imports on master:

julia> @time_imports using Optimization
   ...
   1589.7 ms  Optimization

This is just barely more than SciMLBase:

julia> @time_imports using SciMLBase
   ...
   1551.4 ms  SciMLBase

So it seems there is little more to do here in this package; Except for removing DiffResults here, reduction of dependencies mostly needs to happen at SciMLBase or its dependencies.

[ChrisRackauckas]

The issues there are JuliaGPU/GPUArrays.jl#409 and JuliaArrays/StaticArrays.jl#1023 (comment) which amounts to approximately 1000ms and 500 ms respectively from GPUArrays.jl and StaticArrays.jl. We need those authors to do something, and have demonstrated exactly what needs to be done.

[sethaxen]

Thanks for the update!

[ChrisRackauckas]

A lot precompiles away now, especially on v1.8. And RecursiveArrayTools + SciMLBase were made lean, so how is it today?

julia> @time_imports using Optimization
      1.7 ms  ┌ TerminalLoggers
      6.3 ms  ┌ ProgressLogging
      9.4 ms  ┌ ProgressMeter
      6.1 ms  ┌ LoggingExtras
      2.1 ms  ┌ ConsoleProgressMonitor
    131.5 ms  Optimization

Boom.

[sethaxen]

A lot precompiles away now, especially on v1.8. And RecursiveArrayTools + SciMLBase were made lean, so how is it today?

Oddly, while for me the load time onn master is much faster than it was before, it still takes 6.5x longer than in your example and loads in more dependencies:

julia> @time_imports using Optimization
     15.6 ms    ┌ MacroTools
     26.5 ms  ┌ ZygoteRules
      0.2 ms    ┌ IteratorInterfaceExtensions
      0.5 ms  ┌ TableTraits
      2.6 ms      ┌ StaticArraysCore
    399.3 ms    ┌ StaticArrays
    401.2 ms  ┌ DiffResults
      0.5 ms  ┌ Compat
      4.6 ms  ┌ OrderedCollections
      0.3 ms  ┌ Requires
      0.3 ms    ┌ Adapt
      1.3 ms    ┌ ArrayInterfaceCore
      1.9 ms  ┌ ArrayInterfaceStaticArraysCore
    109.1 ms  ┌ FillArrays
      0.1 ms  ┌ DataValueInterfaces
      2.4 ms    ┌ DocStringExtensions
     15.0 ms    ┌ RecipesBase
     43.1 ms    ┌ ChainRulesCore
      2.0 ms    ┌ GPUArraysCore
     80.5 ms  ┌ RecursiveArrayTools
      0.1 ms  ┌ Reexport
      4.0 ms    ┌ AbstractTrees
      2.0 ms    ┌ ProgressLogging
      3.5 ms    ┌ LeftChildRightSiblingTrees
     10.2 ms  ┌ TerminalLoggers
      0.7 ms      ┌ DataAPI
      9.3 ms    ┌ Tables
      0.1 ms    ┌ CommonSolve
      0.5 ms    ┌ ConstructionBase
      0.1 ms    ┌ TreeViews
    191.7 ms  ┌ SciMLBase
      3.3 ms  ┌ ProgressMeter
      1.7 ms  ┌ LoggingExtras
      0.5 ms  ┌ ConsoleProgressMonitor
    857.9 ms  Optimization

(jl_k1UwZ5) pkg> st
Status `/tmp/jl_k1UwZ5/Project.toml`
  [7f7a1694] Optimization v3.7.1 `https://github.com/SciML/Optimization.jl.git#master`

julia> versioninfo()
Julia Version 1.8.0-rc1
Commit 6368fdc656 (2022-05-27 18:33 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: 24 × AMD Ryzen 9 5900X 12-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, znver3)
  Threads: 8 on 24 virtual cores

[ChrisRackauckas]

julia> versioninfo()
Julia Version 1.8.0-rc1
Commit 6368fdc656 (2022-05-27 18:33 UTC)
Platform Info:
  OS: Windows (x86_64-w64-mingw32)
  CPU: 32 × AMD Ryzen 9 5950X 16-Core Processor
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, znver3)
  Threads: 1 on 32 virtual cores
Environment:
  JULIA_EDITOR = code
  JULIA_NUM_THREADS =

I think there's something odd with mine, since it wasn't showing all of the dependencies I expected. But I ran it 3 times in new REPLs, so 😅 . But yeah, it's much faster.

And DiffResults can be removed.