Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Volume preserving transformer #136

Merged
merged 23 commits into from
Apr 23, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b93d9bf
Implemented volume-preserving transformer architecture.
benedict-96 Apr 22, 2024
6b1504f
Deleted inverse kernels (this is done by other files now).
benedict-96 Apr 22, 2024
622e160
The attention layer has been removed. Now we only have volume-preserv…
benedict-96 Apr 22, 2024
229599e
Added description of constructor and changed Integer -> Int.
benedict-96 Apr 22, 2024
00f0ab0
Got rid of Lux.
benedict-96 Apr 22, 2024
5e5b878
Iterate_Sympnet -> iterate.
benedict-96 Apr 22, 2024
84b4c74
Got rid of old scripts.
benedict-96 Apr 22, 2024
8cf306e
Added script from our rigid body paper.
benedict-96 Apr 22, 2024
f3a8d0b
routine for mapping network parameters from gpu to cpu.
benedict-96 Apr 22, 2024
1b42cb0
Made manifold layer work with tensors.
benedict-96 Apr 22, 2024
51656f0
This is now entirely replaces by losses.jl.
benedict-96 Apr 22, 2024
5a239d2
Made operations more legible.
benedict-96 Apr 22, 2024
2b8ceaa
Moved files around and added volume_preserving_transformer.
benedict-96 Apr 22, 2024
4d21ab4
Script that generates all the matrix inverses used for the volume-pre…
benedict-96 Apr 22, 2024
f242643
Added various packages needed for the volume-preserving transformer s…
benedict-96 Apr 22, 2024
a7b00ea
Removed duplicate operations.
benedict-96 Apr 22, 2024
8447c96
Various tests for the skew-symmetric matrix.
benedict-96 Apr 22, 2024
be560c7
Cleaned up file.
benedict-96 Apr 22, 2024
ef7c14e
Tests for the resnet layer.
benedict-96 Apr 22, 2024
2bef237
Included new tests.
benedict-96 Apr 22, 2024
ce45521
Adjusted tests to new interface.
benedict-96 Apr 22, 2024
0e67224
Added new tests for cayley transform and the skew-symmetric maps (nee…
benedict-96 Apr 22, 2024
a3d5a06
Renamed folder.
benedict-96 Apr 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions Project.toml
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,6 @@ GeometricEquations = "0.16"
GeometricIntegrators = "0.13"
HDF5 = "0.16, 0.17"
KernelAbstractions = "0.9"
Lux = "0.4, 0.5"
NNlib = "0.8, 0.9"
ProgressMeter = "1"
SafeTestsets = "0.1"
Expand All @@ -55,11 +54,10 @@ Aqua = "4c88cf16-eb10-579e-8560-4a9242c79595"
ChainRulesTestUtils = "cdddcdb0-9152-4a09-a978-84456f9df70a"
GeometricEquations = "c85262ba-a08a-430a-b926-d29770767bf2"
GeometricProblems = "18cb22b4-ad41-5c80-9c5f-710df63fbdc9"
Lux = "b2108857-7c20-44ae-9111-449ecde12c47"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
RungeKutta = "fb486d5c-30a0-4a8a-8415-a8b4ace5a6f7"
SafeTestsets = "1bc83da4-3b8d-516f-aca4-4fe02f6d838f"
Test = "8dfed614-e22c-5e08-85e1-65c5234f0b40"

[targets]
test = ["ChainRulesTestUtils", "Lux", "Random", "SafeTestsets", "Test"]
test = ["ChainRulesTestUtils", "Random", "SafeTestsets", "Test"]
21 changes: 21 additions & 0 deletions code_generation/matrixinverse.jl
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
using Symbolics

function matrix_inverse(n)
@variables A[1:n,1:n]

B = inv(collect(A))

if n ≤ 6
B = simplify.(B)
end

build_function(B, A)[2]
end

for n in 2:8
println("$(n)x$(n)...")
expr = matrix_inverse(n)
str = replace(replace(replace(string(expr), r"#= [^*\s]* =#" => ""), r"\n[\s]*\n" => "\n"), "Num" => "")
str = replace(str, "function (ˍ₋out, A)" => "function matinv$(n)x$(n)(ˍ₋out, A)")
write("matrixinverse_$(n)x$(n).jl", str)
end
4 changes: 1 addition & 3 deletions docs/src/tikz/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -22,8 +22,6 @@ convert_with_pdftocairo: $(MYDIR)/*.pdf
done

png:
pdftocairo -png -r 500 -transp -singlefile logo_with_name.pdf logo_with_name
pdftocairo -png -r 500 -transp -singlefile logo_with_name_dark.pdf logo_with_name_dark
pdftocairo -png -r $(res1) -transp -singlefile adam_optimizer.pdf adam_optimizer
pdftocairo -png -r $(res1) -transp -singlefile adam_optimizer_dark.pdf adam_optimizer_dark
pdftocairo -png -r $(res1) -transp -singlefile general_optimization.pdf general_optimization
Expand All @@ -36,9 +34,9 @@ png:
pdftocairo -png -r $(res2) -transp -singlefile sympnet_architecture_dark.pdf sympnet_architecture_dark
pdftocairo -png -r $(res3) -transp -singlefile structs_visualization.pdf structs_visualization
pdftocairo -png -r $(res3) -transp -singlefile structs_visualization_dark.pdf structs_visualization_dark
pdftocairo -png -r $(res1) -transp -singlefile logo.pdf logo
pdftocairo -png -r 500 -transp -singlefile logo_with_name.pdf logo_with_name
pdftocairo -png -r 500 -transp -singlefile logo_with_name_dark.pdf logo_with_name_dark
pdftocairo -png -r $(res1) -transp -singlefile logo.pdf logo
pdftocairo -png -r $(res1) -transp -singlefile symplectic_autoencoder.pdf symplectic_autoencoder
pdftocairo -png -r $(res1) -transp -singlefile symplectic_autoencoder_dark.pdf symplectic_autoencoder_dark
pdftocairo -png -r $(res1) -transp -singlefile solution_manifold_2.pdf solution_manifold_2
Expand Down
7 changes: 5 additions & 2 deletions docs/src/tutorials/sympnet_tutorial.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,7 +78,10 @@ H:(q,p)\in\mathbb{R}^2 \mapsto \frac{1}{2}p^2-cos(q) \in \mathbb{R}.
Here we generate pendulum data with the script `GeometricMachineLearning/scripts/pendulum.jl`:

```@example sympnet
using GeometricMachineLearning
using GeometricMachineLearning # hide
import Random # hide

Random.seed!(1234)

# load script
include("../../../scripts/pendulum.jl")
Expand Down Expand Up @@ -165,7 +168,7 @@ The train function will change the parameters of the neural networks and gives a

The trainings data `data_q` and `data_p` must be matrices of $\mathbb{R}^{n\times d}$ where $n$ is the length of data and $d$ is the half of the dimension of the system, i.e `data_q[i,j]` is $q_j(t_i)$ where $(t_1,...,t_n)$ are the corresponding time of the training data.

Then we can make prediction. Let's compare the initial data with a prediction starting from the same phase space point using the provided function Iterate_Sympnet:
Then we can make prediction. Let's compare the initial data with a prediction starting from the same phase space point using the provided function `iterate`:

```@example sympnet
ics = (q=qp_data.q[:,1], p=qp_data.p[:,1])
Expand Down
7 changes: 5 additions & 2 deletions scripts/Project.toml
Original file line number Diff line number Diff line change
@@ -1,10 +1,13 @@
[deps]
Flux = "587475ba-b771-5e3f-ad9e-33799f191a9c"
CUDA = "052768ef-5323-5732-b1bb-66c8b64840ba"
GeometricEquations = "c85262ba-a08a-430a-b926-d29770767bf2"
GeometricIntegrators = "dcce2d33-59f6-5b8d-9047-0defad88ae06"
GeometricMachineLearning = "194d25b2-d3f5-49f0-af24-c124f4aa80cc"
GeometricProblems = "18cb22b4-ad41-5c80-9c5f-710df63fbdc9"
GeometricSolutions = "7843afe4-64f4-4df4-9231-049495c56661"
LaTeXStrings = "b964fa9f-0449-5b57-a5c2-d3ea65f4040f"
LinearAlgebra = "37e2e46d-f89d-539d-b4ee-838fcccc9c8e"
Lux = "b2108857-7c20-44ae-9111-449ecde12c47"
Plots = "91a5bcdd-55d7-5caf-9e0b-520d859cae80"
ProfileView = "c46f51b8-102a-5cf2-8d2c-8597cb0e0da7"
Random = "9a3f8284-a2c9-5f02-9a11-845980a1fd5c"
Zygote = "e88e6eb3-aa80-5325-afca-941959d7151f"
81 changes: 0 additions & 81 deletions scripts/transformer_coupled_oscillator/auxiliary_functions.jl

This file was deleted.

68 changes: 0 additions & 68 deletions scripts/transformer_coupled_oscillator/generate_data.jl

This file was deleted.

This file was deleted.

71 changes: 0 additions & 71 deletions scripts/transformer_coupled_oscillator/train_transformer.jl

This file was deleted.

Loading
Loading