Skip to content

Commit

Permalink
Fix figure paths
Browse files Browse the repository at this point in the history
  • Loading branch information
ludwigbothmann authored and github-actions[bot] committed Jan 20, 2025
1 parent 48c487c commit d00b64a
Show file tree
Hide file tree
Showing 6 changed files with 77 additions and 77 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -43,8 +43,8 @@
\vspace{-0.4cm}
GD with medium $\alpha=2\cdot10^{-3}$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_reg_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_reg_coef_med.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_reg_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_reg_coef_med.pdf}\\
\begin{footnotesize}
Irreducible error due to additive noise is $\sigma=1$. Dotted lines indicate global minimizers.
\end{footnotesize}
Expand All @@ -56,8 +56,8 @@
\vspace{-0.4cm}
GD with medium $\alpha=2\cdot10^{-3}$ and bad conditioning (corr. features):
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_reg_med_lr_corr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_reg_coef_med_corr.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_reg_med_lr_corr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_reg_coef_med_corr.pdf}\\
\begin{footnotesize}
Irreducible error due to additive noise is $\sigma=1$. Dotted lines indicate global minimizers.
\end{footnotesize}
Expand All @@ -72,8 +72,8 @@
\vspace{-0.4cm}
GD with (too small) $\alpha=3\cdot10^{-4}$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_reg_small_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_reg_coef_small.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_reg_small_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_reg_coef_small.pdf}\\
\begin{footnotesize}
Irreducible error due to additive noise is $\sigma=1$. Dotted lines indicate global minimizers.
\end{footnotesize}
Expand All @@ -85,8 +85,8 @@
\vspace{-0.5cm}
GD with large $\alpha=1.5$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_reg_large_lr_iters.pdf} \\
\includegraphics[width=0.7\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_reg_coef_large.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_reg_large_lr_iters.pdf} \\
\includegraphics[width=0.7\textwidth]{figure_man/simu_linmod/GD_reg_coef_large.pdf}\\
\begin{footnotesize}
Irreducible error due to additive noise is $\sigma=1$. Dotted lines indicate global minimizers.
\end{footnotesize}
Expand All @@ -101,8 +101,8 @@
\vspace{-0.4cm}
SGD with medium $\alpha=2\cdot10^{-3}$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_reg_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_reg_coef_med.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_reg_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_reg_coef_med.pdf}\\
\begin{footnotesize}
%Irreducible error due to additive noise is $\sigma=1$
\end{footnotesize}
Expand All @@ -115,8 +115,8 @@
\vspace{-0.4cm}
SGD with medium $\alpha=2\cdot10^{-3}$ and bad conditioning (corr. features):
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_reg_med_lr_corr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_reg_coef_med_corr.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_reg_med_lr_corr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_reg_coef_med_corr.pdf}\\
\begin{footnotesize}
Irreducible error due to additive noise is $\sigma=1$
\end{footnotesize}
Expand All @@ -130,8 +130,8 @@
\vspace{-0.4cm}
SGD with small $\alpha=3\cdot10^{-4}$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_reg_small_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_reg_coef_small.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_reg_small_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_reg_coef_small.pdf}\\
\begin{footnotesize}
Irreducible error due to additive noise is $\sigma=1$
\end{footnotesize}
Expand All @@ -145,8 +145,8 @@
\vspace{-0.4cm}
SGD with large $\alpha=1 \cdot 10^{-2}$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_reg_large_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_reg_coef_large.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_reg_large_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_reg_coef_large.pdf}\\
\begin{footnotesize}
Irreducible error due to additive noise is $\sigma=1$
\end{footnotesize}
Expand Down Expand Up @@ -246,8 +246,8 @@
\vspace{-0.4cm}
GD with medium $\alpha=0.25$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_coef_med.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_coef_med.pdf}\\
\begin{footnotesize}
Dotted lines indicate global minimizers.
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
Expand All @@ -261,8 +261,8 @@
\vspace{-0.4cm}
GD with medium $\alpha=0.25$ and bad conditioning (corr. features):
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_med_lr_corr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_coef_med_corr.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_med_lr_corr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_coef_med_corr.pdf}\\
\begin{footnotesize}
Dotted lines indicate global minimizers.
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
Expand All @@ -277,8 +277,8 @@
\vspace{-0.4cm}
GD with small $\alpha=0.025$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_small_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_coef_small.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_small_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_coef_small.pdf}\\
\begin{footnotesize}
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
\end{footnotesize}
Expand All @@ -292,8 +292,8 @@
\vspace{-0.5cm}
GD with large $\alpha=10$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_large_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_coef_large.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_large_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_coef_large.pdf}\\
\begin{footnotesize}
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
\end{footnotesize}
Expand All @@ -309,8 +309,8 @@
\vspace{-0.4cm}
SGD with medium $\alpha=0.03$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_log_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_log_coef_med.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_log_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_log_coef_med.pdf}\\
\begin{footnotesize}
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
\end{footnotesize}
Expand All @@ -323,8 +323,8 @@
\vspace{-0.4cm}
SGD with medium $\alpha=5\cdot10^{-2}$ and bad conditioning (corr. features):
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_log_med_lr_corr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_log_coef_med_corr.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_log_med_lr_corr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_log_coef_med_corr.pdf}\\
\begin{footnotesize}
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
\end{footnotesize}
Expand All @@ -338,8 +338,8 @@
\vspace{-0.4cm}
SGD with small $\alpha=3\cdot10^{-4}$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_log_small_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_log_coef_small.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_log_small_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_log_coef_small.pdf}\\
\begin{footnotesize}
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
\end{footnotesize}
Expand All @@ -353,8 +353,8 @@
\vspace{-0.4cm}
SGD with large $\alpha=0.3$ and indep. features:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_log_large_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/SGD_log_coef_large.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_log_large_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/SGD_log_coef_large.pdf}\\
\begin{footnotesize}
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
\end{footnotesize}
Expand Down Expand Up @@ -383,7 +383,7 @@
\vspace{-0.3cm}
Mini-batch SGD with $\alpha \in \{0.01, 0.1\}$ for 100 epochs (5000 iterations):
\begin{figure}
\includegraphics[width=1.0\textwidth]{slides/04-multivariate-first-order/figure_man/simu_mnist/SGD_compar.pdf} \\
\includegraphics[width=1.0\textwidth]{figure_man/simu_mnist/SGD_compar.pdf} \\
%\begin{footnotesize}
% Irreducible error due to additive noise is $\sigma=1$
%\end{footnotesize}
Expand All @@ -404,7 +404,7 @@
\vspace{-0.2cm}
Why is it not a good idea to use GD in most DL applications? SGD is much faster. Compare runtime of mini-batch SGD (batch size=$100$) with GD (constant $\alpha=0.01$ without momentum for $t_{\text{max}}=5000$ iterations):
\begin{figure}
\includegraphics[width=1.0\textwidth]{slides/04-multivariate-first-order/figure_man/simu_mnist/SGD_GD_compar.pdf} \\
\includegraphics[width=1.0\textwidth]{figure_man/simu_mnist/SGD_GD_compar.pdf} \\
%\begin{footnotesize}
% Irreducible error due to additive noise is $\sigma=1$
%\end{footnotesize}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -89,8 +89,8 @@
\vspace{-0.3cm}
Recall comparison of GD variants on log. reg. in last chapter:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/04-multivariate-first-order/figure_man/simu_linmod/GD_log_coef_med.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_med_lr_iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu_linmod/GD_log_coef_med.pdf}\\
\begin{footnotesize}
Dotted lines indicate global minimizers.
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
Expand All @@ -105,8 +105,8 @@
\vspace{-0.4cm}
Let's run GD vs. NR for \textbf{$1000$ steps} (independent features):
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/05-multivariate-second-order/figure_man/simu-newton/NR_GD_log_indep_1000iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/05-multivariate-second-order/figure_man/simu-newton/NR_GD_log_coef_1000indep.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu-newton/NR_GD_log_indep_1000iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu-newton/NR_GD_log_coef_1000indep.pdf}\\
\begin{footnotesize}
Dotted lines indicate global minimizers.
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
Expand All @@ -120,8 +120,8 @@
\vspace{-0.4cm}
Let's run the same configuration only for \textbf{$50$ steps} to see clearer picture:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/05-multivariate-second-order/figure_man/simu-newton/NR_GD_log_indep_50iters.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/05-multivariate-second-order/figure_man/simu-newton/NR_GD_log_coef_50indep.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu-newton/NR_GD_log_indep_50iters.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu-newton/NR_GD_log_coef_50indep.pdf}\\
\begin{footnotesize}
Dotted lines indicate global minimizers.
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
Expand All @@ -136,7 +136,7 @@
{\small Clearly, NR makes more progress than GD per iteration. OTOH Newton steps are much more expensive than GD updates\\
$\Rightarrow$ How do NR and GD compare wrt runtime instead of iterations (50 steps)?}
\begin{figure}
\includegraphics[width=1.0\textwidth]{slides/05-multivariate-second-order/figure_man/simu-newton/NR_GD_runtime_comparison.pdf} \\
\includegraphics[width=1.0\textwidth]{figure_man/simu-newton/NR_GD_runtime_comparison.pdf} \\
%\begin{footnotesize}
% Irreducible error due to additive noise is $\sigma=1$
%\end{footnotesize}
Expand All @@ -150,8 +150,8 @@
\vspace{-0.4cm}
In case of correlated features the results are very similar:
\begin{figure}
\includegraphics[width=0.8\textwidth]{slides/05-multivariate-second-order/figure_man/simu-newton/NR_GD_log_indep_50iters_corr.pdf} \\
\includegraphics[width=0.8\textwidth]{slides/05-multivariate-second-order/figure_man/simu-newton/NR_GD_log_coef_50indep_corr.pdf}\\
\includegraphics[width=0.8\textwidth]{figure_man/simu-newton/NR_GD_log_indep_50iters_corr.pdf} \\
\includegraphics[width=0.8\textwidth]{figure_man/simu-newton/NR_GD_log_coef_50indep_corr.pdf}\\
\begin{footnotesize}
Dotted lines indicate global minimizers.
%Dashed line in test loss indicates irreducible error due to $\sigma=1$
Expand All @@ -166,7 +166,7 @@
\bigskip
\begin{figure}
\includegraphics[width=1.0\textwidth]{slides/05-multivariate-second-order/figure_man/simu-newton/NR_GD_runtime_comparison_corr.pdf} \\
\includegraphics[width=1.0\textwidth]{figure_man/simu-newton/NR_GD_runtime_comparison_corr.pdf} \\
%\begin{footnotesize}
% Irreducible error due to additive noise is $\sigma=1$
%\end{footnotesize}
Expand Down
Loading

0 comments on commit d00b64a

Please sign in to comment.