Performance of v1 and v2 on metabolism variables

[jsadler2]

The question of the performance of v1 and v2 came up in the reviews. I did a little analysis on this. Below is a table of summary statistics of the NSE's for each of the sites with metabolism estimates for GPP.

	count	mean	std	min	25%	50%	75%	max
('BC_24', 'v1 - Process-Informed Multitask')	10	0.183483	0.041765	0.148315	0.155368	0.173863	0.184158	0.277129
('BC_24', 'v2 - Process-Dependent Multitask')	10	0.121001	0.240079	-0.333508	-0.031117	0.137663	0.247184	0.451938
('BC_40', 'v1 - Process-Informed Multitask')	10	-0.0016963	0.040878	-0.064888	-0.0323425	-0.0013305	0.027947	0.059083
('BC_40', 'v2 - Process-Dependent Multitask')	10	0.0858435	0.120403	-0.101917	0.0097475	0.0528945	0.172728	0.268014
('BC_53', 'v1 - Process-Informed Multitask')	10	-0.129638	0.0973996	-0.323575	-0.168624	-0.116805	-0.078289	0.040102
('BC_53', 'v2 - Process-Dependent Multitask')	10	-0.111172	0.396592	-0.687346	-0.412103	-0.171459	0.199471	0.5041
('SR_40', 'v1 - Process-Informed Multitask')	10	-0.42923	0.173224	-0.76964	-0.528927	-0.394296	-0.317387	-0.160238
('SR_40', 'v2 - Process-Dependent Multitask')	10	-0.561778	0.320169	-1.0773	-0.708186	-0.482284	-0.338313	-0.213263

And ER:

	count	mean	std	min	25%	50%	75%	max
('BC_24', 'v1 - Process-Informed Multitask')	10	0.379391	0.042008	0.292413	0.363698	0.375451	0.405685	0.447272
('BC_24', 'v2 - Process-Dependent Multitask')	10	0.265132	0.18703	-0.12138	0.174873	0.248651	0.419316	0.487525
('BC_40', 'v1 - Process-Informed Multitask')	10	-0.0487092	0.0424984	-0.114341	-0.070662	-0.058703	-0.0354472	0.031145
('BC_40', 'v2 - Process-Dependent Multitask')	10	-0.0402173	0.0880268	-0.158404	-0.113148	-0.0345635	0.020053	0.089052
('BC_53', 'v1 - Process-Informed Multitask')	10	-0.0912459	0.21411	-0.319095	-0.268808	-0.096497	-0.0248843	0.380792
('BC_53', 'v2 - Process-Dependent Multitask')	10	-0.412079	1.0233	-2.56067	-0.938191	-0.0640765	0.322946	0.490124
('SR_40', 'v1 - Process-Informed Multitask')	10	0.104795	0.0148806	0.085017	0.0922185	0.103179	0.117831	0.125553
('SR_40', 'v2 - Process-Dependent Multitask')	10	0.0241732	0.10121	-0.155439	-0.0464302	0.068531	0.0946857	0.131504

Here are some example predictions at two of the sites (BC_24 which had the highest metrics for GPP and ER, and BC_53 and SR_40 which were the worst at predicting ER and GPP, respectively). I included all 10 replicates because they vary quite a lot replicate to replicate:

[jsadler2]

Overall it seems like most of them get the trends pretty well, but they don't go after the extremes at all.

Another interesting thing is that in some of the V2 models, (e.g., replicate 2 for ER and 6 for GPP) the trends are backwards. What I'm pretty sure happened is that the random starting weights were set in a way that the algorithm settled on having the sign flipped in the second dense layer so that the output of the GPP/ER was actually the negative of what it should be. It seems like the loss in performance for that one variable wasn't enough for the algorithm to change the weights enough to flip the sign to the correct one.