Rachaelj #68
Rachaelj #68
Conversation
| for j in range(Ytrain.shape[1]): | ||
| x=linear_model.Lasso(alpha=a).fit(Xtrain, Ytrain[:, j]) | ||
| mdl.append(x) | ||
| ypred = mdl[i].predict(Xtest) |
There was a problem hiding this comment.
I would move this code to the inner loop and predict with x instead - the reason is because if you evaluate with mdl[i], you're not evaluating all the models you just trained. Instead, if you're on the first alpha, you'll only train 1 model - mdl[0].
| x=linear_model.Lasso(alpha=a).fit(Xtrain, Ytrain[:, j]) | ||
| mdl.append(x) | ||
| ypred = mdl[i].predict(Xtest) | ||
| error.append(mean_squared_error(ypred,Ytest[:, j])) |
There was a problem hiding this comment.
Same thing here for the same exact reason.
There was a problem hiding this comment.
Then to save on memory, write some code to update your model list. The logic is the following:
- Evaluate models with first alpha
- On the next iteration, save your models and errors in another object
- Write code that will see if the MSE in the current alpha level is less than the MSE in the previous alpha level
- If it is, replace the model object in your original list, and update the alpha associated with this model
- Otherwise, don't update and continue
| print(error) | ||
| minmse_index = error.index(min(error)) | ||
| best_alpha = alphas[minmse_index] | ||
| print(best_alpha) |
There was a problem hiding this comment.
This is why all of your models have a single alpha value. But remember - each model corresponding to each target variable should have a unique "best" alpha. The comments above should resolve this.
|
|
||
| GCP2MET_models = [] | ||
| for i in range(Ytrain.shape[1]): | ||
| mdl_G2M =linear_model.Lasso(alpha=best_alpha).fit(Xtrain, Ytrain[:, i]) |
There was a problem hiding this comment.
In the case described above, you will need to run this code with the arugment alpha=best_alpha[i] to get the correct values.
| # In[7]: | ||
|
|
||
|
|
||
| kf = KFold(n_splits=3) |
There was a problem hiding this comment.
So I had you run hold out previously just to get the concept down. In practice, you don't have to run hold out. You can just report the results from k-fold CV,
|
|
||
| r, p_value = pearsonr(Ypred, Ytest[:, i]) | ||
| # Compute P-value based on r-value | ||
| pval = norm.sf(abs(r))*2 |
There was a problem hiding this comment.
Ah, this may be where things are going wrong. I would run this code once you get a vector of r values out of the loop. I think this will return different values because your distribution will be different. I may be wrong, but it's worth a shot.
upload .py version of linear regression, Lasso, and met2gcp