-
Notifications
You must be signed in to change notification settings - Fork 0
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Understanding Comet code #20
Comments
Just logging here the hparams that are downloaded with CometKiwi from Huggingface
|
Suggest we keep a log of whether we think a parameter will be:
My assumptions are below, but may not be 100% correct and please feel free to edit or discuss.
|
According to the UnifiedMetric docstring:
Presumably there are cases where one wants to load the pretrained model weights and not the COMET checkpoint (not sure if this is an option with the current COMET implementation). It does not make sense to load both so this should always be False if using the COMET checkpoint. |
In the COMET code base, there is a function This function contains the option of whether or not to load the hparams file when creating a model object. So this is where we will want to upload our own hparams file instead or alternatively define the arguments for creating the object from a config file. This is where we should set the training & validation data file paths too I think (currently set later in the code). EDIT: from pytorch lightning documenation: Any arguments specified through **kwargs will override args stored in "hyper_parameters" The code is currently run so that the hparams are not reloaded. |
It looks like you were right about The encoder gets initialized in the base model here. The |
For reference, the |
Bit of a side note, but just wanted to record that the random seed gets set in |
I've gone through the parameters in the table and these are the ones that I think we might want to search / tune ahead of running the experiments:
@radka-j - do you agree? There may also be other parameters we want to tune for the Trainer object (e.g., number of epochs), but just looking at the COMET model here. |
Some notes and questions:
|
Notes on the different encoders:
|
|
Looking through the It is true that there is a default value in the base |
Looking at the method |
We need a better understanding of some aspects of the COMET code base.
A few things to consider:
encoder_model
,pretrained_model
andload_pretrained_weights
loss
argument actually used for anything? Seems that the sentence loss is hard-coded in theinit_losses
methodAs part of this issue we will work out which hyperparameters we want to search and which we want to be able to set ahead of training.
The text was updated successfully, but these errors were encountered: