forked from CGCL-codes/naturalcc
-
Notifications
You must be signed in to change notification settings - Fork 0
/
all.log
283 lines (283 loc) · 49.9 KB
/
all.log
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
nohup: ignoring input
Using backend: pytorch
[2021-09-26 23:25:54] INFO >> Load arguments in /home/wanyao/yang/naturalcc-master/run/retrieval/conv1d/config/csn/all.yml (train.py:302, cli_main())
[2021-09-26 23:25:54] INFO >> {'criterion': 'retrieval_softmax', 'optimizer': 'torch_adam', 'lr_scheduler': 'fixed', 'tokenizer': None, 'bpe': None, 'common': {'no_progress_bar': 0, 'log_interval': 500, 'log_format': 'simple', 'tensorboard_logdir': '', 'seed': 0, 'cpu': 0, 'fp16': 0, 'memory_efficient_fp16': 0, 'fp16_no_flatten_grads': 0, 'fp16_init_scale': 128, 'fp16_scale_window': None, 'fp16_scale_tolerance': 0.0, 'min_loss_scale': 0.0001, 'threshold_loss_scale': None, 'user_dir': None, 'empty_cache_freq': 0, 'all_gather_list_size': 16384, 'task': 'hybrid_retrieval'}, 'dataset': {'num_workers': 1, 'skip_invalid_size_inputs_valid_test': 0, 'max_tokens': None, 'max_sentences': 1000, 'code_max_tokens': 200, 'query_max_tokens': 30, 'required_batch_size_multiple': 8, 'dataset_impl': 'mmap', 'train_subset': 'train', 'valid_subset': 'valid', 'validate_interval': 1, 'fixed_validation_seed': None, 'disable_validation': None, 'max_tokens_valid': None, 'max_sentences_valid': 1000, 'curriculum': 0, 'gen_subset': 'test', 'num_shards': 1, 'shard_id': 0, 'joined_dictionary': 0, 'langs': ['go', 'java', 'javascript', 'ruby', 'python', 'php']}, 'distributed_training': {'distributed_world_size': 1, 'distributed_rank': 0, 'distributed_backend': 'nccl', 'distributed_init_method': None, 'distributed_port': -1, 'device_id': 0, 'distributed_no_spawn': 0, 'ddp_backend': 'c10d', 'bucket_cap_mb': 25, 'fix_batches_to_gpus': None, 'find_unused_parameters': 0, 'fast_stat_sync': 0, 'broadcast_buffers': 0, 'global_sync_iter': 50, 'warmup_iterations': 500, 'local_rank': -1, 'block_momentum': 0.875, 'block_lr': 1, 'use_nbm': 0, 'average_sync': 0}, 'task': {'data': '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all', 'sample_break_mode': 'complete', 'tokens_per_sample': 512, 'mask_prob': 0.15, 'leave_unmasked_prob': 0.1, 'random_token_prob': 0.1, 'freq_weighted_replacement': 0, 'mask_whole_words': 0, 'pooler_activation_fn': 'tanh', 'source_lang': 'code_tokens', 'target_lang': 'docstring_tokens', 'source_aux_lang': 'code_tokens.wo_func', 'target_aux_lang': 'func_name', 'fraction_using_func_name': 0.1, 'load_alignments': 0, 'left_pad_source': 1, 'left_pad_target': 0, 'upsample_primary': 1, 'truncate_source': 0, 'eval_mrr': 1}, 'model': {'arch': 'conv1d_res', 'code_embed_dim': 128, 'code_kernel_size': [16, 16, 16], 'code_residual': True, 'code_layers': [128, 128, 128], 'code_pooling': 'weighted_mean', 'code_activation_fn': 'tanh', 'code_position_encoding': 'learned', 'code_paddding': 'same', 'code_dropout': 0.1, 'query_embed_dim': 128, 'query_kernel_size': [16, 16, 16], 'query_residual': True, 'query_layers': [128, 128, 128], 'query_pooling': 'weighted_mean', 'query_activation_fn': 'tanh', 'query_position_encoding': 'learned', 'query_paddding': 'same', 'query_dropout': 0.1}, 'optimization': {'max_epoch': 300, 'max_update': 0, 'clip_norm': 1, 'sentence_avg': 0, 'update_freq': [1], 'lrs': [0.0005], 'min_lr': -1, 'use_bmuf': 0, 'lr_shrink': 1.0, 'force_anneal': 0, 'warmup_updates': 0, 'end_learning_rate': 0.0, 'power': 1.0, 'total_num_update': 1000000, 'adam': {'adam_betas': '(0.9, 0.999)', 'adam_eps': 1e-08, 'weight_decay': 0, 'use_old_adam': 1}, 'margin': 1, 'clip_norm_version': 'tf_clip_by_global_norm'}, 'checkpoint': {'save_dir': '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints', 'restore_file': 'checkpoint_last.pt', 'reset_dataloader': None, 'reset_lr_scheduler': None, 'reset_meters': None, 'reset_optimizer': None, 'optimizer_overrides': '{}', 'save_interval': 1, 'save_interval_updates': 0, 'keep_interval_updates': 0, 'keep_last_epochs': -1, 'keep_best_checkpoints': -1, 'no_save': 0, 'no_epoch_checkpoints': 1, 'no_last_checkpoints': 0, 'no_save_optimizer_state': None, 'best_checkpoint_metric': 'mrr', 'maximize_best_checkpoint_metric': 1, 'patience': 5}, 'eval': {'path': '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt', 'quiet': 1, 'max_sentences': 1000, 'model_overrides': '{}', 'eval_size': -1}} (train.py:304, cli_main())
[2021-09-26 23:25:54] INFO >> single GPU training... (train.py:333, cli_main())
[2021-09-26 23:25:54] INFO >> loaded 89154 examples from: ['/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.go.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.java.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.javascript.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.ruby.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.python.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.php.code_tokens'] (hybrid_retrieval.py:54, load_tokens_dataset())
[2021-09-26 23:25:54] INFO >> loaded 89154 examples from: ['/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.go.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.java.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.javascript.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.ruby.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.python.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/valid.php.docstring_tokens'] (hybrid_retrieval.py:55, load_tokens_dataset())
[2021-09-26 23:25:54] INFO >> Conv1dRes(
(src_encoders): ModuleDict(
(go): Conv1dResEncoder(
(embed): Embedding(10000, 128)
(weight_layer): Linear(in_features=128, out_features=1, bias=False)
(conv_layers): ModuleList(
(0): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(1): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(2): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
)
)
(java): Conv1dResEncoder(
(embed): Embedding(10000, 128)
(weight_layer): Linear(in_features=128, out_features=1, bias=False)
(conv_layers): ModuleList(
(0): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(1): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(2): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
)
)
(javascript): Conv1dResEncoder(
(embed): Embedding(10000, 128)
(weight_layer): Linear(in_features=128, out_features=1, bias=False)
(conv_layers): ModuleList(
(0): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(1): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(2): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
)
)
(ruby): Conv1dResEncoder(
(embed): Embedding(10000, 128)
(weight_layer): Linear(in_features=128, out_features=1, bias=False)
(conv_layers): ModuleList(
(0): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(1): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(2): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
)
)
(python): Conv1dResEncoder(
(embed): Embedding(10000, 128)
(weight_layer): Linear(in_features=128, out_features=1, bias=False)
(conv_layers): ModuleList(
(0): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(1): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(2): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
)
)
(php): Conv1dResEncoder(
(embed): Embedding(10000, 128)
(weight_layer): Linear(in_features=128, out_features=1, bias=False)
(conv_layers): ModuleList(
(0): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(1): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(2): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
)
)
)
(tgt_encoders): Conv1dResEncoder(
(embed): Embedding(10000, 128)
(weight_layer): Linear(in_features=128, out_features=1, bias=False)
(conv_layers): ModuleList(
(0): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(1): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
(2): Conv2d(1, 128, kernel_size=(16, 128), stride=(1, 1))
)
)
) (train.py:223, single_main())
[2021-09-26 23:25:54] INFO >> model conv1d_res, criterion SearchSoftmaxCriterion (train.py:224, single_main())
[2021-09-26 23:25:54] INFO >> num. model params: 14626048 (num. trained: 14626048) (train.py:225, single_main())
[2021-09-26 23:26:00] INFO >> training on 1 GPUs (train.py:233, single_main())
[2021-09-26 23:26:00] INFO >> max tokens per GPU = None and max sentences per GPU = 1000 (train.py:234, single_main())
[2021-09-26 23:26:00] INFO >> no existing checkpoint found /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_last.pt (ncc_trainers.py:270, load_checkpoint())
[2021-09-26 23:26:00] INFO >> loading train data for epoch 1 (ncc_trainers.py:285, get_train_iterator())
[2021-09-26 23:26:01] INFO >> loaded 1880853 examples from: ['/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.go.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.java.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.javascript.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.ruby.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.python.code_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.php.code_tokens'] (hybrid_retrieval.py:54, load_tokens_dataset())
[2021-09-26 23:26:01] INFO >> loaded 1880853 examples from: ['/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.go.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.java.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.javascript.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.ruby.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.python.docstring_tokens', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.php.docstring_tokens'] (hybrid_retrieval.py:55, load_tokens_dataset())
[2021-09-26 23:26:01] INFO >> loaded 1880853 examples from: ['/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.go.code_tokens.wo_func', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.java.code_tokens.wo_func', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.javascript.code_tokens.wo_func', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.ruby.code_tokens.wo_func', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.python.code_tokens.wo_func', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.php.code_tokens.wo_func'] (hybrid_retrieval.py:67, load_tokens_dataset())
[2021-09-26 23:26:01] INFO >> loaded 1880853 examples from: ['/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.go.func_name', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.java.func_name', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.javascript.func_name', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.ruby.func_name', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.python.func_name', '/home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/train.php.func_name'] (hybrid_retrieval.py:81, load_tokens_dataset())
[2021-09-26 23:26:03] INFO >> NOTE: your device may support faster training with fp16 (ncc_trainers.py:155, _setup_optimizer())
[2021-09-26 23:43:52] INFO >> epoch 001: 500 / 1881 loss=5.805, mrr=73.849, sample_size=1000, wps=470.5, ups=0.47, wpb=1000, bsz=1000, num_updates=500, lr=0.0005, gnorm=1.823, clip=92.6, train_wall=1058, wall=1072 (progress_bar.py:260, log())
[2021-09-27 00:01:06] INFO >> epoch 001: 1000 / 1881 loss=4.356, mrr=253.853, sample_size=1000, wps=483.3, ups=0.48, wpb=1000, bsz=1000, num_updates=1000, lr=0.0005, gnorm=2.459, clip=100, train_wall=1029, wall=2106 (progress_bar.py:260, log())
[2021-09-27 00:18:31] INFO >> epoch 001: 1500 / 1881 loss=3.556, mrr=390.053, sample_size=1000, wps=478.7, ups=0.48, wpb=1000, bsz=1000, num_updates=1500, lr=0.0005, gnorm=2.501, clip=100, train_wall=1039, wall=3151 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 00:31:45] INFO >> epoch 001 | loss 4.277 | mrr 0.28545 | sample_size 1000 | wps 477.7 | ups 0.48 | wpb 1000 | bsz 1000 | num_updates 1880 | lr 0.0005 | gnorm 2.311 | clip 98 | train_wall 3915 | wall 3945 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 00:33:30] INFO >> epoch 001 | valid on 'valid' subset | loss 3.31 | mrr 0.447379 | sample_size 1000 | wps 870 | wpb 1000 | bsz 1000 | num_updates 1880 (progress_bar.py:269, print())
[2021-09-27 00:33:31] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 1 @ 1880 updates, score 0.447379) (writing took 0.533139 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 00:37:42] INFO >> epoch 002: 120 / 1881 loss=3.046, mrr=478.703, sample_size=1000, wps=434.2, ups=0.43, wpb=1000, bsz=1000, num_updates=2000, lr=0.0005, gnorm=2.518, clip=100, train_wall=1036, wall=4302 (progress_bar.py:260, log())
[2021-09-27 00:54:55] INFO >> epoch 002: 620 / 1881 loss=2.722, mrr=533.768, sample_size=1000, wps=484, ups=0.48, wpb=1000, bsz=1000, num_updates=2500, lr=0.0005, gnorm=2.501, clip=100, train_wall=1027, wall=5335 (progress_bar.py:260, log())
[2021-09-27 01:12:08] INFO >> epoch 002: 1121 / 1881 loss=2.543, mrr=564.742, sample_size=1000, wps=484.2, ups=0.48, wpb=1000, bsz=1000, num_updates=3000, lr=0.0005, gnorm=2.434, clip=100, train_wall=1027, wall=6368 (progress_bar.py:260, log())
[2021-09-27 01:29:17] INFO >> epoch 002: 1621 / 1881 loss=2.416, mrr=587.33, sample_size=1000, wps=485.9, ups=0.49, wpb=1000, bsz=1000, num_updates=3500, lr=0.0005, gnorm=2.386, clip=100, train_wall=1024, wall=7397 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 01:38:12] INFO >> epoch 002 | loss 2.544 | mrr 0.564622 | sample_size 1000 | wps 471.5 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 3760 | lr 0.0005 | gnorm 2.433 | clip 100 | train_wall 3855 | wall 7932 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 01:39:56] INFO >> epoch 002 | valid on 'valid' subset | loss 2.688 | mrr 0.552003 | sample_size 1000 | wps 876.3 | wpb 1000 | bsz 1000 | num_updates 3760 | best_mrr 552.003 (progress_bar.py:269, print())
[2021-09-27 01:39:57] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 2 @ 3760 updates, score 0.552003) (writing took 0.954554 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 01:48:15] INFO >> epoch 003: 240 / 1881 loss=2.239, mrr=615.766, sample_size=1000, wps=439.2, ups=0.44, wpb=1000, bsz=1000, num_updates=4000, lr=0.0005, gnorm=2.348, clip=100, train_wall=1023, wall=8535 (progress_bar.py:260, log())
[2021-09-27 02:05:24] INFO >> epoch 003: 740 / 1881 loss=2.135, mrr=633.113, sample_size=1000, wps=486.3, ups=0.49, wpb=1000, bsz=1000, num_updates=4500, lr=0.0005, gnorm=2.335, clip=100, train_wall=1023, wall=9564 (progress_bar.py:260, log())
[2021-09-27 02:22:32] INFO >> epoch 003: 1240 / 1881 loss=2.092, mrr=641.18, sample_size=1000, wps=486.1, ups=0.49, wpb=1000, bsz=1000, num_updates=5000, lr=0.0005, gnorm=2.286, clip=100, train_wall=1023, wall=10592 (progress_bar.py:260, log())
[2021-09-27 02:39:41] INFO >> epoch 003: 1741 / 1881 loss=2.046, mrr=648.693, sample_size=1000, wps=485.8, ups=0.49, wpb=1000, bsz=1000, num_updates=5500, lr=0.0005, gnorm=2.228, clip=100, train_wall=1024, wall=11621 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 02:44:30] INFO >> epoch 003 | loss 2.094 | mrr 0.640527 | sample_size 1000 | wps 472.7 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 5640 | lr 0.0005 | gnorm 2.289 | clip 100 | train_wall 3847 | wall 11910 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 02:46:14] INFO >> epoch 003 | valid on 'valid' subset | loss 2.461 | mrr 0.58794 | sample_size 1000 | wps 878.2 | wpb 1000 | bsz 1000 | num_updates 5640 | best_mrr 587.94 (progress_bar.py:269, print())
[2021-09-27 02:46:15] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 3 @ 5640 updates, score 0.58794) (writing took 0.826783 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 02:58:40] INFO >> epoch 004: 360 / 1881 loss=1.911, mrr=670.789, sample_size=1000, wps=439.3, ups=0.44, wpb=1000, bsz=1000, num_updates=6000, lr=0.0005, gnorm=2.246, clip=100, train_wall=1023, wall=12760 (progress_bar.py:260, log())
[2021-09-27 03:15:48] INFO >> epoch 004: 861 / 1881 loss=1.876, mrr=676.981, sample_size=1000, wps=486, ups=0.49, wpb=1000, bsz=1000, num_updates=6500, lr=0.0005, gnorm=2.19, clip=100, train_wall=1023, wall=13789 (progress_bar.py:260, log())
[2021-09-27 03:32:57] INFO >> epoch 004: 1361 / 1881 loss=1.861, mrr=679.762, sample_size=1000, wps=485.8, ups=0.49, wpb=1000, bsz=1000, num_updates=7000, lr=0.0005, gnorm=2.185, clip=100, train_wall=1024, wall=14818 (progress_bar.py:260, log())
[2021-09-27 03:50:05] INFO >> epoch 004: 1861 / 1881 loss=1.846, mrr=682.455, sample_size=1000, wps=486.7, ups=0.49, wpb=1000, bsz=1000, num_updates=7500, lr=0.0005, gnorm=2.157, clip=100, train_wall=1022, wall=15845 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 03:50:47] INFO >> epoch 004 | loss 1.863 | mrr 0.679232 | sample_size 1000 | wps 472.8 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 7520 | lr 0.0005 | gnorm 2.19 | clip 100 | train_wall 3846 | wall 15887 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 03:52:31] INFO >> epoch 004 | valid on 'valid' subset | loss 2.338 | mrr 0.610136 | sample_size 1000 | wps 877.8 | wpb 1000 | bsz 1000 | num_updates 7520 | best_mrr 610.136 (progress_bar.py:269, print())
[2021-09-27 03:52:31] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 4 @ 7520 updates, score 0.610136) (writing took 0.823987 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 04:09:03] INFO >> epoch 005: 481 / 1881 loss=1.712, mrr=704.005, sample_size=1000, wps=439.2, ups=0.44, wpb=1000, bsz=1000, num_updates=8000, lr=0.0005, gnorm=2.169, clip=100, train_wall=1023, wall=16984 (progress_bar.py:260, log())
[2021-09-27 04:26:12] INFO >> epoch 005: 981 / 1881 loss=1.717, mrr=703.159, sample_size=1000, wps=486, ups=0.49, wpb=1000, bsz=1000, num_updates=8500, lr=0.0005, gnorm=2.139, clip=100, train_wall=1023, wall=18012 (progress_bar.py:260, log())
[2021-09-27 04:43:21] INFO >> epoch 005: 1481 / 1881 loss=1.714, mrr=704.335, sample_size=1000, wps=486, ups=0.49, wpb=1000, bsz=1000, num_updates=9000, lr=0.0005, gnorm=2.131, clip=100, train_wall=1023, wall=19041 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 04:57:03] INFO >> epoch 005 | loss 1.712 | mrr 0.704332 | sample_size 1000 | wps 472.7 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 9400 | lr 0.0005 | gnorm 2.136 | clip 100 | train_wall 3846 | wall 19863 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 04:58:47] INFO >> epoch 005 | valid on 'valid' subset | loss 2.255 | mrr 0.623275 | sample_size 1000 | wps 877.5 | wpb 1000 | bsz 1000 | num_updates 9400 | best_mrr 623.275 (progress_bar.py:269, print())
[2021-09-27 04:58:48] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 5 @ 9400 updates, score 0.623275) (writing took 0.789968 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 05:02:18] INFO >> epoch 006: 100 / 1881 loss=1.681, mrr=709.692, sample_size=1000, wps=439.7, ups=0.44, wpb=1000, bsz=1000, num_updates=9500, lr=0.0005, gnorm=2.104, clip=100, train_wall=1022, wall=20178 (progress_bar.py:260, log())
[2021-09-27 05:19:28] INFO >> epoch 006: 600 / 1881 loss=1.593, mrr=724.158, sample_size=1000, wps=485.6, ups=0.49, wpb=1000, bsz=1000, num_updates=10000, lr=0.0005, gnorm=2.118, clip=100, train_wall=1024, wall=21208 (progress_bar.py:260, log())
[2021-09-27 05:36:38] INFO >> epoch 006: 1101 / 1881 loss=1.609, mrr=721.399, sample_size=1000, wps=485.4, ups=0.49, wpb=1000, bsz=1000, num_updates=10500, lr=0.0005, gnorm=2.103, clip=100, train_wall=1024, wall=22238 (progress_bar.py:260, log())
[2021-09-27 05:53:47] INFO >> epoch 006: 1601 / 1881 loss=1.615, mrr=720.572, sample_size=1000, wps=486, ups=0.49, wpb=1000, bsz=1000, num_updates=11000, lr=0.0005, gnorm=2.084, clip=100, train_wall=1023, wall=23267 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 06:03:24] INFO >> epoch 006 | loss 1.604 | mrr 0.722235 | sample_size 1000 | wps 472.3 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 11280 | lr 0.0005 | gnorm 2.098 | clip 100 | train_wall 3850 | wall 23844 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 06:05:08] INFO >> epoch 006 | valid on 'valid' subset | loss 2.189 | mrr 0.636428 | sample_size 1000 | wps 877.1 | wpb 1000 | bsz 1000 | num_updates 11280 | best_mrr 636.428 (progress_bar.py:269, print())
[2021-09-27 06:05:09] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 6 @ 11280 updates, score 0.636428) (writing took 0.773826 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 06:12:45] INFO >> epoch 007: 220 / 1881 loss=1.552, mrr=730.806, sample_size=1000, wps=439.1, ups=0.44, wpb=1000, bsz=1000, num_updates=11500, lr=0.0005, gnorm=2.069, clip=100, train_wall=1024, wall=24406 (progress_bar.py:260, log())
[2021-09-27 06:29:54] INFO >> epoch 007: 721 / 1881 loss=1.505, mrr=738.526, sample_size=1000, wps=486.2, ups=0.49, wpb=1000, bsz=1000, num_updates=12000, lr=0.0005, gnorm=2.07, clip=100, train_wall=1023, wall=25434 (progress_bar.py:260, log())
[2021-09-27 06:47:03] INFO >> epoch 007: 1221 / 1881 loss=1.522, mrr=735.89, sample_size=1000, wps=485.9, ups=0.49, wpb=1000, bsz=1000, num_updates=12500, lr=0.0005, gnorm=2.08, clip=100, train_wall=1024, wall=26463 (progress_bar.py:260, log())
[2021-09-27 07:04:10] INFO >> epoch 007: 1721 / 1881 loss=1.529, mrr=734.831, sample_size=1000, wps=486.7, ups=0.49, wpb=1000, bsz=1000, num_updates=13000, lr=0.0005, gnorm=2.064, clip=100, train_wall=1022, wall=27490 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 07:09:40] INFO >> epoch 007 | loss 1.515 | mrr 0.736995 | sample_size 1000 | wps 472.8 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 13160 | lr 0.0005 | gnorm 2.071 | clip 100 | train_wall 3846 | wall 27820 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 07:11:24] INFO >> epoch 007 | valid on 'valid' subset | loss 2.165 | mrr 0.64182 | sample_size 1000 | wps 876.1 | wpb 1000 | bsz 1000 | num_updates 13160 | best_mrr 641.82 (progress_bar.py:269, print())
[2021-09-27 07:11:25] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 7 @ 13160 updates, score 0.64182) (writing took 0.784883 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 07:23:08] INFO >> epoch 008: 340 / 1881 loss=1.448, mrr=748.409, sample_size=1000, wps=439.6, ups=0.44, wpb=1000, bsz=1000, num_updates=13500, lr=0.0005, gnorm=2.05, clip=100, train_wall=1022, wall=28628 (progress_bar.py:260, log())
[2021-09-27 07:40:16] INFO >> epoch 008: 840 / 1881 loss=1.436, mrr=750.206, sample_size=1000, wps=486.4, ups=0.49, wpb=1000, bsz=1000, num_updates=14000, lr=0.0005, gnorm=2.062, clip=100, train_wall=1022, wall=29656 (progress_bar.py:260, log())
[2021-09-27 07:57:25] INFO >> epoch 008: 1341 / 1881 loss=1.456, mrr=746.893, sample_size=1000, wps=485.7, ups=0.49, wpb=1000, bsz=1000, num_updates=14500, lr=0.0005, gnorm=2.052, clip=100, train_wall=1024, wall=30685 (progress_bar.py:260, log())
[2021-09-27 08:14:34] INFO >> epoch 008: 1841 / 1881 loss=1.463, mrr=745.681, sample_size=1000, wps=485.9, ups=0.49, wpb=1000, bsz=1000, num_updates=15000, lr=0.0005, gnorm=2.053, clip=100, train_wall=1023, wall=31714 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 08:15:57] INFO >> epoch 008 | loss 1.445 | mrr 0.748855 | sample_size 1000 | wps 472.8 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 15040 | lr 0.0005 | gnorm 2.054 | clip 100 | train_wall 3846 | wall 31797 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 08:17:41] INFO >> epoch 008 | valid on 'valid' subset | loss 2.134 | mrr 0.647293 | sample_size 1000 | wps 877.1 | wpb 1000 | bsz 1000 | num_updates 15040 | best_mrr 647.293 (progress_bar.py:269, print())
[2021-09-27 08:17:42] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 8 @ 15040 updates, score 0.647293) (writing took 0.861310 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 08:33:32] INFO >> epoch 009: 461 / 1881 loss=1.364, mrr=762.326, sample_size=1000, wps=439.4, ups=0.44, wpb=1000, bsz=1000, num_updates=15500, lr=0.0005, gnorm=2.034, clip=100, train_wall=1023, wall=32852 (progress_bar.py:260, log())
[2021-09-27 08:50:40] INFO >> epoch 009: 961 / 1881 loss=1.385, mrr=758.271, sample_size=1000, wps=486.4, ups=0.49, wpb=1000, bsz=1000, num_updates=16000, lr=0.0005, gnorm=2.052, clip=100, train_wall=1022, wall=33880 (progress_bar.py:260, log())
[2021-09-27 09:07:49] INFO >> epoch 009: 1461 / 1881 loss=1.398, mrr=756.686, sample_size=1000, wps=485.9, ups=0.49, wpb=1000, bsz=1000, num_updates=16500, lr=0.0005, gnorm=2.033, clip=100, train_wall=1023, wall=34909 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 09:22:13] INFO >> epoch 009 | loss 1.388 | mrr 0.758146 | sample_size 1000 | wps 472.8 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 16920 | lr 0.0005 | gnorm 2.038 | clip 100 | train_wall 3845 | wall 35773 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 09:23:58] INFO >> epoch 009 | valid on 'valid' subset | loss 2.108 | mrr 0.653828 | sample_size 1000 | wps 873.4 | wpb 1000 | bsz 1000 | num_updates 16920 | best_mrr 653.828 (progress_bar.py:269, print())
[2021-09-27 09:23:59] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 9 @ 16920 updates, score 0.653828) (writing took 0.810863 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 09:26:49] INFO >> epoch 010: 80 / 1881 loss=1.396, mrr=756.949, sample_size=1000, wps=438.6, ups=0.44, wpb=1000, bsz=1000, num_updates=17000, lr=0.0005, gnorm=2.023, clip=100, train_wall=1022, wall=36049 (progress_bar.py:260, log())
[2021-09-27 09:44:00] INFO >> epoch 010: 581 / 1881 loss=1.314, mrr=770.437, sample_size=1000, wps=485.1, ups=0.49, wpb=1000, bsz=1000, num_updates=17500, lr=0.0005, gnorm=2.038, clip=100, train_wall=1024, wall=37080 (progress_bar.py:260, log())
[2021-09-27 10:01:11] INFO >> epoch 010: 1081 / 1881 loss=1.334, mrr=767.077, sample_size=1000, wps=484.9, ups=0.48, wpb=1000, bsz=1000, num_updates=18000, lr=0.0005, gnorm=2.033, clip=100, train_wall=1025, wall=38111 (progress_bar.py:260, log())
[2021-09-27 10:18:21] INFO >> epoch 010: 1581 / 1881 loss=1.353, mrr=764.006, sample_size=1000, wps=485.2, ups=0.49, wpb=1000, bsz=1000, num_updates=18500, lr=0.0005, gnorm=2.027, clip=100, train_wall=1023, wall=39142 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 10:28:40] INFO >> epoch 010 | loss 1.337 | mrr 0.766706 | sample_size 1000 | wps 471.5 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 18800 | lr 0.0005 | gnorm 2.031 | clip 100 | train_wall 3850 | wall 39760 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 10:30:25] INFO >> epoch 010 | valid on 'valid' subset | loss 2.104 | mrr 0.655996 | sample_size 1000 | wps 876.3 | wpb 1000 | bsz 1000 | num_updates 18800 | best_mrr 655.996 (progress_bar.py:269, print())
[2021-09-27 10:30:26] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 10 @ 18800 updates, score 0.655996) (writing took 0.865901 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 10:37:23] INFO >> epoch 011: 200 / 1881 loss=1.317, mrr=770.329, sample_size=1000, wps=437.8, ups=0.44, wpb=1000, bsz=1000, num_updates=19000, lr=0.0005, gnorm=2.016, clip=100, train_wall=1024, wall=40284 (progress_bar.py:260, log())
[2021-09-27 10:54:37] INFO >> epoch 011: 700 / 1881 loss=1.27, mrr=777.631, sample_size=1000, wps=484, ups=0.48, wpb=1000, bsz=1000, num_updates=19500, lr=0.0005, gnorm=2.014, clip=100, train_wall=1025, wall=41317 (progress_bar.py:260, log())
[2021-09-27 11:11:47] INFO >> epoch 011: 1200 / 1881 loss=1.302, mrr=772.281, sample_size=1000, wps=485.4, ups=0.49, wpb=1000, bsz=1000, num_updates=20000, lr=0.0005, gnorm=2.039, clip=100, train_wall=1024, wall=42347 (progress_bar.py:260, log())
[2021-09-27 11:28:58] INFO >> epoch 011: 1701 / 1881 loss=1.318, mrr=769.934, sample_size=1000, wps=484.6, ups=0.48, wpb=1000, bsz=1000, num_updates=20500, lr=0.0005, gnorm=2.028, clip=100, train_wall=1025, wall=43378 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 11:35:08] INFO >> epoch 011 | loss 1.293 | mrr 0.773882 | sample_size 1000 | wps 471.4 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 20680 | lr 0.0005 | gnorm 2.023 | clip 100 | train_wall 3850 | wall 43749 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 11:36:54] INFO >> epoch 011 | valid on 'valid' subset | loss 2.082 | mrr 0.660139 | sample_size 1000 | wps 874.7 | wpb 1000 | bsz 1000 | num_updates 20680 | best_mrr 660.139 (progress_bar.py:269, print())
[2021-09-27 11:36:55] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 11 @ 20680 updates, score 0.660139) (writing took 0.952836 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 11:47:59] INFO >> epoch 012: 320 / 1881 loss=1.249, mrr=781.206, sample_size=1000, wps=438.4, ups=0.44, wpb=1000, bsz=1000, num_updates=21000, lr=0.0005, gnorm=2.013, clip=100, train_wall=1022, wall=44519 (progress_bar.py:260, log())
[2021-09-27 12:05:11] INFO >> epoch 012: 820 / 1881 loss=1.24, mrr=782.72, sample_size=1000, wps=484.6, ups=0.48, wpb=1000, bsz=1000, num_updates=21500, lr=0.0005, gnorm=2.043, clip=100, train_wall=1025, wall=45551 (progress_bar.py:260, log())
[2021-09-27 12:22:24] INFO >> epoch 012: 1321 / 1881 loss=1.269, mrr=777.772, sample_size=1000, wps=484, ups=0.48, wpb=1000, bsz=1000, num_updates=22000, lr=0.0005, gnorm=2.015, clip=100, train_wall=1025, wall=46584 (progress_bar.py:260, log())
[2021-09-27 12:39:35] INFO >> epoch 012: 1821 / 1881 loss=1.283, mrr=776.101, sample_size=1000, wps=484.7, ups=0.48, wpb=1000, bsz=1000, num_updates=22500, lr=0.0005, gnorm=2.002, clip=100, train_wall=1024, wall=47615 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 12:41:39] INFO >> epoch 012 | loss 1.256 | mrr 0.780254 | sample_size 1000 | wps 471.1 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 22560 | lr 0.0005 | gnorm 2.017 | clip 100 | train_wall 3852 | wall 47739 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 12:43:25] INFO >> epoch 012 | valid on 'valid' subset | loss 2.062 | mrr 0.663204 | sample_size 1000 | wps 875.8 | wpb 1000 | bsz 1000 | num_updates 22560 | best_mrr 663.204 (progress_bar.py:269, print())
[2021-09-27 12:43:25] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 12 @ 22560 updates, score 0.663204) (writing took 0.807100 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 12:58:39] INFO >> epoch 013: 440 / 1881 loss=1.196, mrr=790.353, sample_size=1000, wps=437.1, ups=0.44, wpb=1000, bsz=1000, num_updates=23000, lr=0.0005, gnorm=1.991, clip=100, train_wall=1025, wall=48759 (progress_bar.py:260, log())
[2021-09-27 13:15:52] INFO >> epoch 013: 940 / 1881 loss=1.214, mrr=786.747, sample_size=1000, wps=484, ups=0.48, wpb=1000, bsz=1000, num_updates=23500, lr=0.0005, gnorm=2.013, clip=100, train_wall=1025, wall=49792 (progress_bar.py:260, log())
[2021-09-27 13:33:04] INFO >> epoch 013: 1441 / 1881 loss=1.238, mrr=782.731, sample_size=1000, wps=484.7, ups=0.48, wpb=1000, bsz=1000, num_updates=24000, lr=0.0005, gnorm=2.004, clip=100, train_wall=1024, wall=50824 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 13:48:13] INFO >> epoch 013 | loss 1.222 | mrr 0.785691 | sample_size 1000 | wps 470.7 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 24440 | lr 0.0005 | gnorm 2.001 | clip 100 | train_wall 3854 | wall 51733 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 13:49:59] INFO >> epoch 013 | valid on 'valid' subset | loss 2.063 | mrr 0.666299 | sample_size 1000 | wps 870.4 | wpb 1000 | bsz 1000 | num_updates 24440 | best_mrr 666.299 (progress_bar.py:269, print())
[2021-09-27 13:49:59] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 13 @ 24440 updates, score 0.666299) (writing took 0.950691 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 13:52:08] INFO >> epoch 014: 60 / 1881 loss=1.237, mrr=783.478, sample_size=1000, wps=437, ups=0.44, wpb=1000, bsz=1000, num_updates=24500, lr=0.0005, gnorm=1.984, clip=100, train_wall=1025, wall=51968 (progress_bar.py:260, log())
[2021-09-27 14:09:20] INFO >> epoch 014: 561 / 1881 loss=1.166, mrr=794.922, sample_size=1000, wps=484.4, ups=0.48, wpb=1000, bsz=1000, num_updates=25000, lr=0.0005, gnorm=1.997, clip=100, train_wall=1024, wall=53000 (progress_bar.py:260, log())
[2021-09-27 14:26:33] INFO >> epoch 014: 1061 / 1881 loss=1.189, mrr=791.137, sample_size=1000, wps=484.1, ups=0.48, wpb=1000, bsz=1000, num_updates=25500, lr=0.0005, gnorm=2.008, clip=100, train_wall=1025, wall=54033 (progress_bar.py:260, log())
[2021-09-27 14:43:45] INFO >> epoch 014: 1561 / 1881 loss=1.21, mrr=787.633, sample_size=1000, wps=484.2, ups=0.48, wpb=1000, bsz=1000, num_updates=26000, lr=0.0005, gnorm=1.993, clip=100, train_wall=1025, wall=55065 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 14:54:47] INFO >> epoch 014 | loss 1.193 | mrr 0.790461 | sample_size 1000 | wps 470.8 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 26320 | lr 0.0005 | gnorm 1.995 | clip 100 | train_wall 3853 | wall 55727 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 14:56:32] INFO >> epoch 014 | valid on 'valid' subset | loss 2.056 | mrr 0.666254 | sample_size 1000 | wps 871.6 | wpb 1000 | bsz 1000 | num_updates 26320 | best_mrr 666.254 (progress_bar.py:269, print())
[2021-09-27 14:56:32] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_last.pt (epoch 14 @ 26320 updates, score 0.666254) (writing took 0.696447 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 15:02:49] INFO >> epoch 015: 181 / 1881 loss=1.183, mrr=792.031, sample_size=1000, wps=437, ups=0.44, wpb=1000, bsz=1000, num_updates=26500, lr=0.0005, gnorm=1.975, clip=100, train_wall=1025, wall=56210 (progress_bar.py:260, log())
[2021-09-27 15:20:00] INFO >> epoch 015: 681 / 1881 loss=1.144, mrr=798.484, sample_size=1000, wps=485.2, ups=0.49, wpb=1000, bsz=1000, num_updates=27000, lr=0.0005, gnorm=2.002, clip=100, train_wall=1023, wall=57240 (progress_bar.py:260, log())
[2021-09-27 15:37:14] INFO >> epoch 015: 1181 / 1881 loss=1.17, mrr=794.038, sample_size=1000, wps=483.7, ups=0.48, wpb=1000, bsz=1000, num_updates=27500, lr=0.0005, gnorm=2.005, clip=100, train_wall=1026, wall=58274 (progress_bar.py:260, log())
[2021-09-27 15:54:28] INFO >> epoch 015: 1681 / 1881 loss=1.188, mrr=791.289, sample_size=1000, wps=483.5, ups=0.48, wpb=1000, bsz=1000, num_updates=28000, lr=0.0005, gnorm=1.979, clip=100, train_wall=1026, wall=59308 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 16:01:21] INFO >> epoch 015 | loss 1.165 | mrr 0.794983 | sample_size 1000 | wps 470.7 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 28200 | lr 0.0005 | gnorm 1.991 | clip 100 | train_wall 3853 | wall 59721 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 16:03:06] INFO >> epoch 015 | valid on 'valid' subset | loss 2.066 | mrr 0.668489 | sample_size 1000 | wps 874.7 | wpb 1000 | bsz 1000 | num_updates 28200 | best_mrr 668.489 (progress_bar.py:269, print())
[2021-09-27 16:03:07] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 15 @ 28200 updates, score 0.668489) (writing took 0.924813 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 16:13:31] INFO >> epoch 016: 300 / 1881 loss=1.138, mrr=799.553, sample_size=1000, wps=437.4, ups=0.44, wpb=1000, bsz=1000, num_updates=28500, lr=0.0005, gnorm=1.983, clip=100, train_wall=1024, wall=60451 (progress_bar.py:260, log())
[2021-09-27 16:30:44] INFO >> epoch 016: 801 / 1881 loss=1.126, mrr=801.471, sample_size=1000, wps=484, ups=0.48, wpb=1000, bsz=1000, num_updates=29000, lr=0.0005, gnorm=1.982, clip=100, train_wall=1025, wall=61484 (progress_bar.py:260, log())
[2021-09-27 16:47:54] INFO >> epoch 016: 1301 / 1881 loss=1.145, mrr=798.005, sample_size=1000, wps=485.5, ups=0.49, wpb=1000, bsz=1000, num_updates=29500, lr=0.0005, gnorm=1.969, clip=100, train_wall=1023, wall=62514 (progress_bar.py:260, log())
[2021-09-27 17:05:05] INFO >> epoch 016: 1801 / 1881 loss=1.176, mrr=793.082, sample_size=1000, wps=485, ups=0.48, wpb=1000, bsz=1000, num_updates=30000, lr=0.0005, gnorm=1.986, clip=100, train_wall=1024, wall=63545 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 17:07:51] INFO >> epoch 016 | loss 1.142 | mrr 0.798772 | sample_size 1000 | wps 471.2 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 30080 | lr 0.0005 | gnorm 1.978 | clip 100 | train_wall 3851 | wall 63711 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 17:09:36] INFO >> epoch 016 | valid on 'valid' subset | loss 2.056 | mrr 0.669902 | sample_size 1000 | wps 875.3 | wpb 1000 | bsz 1000 | num_updates 30080 | best_mrr 669.902 (progress_bar.py:269, print())
[2021-09-27 17:09:37] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 16 @ 30080 updates, score 0.669902) (writing took 0.973867 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 17:24:08] INFO >> epoch 017: 420 / 1881 loss=1.094, mrr=806.552, sample_size=1000, wps=437.5, ups=0.44, wpb=1000, bsz=1000, num_updates=30500, lr=0.0005, gnorm=1.945, clip=100, train_wall=1024, wall=64688 (progress_bar.py:260, log())
[2021-09-27 17:41:20] INFO >> epoch 017: 921 / 1881 loss=1.107, mrr=804.594, sample_size=1000, wps=484.6, ups=0.48, wpb=1000, bsz=1000, num_updates=31000, lr=0.0005, gnorm=1.983, clip=100, train_wall=1024, wall=65720 (progress_bar.py:260, log())
[2021-09-27 17:58:33] INFO >> epoch 017: 1421 / 1881 loss=1.131, mrr=801.047, sample_size=1000, wps=484, ups=0.48, wpb=1000, bsz=1000, num_updates=31500, lr=0.0005, gnorm=1.971, clip=100, train_wall=1024, wall=66753 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 18:14:23] INFO >> epoch 017 | loss 1.118 | mrr 0.802815 | sample_size 1000 | wps 470.9 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 31960 | lr 0.0005 | gnorm 1.967 | clip 100 | train_wall 3852 | wall 67703 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 18:16:09] INFO >> epoch 017 | valid on 'valid' subset | loss 2.059 | mrr 0.671523 | sample_size 1000 | wps 871 | wpb 1000 | bsz 1000 | num_updates 31960 | best_mrr 671.523 (progress_bar.py:269, print())
[2021-09-27 18:16:10] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 17 @ 31960 updates, score 0.671523) (writing took 0.955758 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 18:17:37] INFO >> epoch 018: 40 / 1881 loss=1.142, mrr=798.556, sample_size=1000, wps=436.9, ups=0.44, wpb=1000, bsz=1000, num_updates=32000, lr=0.0005, gnorm=1.96, clip=100, train_wall=1025, wall=67897 (progress_bar.py:260, log())
[2021-09-27 18:34:48] INFO >> epoch 018: 541 / 1881 loss=1.064, mrr=811.939, sample_size=1000, wps=484.7, ups=0.48, wpb=1000, bsz=1000, num_updates=32500, lr=0.0005, gnorm=1.964, clip=100, train_wall=1024, wall=68929 (progress_bar.py:260, log())
[2021-09-27 18:52:01] INFO >> epoch 018: 1041 / 1881 loss=1.096, mrr=805.919, sample_size=1000, wps=484, ups=0.48, wpb=1000, bsz=1000, num_updates=33000, lr=0.0005, gnorm=1.975, clip=100, train_wall=1025, wall=69962 (progress_bar.py:260, log())
[2021-09-27 19:09:13] INFO >> epoch 018: 1541 / 1881 loss=1.118, mrr=802.474, sample_size=1000, wps=484.6, ups=0.48, wpb=1000, bsz=1000, num_updates=33500, lr=0.0005, gnorm=1.947, clip=100, train_wall=1024, wall=70993 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 19:20:55] INFO >> epoch 018 | loss 1.1 | mrr 0.805745 | sample_size 1000 | wps 471 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 33840 | lr 0.0005 | gnorm 1.963 | clip 100 | train_wall 3852 | wall 71695 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 19:22:40] INFO >> epoch 018 | valid on 'valid' subset | loss 2.055 | mrr 0.673052 | sample_size 1000 | wps 877.4 | wpb 1000 | bsz 1000 | num_updates 33840 | best_mrr 673.052 (progress_bar.py:269, print())
[2021-09-27 19:22:41] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 18 @ 33840 updates, score 0.673052) (writing took 1.089699 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 19:28:16] INFO >> epoch 019: 161 / 1881 loss=1.099, mrr=806.025, sample_size=1000, wps=437.5, ups=0.44, wpb=1000, bsz=1000, num_updates=34000, lr=0.0005, gnorm=1.949, clip=100, train_wall=1024, wall=72136 (progress_bar.py:260, log())
[2021-09-27 19:45:28] INFO >> epoch 019: 661 / 1881 loss=1.058, mrr=812.96, sample_size=1000, wps=484.3, ups=0.48, wpb=1000, bsz=1000, num_updates=34500, lr=0.0005, gnorm=1.96, clip=100, train_wall=1025, wall=73169 (progress_bar.py:260, log())
[2021-09-27 20:02:40] INFO >> epoch 019: 1161 / 1881 loss=1.089, mrr=807.744, sample_size=1000, wps=484.8, ups=0.48, wpb=1000, bsz=1000, num_updates=35000, lr=0.0005, gnorm=1.963, clip=100, train_wall=1024, wall=74200 (progress_bar.py:260, log())
[2021-09-27 20:19:53] INFO >> epoch 019: 1661 / 1881 loss=1.103, mrr=805.148, sample_size=1000, wps=484, ups=0.48, wpb=1000, bsz=1000, num_updates=35500, lr=0.0005, gnorm=1.942, clip=100, train_wall=1025, wall=75233 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 20:27:28] INFO >> epoch 019 | loss 1.082 | mrr 0.808799 | sample_size 1000 | wps 470.9 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 35720 | lr 0.0005 | gnorm 1.95 | clip 100 | train_wall 3852 | wall 75688 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 20:29:13] INFO >> epoch 019 | valid on 'valid' subset | loss 2.064 | mrr 0.673286 | sample_size 1000 | wps 871.1 | wpb 1000 | bsz 1000 | num_updates 35720 | best_mrr 673.286 (progress_bar.py:269, print())
[2021-09-27 20:29:14] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 19 @ 35720 updates, score 0.673286) (writing took 1.015075 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 20:38:57] INFO >> epoch 020: 280 / 1881 loss=1.06, mrr=812.418, sample_size=1000, wps=437.1, ups=0.44, wpb=1000, bsz=1000, num_updates=36000, lr=0.0005, gnorm=1.943, clip=100, train_wall=1024, wall=76377 (progress_bar.py:260, log())
[2021-09-27 20:56:08] INFO >> epoch 020: 781 / 1881 loss=1.047, mrr=814.092, sample_size=1000, wps=484.7, ups=0.48, wpb=1000, bsz=1000, num_updates=36500, lr=0.0005, gnorm=1.96, clip=100, train_wall=1025, wall=77409 (progress_bar.py:260, log())
[2021-09-27 21:13:22] INFO >> epoch 020: 1281 / 1881 loss=1.074, mrr=809.668, sample_size=1000, wps=483.8, ups=0.48, wpb=1000, bsz=1000, num_updates=37000, lr=0.0005, gnorm=1.961, clip=100, train_wall=1025, wall=78442 (progress_bar.py:260, log())
[2021-09-27 21:30:33] INFO >> epoch 020: 1781 / 1881 loss=1.088, mrr=807.464, sample_size=1000, wps=484.7, ups=0.48, wpb=1000, bsz=1000, num_updates=37500, lr=0.0005, gnorm=1.952, clip=100, train_wall=1024, wall=79474 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 21:34:00] INFO >> epoch 020 | loss 1.063 | mrr 0.811605 | sample_size 1000 | wps 470.9 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 37600 | lr 0.0005 | gnorm 1.954 | clip 100 | train_wall 3852 | wall 79681 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 21:35:45] INFO >> epoch 020 | valid on 'valid' subset | loss 2.059 | mrr 0.674556 | sample_size 1000 | wps 875.6 | wpb 1000 | bsz 1000 | num_updates 37600 | best_mrr 674.556 (progress_bar.py:269, print())
[2021-09-27 21:35:46] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_best.pt (epoch 20 @ 37600 updates, score 0.674556) (writing took 0.955308 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 21:49:37] INFO >> epoch 021: 400 / 1881 loss=1.028, mrr=818.048, sample_size=1000, wps=437.1, ups=0.44, wpb=1000, bsz=1000, num_updates=38000, lr=0.0005, gnorm=1.941, clip=100, train_wall=1024, wall=80618 (progress_bar.py:260, log())
[2021-09-27 22:06:50] INFO >> epoch 021: 900 / 1881 loss=1.033, mrr=816.76, sample_size=1000, wps=484, ups=0.48, wpb=1000, bsz=1000, num_updates=38500, lr=0.0005, gnorm=1.951, clip=100, train_wall=1025, wall=81651 (progress_bar.py:260, log())
[2021-09-27 22:24:02] INFO >> epoch 021: 1401 / 1881 loss=1.065, mrr=811.281, sample_size=1000, wps=484.7, ups=0.48, wpb=1000, bsz=1000, num_updates=39000, lr=0.0005, gnorm=1.955, clip=100, train_wall=1024, wall=82682 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 22:40:34] INFO >> epoch 021 | loss 1.049 | mrr 0.81412 | sample_size 1000 | wps 470.7 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 39480 | lr 0.0005 | gnorm 1.949 | clip 100 | train_wall 3852 | wall 83675 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 22:42:20] INFO >> epoch 021 | valid on 'valid' subset | loss 2.077 | mrr 0.673722 | sample_size 1000 | wps 871.5 | wpb 1000 | bsz 1000 | num_updates 39480 | best_mrr 673.722 (progress_bar.py:269, print())
[2021-09-27 22:42:20] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_last.pt (epoch 21 @ 39480 updates, score 0.673722) (writing took 0.542846 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 22:43:06] INFO >> epoch 022: 20 / 1881 loss=1.077, mrr=809.29, sample_size=1000, wps=436.8, ups=0.44, wpb=1000, bsz=1000, num_updates=39500, lr=0.0005, gnorm=1.943, clip=100, train_wall=1025, wall=83827 (progress_bar.py:260, log())
[2021-09-27 23:00:19] INFO >> epoch 022: 520 / 1881 loss=1.002, mrr=821.962, sample_size=1000, wps=484.2, ups=0.48, wpb=1000, bsz=1000, num_updates=40000, lr=0.0005, gnorm=1.944, clip=100, train_wall=1024, wall=84859 (progress_bar.py:260, log())
[2021-09-27 23:17:31] INFO >> epoch 022: 1021 / 1881 loss=1.03, mrr=817.412, sample_size=1000, wps=484.6, ups=0.48, wpb=1000, bsz=1000, num_updates=40500, lr=0.0005, gnorm=1.95, clip=100, train_wall=1024, wall=85891 (progress_bar.py:260, log())
[2021-09-27 23:34:43] INFO >> epoch 022: 1521 / 1881 loss=1.05, mrr=813.902, sample_size=1000, wps=484.4, ups=0.48, wpb=1000, bsz=1000, num_updates=41000, lr=0.0005, gnorm=1.951, clip=100, train_wall=1025, wall=86923 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-27 23:47:06] INFO >> epoch 022 | loss 1.033 | mrr 0.81683 | sample_size 1000 | wps 471 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 41360 | lr 0.0005 | gnorm 1.944 | clip 100 | train_wall 3852 | wall 87666 (progress_bar.py:269, print())
Using backend: pytorch
[2021-09-27 23:48:52] INFO >> epoch 022 | valid on 'valid' subset | loss 2.071 | mrr 0.673996 | sample_size 1000 | wps 869.4 | wpb 1000 | bsz 1000 | num_updates 41360 | best_mrr 673.996 (progress_bar.py:269, print())
[2021-09-27 23:48:52] INFO >> saved checkpoint /home/wanyao/ncc_data/codesearchnet/retrieval/data-mmap/all/conv1d_res/checkpoints/checkpoint_last.pt (epoch 22 @ 41360 updates, score 0.673996) (writing took 0.601727 seconds) (checkpoint_utils.py:79, save_checkpoint())
[2021-09-27 23:53:46] INFO >> epoch 023: 140 / 1881 loss=1.034, mrr=816.468, sample_size=1000, wps=437.5, ups=0.44, wpb=1000, bsz=1000, num_updates=41500, lr=0.0005, gnorm=1.922, clip=100, train_wall=1025, wall=88066 (progress_bar.py:260, log())
[2021-09-28 00:10:56] INFO >> epoch 023: 640 / 1881 loss=0.992, mrr=823.581, sample_size=1000, wps=485.6, ups=0.49, wpb=1000, bsz=1000, num_updates=42000, lr=0.0005, gnorm=1.942, clip=100, train_wall=1024, wall=89096 (progress_bar.py:260, log())
[2021-09-28 00:28:05] INFO >> epoch 023: 1141 / 1881 loss=1.024, mrr=818.608, sample_size=1000, wps=485.6, ups=0.49, wpb=1000, bsz=1000, num_updates=42500, lr=0.0005, gnorm=1.929, clip=100, train_wall=1024, wall=90125 (progress_bar.py:260, log())
[2021-09-28 00:45:13] INFO >> epoch 023: 1641 / 1881 loss=1.044, mrr=814.832, sample_size=1000, wps=486.3, ups=0.49, wpb=1000, bsz=1000, num_updates=43000, lr=0.0005, gnorm=1.938, clip=100, train_wall=1023, wall=91154 (progress_bar.py:260, log())
Using backend: pytorch
[2021-09-28 00:53:27] INFO >> epoch 023 | loss 1.021 | mrr 0.818897 | sample_size 1000 | wps 472.2 | ups 0.47 | wpb 1000 | bsz 1000 | num_updates 43240 | lr 0.0005 | gnorm 1.935 | clip 100 | train_wall 3849 | wall 91647 (progress_bar.py:269, print())