Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

terminate called after throwing an instance of 'c10::Error' what(): [enforce fail at inline_container.cc:319] . unexpected pos 2302963008 vs 2302962896 frame #0: c10::ThrowEnforceNotMet(char const*, int, char const*, std::string const&, void const*) + 0x55 (0x15158b1320a5 in /usr/local/lib/python3.8/dist-packages/torch/lib/libc10.so) #63

Open
lldd1001 opened this issue Apr 20, 2024 · 0 comments

Comments

@lldd1001
Copy link

frame #1: + 0x34396bc (0x1514c3a796bc in /usr/local/lib/python3.8/dist-packages/torch/lib/libtorch_cpu.so)
frame #2: mz_zip_writer_add_mem_ex_v2 + 0x5b1 (0x1514c3a73a91 in /usr/local/lib/python3.8/dist-packages/torch/lib/libtorch_cpu.so)
frame #3: caffe2::serialize::PyTorchStreamWriter::writeRecord(std::string const&, void const*, unsigned long, bool) + 0xb9 (0x1514c3a7ac79 in /usr/local/lib/python3.8/dist-packages/torch/lib/libtorch_cpu.so)
frame #4: caffe2::serialize::PyTorchStreamWriter::writeEndOfFile() + 0x2c3 (0x1514c3a7b143 in /usr/local/lib/python3.8/dist-packages/torch/lib/libtorch_cpu.so)
frame #5: caffe2::serialize::PyTorchStreamWriter::~PyTorchStreamWriter() + 0x125 (0x1514c3a7b3b5 in /usr/local/lib/python3.8/dist-packages/torch/lib/libtorch_cpu.so)
frame #6: + 0x6d90a5 (0x15155c0700a5 in /usr/local/lib/python3.8/dist-packages/torch/lib/libtorch_python.so)
frame #7: + 0x36658f (0x15155bcfd58f in /usr/local/lib/python3.8/dist-packages/torch/lib/libtorch_python.so)
frame #8: + 0x36747f (0x15155bcfe47f in /usr/local/lib/python3.8/dist-packages/torch/lib/libtorch_python.so)
frame #9: /usr/bin/python3.8() [0x5ec780]
frame #10: /usr/bin/python3.8() [0x5441f8]
frame #11: /usr/bin/python3.8() [0x54424a]
frame #12: _PyEval_EvalFrameDefault + 0x286b (0x56d1eb in /usr/bin/python3.8)
frame #13: _PyFunction_Vectorcall + 0x1b6 (0x5f6836 in /usr/bin/python3.8)
frame #14: _PyEval_EvalFrameDefault + 0x72e (0x56b0ae in /usr/bin/python3.8)
frame #15: _PyEval_EvalCodeWithName + 0x26a (0x56939a in /usr/bin/python3.8)
frame #16: _PyFunction_Vectorcall + 0x393 (0x5f6a13 in /usr/bin/python3.8)
frame #17: _PyEval_EvalFrameDefault + 0x190c (0x56c28c in /usr/bin/python3.8)
frame #18: _PyFunction_Vectorcall + 0x1b6 (0x5f6836 in /usr/bin/python3.8)
frame #19: _PyEval_EvalFrameDefault + 0x85a (0x56b1da in /usr/bin/python3.8)
frame #20: _PyEval_EvalCodeWithName + 0x26a (0x56939a in /usr/bin/python3.8)
frame #21: _PyFunction_Vectorcall + 0x393 (0x5f6a13 in /usr/bin/python3.8)
frame #22: _PyEval_EvalFrameDefault + 0x56b5 (0x570035 in /usr/bin/python3.8)
frame #23: _PyFunction_Vectorcall + 0x1b6 (0x5f6836 in /usr/bin/python3.8)
frame #24: _PyEval_EvalFrameDefault + 0x72e (0x56b0ae in /usr/bin/python3.8)
frame #25: _PyFunction_Vectorcall + 0x1b6 (0x5f6836 in /usr/bin/python3.8)
frame #26: PyObject_Call + 0x1f7 (0x5f3547 in /usr/bin/python3.8)
frame #27: _PyEval_EvalFrameDefault + 0x1f4d (0x56c8cd in /usr/bin/python3.8)
frame #28: _PyEval_EvalCodeWithName + 0x26a (0x56939a in /usr/bin/python3.8)
frame #29: _PyFunction_Vectorcall + 0x393 (0x5f6a13 in /usr/bin/python3.8)
frame #30: _PyEval_EvalFrameDefault + 0x72e (0x56b0ae in /usr/bin/python3.8)
frame #31: _PyFunction_Vectorcall + 0x1b6 (0x5f6836 in /usr/bin/python3.8)
frame #32: PyObject_Call + 0x1f7 (0x5f3547 in /usr/bin/python3.8)
frame #33: _PyEval_EvalFrameDefault + 0x1f4d (0x56c8cd in /usr/bin/python3.8)
frame #34: _PyEval_EvalCodeWithName + 0x26a (0x56939a in /usr/bin/python3.8)
frame #35: _PyFunction_Vectorcall + 0x393 (0x5f6a13 in /usr/bin/python3.8)
frame #36: _PyEval_EvalFrameDefault + 0x56b5 (0x570035 in /usr/bin/python3.8)
frame #37: /usr/bin/python3.8() [0x6b2b5c]
frame #38: _PyEval_EvalFrameDefault + 0x72e (0x56b0ae in /usr/bin/python3.8)
frame #39: _PyEval_EvalCodeWithName + 0x26a (0x56939a in /usr/bin/python3.8)
frame #40: PyEval_EvalCode + 0x27 (0x68d047 in /usr/bin/python3.8)
frame #41: /usr/bin/python3.8() [0x67e351]
frame #42: /usr/bin/python3.8() [0x67e3cf]
frame #43: /usr/bin/python3.8() [0x67e471]
frame #44: PyRun_SimpleFileExFlags + 0x197 (0x67e817 in /usr/bin/python3.8)
frame #45: Py_RunMain + 0x212 (0x6b6fe2 in /usr/bin/python3.8)
frame #46: Py_BytesMain + 0x2d (0x6b736d in /usr/bin/python3.8)
frame #47: __libc_start_main + 0xf3 (0x15158ed860b3 in /usr/lib/x86_64-linux-gnu/libc.so.6)
frame #48: _start + 0x2e (0x5fa5ce in /usr/bin/python3.8)

train.sh: line 21: 7442 Aborted (core dumped) fairseq-train ../../data/DDI/relis-bin --save-dir ${SAVE_DIR} --user-dir ../../src --finetune-from-model ../../checkpoints/Pre-trained-BioGPT/checkpoint.pt --task language_modeling_prompt --arch transformer_lm_prompt_biogpt --share-decoder-input-output-embed --decoder-learned-pos --optimizer adam --adam-betas '(0.9, 0.98)' --weight-decay 0.01 --clip-norm 0.0 --lr 1e-4 --lr-scheduler inverse_sqrt --warmup-updates 500 --warmup-init-lr 1e-07 --tokens-per-sample 1024 --max-source-positions 640 --max-target-positions 1024 --max-tokens 1024 --update-freq 32 --skip-invalid-size-inputs-valid-test --max-epoch 100 --keep-last-epochs 5 --learned-prompt 9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant