Timeout Issue with POD5 conversion

[jainronit]

Hi,

I am trying to convert a FAST5 file to POD5, but the program just hangs and eventually gives a timeout error. This is the command I am running:

pod5 convert fast5 FAY49394_pass_0b33d99d_65450f2d_386.fast5 --output converted.pod5

And this is the error I am getting:

Here is the output log of the command:
2024-06-24--23-56-22-main-pod5.log

[HalfPhoton]

Hi @jainronit,
This looks like it could be an issue with the fast5 file.

Can you try the script here to find issues in the fast5 file?

[jainronit]

Hi @HalfPhoton,
I ran the script on the 1 FAST5 file that I am trying to convert and it seems that there is no issue. The output was:

Found 0 issues from 1 input paths

Do you know if there is any further troubleshooting I could do? I also tried running this FAST5 file through the online tool for FAST5 to POD5 conversion, but got an error:

[HalfPhoton]

Do you have any other log files other than ...main-pod5.log?
There should be another one for the conversion sub-process which might be insightful.

Kind regards,
Rich

[jainronit]

Hi @HalfPhoton,

I don't see any other log files other than the main-pod5.log one that I provided. I just prefaced my command with POD5_DEBUG=1 and only 1 log file was produced. Is there some other command I should be using?

[HalfPhoton]

Hmmm. This is very unusual.
Your fast5 file seems to be causing issues. Are you able to share it?

[jainronit]

Yes, I can share it! Github won't let me attach it directly as a file so is there another way I could send it to you (i.e. email, messaging, etc.)?

[HalfPhoton]

To securely send the data to use, can I ask you to contact the ONT Technical Support team via https://nanoporetech.com/support

If you click Contact Support > on that page you will be directed to the chatbot, where you can go to
"See all topics" -> "Software" -> "MinKNOW" -> "Linux" -> choose anything -> scroll down and hit "I still need support" -> "Technical/Product Support" -> enter your email -> "Create a support ticket".

You will then be able to log a support ticket with the team. If you can attach your log file and reference this github issue and mention that Dorado team has requested this, Technical Support will be able to forward this data to me quickly.

Thanks

[jainronit]

I went to create a ticket, but it still won't let me attach a fast5 file. Is there some file conversion I need to do before sending the fast5 file, or is there another way to send it?

[HalfPhoton]

Hi @jainronit, apologies for how long this is taking.

Can you submit the TS ticket and request that they provide you with a method of uploading the file to them. This should make its way to me then 🤞.

I'll contact TS to get this moving ASAP.

Thanks for your patience.

Kind regards,
Rich

[jainronit]

Hi @HalfPhoton, no worries, thanks for all your help! I just submitted the TS ticket requesting a method of sending the FAST5 file over to you. Let me know once they get in touch with you and hopefully I'll be able to get the FAST5 file sent to you then.

[HalfPhoton]

@jainronit,
I've found a better way to support you.
I've sent you an email ending in .edu which should contain a URL to securely upload the data to ONT servers which I can reach.

Internal reference: 317063CA055919242B

Kind regards,
Rich

[jainronit]

@HalfPhoton Perfect, thanks so much! I just uploaded the FAST5 file to the ONT server.

[HalfPhoton]

@jainronit , I emailed you directly with more information

[HalfPhoton]

Hi @jainronit,

I've received your file and I was able to convert it first-time

~/w/pod5_issue_134 [1]$ pod5 convert fast5 FAY49394_pass_0b33d99d_65450f2d_386.fast5 -o foo.pod5 -f
Converting 1 Fast5s: 100%|#####################| 4000/4000 [00:02<00:00, 1881.07Reads/s]
~/w/pod5_issue_134 $ pod5 view foo.pod5 | head -n 2
read_id	filename	...
0005cf56-82e9-469d-8494-af70a62a4f02	foo.pod5	...

Can you tell me more about your system?

[jainronit]

Hi @HalfPhoton,

I ran the command you used and also was able to convert the FAST5 file successfully--it seems that adding the -f flag fixed the issue. Could you share what that flag's functionality is?

Also, I am still getting the same "memory access out of bounds" error when I try to perform the conversion using the online tool--do you know what the issue there might be?

[HalfPhoton]

pod5 merge fast5 --help
...
  -f, --force-overwrite
                        Overwrite destination files (default: False)

This should have had no effect on the merge process as it just deletes the output file if it already exists. This is only there because I ran it myself multiple times to double check that it worked every time and left it in the message.

Can you try again without -f and setting a new output filename? If this doesn't work then this is very unusual.

I can ask the online tool devs to take a look at the file you sent me to see what the issue is.

Happy to hear that you're unblocked though! That's a win even if we aren't sure why 😄

Best regards,
Rich

[jainronit]

I deleted the POD5 file that was created and re-ran the exact same command (with the -f flag), but this time it doesn't work and just hangs again:

I'm rather confused on how that's possible as this is the same command that just ran successfully a couple minutes ago, so there seems to be some kind of stochasticity at play here? Not sure what to make of it.

For context, I'm running this command in a high-performance computing cluster environment without allocation of extra resources. Perhaps it's a memory issue and I need to allocate a compute node? Where are you running your commands from?

[HalfPhoton]

I see (base) - Are you running from a conda environment and not a python venv? We do see issues when people use conda environments. POD5 doesn't have a conda package maintained by ONT yet.

Please try again using a python venv:

python3.10 -m venv venv --prompt="pod5"
source venv/bin/activate
pod5 convert fast5 FAY49394_pass_0b33d99d_65450f2d_386.fast5 --output not_conda.pod5

[jainronit]

I tried again using a python venv, but now I just get an "aborted" message:

[HalfPhoton]

And this generated no logs with POD5_DEBUG=1?
Can you share you system information please?

[jainronit]

No, there doesn't seem to be a log. I am running this on a MacOS Sonoma Version 14.3 with 16 GB memory.