Overflow of timestamps in KS4

[seto0425]

Describe the issue:

Hi! I am using Kilosort 4.0.10 to perform spike sorting on data recorded with a 64-channel silicon probe for over 40 hours (sampling rate: 20k). During this process, I encountered an issue with the timestamps, as seen in the Amplitude View, Feature View, and Firing Rate View (probably due to overflow). It seems that this issue is caused by settings of the data type in KS4, and I believe it can be improved by changing to a larger data type (int64 or uint32). However, I am not sure where to modify the code to fix this issue. Could you please guide me on which part of the code should be changed?
Thank you in advance!

Reproduce the bug:

No response

Error message:

No response

Version information:

Kilosort 4.0.10

Context for the issue:

No response

Experiment information:

No response

[jacobpennington]

Hello,

Spike times are already saved as int64 by Kilosort. Can you please try checking the spike times in a script / notebook instead of looking at Phy? It does look like an overflow is happening, but that might be happening in Phy.

Something like this should confirm if the values are stored correctly:

import numpy as np
import matplotlib.pyplot as plt

# Replace path with wherever your results were saved
st = np.load('C:/users/jacob/.kilosort/.test_data/kilosort4/spike_times.npy')
_ = plt.hist(st/30000, bins=100)

[seto0425]

Thank you. I tried to check the spike times as shown below. The values may indicate overflow, but the data type is int64 in the spike_time. If there are any other possible solutions, please let me know.

[jacobpennington]

Thanks, looking into it.

[jacobpennington]

@seto0425 Sorry for the delay, this has been a difficult one. I fixed a related overflow in an earlier part of the pipeline, but I have still not found the source of this. Every variable assignment and intermediate computation I can find that deals with spike times uses a 64-bit data type so far as I can tell. So, I have a couple follow-up questions:

1. Are there any other steps you're applying before or after sorting, like using SpikeInterface or doing your own post-processing?

2. Would it be feasible for you to sort it again with some modifications to the code? I would make a new branch for you with the required changes, which are a bunch of log statements to narrow down which step the overflow is happening at.

[seto0425]

Thank you for your reply.

1. I am not using SpikeInterface or doing any of my own post-processing. The data I am using is in the original dat file recorded with the OpenEphys GUI.
2. Of course, it is possible. Please advise on how to proceed.

[jacobpennington]

Thank you! These would be the steps to follow, please let me know if you need any clarifications:

Download repository and install in editable mode in a new kilosort environment (modifying the last line if needed for your cuda version):

git clone [email protected]:MouseLand/Kilosort.git
conda create --name kilosort_new python=3.9
conda activate kilosort_new
pip install -e Kilosort[gui]
pip uninstall torch
conda install pytorch pytorch-cuda=11.8 -c pytorch -c nvidia

Checkout branch for this issue:

git checkout issue_715

Then run Kilosort4 however you did before. After sorting, there should be a kilosort4.log file in the results directory that you can upload here.

[seto0425]

I will send you the Kilosort4.log file. Is this okay with you? Also, I recently noticed that there were no issues related to overflow in the drift plot displayed during Kilosort running.

kilosort4.log

[jacobpennington]

Was that the correct log file? It looks like that one stopped with an error very quickly, during drift correction.

[seto0425]

I apologize for the delayed response due to another issue. After updating from KS4.0.10 to the new version, the program no longer runs correctly and encounters an error, even though I am using the same recording data files as before. I have tried KS4.0.12 and 4.0.14, but both seem to stop with the same error.
I apologize for the delayed response due to another issue. After updating from KS4.0.10 to the new version, the program no longer runs correctly and encounters an error, even though I am using the same recording data files as before. I have tried KS4.0.12 and 4.0.14, but both seem to stop with the same error.

I am attaching the 'kilosort.log' file and the contents of the ‘Message log box’ obtained from the GUI (Kilosort version 4.0.14.dev2) for your reference.

I am currently trying to resolve this issue on my end as well. Although it deviates from the main topic of this thread, if you have any solutions for this error, could you please let me know?

kilosort4_0_14_dev_20240731.log
Message log box_KS4_0_14_dev_20240731.txt

[jacobpennington]

@seto0425 Sorry for the delay. I just pushed a fix for this, it will be live on pypi later today as v4.0.16.