Error running mean_waveforms module from curated kilosort output

[edwardyan95]

Hi, I was trying to use the mean_waveforms module and I get the following error towards the end of the analysis.

peak_channels[cluster_idx],
IndexError: index 539 is out of bounds for axis 0 with size 539

From what I can tell, it results from the fact that template.npy isn't updated after manual curation in phy, so merging and spliting gives rise to new cluster ids that are not in the original template.npy. The size of peak_channels depends on the size of template (line 87, extract_waveforms.py) but the cluster_ids are from spike_clusters which are changed following curation(line 79, extract_waveforms.py). Currently my workaround is to run the module on the original kilosort output and manually map the original clusters to the manually curated clusters. I'm curious if there's any way that I can run the module on curated kilosort output directly?

Another thing that I noticed is that on line 195 in utils.py, the cluster_quality is inferred from 'group' column of the cluster_group.tsv. However, it seems the kilosort now outputs cluster_group.tsv with 'KSLabel' instead of 'group'.

I really appreciate your work and it has helped a lot with my research, so just want to say thank you!

[jsiegle]

Thanks, this is all helpful to know! Can you share a dataset that's causing these errors? That will make it much easier to troubleshoot.

[edwardyan95]

Hi, my dataset is really large, so it's difficult to share for now. I will record a short session and then share the data with you. Thanks a lot!

[FilippoHei]

I hav ethe same issue, when running the quality metrics on curated data, due to newly splitted and merged data. Any work-around on that?

[edwardyan95]

My work around is to add templates for manually added clusters (during curation) to the spike_templates matrix. I can send you the code if you want

[moscraciunxxx]

@edwardyan95 could you share this with me please ? I thought this might be one of the solutions on this matter....

[edwardyan95]

Hi, sure, you should be able to access it in my public forked repository. I only changed code in main.py file under ../modules/mean_waveforms/

[moscraciunxxx]

@edwardyan95 let ma have a look at it then. Thanks!

[moscraciunxxx]

Hello, I have modified the PHY’ GUI actual code files and now I got what I needed.  Let me know if this works for you. Regards, Vitalie Cervinschi, MSc graduate, Goldsmiths, University of London Tuesday, 11 July 2023 at 08:07 -07:00 from ***@***.*** ***@***.***>:

…

>My work around is to add templates for manually added clusters (during curation) to the spike_templates matrix. I can send you the code if you want Hi, I have the same issue. It will be greatly helpful if you can share it with me. Thanks! Shivam — Reply to this email directly, view it on GitHub , or unsubscribe . You are receiving this because you commented. Message ID: <AllenInstitute/ecephys_spike_sorting/issues/56/1631003857 @ github . com>

[XiaochenFu]

I got the same issue after splitting the unit clusters in phy. Would there be a easy way to fix this without doing curation again?

[edwardyan95]

I got the same issue after splitting the unit clusters in phy. Would there be a easy way to fix this without doing curation again?

Please see above comment, you can see my shared repository with an added block of code which should resolve this