Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Preprocessing patch and added visualizations #158

Open
4 tasks
katiekly opened this issue Mar 13, 2025 · 1 comment
Open
4 tasks

Preprocessing patch and added visualizations #158

katiekly opened this issue Mar 13, 2025 · 1 comment
Labels
bug Something isn't working

Comments

@katiekly
Copy link
Collaborator

katiekly commented Mar 13, 2025

We found that data with low confidence values aren't being filtered from thresholding. This may be because the preprocessing function isn't saving correctly. We found that the processed data is exactly the same as the raw data.

  • Please check/fix that the final processed data is saving to file.

In addition, we believe that the z-scoring of the data is incorrectly applied to each body part as shown below. All the signals in the processed timeseries shouldn't be about 0, implying that the animal's posture isn't preserved. The z-score should be applied to the animal's body parts as a whole to preserve the body part spatial distances by using the distance between the reference points to scale all body parts. Centered reference point should be zeroed.

Image

  • Please update how z-score is applied to preserve animal's posture.

We'd also like to add more visualizations in the scatter and timeseries figures for more clarity.

  • Add 4th column of scatter plots displaying the transformation of the data after the 4th step of preprocessing (outlier_cleaning() + savgal_filtering()). This will help us visualize how the z-scoring of each body part is affecting the animal's posture.
  • Add thresholding only timeseries figure for both x and y after plotting the raw timeseries figures.
@katiekly katiekly added the bug Something isn't working label Mar 13, 2025
@katiekly
Copy link
Collaborator Author

data saved, saves each intermediate steps of the preprocessing

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant