Rapids Execution - Getting Errors

[sureshksampath]

Hello,
I have completed the configuration steps and executed the Rapids package as per the instructions, however I got some errors. I have manually copied the Fitbit CSV files in the directory: /rapids/data/external/fitbit_csv. Attached are the config.yaml and log files. Appreciate if you could provide any pointers.

Thanks,
Suresh
config.yaml.txt
2023-06-06T194603.987986.snakemake.log

[jenniferfedor]

Hi @sureshksampath,

Thank you for attaching these files. Unfortunately the log does not record the actual error message, but because the error occurred during the pull_wearable_data rule and based on your config file, I believe the issue is that the CONTAINER parameters in the config file have not been changed to the CSV files containing all participants' raw data for each Fitbit sensor. For example, for the Fitbit calories intraday sensor, your config file should look something like the following:

FITBIT_CALORIES_INTRADAY:
  CONTAINER: fitbit_calories_intraday_raw.csv

Please feel free to refer to the data stream configuration and Fitbit parsed CSV data stream sections of our documentation and let us know if you have additional questions.

[sureshksampath]

Thanks @jenniferfedor! I am able to run the Rapids package without any errors after making some minor corrections to the Config.yaml file - although some output files have not been generated. Attached are the Config and log files. Appreciate all your help!

config.yaml.txt
2023-06-07T144154.812857.snakemake.log

changes

[jenniferfedor]

Hi @sureshksampath, that’s great! Could you please let us know which output files you were expecting but were not generated? Currently your config file is set up to compute only Fitbit calories intraday features and, based on the log, it looks like the appropriate set of output files was created. If you were expecting features to be computed for other Fitbit sensors, for each of those sensors, you will need to update the following in your config file:

• Change the CONTAINER parameter to the CSV file containing the raw data for all participants (please note that an exception to this is Fitbit data yield)
• Change the COMPUTE flag for the feature provider(s) of interest to True

So for example, if you would also like to compute features for the Fitbit heart rate summary sensor, the corresponding section of your config file should look something like the following:

FITBIT_HEARTRATE_SUMMARY:
  CONTAINER: fitbit_heartrate_summary_raw.csv
  PROVIDERS:
    RAPIDS:
      COMPUTE: True

The Behavioral Features section of our documentation explains the structure of the sensor feature sections in the config file in case that is helpful. Thanks!

[sureshksampath]

@jenniferfedor: Thanks again for your prompt response! I wanted to first try out for one of the behavioral features "FITBIT_CALORIES_INTRADAY", and if it works, I can do for the others. The following output files which are part of process_time_segments rule are mentioned in the log file, however they aren't generated:
data/interim/time_segments/p02_time_segments.csv, data/interim/time_segments/p02_time_segments_labels.csv, data/interim/time_segments/p01_time_segments.csv, data/interim/time_segments/p01_time_segments_labels.csv

[jenniferfedor]

Thank you for clarifying, @sureshksampath. I’m not sure how that is possible given that those files are inputs for subsequent steps of the pipeline (e.g., fitbit_calories_intraday_r_features rule) that were executed successfully according to your log file. I also checked several of our own studies for which we’ve recently processed features and those files are present for our participants in all cases.

Could you please confirm that those files indeed do not exist anywhere within your rapids directory (e.g., with find . -name *time_segments*) and if not, could you please upload the following files so we can attempt to replicate your results? Thanks!

• data/external/participant_files/p01.yaml
• data/external/participant_files/p02.yaml
• data/external/timesegments_periodic_daily.csv
• data/external/fitbit_csv/fitbit_calories_intraday_raw.csv (if this differs from our test data)

[sureshksampath]

Hello @jenniferfedor - I just verified and these files do exist with some content as below. Sorry about any confusion. Thanks again!

(rapids) root@03795c0f019f:/rapids# cat data/interim/time_segments/p02_time_segments.csv
label,start_time,length,repeats_on,repeats_value,overlap_duration,overlap_id,original_label
daily_RR0SS,00:00:00,23H 59M 59S,every_day,0,0D,0,daily
(rapids) root@03795c0f019f:/rapids#
(rapids) root@03795c0f019f:/rapids# cat data/interim/time_segments/p02_time_segments_labels.csv
label
daily_RR0SS
(rapids) root@03795c0f019f:/rapids#
(rapids) root@03795c0f019f:/rapids# cat data/interim/time_segments/p01_time_segments.csv
label,start_time,length,repeats_on,repeats_value,overlap_duration,overlap_id,original_label
daily_RR0SS,00:00:00,23H 59M 59S,every_day,0,0D,0,daily
(rapids) root@03795c0f019f:/rapids#
(rapids) root@03795c0f019f:/rapids# cat data/interim/time_segments/p01_time_segments.csv
label,start_time,length,repeats_on,repeats_value,overlap_duration,overlap_id,original_label
daily_RR0SS,00:00:00,23H 59M 59S,every_day,0,0D,0,daily
(rapids) root@03795c0f019f:/rapids#