Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature/reorganize dewrangle #6

Merged
merged 7 commits into from
Apr 2, 2024
Merged

Conversation

sickler-alex
Copy link
Contributor

Pull Request Name

  • closes #xxxx
  • README entry added if new functionality
  • fixup commits are appropriately squashed

[Description here]

This PR adds functions for checking Dewrangle job status and download results. It also reorganizes the Dewrangle functions to put common functions in a separate file then the ones handling hashing volumes and downloading jobs.

@calkinsh
Copy link

calkinsh commented Apr 1, 2024

is this related to d3b-420?

@HuangXiaoyan0106
Copy link
Contributor

@sickler-alex I got this error when I run d3b dewrangle list_jobs

% d3b dewrangle list_jobs -study sd-86n4tk8ekn -bucket d3b-study-us-east-1-prd-sd-32sc9wns
Vm9sdW1lOmNsc3pxODk4cDAwMDI4c29qN3VtODk2cmc=
Traceback (most recent call last):
  File "/Users/xiaoyan/opt/anaconda3/bin/d3b", line 8, in <module>
    sys.exit(main())
  File "/Users/xiaoyan/opt/anaconda3/lib/python3.8/site-packages/d3b_dff_cli/cli.py", line 149, in main
    args.func(args)
  File "/Users/xiaoyan/opt/anaconda3/lib/python3.8/site-packages/d3b_dff_cli/modules/dewrangle/list_jobs.py", line 70, in main
    list_volume_jobs(
  File "/Users/xiaoyan/opt/anaconda3/lib/python3.8/site-packages/d3b_dff_cli/modules/dewrangle/list_jobs.py", line 28, in list_volume_jobs
    jobs = hf.get_volume_jobs(client, volume_id)
  File "/Users/xiaoyan/opt/anaconda3/lib/python3.8/site-packages/d3b_dff_cli/modules/dewrangle/helper_functions.py", line 385, in get_volume_jobs
    comp = datetime.strptime(
TypeError: strptime() argument 1 must be str, not None

@sickler-alex
Copy link
Contributor Author

@HuangXiaoyan0106 try again. It looks like there was a group of jobs running today. I'm not sure what timezone Dewrangle uses. Maybe those jobs were running when you were testing this? The command is working for me now.

Copy link
Contributor

@HuangXiaoyan0106 HuangXiaoyan0106 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It works for me.

@sickler-alex sickler-alex merged commit d554e7c into master Apr 2, 2024
@sickler-alex sickler-alex deleted the feature/reorganize_dewrangle branch April 2, 2024 14:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants