-
Notifications
You must be signed in to change notification settings - Fork 2
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
size of runs #86
Comments
|
@coderdj Ciao Dan, I'm trying to understand how to implement the "size" entry on database starting from your example. We can calculate the total size of raw data and put here the final value for the update: But one thing is not clear to me. How you calculate the size of the raw data. Of course will be the sum of all *.zip files, but I see from your code that you can print this command I see in python manual that there is in Maybe I can write a little function that calculate this value. For the moment I'm trying to run you example but give me this error: I need some particular permission from my account?? |
Hi Francesco,
I don't understand the link. The specific line you link is only invoked in case of a checksum error, right? So exactly that spot would no be the place to put this. In general putting the size calculation in the initial checksum creation is probably the best place.
Luca sent me a list of runs and their sizes in csv. All I did was put this information into the run docs. It was not for all runs and some of the sizes were only precise to two decimals (in units of GB, but I put them in the runs DB in bytes). I think if we don't have every file in there this may be of limited use, but I don't know how to access the rest of the runs.
You want the size of the directory calculated recursively. You'll probably need a small function to do it. You can always look online, there are a lot of examples how to do this.
Well, the name or service "gw:27017" is not known to your PC. That's the runs DB address from within the DAQ. Just replace it with whatever address you usually use for connecting to the runs database. ciao, |
@lucrlom You should make a new Dan's example script shows what field you need to fill ( This task should of course eventually be running on |
@pdeperio @lucrlom are you waiting for me to comment on this? There's tons of example snippets in cax already, right? For example the code used to add a checksum is analog: https://github.com/XENON1T/cax/blob/master/cax/tasks/checksum.py#L97 |
We would need CAX to calculate the size of a run (in GB) and fill a column of the runDB with this information. This is very important for computing in order to estimate how much space is needed to transfer entire campaigns in the future. Dan Coderre told to contact him to create the proper column in the DB. (Someone suggested that a good moment to do this would do when you do checksum...)
The text was updated successfully, but these errors were encountered: