add more sites

[erikarasnick]

No description provided.

[erikarasnick]

@cole-brokamp can you check that the new isochrone files are at s3://geomarker/drivetime/isochrones/ (one of the new ones should be vandy_isochrones.rds) and that the permissions are set to public for all of them? I need to renew my transmit subscription, so I don't have easy access to s3 right now to check. Thanks.

[cole-brokamp]

I'm trying to wean myself off of Transmit too. Here's an aws cli command that will do that:

aws s3 ls s3://geomarker/drivetime/isochrones/ | grep vandy

It returned nothing for me, so I did this one to double check:

aws s3 ls s3://geomarker/drivetime/isochrones/ | grep cchmc
2022-12-16 10:21:48     195502 cchmc_isochrones.rds

That did return what I expected, so I think that it is not available.

[cole-brokamp]

I'm trying to wean myself off of Transmit too. Here's an aws cli command that will do that:

aws s3 ls s3://geomarker/drivetime/isochrones/ | grep vandy

It returned nothing for me, so I did this one to double check:

aws s3 ls s3://geomarker/drivetime/isochrones/ | grep cchmc
2022-12-16 10:21:48     195502 cchmc_isochrones.rds

That did return what I expected, so I think that it is not available.

Doing more digging and it is available, but not in the isochrones folder:

> aws s3 ls --recursive s3://geomarker/drivetime/ | grep vandy
2023-08-22 09:38:10     301888 drivetime/vandy_isochrones.rds

[cole-brokamp]

Everything in s3://geomarker/drivetime is private, except for all files in the isochrones subfolder are public. These seem to be a duplicate of the ones in the drivetime folder, without the new ones, e.g. vandy.

[erikarasnick]

I tried to move the whole isochrones folder but I think I misunderstood the usage of recursive. I think they are there now. what is the easiest way to delete the isochrones I put in the drivetime folder?

[cole-brokamp]

aws s3 rm is good to use for this. Run with --dry-run to see the list of files you will delete with that command first:

aws s3 rm --dryrun --recursive s3://geomarker/drivetime/isochrones

Then you can run without the --dryrun after you verify the output is what you want to delete, (which it looks like it is, but I'll let you run the command).

[erikarasnick]

I think I want to keep the ones in the isochrones folder and delete the ones from the drivetime folder. i.e.,

delete: s3://geomarker/drivetime/al_isochrones.rds
but keep: s3://geomarker/drivetime/isochrones/al_isochrones.rds

[erikarasnick]

also, is there a way to check the permissions using the cli?
I'm still getting invalid response status 403

[erikarasnick]

note: cleveland clinic address provided by pac3 here was already included

[cole-brokamp]

Not without going a little deeper with the aws cli tool. I made those changes in the AWS S3 console -- you don't have access to that right now, but I think we can change that if you want to use it in the future.

[cole-brokamp]

I think the error messages are a little deceiving here because all of the Dockerfile steps are run in parallel. If one of them errors, then all other ones are canceled, and all of those 403 errors appear. This usually isn't a problem, but might be an issue with this many large number of Dockerfile instructions steps. I figured out the offending line by commenting out all but one of the ADD steps and narrowed it down to center_addresses.csv. I fixed permissions on that now and I'll retrigger our checks here. It looks to be building locally okay for me right now.

[erikarasnick]

Oh, good catch. I didn't think about the center_addresses file. It looks like now we are having package installation problems. Specifically for the openrouteservice package, which isn't actually needed to run the container, only to download the isochrones. Should we just remove it from renv.lock?

[cole-brokamp]

I would avoid manually editing renv.lock since we won't be able to reproduce that. We should try to find a way to avoid renv::snapshot() from picking up package dependencies that are not needed by the docker image: https://rstudio.github.io/renv/reference/dependencies.html#ignoring-files

[brandtaj24]

We need drivetime codes created for these 5 PAC3 centers. If time or resources is a concern, could we please prioritize creating a drivetime code for Vanderbilt? Their files are prepped and ready to run.

PAC3 Need DriveTime Codes Sep 2023.xlsx

[cole-brokamp]

We need drivetime codes created for these 5 PAC3 centers. If time or resources is a concern, could we please prioritize creating a drivetime code for Vanderbilt? Their files are prepped and ready to run.

PAC3 Need DriveTime Codes Sep 2023.xlsx

Thanks for the update, we will get to adding these soon!

[cole-brokamp]

continued in #16