Update to income metric. #456
Conversation
…e AWS method for data storage and check data on final evaluation forms
| --- | ||
| title: "Run employment" | ||
| date: today | ||
| format:html |
There was a problem hiding this comment.
| format:html | |
| format: html |
| quarto_render_move <- function( | ||
| input, | ||
| output_file = NULL, | ||
| output_dir = NULL, | ||
| ... | ||
| ) { | ||
|
|
||
| # Get all the input / output file names and paths | ||
| x <- quarto::quarto_inspect(input) | ||
| output_format <- names(x$formats) | ||
| output <- x$formats[[output_format]]$pandoc$`output-file` | ||
| if (is.null(output_file)) { output_file <- output } | ||
| input_dir <- dirname(input) | ||
| if (is.null(output_dir)) { output_dir <- input_dir } | ||
| output_path_from <- file.path(input_dir, output) | ||
| output_path_to <- file.path(output_dir, output_file) | ||
|
|
||
| # Render qmd file to input_dir | ||
| quarto::quarto_render(input = input, ... = ...) | ||
|
|
||
| # If output_dir is different from input_dir, copy the rendered output | ||
| # there and delete the original file | ||
| if (input_dir != output_dir) { | ||
|
|
||
| # Try to make the folder if it doesn't yet exist | ||
| if (!dir.exists(output_dir)) { dir.create(output_dir) } | ||
|
|
||
| # Now move the output to the output_dir and remove the original output | ||
| file.copy( | ||
| from = output_path_from, | ||
| to = output_path_to, | ||
| overwrite = TRUE | ||
| ) | ||
| file.remove(output_path_from) | ||
|
|
||
| # If the output_dir is the same as input_dir, but the output_file | ||
| # has a different name from the input file, then just rename it | ||
| } else if (output_file != output) { | ||
| file.rename(from = output_path_from, to = output_path_to) | ||
| } | ||
| } |
There was a problem hiding this comment.
I do really like this parametrized report approach and think in the next round of updates we should encourage it + add guidance. This quarto_render_move function is excellent and I think it should belong in the functions folder so all metrics can use it. With the current approach it will be duplicated in many folders in the repo!
This comment is more food for thought for 2026, no action needed currently
|
|
||
| + The ACS micro data used to create this metric is large and will take time to read in. It is strongly recommended that you use a server with large computing power to run this. | ||
| + Please check that no folders in the filepath that you have cloned this repostiory to include the acronym "UMF" - this will throw an error from the extract_ipums function. To check your file path you can use the function here::here(). | ||
| *User warning* The ACS micro data used to create this metric is large and will take time to read in. It is strongly recommended that you use a server with significant computing power to run this. |
There was a problem hiding this comment.
Adding a recommended server or listing one that was used successfully to update the metric would be helpful for posterity!
| @@ -0,0 +1,99 @@ | |||
| --- | |||
| title: "Run employment" | |||
There was a problem hiding this comment.
| title: "Run employment" | |
| title: "Run income" |
|
@malcalakovalski Apologize this intitial PR was a bit half-baked and did not include subgroups - they have been added. As well as the final evaluations. You should be good to proceed with your review. |
There was a problem hiding this comment.
Everything looks great except that I would incorporate the logic of functions/testing/evaluate_final_data_srv_quantile.R into the evaluate_final_data function used across the project. The new function is the same except that it allows for NA values for lb and ub in confidence intervals. However, this could be handled by adding an argument for whether the data is survey data or not in the function and do if else logic based on that. That's okay for this round, but I'm flagging this for the future!
This PR updates the income metric. Includes a back-update for years going back to 2014 and brings in 2023 census data + adds in a gender subgroup.
A) Addition of the gender subgroup is notable, please have a look at this in the county and place sub files.
B) Similar to your comment Manu, this codes major changes reflect other ACS metrics in the use of the aws.s3 system. Checking if the application is consistent and looking out for potential bugs/errors would be great.