Path problem creating HDSM

[rnkazman]

When I try to run the step that creates the JSON HDSM in dv8_showcase I am getting another path problem. I am not sure what to do here.

hdsmj_path <- transform_gitlog_to_hdsmj(project_git,

•                                     hdsmj_path = file.path(project_path,paste0(project_name,"-hdsm.json")))
  

Processed 9944 groups out of 9944. 100% done. Time elapsed: 7s. ETA: 0s.
Processed 41720 groups out of 41720. 100% done. Time elapsed: 3s. ETA: 0s.
Warning: cannot open file '../../analysis/pdfbox/dv8//-hdsm.json': No such file or directoryError in file(con, "w") : cannot open the connection

[carlosparadis]

Warning says the file specified on hdsmj_path does not exist. You can open the folder on your computer to check that this is indeed the case. I suspect the issue lies in the project_name variable not being specified and being an empty string. Modify it so it matches the filepath. I don't have enough information to diagnose from the above alone. It could also be the file was not created along the pipeline.

Again, I think running the notebook first with the default config would make it easier to understand how the code works, which was calculator.yml. The dv8-console dumps many intermediate files on every step that Kaiaulu needs to knit together, the functions are mostly syscalls to dv8-console. The Notebooks are intended to be run one code block at a time to familiarize with what is going in the pipeline or make more fine changes. Project-name is specified automatically in line 48:

kaiaulu/vignettes/dv8_showcase.Rmd

Line 48 in ac522b6

project_name <- stringi::stri_split_regex(project_path,pattern = "/")[[1]]

p.s.: Rather than pasting the code here, you can left click the line of code in the source code and select "copy permalink" so I know what portion you are referring to. GitHub will auto-embed as you see above.

[carlosparadis]

Another guess: Does the folder path exists?

I.e. if you go to the analysis folder, is there a pdfbox folder and a dv8 folder within it to match analysis/pdfbox/dv8? I would still check project_name, otherwise your files will be missing the project prefix. You can just assign any string to line 48 if it is easier.

[carlosparadis]

Confirmed on call the analysis folder was part of the issue, but there is also a bug on the one liner of the notebook that tries to get the project_name variable from the folder path. See #338 for details.

[rnkazman]

project_name is defined in dv8_showcase on lines 47-49. It takes project_path as input. The project_path variable has the value ../../analysis/pdfbox/dv8/ which it gets from the config file (via the conf variable). But the subsequent lines of code do not work as expected:

project_name <- stringi::stri_split_regex(project_path,pattern = "/")[[1]]
project_name <- project_name[length(project_name)]

This results in project_name being an empty string. This suggests that get_dv8_folder_path is not returning the correct path, or that the path is not set up correctly in my filesystem. Once again the intended folder structure is baffling to me. Was I supposed to create an "analysis" folder? Where is this documented?

[carlosparadis]

You are "not supposed" to create any folder path. You should specify them on the project configuration file:

kaiaulu/conf/calculator.yml

Line 161 in ac522b6

folder_path: ../../analysis/Calculator/dv8/

DV8 also creates its own folder structure when addressing architectural flaws. It is neither documented, nor do I have control.

[carlosparadis]

One more note:

Every Notebook defines the paths it needs at the start. This is also true for DV8:

kaiaulu/vignettes/dv8_showcase.Rmd

Lines 32 to 57 in ac522b6

	```{r}
	tool <- parse_config("../tools.yml")
	conf <- parse_config("../conf/calculator.yml")
	perceval_path <- get_tool_project("perceval", tool)
	dv8_path <- get_tool_project("dv8", tool)
	# Gitlog parameters
	git_repo_path <- get_git_repo_path(conf)
	# Depends parameters
	depends_jar_path <- get_tool_project("depends", tool)
	language <- get_depends_code_language(conf)
	keep_dependencies_type <- get_depends_keep_dependencies_type(conf)
	# DV8 parameters
	project_path <- get_dv8_folder_path(conf)
	project_name <- stringi::stri_split_regex(project_path,pattern = "/")[[1]]
	project_name <- project_name[length(project_name)]
	flaws_params <- get_dv8_flaws_params(conf)
	# Filters
	file_extensions <- get_file_extensions(conf)
	substring_filepath <- get_substring_filepath(conf)
	filter_commit_size <- get_filter_commit_size(conf)
	```

You will also see the analysis path is defined here:

kaiaulu/vignettes/dv8_showcase.Rmd

Line 36 in ac522b6

dv8_path <- get_tool_project("dv8", tool)

You can also read the getter function path for more detail using the ?get_tool_project R notation:

kaiaulu/R/config.R

Lines 879 to 900 in ac522b6

	#' Returns the path to the folder used to store files for DV8 analysis.
	#'
	#' @description This function returns the path to the folder that will be
	#' used to store various intermediate files for DV8 analysis, that is specified
	#' in the input parameter `config_file`. The input, `config_file` must be a
	#' parsed configuration file. The function will inform the user if the path
	#' path to the folder for intermediate file storage for DV8 analysis exists in
	#' the parsed configuration file, `config_file`.
	#'
	#' @param config_file The parsed configuration file obtained from \code{\link{parse_config}}.
	#' @return The DV8 project folder path.
	#' @export
	get_dv8_folder_path <- function(config_file) {
	project_path <- config_file[["tool"]][["dv8"]][["folder_path"]]
	if (is.null(project_path)) {
	warning("Attribute does not exist in the configuration file.")
	}
	return(project_path)
	}

I will reiterate just in case: If you are using a Notebook, the expectation is to try to learn the functions and follow along with the notebook. It is not intended as a one-click solution to work on every pipeline. This is true for every R package. One button solutions exist on the /exec folder, which defines CLI. Therefore, if you are taking the approach of "Run All", find error, try to fix, "Run All", then there is a mismatch between what Notebooks are for vs what you are hoping to find. The CLI is the preferred way to do that. You trade flexibility for learning curve.

Libraries like ggplot2 in R also define different abstractions. You can either use qplot() for simplified usage, or ggplot, which is much lower level API. So long you are using Notebooks, the expectation is that you will get more familiar with how the API is designed, since there is consistency at function level.