-
Notifications
You must be signed in to change notification settings - Fork 12
/
Copy pathREADME.Rmd
387 lines (294 loc) · 16.9 KB
/
README.Rmd
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
---
output:
md_document:
variant: markdown_github
pdf_document: default
html_document: default
---
<!-- README.md is generated from README.Rmd. Please edit that file -->
# secuTrialR
`r badger::badge_custom("dev version", as.character(packageVersion("secuTrialR")), "blue", "https://github.com/SwissClinicalTrialOrganisation/secuTrialR")` [](https://cran.r-project.org/package=secuTrialR) [](https://github.com/SwissClinicalTrialOrganisation/secuTrialR/actions)
An R package to handle data from the clinical data management system (CDMS) [secuTrial](https://www.secutrial.com/en/).
## Installing from GitHub with devtools
Please note that `R versions >= 3.5` should be used to run `secuTrialR`.
```{r, eval = FALSE}
devtools::install_github("SwissClinicalTrialOrganisation/secuTrialR")
```
## Recommended export options
While the package strives to allow loading of as many types of secuTrial data exports
as possible, there are certain export options which are less likely to cause issues.
If possible it is suggested to export data which adheres to a suggested option set.
Thus, we suggest to work with exports which:
- are **zipped**
- are **English**
- have **reference values** stored **in a separate table**
- contain **Add-IDs**, **centre information**, **structure information**, **form status**, **project setup**
- do **NOT** have the **meta data duplicated** into all tables
- are **UTF-8** encoded
- are **"CSV format"** or **"CSV format for MS Excel"**
- do **NOT** contain form **data of hidden fields**
If you use `read_secuTrial()` to read your export then it will inform you regarding deviations.
We also recommend using short names when exporting your data. Some users have reported issues importing data with long names, but the issues do not exist when using short names. That may (or may not) be related to upgarding SecuTrial.
## Basic usage
An extensive applied manual/vignette is available
[here](https://github.com/SwissClinicalTrialOrganisation/secuTrialR/blob/master/vignettes/secuTrialR-package-vignette.pdf)
and probably the best place to get started.
Load the package
```{r, echo = TRUE, warning=FALSE, message=FALSE}
library(secuTrialR)
```
Load a dataset
```{r}
export_location <- system.file("extdata", "sT_exports", "lnames",
"s_export_CSV-xls_CTU05_long_ref_miss_en_utf8.zip",
package = "secuTrialR")
ctu05 <- read_secuTrial(export_location)
```
This will load all sheets from the export into an object of class `secuTrialdata`, which is basically a list. It will always contain `export_details` (which are parsed from the HTML ExportOptions file that secuTrial generates). By default, it will also contain all other files in the dataset. secuTrialR automatically strips file names of dates. The new file names can be seen via `ctu05$export_options$data_names`. The function also adds [labels to variables](#variable-labels) and data.frames, converts [categorical variables to `factor`s](#prepare-factors) and ensures that [dates are `Date`s and date-times are `POSIXct`](#prepare-dates).
`read_secuTrial` is a wrapper for the functions described below, so it is possible to achieve more flexibility by using the individual functions (if necessary).
Individual tables can be extracted from the `ctu05` object via `tab <- ctu05$tab`, where `tab` is the table of interest.
<details><summary>Wrapped functions</summary>
#### Load the dataset
```{r}
# prepare path to example export
export_location <- system.file("extdata", "sT_exports", "BMD",
"s_export_CSV-xls_BMD_short_en_utf8.zip",
package = "secuTrialR")
# load all export data
bmd_export <- read_secuTrial_raw(data_dir = export_location)
# load a second dataset
export_location <- system.file("extdata", "sT_exports", "lnames",
"s_export_CSV-xls_CTU05_long_ref_miss_en_utf8.zip",
package = "secuTrialR")
ctu05_raw <- read_secuTrial_raw(export_location)
# View names of the bmd_export object
names(bmd_export)
```
`read_secuTrial_raw` returns an object of class `secuTrialdata`, which is basically a list. It will always contain `export_details` (which are parsed from the HTML ExportOptions file that secuTrial generates). By default, it will also contain all other files in the dataset. secuTrialR automatically strips file names of dates. The new file names can be seen via `bmd_export$export_options$data_names`.
<!-- DEDICATED ACCESSOR FUNCTION FOR DATA_NAMES? might already be implemented in the print method -->
`bmd_export` is a list, with class `secuTrialdata`. To prevent it from printing all data to the console, a special print method returns some useful information about the objects within `bmd_export` instead. The information returned includes the original file name in the datafile, it's name in the `secuTrialdata` object, together with the number of rows and columns and a column indicating whether the object is metadata or not:
```{r}
bmd_export
```
Individual tables can be extracted from the `bmd_export` object via `tab <- bmd_export$tab`, where `tab` is the table of interest.
<!-- accessor function? -->
#### Variable labels
For creating tables, it is often useful to have access to variable labels. secuTrialR supports two main methods for handling them - a named list, or via variable attributes. The list approach works as follows.
```{r}
labs <- labels_secuTrial(bmd_export)
# query the list with the variable name of interest
labs[["age"]]
```
The attribute based approach adds labels as an attribute to a variable, which can then be accessed via `label(var)`.
```{r}
labelled <- label_secuTrial(bmd_export)
label(labelled$bmd$age)
```
Labels can be added to new variables or changed via
```{r}
label(labelled$bmd$age) <- "Age (years)"
label(labelled$bmd$age)
```
Where units have been defined in the SecuTrial database, they can be accessed or changed analogously (here, age had no unit assigned, but we can add one).
```{r}
units(labelled$bmd$age)
units(labelled$bmd$age) <- "years"
units(labelled$bmd$age)
```
There is a drawback to the attribute based approach - labels will not be propagated if variables are derived and may be lost if variables are edited.
Currently, `label_secuTrial` should be used prior to `dates_secuTrial` or `factorize_secuTrial` so that labels and units are propagated to factor and date variables.
#### Prepare factors
It is often useful to have categorical variables as factors (R knows how to handle factors). secuTrialR can prepare factors easily.
```{r, error=TRUE}
factors <- factorize_secuTrial(ctu05_raw)
```
This functions loops through each table of the dataset, creating new factor variables where necessary. The new variables are the same as the original but with `.factor` appended (i.e. a new variable called `sex.factor` would be added to the relevant form).
```{r}
# original variable
str(factors$ctu05baseline$gender)
# factor
str(factors$ctu05baseline$gender.factor)
# cross tabulation
table(original = factors$ctu05baseline$gender, factor = factors$ctu05baseline$gender.factor)
```
#### Prepare dates
Date(time)s are a very common data type. They cannot be easily used though in their export format. This is also easily rectified in secuTrialR:
```{r}
dates <- dates_secuTrial(ctu05_raw)
```
Date variables are converted to `Date` class, and datetimes are converted to `POSIXct` class. Rather than overwriting the original variable, new variables are added with the new class. This is a safetly mechanism in case `NA`s are accidentally created.
```{r}
dates$ctu05baseline[c(1, 7), c("aspirin_start", "aspirin_start.date",
"hiv_date", "hiv_date.datetime")]
```
secuTrial exports containing date variables sometimes include incomplete dates. e.g. the day or the month may be missing.
During date conversion (i.e. `dates_secuTrial()`) `secuTrialR` currently creates `NA`s from such incomplete date entries.
Incomplete dates are not approximated to exact dates, since this can lead to false conclusions and biases.
Users are, however, informed about this behaviour with a `warning()`. Subsequent approximation of incomplete dates can be manually performed.
Recommended literature on incomplete dates/date imputation:\
[Dubois and Hebert 2001](https://www.cambridge.org/core/services/aop-cambridge-core/content/view/F50311F9FFAB56176CDDC9FFBF66F655/S1041610202008025a.pdf/imputation_of_missing_dates_of_death_or_institutionalization_for_timetoevent_analyses_in_the_canadian_study_of_health_and_aging.pdf) \
[Bowman 2006](https://www.lexjansen.com/phuse/2006/po/PO11.pdf) \
#### Recommended approach if not using `read_secuTrial`
```{r, eval=FALSE}
f <- "PATH_TO_FILE"
d <- read_secuTrial_raw(f)
l <- label_secuTrial(d)
fa <- factorize_secuTrial(l)
dat <- dates_secuTrial(fa)
# or, if you like pipes
library(magrittr)
f <- "PATH_TO_FILE"
d <- read_secuTrial_raw(f)
dat <- d %>%
label_secuTrial() %>%
factorize_secuTrial() %>%
dates_secuTrial()
```
</details>
### Exploratory helpers
`secuTrialR` has a couple of functions to help get to grips with a secuTrial data export. They are intended to be used in an exploratory manner only.
#### as.data.frame
Working with a list can be tiresome so `secuTrialR` provides a `as.data.frame` method to save the `data.frames` in the list to an environment of your choice.
As a demonstration, we'll create a new environment (`env`) and create the `data.frame`s in there. In practice, using `.GlobalEnv` would probably be more useful.
```{r}
env <- new.env()
ls(env)
names(ctu05)
as.data.frame(ctu05, envir = env)
ls(env)
```
There are also options for selecting specific forms (option `data.frames`), changing names based on a named vector (option `data.frames`) or regex (options `regex` and `rep`), and specifying whether metadata objects should be returned (option `meta`).
#### Recruitment over time
Recruitment is an important cornerstone for every clinical trial. `secuTrialR` allows for straigt forward visualizion of recuitment
over time for a given export file.
```{r, eval = TRUE}
# show plot
# note that there is no line for Universitätsspital
# Basel because only one participant is registered for this centre
plot_recruitment(ctu05, cex = 1.5, rm_regex = "\\(.*\\)$")
# return the plot data
plot_recruitment(ctu05, return_data = TRUE)
```
Furthermore, recruitment per year and center can be returned.
```{r, eval = TRUE}
annual_recruitment(ctu05, rm_regex = "\\(.*\\)$")
```
#### Form status summary statistics
If you are not sure about how complete the data in you export is, it may be useful to get a quick overview of how well the forms
have been filled.
```{r, eval = TRUE}
count_summary <- form_status_summary(ctu05)
tail(count_summary)
```
As you can see, the majority of forms has been completeley filled. None of the forms were saved empty, with warnings or with errors.
For a more participant id centered statistic you can perform the following.
```{r, eval = FALSE}
form_status_counts(ctu05)
```
This will give you a count based overview per participant id and form. Please note that both `form_status_summary`
and `form_status_counts` only work with saved forms since unsaved form data is not available in secuTrial exports.
#### Visit plan
secuTrialR can provide a depiction of the visit structure, although only where the visit plan is fixed:
```{r, eval = FALSE}
vs <- visit_structure(ctu05)
plot(vs)
```
<!-- PLOT METHOD DIRECTLY FOR secuTrialdata objects? -->
#### Data dictionary
It can be difficult to find the variable you're looking for. secuTrialR provides the `dictionary_secuTrial` function to help:
```{r}
head(dictionary_secuTrial(ctu05))
```
#### Linking different forms
Linkages amongst forms can be explored with the `links_secuTrial` function. This relies on the `igraph` package to create a network. It is possible to interact with the network, e.g. move nodes around in order to read the labels better. The device ID is returned to the console, but can be ignored. Forms are plotted in deep yellow, variables in light blue.
```{r, eval=FALSE}
links_secuTrial(bmd_export)
```
<!-- Figure has to be generated outside of the Rmd file - resize the window and select view/"fit to screen", export it to a PDF and then convert it to a PNG -->
#### Sampling random participants
During study monitoring it is common practice to check random participants from a study database. These
participants should be retrieved in a reproducible fashion. The below function allows this for a loaded
secuTrial data export.
```{r}
# retrieve at least 25 percent of participants recorded after March 18th 2019
# from the centres "Inselspital Bern" and "Charité Berlin"
return_random_participants(ctu05, percent = 0.25, seed = 1337, date = "2019-03-18",
centres = c("Inselspital Bern (RPACK)", "Charité Berlin (RPACK)"))
```
## For contributors
### Testing with devtools
```{r, eval = FALSE}
# run tests
devtools::test("secuTrialR")
# spell check -> will contain some technical terms beyond the below list which is fine
ignore_words <- c("AdminTool", "allforms", "casenodes", "CDMS", "codebook",
"codebooks", "datetime" ,"dir" ,"Hmisc" ,"igraph",
"labelled", "mnp", "savedforms", "secutrial", "secuTrial",
"secuTrialdata", "tcltk", "tibble")
devtools::spell_check("secuTrialR", ignore = ignore_words)
```
### Linting with lintr
```{r, eval = FALSE}
# lint the package -> should be clean
library(lintr)
lint_package("secuTrialR", linters = with_defaults(camel_case_linter = NULL,
object_usage_linter = NULL,
line_length_linter(125)))
```
### Building the vignette
```{r, eval = FALSE}
library(rmarkdown)
render("vignettes/secuTrialR-package-vignette.Rmd",
output_format=c("pdf_document"))
```
### Generating the README file
The README file is automatically generated on GitHub via a GitHub action.
### Handling dependencies
Dependencies to other R packages are to be declared in the `DESCRIPTION` file under `Imports:` and in
the specific `roxygen2` documentation of the functions relying on the dependency. It is suggested to
be as explicit as possible. i.e. Just import functions that are needed and not entire packages.
Example to import `str_match` `str_length` `str_wrap` from the `stringr` package (see [read_secuTrial_raw.R](R/read_secuTrial_raw.R)):
```{r, eval = FALSE}
#' @importFrom stringr str_match str_length str_wrap
```
### Preparing a release on CRAN
```bash
# build the package archive
R CMD build secuTrialR
# check the archive (should return "Status: OK", no WARNINGs, no NOTEs)
# in this example for version 0.9.0
R CMD check secuTrialR_0.9.0.tar.gz
```
### Versioning and releases
The version number is made up of three digits. The first digit
is reserved for major releases which may break backwards compatibility.
The second and third digits are used for medium and minor changes respectively.
Versions released on CRAN will be tagged and saved as releases on GitHub.
The version released on CRAN is regarded as the stable version while
the master branch on GitHub is regarded as the current development version.
#### Release checklist
Compile/Update:
* README.Rmd
* vignette
* pkgdown page
* NEWS.md
### Guidelines for contributors
Requests for new features and bug fixes should first be documented as an [Issue](https://github.com/SwissClinicalTrialOrganisation/secuTrialR/issues) on GitHub.
Subsequently, in order to contribute to this R package you should fork the main repository.
After you have made your changes please run the
[tests](README.md#testing-with-devtools)
and
[lint](README.md#linting-with-lintr) your code as
indicated above. Please also increment the version number and recompile the `README.md` to increment the dev-version badge (requires installing the package after editing the `DESCRIPTION` file). If all tests pass and linting confirms that your
coding style conforms you can send a pull request (PR). Changes should also be mentioned in the `NEWS` file.
The PR should have a description to help the reviewer understand what has been
added/changed. New functionalities must be thoroughly documented, have examples
and should be accompanied by at least one [test](tests/testthat/) to ensure long term
robustness. The PR will only be reviewed if all travis checks are successful.
The person sending the PR should not be the one merging it.
A depiction of the core functionalities for loading can be found [here](inst/extdata/graphics/secuTrialR.png).
### Citation [](https://doi.org/10.21105/joss.02816)
If you use and benefit from `secuTrialR` in your work please cite it as:
Wright et al., (2020). secuTrialR: Seamless interaction with clinical trial databases in R.
Journal of Open Source Software, 5(55), 2816, https://doi.org/10.21105/joss.02816