Error messages being squelched

[DomBennett]

Hi!

I'm developing an R package that creates a MonetDBLite database of biological sequence data (https://github.com/ropensci/restez). Users can run automated functions for downloading large amounts of sequence data and store it in the database.

Recently, one user encountered the following error:

: Invalid argument
simpleError in gsub("\n", " ", res, fixed = TRUE): Unable to execute statement 'ROLLBACK'.
Server says 'SQLException:sql.trans:2DM30!ROLLBACK: not allowed in auto commit mode'.>
 
Error in gsub("\n", " ", res, fixed = TRUE) : 
  input string 1 is invalid UTF-8

The error appears to be due to an unexpected encoding for the incoming character string to gsub. I suspected it was due to foreign language symbols (the user's computer is in French). Initially, I was able to recreate the error on a Windows machine in Swedish. I was then able to fix the gsub error by re-encoding the incoming string for gsub in the monetdb_embedded_query function using iconv (see updated function below). Now when re-running the original user's code (you can recreate the error with this script), I get a message about what is exactly failing (partly in Swedish).

ftruncate: Invalid argument
Error in .local(conn, statement, ...) : 
  Unable to execute statement 'COMMIT'.
 ].ver says 'ERROR: HEAPextend: failed to extend for 05\516.theap: GDKmremap() failed ' [#OS: Enheten k‰nner inte igen kommandot.

.... which indicates that we're asking too much of our machine.

If I have diagnosed the problem correctly, could the monetdb_embedded_query function be updated to handle foreign language characters? Below is my updated version of the function, note the addition of iconv lines.

Many thanks,
Dom

monetdb_embedded_query <- function(conn, query, execute=TRUE, resultconvert=TRUE, int64=FALSE) {
  print('My function')
  if (!inherits(conn, MonetDBLite:::classname)) {
    stop("Invalid connection")
  }
  query <- as.character(query)
  if (length(query) != 1) {
    stop("Need a single query as parameter.")
  }
  if (!MonetDBLite:::monetdb_embedded_env$is_started) {
    stop("Call monetdb_embedded_startup() first")
  }
  if (MonetDBLite:::monetdb_embedded_env$started_dir != ":memory:" &&
      !dir.exists(file.path(MonetDBLite:::monetdb_embedded_env$started_dir, "bat"))) {
    stop("Someone killed all the BATs! Call Brigitte Bardot!")
  }
  execute <- as.logical(execute)
  if (length(execute) != 1) {
    stop("Need a single execute flag as parameter.")
  }
  resultconvert <- as.logical(resultconvert)
  if (length(resultconvert) != 1) {
    stop("Need a single resultconvert flag as parameter.")
  }
  int64 <- as.logical(int64)
  if (length(resultconvert) != 1) {
    stop("Need a single int64 flag as parameter.")
  }
  if (int64 && !requireNamespace("bit64", quietly = TRUE)) {
    stop("Need bit64 package for integer64 support")
  }
  
  # make sure the query is terminated
  query <- paste(query, "\n;", sep="")
  res <- .Call(MonetDBLite:::monetdb_query_R, conn, query, execute, resultconvert, interactive() &&
                 getOption("monetdb.progress", FALSE), int64)
  resp <- list()
  if (is.character(res)) { # error
    resp$type <- "!" # MSG_MESSAGE
    res <- iconv(res, to = 'UTF-8')
    resp$message <- gsub("\n", " ", res, fixed=TRUE)
  }
  if (is.numeric(res)) { # no result set, but successful
    resp$type <- 2 # Q_UPDATE
    resp$rows <- res
  }
  if (is.list(res)) {
    resp$type <- 1 # Q_TABLE
    if ("__prepare" %in% names(attributes(res))) {
      resp$type <- Q_PREPARE
      resp$prepare = attr(res, "__prepare")
      attr(res, "__prepare") <- NULL
    }
    attr(res, "row.names") <- c(NA_integer_, as.integer(-1 * attr(res, "__rows")))
    class(res) <- "data.frame"
    names(res) <- gsub("\\", "", names(res), fixed=T)
    resp$tuples <- res
  }
  resp
}

[zeyus]

tried recompiling with this change and it still fails with

: Invalid argument
Error in .local(conn, statement, ...) : 
  Unable to execute statement 'COMMIT'.
 ].ver says 'ERROR: HEAPextend: failed to extend for 06\640.theap: GDKmremap() failed ' [#OS: The device does not recognize the command.
Error: callr subprocess failed: Unable to execute statement 'COMMIT'.
 ].ver says 'ERROR: HEAPextend: failed to extend for 06\640.theap: GDKmremap() failed ' [#OS: The device does not recognize the command.
Type .Last.error.trace to see where the error occured


r$> .Last.error.trace

 Stack trace:

 Process 9680:
 1. restez:::db_create()
 2. restez:::gb_build2(dpth = dpth, seq_files = seq_files, max_length = max_length,  ...
 3. callr::r(func = function(restez_path, gb_build, dpth, seq_files,  ...
 4. callr:::get_result(output = out, options)
 5. throw(newerr, parent = remerr[[2]])

 x callr subprocess failed: Unable to execute statement 'COMMIT'.
 ]. er says 'ERROR: HEAPextend: failed to extend for 06\640.theap: GDKmremap() failed ' [#OS: The device does not recognize the command.

 Process 15404:
 17. (function (restez_path, gb_build, dpth, seq_files, max_length,  ...
 18. restez:::gb_build(dpth = dpth, seq_files = seq_files, max_length = max_length,  ...
 19. restez:::gb_sql_add(df = df)
 20. DBI::dbWriteTable(conn = connection, name = "nucleotide", value = df,  ...
 21. DBI::dbWriteTable(conn = connection, name = "nucleotide", value = df,  ...
 22. MonetDBLite:::.local(conn, name, value, ...)
 23. DBI:::dbCommit(conn)
 24. MonetDBLite:::dbCommit(conn)
 25. DBI:::dbSendQuery(conn, "COMMIT")
 26. MonetDBLite:::dbSendQuery(conn, "COMMIT")
 27. MonetDBLite:::.local(conn, statement, ...)
 28. base:::stop("Unable to execute statement '", statement, "'.\nServer says '",  ...
 29. base:::.handleSimpleError(function (e)  ...
 30. h(simpleError(msg, call))

 x Unable to execute statement 'COMMIT'.
 ]. er says 'ERROR: HEAPextend: failed to extend for 06\640.theap: GDKmremap() failed ' [#OS: The device does not recognize the command.

[zeyus]

For context, the db_create() step completes successfully in Ubuntu on WSL (no issue with the particular file that failed in Windows), so I can work around it for now but clearly points to a Windows only issue. My windows locale is en_DK.