Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: SantanderMetGroup/loadeR
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v1.4.15
Choose a base ref
...
head repository: SantanderMetGroup/loadeR
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: master
Choose a head ref

Commits on Aug 1, 2019

  1. Copy the full SHA
    144868d View commit details

Commits on Aug 14, 2019

  1. Copy the full SHA
    da03ac4 View commit details
  2. changes for version 1.5.0

    miturbide committed Aug 14, 2019
    Copy the full SHA
    e6cc91a View commit details
  3. Copy the full SHA
    dd5983c View commit details

Commits on Sep 18, 2019

  1. Copy the full SHA
    eed05d1 View commit details

Commits on Sep 23, 2019

  1. Copy the full SHA
    2dd4e8d View commit details
  2. Copy the full SHA
    077b7d6 View commit details

Commits on Oct 19, 2019

  1. solve conflicts

    Merge branch 'devel' into mai-devel
    
    Conflicts:
    	R/datasets.R
    miturbide committed Oct 19, 2019
    Copy the full SHA
    36c9dae View commit details

Commits on Oct 27, 2019

  1. Copy the full SHA
    f085132 View commit details
  2. Copy the full SHA
    b8be0bf View commit details

Commits on Nov 28, 2019

  1. Copy the full SHA
    eb584ea View commit details

Commits on Dec 5, 2019

  1. Copy the full SHA
    3c1ba57 View commit details

Commits on Jan 6, 2020

  1. updates for v1.6.0

    miturbide committed Jan 6, 2020
    Copy the full SHA
    7e52b84 View commit details

Commits on Jan 29, 2020

  1. Copy the full SHA
    d636a29 View commit details

Commits on Jan 31, 2020

  1. Copy the full SHA
    a6dc116 View commit details
  2. Copy the full SHA
    7a3b4f3 View commit details

Commits on Feb 4, 2020

  1. Copy the full SHA
    088a3da View commit details

Commits on Feb 22, 2020

  1. Copy the full SHA
    c9f1b21 View commit details
  2. Copy the full SHA
    6b73e25 View commit details
  3. Copy the full SHA
    867a6dc View commit details
  4. Copy the full SHA
    b0e54af View commit details
  5. Copy the full SHA
    296bd3a View commit details

Commits on Mar 13, 2020

  1. minor change

    miturbide committed Mar 13, 2020
    Copy the full SHA
    398c92b View commit details

Commits on Mar 17, 2020

  1. Copy the full SHA
    89ea7b8 View commit details

Commits on Mar 23, 2020

  1. changes in news and description

    Jorge Bano Medina authored and Jorge Bano Medina committed Mar 23, 2020
    Copy the full SHA
    aa4af17 View commit details
  2. update man

    Jorge Bano Medina authored and Jorge Bano Medina committed Mar 23, 2020
    Copy the full SHA
    2369eae View commit details

Commits on Apr 27, 2020

  1. add NetCDF reading

    miturbide committed Apr 27, 2020
    Copy the full SHA
    7ee0942 View commit details

Commits on May 1, 2020

  1. Copy the full SHA
    c093e99 View commit details

Commits on May 11, 2020

  1. add version to the inventory

    only load in session requested subset in loadStationData
    miturbide committed May 11, 2020
    Copy the full SHA
    2f23b51 View commit details

Commits on May 21, 2020

  1. Copy the full SHA
    9b210eb View commit details
  2. Copy the full SHA
    11164bc View commit details

Commits on May 22, 2020

  1. bug fixes

    miturbide committed May 22, 2020
    Copy the full SHA
    746a57f View commit details

Commits on May 23, 2020

  1. Doc update

    jbedia committed May 23, 2020
    Copy the full SHA
    051c2b6 View commit details
  2. Copy the full SHA
    06ed580 View commit details
  3. Copy the full SHA
    990c363 View commit details

Commits on May 28, 2020

  1. Copy the full SHA
    c99d74d View commit details
  2. bug fix

    miturbide committed May 28, 2020
    Copy the full SHA
    427411e View commit details

Commits on May 29, 2020

  1. Doc update

    jbedia committed May 29, 2020
    Copy the full SHA
    245c635 View commit details

Commits on Jun 15, 2020

  1. Copy the full SHA
    87ff16a View commit details
  2. Minor updates in dataInventory

    jbedia committed Jun 15, 2020
    Copy the full SHA
    74c25bd View commit details
  3. Copy the full SHA
    f6db24f View commit details
  4. Include 2D coordinates

    jbedia committed Jun 15, 2020
    Copy the full SHA
    b2f1a51 View commit details
  5. Doc update

    jbedia committed Jun 15, 2020
    Copy the full SHA
    f56130d View commit details

Commits on Jun 16, 2020

  1. Copy the full SHA
    a8c6dbe View commit details
  2. Copy the full SHA
    4a0121c View commit details
  3. README update

    jbedia committed Jun 16, 2020
    Copy the full SHA
    3b13234 View commit details
  4. Doc update

    jbedia committed Jun 16, 2020
    Copy the full SHA
    98d3be2 View commit details
  5. Doc update

    jbedia committed Jun 16, 2020
    Copy the full SHA
    0bf5d47 View commit details
  6. Copy the full SHA
    518d340 View commit details
  7. Copy the full SHA
    1aaa7ea View commit details
Showing with 1,627 additions and 1,197 deletions.
  1. +1 −1 .gitignore
  2. +17 −18 DESCRIPTION
  3. +4 −6 NAMESPACE
  4. +33 −2 NEWS
  5. +103 −0 R/dataInventory.NetCDF.R
  6. +115 −28 R/dataInventory.R
  7. +5 −9 R/datasets.R
  8. +2 −2 R/dictionaryLookup.R
  9. +39 −3 R/findPointXYindex.R
  10. +382 −176 R/getLatLonDomain.R
  11. +14 −3 R/getLatLonDomainStations.R
  12. +13 −4 R/getMemberDomain.R
  13. +7 −7 R/getTimeDomain.R
  14. +1 −1 R/getVerticalLevelPars.R
  15. +25 −7 R/loadCircularGridData.R
  16. +18 −3 R/loadDecadalForecast.R
  17. +290 −196 R/loadGridData.R
  18. +2 −3 R/loadSeasonalForecast.R
  19. +181 −102 R/loadStationData.R
  20. +6 −0 R/login.R
  21. +0 −51 R/loginUDG.R
  22. +77 −18 R/makeSubset.R
  23. +6 −2 R/makeSubset.seasonal.R
  24. +1 −1 R/onAttach.R
  25. +6 −2 R/revArrayLatDim.R
  26. +21 −11 R/scanVarDimensions.R
  27. +0 −54 R/vocabulary.R
  28. +11 −8 README.md
  29. +2 −259 inst/datasets.txt
  30. +3 −3 man-roxygen/templateDicDetails.R
  31. +6 −10 man/{UDG.datasets.Rd → C4R.datasets.Rd}
  32. +0 −31 man/C4R.vocabulary.Rd
  33. +0 −47 man/C4R.vocabulary.update.Rd
  34. +1 −1 man/adjustDates.Rd
  35. +1 −1 man/check.dictionary.Rd
  36. +3 −3 man/dataInventory.NetCDF.Rd
  37. +25 −0 man/dataInventory.NetCDF.ts.Rd
  38. +2 −0 man/dataInventory.Rd
  39. +1 −1 man/dictionaryLookup.Rd
  40. +1 −1 man/findPointXYindex.Rd
  41. +1 −2 man/getForecastTimeDomain.Rd
  42. +1 −1 man/getLatLonDomain.Rd
  43. +1 −1 man/getMemberDomain.Rd
  44. +11 −2 man/getTimeDomain.Rd
  45. +0 −8 man/getTimeDomainStations.Rd
  46. +2 −2 man/getVerticalLevelPars.Rd
  47. +3 −2 man/hurs2huss.Rd
  48. +26 −13 man/loadCircularGridData.Rd
  49. +19 −8 man/loadDecadalForecast.Rd
  50. +30 −13 man/loadGridData.Rd
  51. +16 −3 man/loadGridDataset.Rd
  52. +17 −6 man/loadSeasonalForecast.Rd
  53. +23 −8 man/loadStationData.Rd
  54. +0 −31 man/loginUDG.Rd
  55. +14 −5 man/makeAggregatedDataset.Rd
  56. +2 −2 man/makeSubset.Rd
  57. +8 −2 man/makeSubset.decadal.Rd
  58. +11 −2 man/makeSubset.seasonal.Rd
  59. +3 −2 man/mslp2ps.Rd
  60. +3 −0 man/ndays.Rd
  61. +0 −2 man/revArrayLatDim.Rd
  62. +7 −4 man/scanVarDimensions.Rd
  63. +3 −2 man/tdps2hurs.Rd
  64. +1 −1 man/timeUnits.Rd
2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
@@ -16,4 +16,4 @@ ignore
.Rbuildignore
.Rprofile
.directory

.DS_Store
35 changes: 17 additions & 18 deletions DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,36 +1,35 @@
Package: loadeR
Depends:
R(>= 3.1.0),
R(>= 3.5.0),
rJava,
loadeR.java
loadeR.java,
climate4R.UDG(>= 0.2.0)
Imports:
utils,
abind,
RCurl
Suggests:
loadeR.ECOMS,
transformeR,
visualizeR,
downscaleR
Type: Package
Title: A climate4R package for data access <http://meteo.unican.es/climate4r>
Version: 1.4.15
Date: 2019-07-30
Authors@R: as.person(c(
"Santander Meteorology Group <http://meteo.unican.es> [cph]",
"Joaquín Bedia <bediaj@unican.es> [aut,cre]",
"Antonio Cofiño <antonio.cofino@unican.es> [ctb]",
"Matteo de Felice <matteo.defelice@enea.it> [ctb]",
"Jesus Fernández <fernandej@unican.es> [ctb]",
"José Manuel Gutiérrez <gutierjm@unican.es> [ctb]",
"Sixto Herrera <herreras@unican.es> [ctb]",
"Maialen Iturbide <miturbide@ifca.unican.es> [aut]",
"Etienne Tourigny <etienne.tourigny@bsc.es> [ctb]",
"Max Tuni <max@predictia.es> [ctb]",
"Manuel Vega <manuel.vega@unican.es> [ctb]"))
Version: 1.8.2
Date: 2024-06-04
Authors@R: c(person("Bedia", "Joaquín", email = "bediaj@unican.es", role = c("aut","cre"), comment = c(ORCID = "0000-0001-6219-4312")),
person("Antonio", "Cofiño", email = "antonio.cofino@unican.es", role = "ctb"),
person("Matteo", "de Felice", email = "matteo.defelice@enea.it", role = "ctb"),
person("Jesús", "Fernández", email = "fernandej@unican.es", role = "ctb"),
person("José Manuel", "Gutiérrez", email = "gutierjm@unican.es", role = "ctb"),
person("Sixto", "Herrera", email = "herreras@unican.es", role = "aut"),
person("Maialen", "Iturbide", email = "miturbide@ifca.unican.es", role = "aut", comment = c(ORCID = "0000-0002-5048-0941")),
person("Etienne", "Tourigny", email = "etienne.tourigny@bsc.es", role = "ctb"),
person("Max", "Tuni", email = "max@predictia.es", role = "ctb"),
person("Manuel", "Vega", email = "manuel.vega@unican.es", role = "ctb"),
person("Santander Meteorology Group", comment = "<http://www.meteo.unican.es/en>", role = "cph"))
BugReports: https://github.com/SantanderMetGroup/loadeR/issues
URL: https://github.com/SantanderMetGroup/loadeR/wiki
Encoding: UTF-8
Description: Enables harmonized access to either remote climate databases via OPeNDAP or to locally stored data, allowing the retrieval of user-defined dimensional slices from large climate datasets. For more details see Iturbide et al 2019 <http://dx.doi.org/10.1016/j.envsoft.2018.09.009>.
License: file LICENSE
RoxygenNote: 6.1.1
RoxygenNote: 7.3.1
10 changes: 4 additions & 6 deletions NAMESPACE
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
# Generated by roxygen2: do not edit by hand

export(C4R.vocabulary)
export(C4R.vocabulary.update)
export(UDG.datasets)
export(C4R.datasets)
export(adjustDates)
export(adjustDates.forecast)
export(adjustRCMgrid)
@@ -24,7 +22,6 @@ export(loadGridData)
export(loadGridDataset)
export(loadSeasonalForecast)
export(loadStationData)
export(loginUDG)
export(makeAggregatedDataset)
export(makeSubset)
export(mslp2ps)
@@ -40,17 +37,19 @@ export(writeStationData)
importFrom(RCurl,getURL)
importFrom(RCurl,url.exists)
importFrom(abind,abind)
importFrom(climate4R.UDG,C4R.vocabulary)
importFrom(climate4R.UDG,UDG.datasets)
importFrom(graphics,lines)
importFrom(graphics,text)
importFrom(loadeR.java,javaCalendarDate2rPOSIXlt)
importFrom(loadeR.java,javaString2rChar)
importFrom(rJava,.jnew)
importFrom(rJava,.jnull)
importFrom(rJava,J)
importFrom(stats,median)
importFrom(stats,na.exclude)
importFrom(stats,na.omit)
importFrom(stats,setNames)
importFrom(utils,URLencode)
importFrom(utils,globalVariables)
importFrom(utils,head)
importFrom(utils,packageDescription)
@@ -59,4 +58,3 @@ importFrom(utils,read.csv)
importFrom(utils,tail)
importFrom(utils,unzip)
importFrom(utils,write.csv)
importFrom(utils,write.table)
35 changes: 33 additions & 2 deletions NEWS
Original file line number Diff line number Diff line change
@@ -67,5 +67,36 @@ For earlier version changes, see the GitHub Releases [Tab](https://github.com/Sa
## v1.4.15 (30 Jul 2019)
* Bug fix related to loading datasets that contain the level dimension



## v1.5.0 (13 Aug 2019)
* Functions loginUDG, UDG.datasets and C4R.vocabulary deprecated and now available in package climate4R.UDG

## v1.6.0 (5 Jan 2020)
* Add climate4R.UDG dependency (for accessing UDG datasets)
* Do not allow requests of years out of boundaries in loadGridData

## v1.6.1 (23 Mar 2020)
* Minor bug fixes and and enhancements

## v1.7.0 (18 Sep 2020)
* Enhancements in NetCDF support for loading station datasets via loadStationData
* Several improvements and bug fixes in loadGridData for rotated grids
* Include 'Shape' and 'MemorySizeMb' attributes in dataInventory for NetCDF datasets
* Include 2D coordinates of irregular grids in dataInventories
* Minor bug fixes and enhancements for dealing with ensemble axes
* loginUDG, C4R.vocabulary and C4R.vocabulary.update utilities moved to climate4R.UDG dependency
* Other minor bug fixes and documentation updates

## v1.7.1 (05 Jul 2021)
* Minor bug fixes and and enhancements

## v1.8.0 (21 Jun 2023)
* Minor fix in loadSeasonalForecast output attributes (resX and resY)
* Bug fix in dataInventory: exact variable name retrieval by grep
* Adapt to new class denomination for matrix objects in R version 4.X

## v1.8.1 (22 Jun 2023)
* Update of the DESCRIPTION file

## v1.8.2 (04 Jun 2024)
* Minor changes in loadSeasonalForecast to handle non-standard dimension names

103 changes: 103 additions & 0 deletions R/dataInventory.NetCDF.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,103 @@
#' @title Inventory of a gridded dataset
#' @description Returns a list with summary information about the variables stored in a gridded dataset.
#' Sub-routine of \code{dataInventory}
#' @param dataset A full path to the file describing the dataset (NcML)
#' @return A (named) list whose length is determined by the number of variables stored in the dataset,
#' its names corresponding to the short names of the variables.
#' For each variable, information on the variable long name, data type, units, data size (in Mb) and
#' characteristics of its dimensions is provided.
#' @author J. Bedia, S. Herrera
#' @keywords internal
#' @importFrom rJava J


dataInventory.NetCDF <- function(dataset) {
gds <- J("ucar.nc2.dt.grid.GridDataset")$open(dataset)
nc.dataset <- gds$getNetcdfDataset()
varNames <- unlist(strsplit(gsub("\\[|]|\\s", "", gds$getGrids()$toString()), ","))
rm.ind <- grep("^lon|^lat", varNames)
if (length(rm.ind) > 0) {
varNames <- varNames[-rm.ind]
}
if (length(varNames) == 0) {
stop("No variables found", call. = FALSE)
} else {
var.list <- vector(mode = "list", length = length(varNames))
for (i in 1:length(varNames)) {
message("[", Sys.time(), "] Retrieving info for \'", varNames[i], "\' (", length(varNames) - i, " vars remaining)")
datavar <- gds$getDataVariable(varNames[i])
description <- datavar$getDescription()
varName <- datavar$getShortName()
version <- tryCatch({trimws(datavar$findAttribute("version")$getValues()$toString())}, error = function(e){NA})
dataType <- datavar$getDataType()$toString()
element.size <- datavar$getDataType()$getSize()
data.number <- datavar$getSize()
dataSize <- element.size * data.number * 1e-06
units <- datavar$getUnitsString()
shape <- datavar$getShape()
grid <- gds$findGridByShortName(varName)
dim.list <- scanVarDimensions(grid)
if (grid$getXDimensionIndex() >= 0) {
xDimIndex <- grid$getXDimensionIndex() + 1
if (length(dim.list[[xDimIndex]]$Shape) > 1) {
dimX.list <- list()
length(dimX.list) <- length(dim.list[[xDimIndex]]$Shape)
for (v in c(1:length(dimX.list))) {
dimX.list[[v]] <- tryCatch({
Xcoord <- nc.dataset$findVariable(dim.list[[xDimIndex]]$Coordinates[v])
Xunits <- Xcoord$getUnitsString()
XType <- Xcoord$getAxisType()$toString()
XAxisShape <- Xcoord$getShape()
values <- Xcoord$getCoordValues()
aux.XAxis <- Xcoord$getDimensionsString()
list("Type" = XType, "Units" = Xunits,
"Values" = values, "Shape" = XAxisShape,"Coordinates" = aux.XAxis)
}, error = function(e) {
list("Type" = NULL, "Units" = NULL,
"Values" = NULL, "Shape" = NULL,
"Coordinates" = NULL)
})
names(dimX.list)[v] <- dim.list[[xDimIndex]]$Coordinates[v] # "lon"
}
dim.list[[xDimIndex]]$Dimensions <- dimX.list
}
}
if (grid$getYDimensionIndex() >= 0) {
yDimIndex <- grid$getYDimensionIndex() + 1
if (length(dim.list[[yDimIndex]]$Shape) > 1) {
dimX.list <- list()
length(dimX.list) <- length(dim.list[[yDimIndex]]$Shape)
for (v in c(1:length(dimX.list))) {
dimX.list[[v]] <- tryCatch({
Xcoord <- nc.dataset$findVariable(dim.list[[yDimIndex]]$Coordinates[v])
Xunits <- Xcoord$getUnitsString()
XType <- Xcoord$getAxisType()$toString()
XAxisShape <- Xcoord$getShape()
values <- Xcoord$getCoordValues()
aux.XAxis <- Xcoord$getDimensionsString()
list("Type" = XType, "Units" = Xunits,
"Values" = values, "Shape" = XAxisShape,"Coordinates" = aux.XAxis)
}, error = function(e) {
list("Type" = NULL, "Units" = NULL,
"Values" = NULL, "Shape" = NULL,
"Coordinates" = NULL)
})
names(dimX.list)[v] <- dim.list[[yDimIndex]]$Coordinates[v] # "lon"
}
dim.list[[yDimIndex]]$Dimensions <- dimX.list
}
}
var.list[[i]] <- list("Description" = description,
"DataType" = dataType,
"Shape" = shape,
"Units" = units,
"DataSizeMb" = dataSize,
"Version" = version,
"Dimensions" = dim.list)
}
names(var.list) <- varNames
}
gds$close()
return(var.list)
}
# End
Loading