Skip to content

Commit

Permalink
g.download.project: rename g.download.location (OSGeo#4187)
Browse files Browse the repository at this point in the history
  • Loading branch information
@petrasovaa
petrasovaa authored Aug 20, 2024
1 parent e1b289f commit 037a5ed
Show file tree
Hide file tree
Showing 9 changed files with 224 additions and 147 deletions.
2 changes: 1 addition & 1 deletion .github/workflows/macos.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -88,7 +88,7 @@ jobs:
shell: bash -el {0}
run: |
grass --tmp-project XY --exec \
g.download.location url=${{ env.SampleData }} path=$HOME
g.download.project url=${{ env.SampleData }} path=$HOME
grass --tmp-project XY --exec \
python3 -m grass.gunittest.main \
--grassdata $HOME --location nc_spm_full_v2alpha2 --location-type nc \
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/test_thorough.bat
View file
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
set grass=%1
set python=%2

call %grass% --tmp-project XY --exec g.download.location url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path=%USERPROFILE%
call %grass% --tmp-project XY --exec g.download.project url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path=%USERPROFILE%
call %grass% --tmp-project XY --exec %python% -m grass.gunittest.main --grassdata %USERPROFILE% --location nc_spm_full_v2alpha2 --location-type nc --min-success 80
2 changes: 1 addition & 1 deletion .github/workflows/test_thorough.sh
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
set -e

grass --tmp-project XY --exec \
g.download.location url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path=$HOME
g.download.project url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path=$HOME

grass --tmp-project XY --exec \
python3 -m grass.gunittest.main \
Expand Down
1 change: 1 addition & 0 deletions scripts/Makefile
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ SUBDIRS = \
db.test \
db.univar \
g.download.location \
g.download.project \
g.extension \
g.extension.all \
g.manual \
Expand Down
43 changes: 6 additions & 37 deletions scripts/g.download.location/g.download.location.html
View file
Original file line number Diff line number Diff line change
@@ -1,47 +1,16 @@
<h2>DESCRIPTION</h2>

<em>g.download.location</em> downloads an archived (e.g.,
<code>.zip</code> or <code>.tar.gz</code>) project (previously called
location) from a given URL
and unpacks it to a specified or current GRASS GIS Spatial Database.
URL can be also a local file on the disk.

If the archive contains a directory which contains a project, the module
will recognize that and use the project automatically.
The first directory which is a project is used.
Other projects or any other files are ignored.

<h2>EXAMPLES</h2>

<h3>Download the full GRASS GIS sample project within a running session</h3>

Download and unpack the full North Carolina sample project into the user's
HOME directory:

<div class="code"><pre>
g.download.location url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path=$HOME
</pre></div>

<h3>Download the full GRASS GIS sample project in a temporary session</h3>

In a temporary session, download and unpack the full North Carolina sample project
into the user's HOME directory:

<div class="code"><pre>
grass --tmp-location XY --exec g.download.location url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path=$HOME
</pre></div>

<em>g.download.location</em> has been renamed to
<a href="g.download.project.html">g.download.project</a>
and exists for backwards compatibility reasons.
It will be removed in the next major version.

<h2>SEE ALSO</h2>

<em>
<a href="g.mapset.html">g.mapset</a>,
<a href="g.mapsets.html">g.mapsets</a>,
<a href="r.proj.html">r.proj</a>,
<a href="v.proj.html">v.proj</a>,
<a href="g.proj.all.html">g.proj.all</a>
<a href="g.download.project.html">g.download.project</a>
</em>

<h2>AUTHOR</h2>

Vaclav Petras, <a href="http://geospatial.ncsu.edu/osgeorel/">NCSU GeoForAll Lab</a>
Vaclav Petras, <a href="http://geospatial.ncsu.edu/geoforall/">NCSU GeoForAll Lab</a>
115 changes: 8 additions & 107 deletions scripts/g.download.location/g.download.location.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -3,20 +3,20 @@
#
# MODULE: g.download.location
# AUTHOR(S): Vaclav Petras <wenzeslaus gmail com>
# PURPOSE: Download and extract location from web
# COPYRIGHT: (C) 2017 by the GRASS Development Team
# PURPOSE: Download and extract project (location) from web
# COPYRIGHT: (C) 2017-2024 by the GRASS Development Team
#
# This program is free software under the GNU General
# Public License (>=v2). Read the file COPYING that
# comes with GRASS for details.
#
#############################################################################

"""Download GRASS Locations"""
"""Download GRASS projects"""

# %module
# % label: Download GRASS Location from the web
# % description: Get GRASS Location from an URL or file path
# % label: Download GRASS project (location) from the web
# % description: Get GRASS project from an URL or file path
# % keyword: general
# % keyword: data
# % keyword: download
Expand All @@ -26,7 +26,7 @@
# % key: url
# % multiple: no
# % type: string
# % label: URL of the archive with a location to be downloaded
# % label: URL of the archive with a project to be downloaded
# % description: URL of ZIP, TAR.GZ, or other similar archive
# % required: yes
# %end
Expand All @@ -42,111 +42,12 @@
# % multiple: no
# %end

import atexit
import os
import shutil
from pathlib import Path

import grass.script as gs
from grass.grassdb.checks import is_location_valid
from grass.script.utils import try_rmdir
from grass.utils.download import DownloadError, download_and_extract, name_from_url


def find_location_in_directory(path, recurse=0):
"""Return path to location in one of the subdirectories or None
The first location found is returned. The expected usage is looking for one
location somewhere nested in subdirectories.
By default only the immediate subdirectories of the provided directory are
tested, but with ``recurse >= 1`` additional levels of subdirectories
are tested for being locations.
Directory names are sorted to provide a stable result.
:param path: Path to the directory to search
:param recurse: How many additional levels of subdirectories to explore
"""
assert recurse >= 0
full_paths = [os.path.join(path, i) for i in os.listdir(path)]
candidates = sorted([i for i in full_paths if os.path.isdir(i)])
for candidate in candidates:
if is_location_valid(candidate):
return candidate
if recurse:
for candidate in candidates:
result = find_location_in_directory(candidate, recurse - 1)
if result:
return result
return None


def location_name_from_url(url):
"""Create location name from URL"""
return gs.legalize_vector_name(name_from_url(url))


def main(options, unused_flags):
"""Download and copy location to destination"""
url = options["url"]
name = options["name"]
database = options["path"]

if not database:
# Use the current database path.
database = gs.gisenv()["GISDBASE"]
if not name:
name = location_name_from_url(url)
destination = Path(database) / name

if destination.exists():
gs.fatal(
_(
"Location named <{name}> already exists in <{directory}>, download canceled"
).format(name=name, directory=database)
)

gs.message(_("Downloading and extracting..."))
try:
directory = download_and_extract(url)
if not directory.is_dir():
gs.fatal(_("Archive contains only one file and no mapset directories"))
atexit.register(lambda: try_rmdir(directory))
except DownloadError as error:
gs.fatal(_("Unable to get the location: {error}").format(error=error))
if not is_location_valid(directory):
gs.verbose(_("Searching for valid location..."))
# This in fact deal with location being on the third level of directories
# thanks to how the extraction functions work (leaving out one level).
result = find_location_in_directory(directory, recurse=1)
if result:
# We just want to show relative path in the message.
# The relative path misses the root directory (name), because we
# loose it on the way. (We should use parent directory to get the
# full relative path, but the directory name is different now.
# This is the consequence of how the extract functions work.)
relative = os.path.relpath(result, start=directory)
gs.verbose(
_("Location found in a nested directory '{directory}'").format(
directory=relative
)
)
directory = result
else:
# The list is similarly misleading as the relative path above
# as it misses the root directory, but it still should be useful.
files_and_dirs = os.listdir(directory)
gs.fatal(
_(
"The downloaded file is not a valid GRASS Location."
" The extracted file contains these files and directories:"
"\n{files_and_dirs}"
).format(files_and_dirs=" ".join(files_and_dirs))
)
gs.verbose(_("Copying to final destination..."))
shutil.copytree(src=directory, dst=destination)
gs.message(_("Path to the location now <{path}>").format(path=destination))
"""Download and copy project to destination"""
gs.run_command("g.dowload.project", **options)


if __name__ == "__main__":
Expand Down
7 changes: 7 additions & 0 deletions scripts/g.download.project/Makefile
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
MODULE_TOPDIR = ../..

PGM = g.download.project

include $(MODULE_TOPDIR)/include/Make/Script.make

default: script
46 changes: 46 additions & 0 deletions scripts/g.download.project/g.download.project.html
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
<h2>DESCRIPTION</h2>

<em>g.download.project</em> downloads an archived (e.g.,
<code>.zip</code> or <code>.tar.gz</code>) project (previously called
location) from a given URL
and unpacks it to a specified or current GRASS GIS Spatial Database.
URL can be also a local file on the disk.

If the archive contains a directory which contains a project, the module
will recognize that and use the project automatically.
The first directory which is a project is used.
Other projects or any other files are ignored.

<h2>EXAMPLES</h2>

<h3>Download the full GRASS GIS sample project within a running session</h3>

Download and unpack the full North Carolina sample project into the user's
HOME directory:

<div class="code"><pre>
g.download.project url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path=$HOME
</pre></div>

<h3>Download the full GRASS GIS sample project in a temporary session</h3>

In a temporary session, download and unpack the full North Carolina sample project
into the user's HOME directory:

<div class="code"><pre>
grass --tmp-project XY --exec g.download.project url=https://grass.osgeo.org/sampledata/north_carolina/nc_spm_full_v2alpha2.tar.gz path=$HOME
</pre></div>

<h2>SEE ALSO</h2>

<em>
<a href="g.mapset.html">g.mapset</a>,
<a href="g.mapsets.html">g.mapsets</a>,
<a href="r.proj.html">r.proj</a>,
<a href="v.proj.html">v.proj</a>,
<a href="g.proj.all.html">g.proj.all</a>
</em>

<h2>AUTHOR</h2>

Vaclav Petras, <a href="http://geospatial.ncsu.edu/geoforall/">NCSU GeoForAll Lab</a>
Loading

0 comments on commit 037a5ed

Please sign in to comment.