Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

style: Fix unused-import (F401) #4521

Merged
merged 30 commits into from
Oct 23, 2024
Merged
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
7092055
t.rast.accumulate: Remove unused SimpleModule import in test_accumula…
echoix Oct 6, 2024
b7d485b
temporal: Fix unused-import (F401) in testsuite
echoix Oct 7, 2024
7a15c41
grass.temporal: Fix unused-import (F401)
echoix Oct 7, 2024
089d5f2
grass.benchmark: Fix unused-import (F401) by defining __all__
echoix Oct 7, 2024
84510ac
grass.pygrass.vector: Fix unused-import (F401) for testsuite
echoix Oct 7, 2024
75f6c14
grass.pygrass.vector: Sort imports
echoix Oct 7, 2024
c333b84
vector: Fix unused-import (F401) for testsuite
echoix Oct 7, 2024
eac4d5a
scripts: Fix unused-import (F401) for testsuite
echoix Oct 7, 2024
742e2ee
raster: Fix unused-import (F401) for testsuite
echoix Oct 7, 2024
c7371e3
v.fill.holes: Fix unused-import (F401)
echoix Oct 7, 2024
6d6a5e0
doc: Fix unused-import (F401)
echoix Oct 7, 2024
147d5a1
grass.semantic_label: Fix unused-import (F401) by defining __all__ in…
echoix Oct 7, 2024
562bdca
grass.pygrass.rpc: Fix unused-import (F401) in __init__
echoix Oct 7, 2024
b05d27c
grass.pygrass.modules.interface: Fix unused-import (F401) by defining…
echoix Oct 7, 2024
28b7194
grass.pygrass.modules.interface: Apply isort
echoix Oct 7, 2024
64567c2
grass.pygrass.modules.grid: Fix unused-import (F401) by defining __al…
echoix Oct 7, 2024
cdc4380
grass.pygrass.modules: Fix unused-import (F401) by defining __all__ i…
echoix Oct 7, 2024
6a6aa63
grass.pygrass.modules: Apply isort
echoix Oct 7, 2024
1c13ab0
grass.imaging: Fix unused-import (F401) by defining __all__ in __init__
echoix Oct 7, 2024
d2becc9
grass.imaging: Apply isort
echoix Oct 7, 2024
cd8f4dc
grass.jupyter: Fix unused-import (F401) by defining __all__ in __init__
echoix Oct 7, 2024
c83476d
grass.jupyter: Apply isort
echoix Oct 7, 2024
2a62682
pygrass: Ignore F401 (unused imports)
echoix Oct 14, 2024
b8fd615
lib/gis: Fix F401 unused platform import in test_gis_lib_getl
echoix Oct 14, 2024
06a9dd5
grass.script: Ignore F401 in `__init__.py` for setup import
echoix Oct 14, 2024
422f1bb
style: Remove exclusion for F401 ruff rule
echoix Oct 14, 2024
4ccafbf
style: Remove unneeded flake8 exclusions for F401
echoix Oct 14, 2024
84dc11d
Merge branch 'main' into tests-unused-imports
echoix Oct 18, 2024
fafd964
Apply suggestions from code review
echoix Oct 23, 2024
4a719e3
Merge branch 'main' into tests-unused-imports
echoix Oct 23, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 5 additions & 8 deletions .flake8
Original file line number Diff line number Diff line change
Expand Up @@ -15,11 +15,10 @@ per-file-ignores =
# E501 line too long
# E722 do not use bare 'except'
# W605 invalid escape sequence
# F401 imported but unused
# F821 undefined name 'unicode'
# F841 local variable assigned to but never used
# E741 ambiguous variable name 'l'
__init__.py: F401, F403
__init__.py: F403
man/build_html.py: E501
doc/python/m.distance.py: E501
gui/scripts/d.wms.py: E501
Expand Down Expand Up @@ -67,10 +66,8 @@ per-file-ignores =
python/grass/pygrass/vector/__init__.py: E402
python/grass/pygrass/raster/__init__.py: E402
python/grass/pygrass/vector/__init__.py: E402
python/grass/pygrass/modules/interface/*.py: F401
python/grass/pygrass/modules/grid/*.py: F401
python/grass/pygrass/raster/category.py: E721
python/grass/pygrass/rpc/__init__.py: F401, F403
python/grass/pygrass/rpc/__init__.py: F403
python/grass/pygrass/utils.py: E402
python/grass/temporal/abstract_space_time_dataset.py: E722
python/grass/temporal/c_libraries_interface.py: E722
Expand All @@ -83,7 +80,7 @@ per-file-ignores =
python/grass/temporal/temporal_topology_dataset_connector.py: E722
# Current benchmarks/tests are changing sys.path before import.
# Possibly, a different approach should be taken there anyway.
python/grass/pygrass/tests/benchmark.py: E402, F401, F821
python/grass/pygrass/tests/benchmark.py: E402, F821
# Configuration file for Sphinx:
# Ignoring import/code mix and line length.
# Files not managed by Black
Expand All @@ -92,8 +89,8 @@ per-file-ignores =
# F401 imported but unused
echoix marked this conversation as resolved.
Show resolved Hide resolved
# F403 star import used; unable to detect undefined names
python/grass/*/__init__.py: F401, F403
python/grass/*/*/__init__.py: F401, F403
python/grass/*/*/*/__init__.py: F401, F403
python/grass/*/*/__init__.py: F403
python/grass/*/*/*/__init__.py: F403
# E402 module level import not at top of file
scripts/r.in.wms/wms_gdal_drv.py: E722
scripts/r.in.wms/wms_drv.py: E402, E722
Expand Down
1 change: 0 additions & 1 deletion doc/notebooks/hydrology.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,6 @@
"outputs": [],
"source": [
"# Import Python standard library and IPython packages we need.\n",
"import os\n",
"import subprocess\n",
"import sys\n",
"import csv\n",
Expand Down
1 change: 0 additions & 1 deletion doc/notebooks/jupyter_tutorial.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
"source": [
"# Import Python standard library and IPython packages we need.\n",
"import subprocess\n",
"import time\n",
"import sys\n",
"\n",
"# Ask GRASS GIS where its Python packages are.\n",
Expand Down
40 changes: 39 additions & 1 deletion doc/notebooks/parallelization_tutorial.ipynb
petrasovaa marked this conversation as resolved.
Show resolved Hide resolved
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
"cells": [
{
"cell_type": "markdown",
"id": "7fb27b941602401d91542211134fc71a",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"# Introduction to Parallelization in GRASS GIS\n",
Expand All @@ -10,6 +11,7 @@
},
{
"cell_type": "markdown",
"id": "acae54e37e7d407bbb7b55eff062a284",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"Let's start GRASS to run examples:"
Expand All @@ -18,10 +20,10 @@
{
"cell_type": "code",
"execution_count": null,
"id": "9a63283cbaf04dbcab1f6479b197f3a8",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
"import os\n",
"import subprocess\n",
"import sys\n",
"\n",
Expand All @@ -43,13 +45,15 @@
},
{
"cell_type": "markdown",
"id": "8dd0d8092fe74a7c96281538738b07e2",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"Note: most examples assume we are already in an active GRASS session."
]
},
{
"cell_type": "markdown",
"id": "72eea5119410473aa328ad9291626812",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {
"tags": []
},
Expand All @@ -76,6 +80,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "8edb47106e1a46a883d545849b8ab81b",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -86,13 +91,15 @@
},
{
"cell_type": "markdown",
"id": "10185d26023b46108eb7d9f57d49d2b3",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"The speedup (processing time with 1 core / processing time with N cores) typically does not increase linearly with the number of cores and parallel efficiency (speedup / N cores) decreases when adding cores. See, e.g., [benchmarks for r.neighbors](https://grass.osgeo.org/grass-stable/manuals/r.neighbors.html#performance). This behavior is due to the serial parts of the code (see [Amdahl's law](https://en.wikipedia.org/wiki/Amdahl%27s_law)) and computation overhead. "
]
},
{
"cell_type": "markdown",
"id": "8763a12b2bbd4a93a75aff182afb95dc",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"## Parallelization of workflows\n",
Expand All @@ -105,6 +112,7 @@
},
{
"cell_type": "markdown",
"id": "7623eae2785240b9bd12b16a66d81610",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"### Data-based parallelization\n",
Expand All @@ -114,6 +122,7 @@
},
{
"cell_type": "markdown",
"id": "7cdc8c89c7104fffa095e18ddfef8986",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"The following example shows IDW interpolation split into 4 tiles. In this case, specifying an overlap is needed to get correct results without edge artifacts. Here, the number and size of tiles is automatically derived from the number of cores, but can be specified."
Expand All @@ -122,6 +131,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "b118ea5561624da68c537baed56e602f",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -132,6 +142,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "938c804e27f84196a10c8828c723f798",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -157,6 +168,7 @@
},
{
"cell_type": "markdown",
"id": "504fb2a444614c0babb325280ed9130a",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"The following is the same tool ran in serial:"
Expand All @@ -165,6 +177,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "59bbdb311c014d738909a11f9e486628",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -174,6 +187,7 @@
},
{
"cell_type": "markdown",
"id": "b43b363d81ae4b689946ece5c682cd59",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"There are tools that already integrate tiling. For example, addon [r.mapcalc.tiled](https://grass.osgeo.org/grass-stable/manuals/addons/r.mapcalc.tiled.html) uses the tiling concept for raster algebra computation. More complex algebra expression will increase the speedup of this method."
Expand All @@ -182,6 +196,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "8a65eabff63a45729fe45fb5ade58bdc",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -196,6 +211,7 @@
},
{
"cell_type": "markdown",
"id": "c3933fab20d04ec698c2621248eb3be0",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"### Task-based parallelization\n",
Expand All @@ -205,6 +221,7 @@
},
{
"cell_type": "markdown",
"id": "4dd4641cc4064e0191573fe9c69df29b",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"#### Examples in Python\n",
Expand All @@ -213,6 +230,7 @@
},
{
"cell_type": "markdown",
"id": "8309879909854d7188b41380fd92a7c3",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"In the following example viewsheds from different coordinates are computed in parallel using `multiprocessing.Pool` class. To avoid issues when using multiprocessing from Jupyter Notebook (multiprocessing.Pool does not work with interactive interpreters), we will first write a Python script with main function and then execute it."
Expand All @@ -221,6 +239,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "3ed186c9a28b402fb0bc4494df01f08d",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -247,6 +266,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "cb1e1581032b452c9409d6c6813c49d1",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -257,6 +277,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "379cbbc1e968416e875cc15c1202d7eb",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -265,6 +286,7 @@
},
{
"cell_type": "markdown",
"id": "277c27b1587741f2af2001be3712ef0d",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"#### Examples in Bash\n",
Expand All @@ -274,6 +296,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "db7b79bc585a40fcaf58bf750017e135",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -286,6 +309,7 @@
},
{
"cell_type": "markdown",
"id": "916684f9a58a4a2aa5f864670399430d",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {
"tags": []
},
Expand All @@ -298,6 +322,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "1671c31a24314836a5b85d7ef7fbf015",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -311,20 +336,23 @@
},
{
"cell_type": "markdown",
"id": "33b0902fd34d4ace834912fa1002cf8e",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"See manual pages of GNU Parallel or xargs for more advanced uses. GNU Parallel can be configured to distribute jobs across multiple machines. In that case, use `--exec` interface described below."
]
},
{
"cell_type": "markdown",
"id": "f6fa52606d8c4a75a9b52967216f8f3f",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"### Safe execution of parallel tasks"
]
},
{
"cell_type": "markdown",
"id": "f5a1fa73e5044315a093ec459c9be902",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"While you can execute tasks in parallel within a single mapset, it is *not safe* when your tasks:\n",
Expand All @@ -340,6 +368,7 @@
},
{
"cell_type": "markdown",
"id": "cdf66aed5cc84ca1b48e60bad68798a8",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"#### Executing processes in separate mapsets\n",
Expand All @@ -353,6 +382,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "28d3efd5258a48a79c179ea5c6759f01",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -362,6 +392,7 @@
},
{
"cell_type": "markdown",
"id": "3f9bc0b9dd2c44919cc8dcca39b469f8",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"One of the previous examples that was running within GRASS session in a single mapset can be rewritten so that each task runs in a newly created mapset. Note that by default newly created mapsets use default computational region for that GRASS location (you can use `g.region -s` to modify it). For raster computations, you need to change the computational region for each new mapset if the default one is not desired."
Expand All @@ -370,6 +401,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "0e382214b5f147d187d36a2058b9c724",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -386,6 +418,7 @@
},
{
"cell_type": "markdown",
"id": "5b09d5ef5b5e4bb6ab9b829b10b6a29f",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"In some cases, only a temporary mapset or location is needed, see [examples](https://grass.osgeo.org/grass-stable/manuals/grass.html#batch-jobs-with-the-exec-interface).\n",
Expand All @@ -394,6 +427,7 @@
},
{
"cell_type": "markdown",
"id": "a50416e276a0479cbe66534ed1713a40",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"#### Safely modifying computational region in a single mapset\n",
Expand All @@ -406,6 +440,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "46a27a456b804aa2a380d5edf15a5daf",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand Down Expand Up @@ -435,6 +470,7 @@
{
"cell_type": "code",
"execution_count": null,
"id": "1944c39560714e6e80c856f20744a8e5",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"outputs": [],
"source": [
Expand All @@ -443,13 +479,15 @@
},
{
"cell_type": "markdown",
"id": "d6ca27006b894b04b6fc8b79396e2797",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"#### Safely modifying vectors with attributes in a single mapset"
]
},
{
"cell_type": "markdown",
"id": "f61877af4e7f4313ad8234302950b331",
echoix marked this conversation as resolved.
Show resolved Hide resolved
"metadata": {},
"source": [
"By default vector maps share a single SQLite database file, however SQLite does not support concurrent write access. That poses a problem when modifying vectors with attributes in parallel. While this can be solved by running the computations in separate mapsets, it is also possible to change the default behavior to write attributes of each vector to the vector's individual SQLite file. This behavior can be activated after a new mapset is created with:\n",
Expand Down
1 change: 0 additions & 1 deletion doc/notebooks/solar_potential.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@
"outputs": [],
"source": [
"# Import Python standard library and IPython packages we need.\n",
"import os\n",
"import subprocess\n",
"import sys\n",
"\n",
Expand Down
1 change: 0 additions & 1 deletion doc/notebooks/viewshed_analysis.ipynb
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,6 @@
"outputs": [],
"source": [
"# Import Python standard library and IPython packages we need.\n",
"import os\n",
"import subprocess\n",
"import sys\n",
"\n",
Expand Down
Loading
Loading