Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running Fix Geometries using PyQGIS outside QGIS does not perform as expected #436

Open
1 task done
EricSamson-Tract opened this issue Jun 3, 2024 · 1 comment
Open
1 task done

Running Fix Geometries using PyQGIS outside QGIS does not perform as expected #436

EricSamson-Tract opened this issue Jun 3, 2024 · 1 comment
Labels
bug

Comments

@EricSamson-Tract
Copy link

Solution to issue cannot be found in the documentation.

  • I checked the documentation.

Issue

When running fix geometries from a conda environment with QGIS installed, fix geometries will work on all geometries except for 1 or 2. When running fix geometries within QGIS, it will fix all geometries (based on geos).

`def zonal_histogram_qgis(gdf, raster_path):
temp_dir = tempfile.mkdtemp()
temp_shapefile = os.path.join(temp_dir, 'parcels.shp')
gdf.to_file(temp_shapefile)

input_layer = QgsVectorLayer(temp_shapefile, 'parcels', 'ogr')

fixed_shapefile = os.path.join(temp_dir, 'parcels_fixed.shp')
fix_params = {
    'INPUT': input_layer,
    'METHOD': 1,
    'OUTPUT': fixed_shapefile
}
Processing.runAlgorithm("native:fixgeometries", fix_params)

out_shape = os.path.join(temp_dir, 'zonal_stats.shp')
zone_params = {
    'INPUT_RASTER': raster_path,
    'RASTER_BAND': 1,
    'INPUT_VECTOR': fixed_shapefile2,
    'COLUMN_PREFIX': 'value_',
    'OUTPUT': out_shape
}
feedback = QgsProcessingFeedback()
Processing.runAlgorithm("native:zonalhistogram", zone_params, feedback=feedback)

zonal_gdf = gpd.read_file(out_shape)

return zonal_gdf`


I unfortunately can't attach data. I am running this on a dataset of around 30,000 polygons, derived from parcel data. Running something similar from within the python window in QGIS works as expected, the gui tools as well. I am not sure what is occurring here.

Installed packages

# Name                    Version                   Build  Channel
affine                    2.4.0                    pypi_0    pypi
asttokens                 2.4.1              pyhd8ed1ab_0    conda-forge
attrs                     23.2.0                   pypi_0    pypi
aws-c-auth                0.7.20               h6823eb1_0    conda-forge
aws-c-cal                 0.6.12               hc83774a_0    conda-forge
aws-c-common              0.9.17               h2466b09_0    conda-forge
aws-c-compression         0.2.18               hc83774a_4    conda-forge
aws-c-event-stream        0.4.2               hc6c0aac_10    conda-forge
aws-c-http                0.8.1               hced5053_13    conda-forge
aws-c-io                  0.14.8               hebaacdb_0    conda-forge
aws-c-mqtt                0.10.4               hdafd9a4_2    conda-forge
aws-c-s3                  0.5.8                h7a83f0e_3    conda-forge
aws-c-sdkutils            0.1.16               hc83774a_0    conda-forge
aws-checksums             0.1.18               hc83774a_4    conda-forge
aws-crt-cpp               0.26.8              h4438f58_10    conda-forge
aws-sdk-cpp               1.11.329             h12f3f85_0    conda-forge
azure-core-cpp            1.11.1               h249a519_1    conda-forge
azure-identity-cpp        1.6.0                h91493d7_1    conda-forge
azure-storage-blobs-cpp   12.10.0              h91493d7_1    conda-forge
azure-storage-common-cpp  12.5.0               h91493d7_4    conda-forge
blas                      2.122                       mkl    conda-forge
blas-devel                3.9.0              22_win64_mkl    conda-forge
blosc                     1.21.5               hbd69f2e_1    conda-forge
brotli-python             1.1.0           py312h53d5487_1    conda-forge
bzip2                     1.0.8                hcfcfb64_5    conda-forge
c-ares                    1.28.1               hcfcfb64_0    conda-forge
ca-certificates           2024.2.2             h56e8100_0    conda-forge
cairo                     1.18.0               h1fef639_0    conda-forge
ceres-solver              2.2.0                h0d88682_3    conda-forge
certifi                   2024.2.2           pyhd8ed1ab_0    conda-forge
cfitsio                   4.4.0                h9b0cee5_1    conda-forge
charset-normalizer        3.3.2              pyhd8ed1ab_0    conda-forge
click                     8.1.7                    pypi_0    pypi
click-plugins             1.1.1                    pypi_0    pypi
cligj                     0.7.2                    pypi_0    pypi
colorama                  0.4.6              pyhd8ed1ab_0    conda-forge
comm                      0.2.2              pyhd8ed1ab_0    conda-forge
coverage                  7.5.1           py312h4389bb4_0    conda-forge
curl                      8.7.1                hd5e4a3a_0    conda-forge
dataclasses               0.8                pyhc8e2a94_3    conda-forge
debugpy                   1.8.1           py312h53d5487_0    conda-forge
decorator                 5.1.1              pyhd8ed1ab_0    conda-forge
draco                     1.5.7                h181d51b_0    conda-forge
eigen                     3.4.0                h91493d7_0    conda-forge
et-xmlfile                1.1.0                    pypi_0    pypi
exceptiongroup            1.2.0              pyhd8ed1ab_2    conda-forge
executing                 2.0.1              pyhd8ed1ab_0    conda-forge
exiv2                     0.28.2               hadc2d18_0    conda-forge
expat                     2.6.2                h63175ca_0    conda-forge
fiona                     1.9.6                    pypi_0    pypi
fmt                       10.2.1               h181d51b_0    conda-forge
font-ttf-dejavu-sans-mono 2.37                 hab24e00_0    conda-forge
font-ttf-inconsolata      3.000                h77eed37_0    conda-forge
font-ttf-source-code-pro  2.038                h77eed37_0    conda-forge
font-ttf-ubuntu           0.83                 h77eed37_2    conda-forge
fontconfig                2.14.2               hbde0cde_0    conda-forge
fonts-conda-ecosystem     1                             0    conda-forge
fonts-conda-forge         1                             0    conda-forge
freetype                  2.12.1               hdaf720e_2    conda-forge
freexl                    2.0.0                h8276f4a_0    conda-forge
future                    1.0.0              pyhd8ed1ab_0    conda-forge
gdal                      3.8.5           py312hea5013e_4    conda-forge
geopandas                 0.14.4                   pypi_0    pypi
geos                      3.12.1               h1537add_0    conda-forge
geotiff                   1.7.1               hfbd0779_17    conda-forge
gflags                    2.2.2             ha925a31_1004    conda-forge
glib                      2.80.2               h0df6a38_0    conda-forge
glib-tools                2.80.2               h2f9d560_0    conda-forge
glog                      0.7.0                h9cd36e5_0    conda-forge
greenlet                  3.0.3                    pypi_0    pypi
gsl                       2.7                  hdfb1a43_0    conda-forge
gst-plugins-base          1.24.3               hba88be7_0    conda-forge
gstreamer                 1.24.3               h5006eae_0    conda-forge
hdf4                      4.2.15               h5557f11_7    conda-forge
hdf5                      1.14.3          nompi_h73e8ff5_101    conda-forge
httplib2                  0.22.0             pyhd8ed1ab_0    conda-forge
icu                       73.2                 h63175ca_0    conda-forge
idna                      3.7                pyhd8ed1ab_0    conda-forge
importlib-metadata        7.1.0              pyha770c72_0    conda-forge
importlib_metadata        7.1.0                hd8ed1ab_0    conda-forge
intel-openmp              2024.1.0           h57928b3_965    conda-forge
ipykernel                 6.29.3             pyha63f2e9_0    conda-forge
ipython                   8.24.0             pyh7428d3b_0    conda-forge
jedi                      0.19.1             pyhd8ed1ab_0    conda-forge
jinja2                    3.1.4              pyhd8ed1ab_0    conda-forge
jupyter_client            8.6.1              pyhd8ed1ab_0    conda-forge
jupyter_core              5.7.2           py312h2e8e312_0    conda-forge
kealib                    1.5.3                h6c43f9b_1    conda-forge
khronos-opencl-icd-loader 2023.04.17           h64bf75a_0    conda-forge
krb5                      1.21.2               heb0366b_0    conda-forge
laz-perf                  3.4.0                h91493d7_0    conda-forge
lcms2                     2.16                 h67d730c_0    conda-forge
lerc                      4.0.0                h63175ca_0    conda-forge
libabseil                 20240116.2      cxx17_h63175ca_0    conda-forge
libaec                    1.1.3                h63175ca_0    conda-forge
libarchive                3.7.2                h313118b_1    conda-forge
libblas                   3.9.0              22_win64_mkl    conda-forge
libboost-headers          1.85.0               h57928b3_1    conda-forge
libbrotlicommon           1.1.0                hcfcfb64_1    conda-forge
libbrotlidec              1.1.0                hcfcfb64_1    conda-forge
libbrotlienc              1.1.0                hcfcfb64_1    conda-forge
libcblas                  3.9.0              22_win64_mkl    conda-forge
libclang13                18.1.5          default_hf64faad_0    conda-forge
libcrc32c                 1.1.2                h0e60522_0    conda-forge
libcurl                   8.7.1                hd5e4a3a_0    conda-forge
libdeflate                1.20                 hcfcfb64_0    conda-forge
libexpat                  2.6.2                h63175ca_0    conda-forge
libffi                    3.4.2                h8ffe710_5    conda-forge
libgdal                   3.8.5                hfb9f81c_4    conda-forge
libglib                   2.80.2               h0df6a38_0    conda-forge
libgoogle-cloud           2.24.0               h5e7cea3_0    conda-forge
libgoogle-cloud-storage   2.24.0               hce61461_0    conda-forge
libgrpc                   1.62.2               h5273850_0    conda-forge
libhwloc                  2.10.0          default_h8125262_1001    conda-forge
libiconv                  1.17                 hcfcfb64_2    conda-forge
libintl                   0.22.5               h5728263_2    conda-forge
libintl-devel             0.22.5               h5728263_2    conda-forge
libjpeg-turbo             3.0.0                hcfcfb64_1    conda-forge
libkml                    1.3.0             haf3e7a6_1018    conda-forge
liblapack                 3.9.0              22_win64_mkl    conda-forge
liblapacke                3.9.0              22_win64_mkl    conda-forge
libnetcdf                 4.9.2           nompi_h07c049d_113    conda-forge
libogg                    1.3.4                h8ffe710_1    conda-forge
libpng                    1.6.43               h19919ed_0    conda-forge
libpq                     16.3                 hab9416b_0    conda-forge
libprotobuf               4.25.3               h503648d_0    conda-forge
libre2-11                 2023.09.01           hf8d8778_2    conda-forge
librttopo                 1.1.0               h94c4f80_15    conda-forge
libsodium                 1.0.18               h8d14728_1    conda-forge
libspatialindex           1.9.3                h5a68840_5    conda-forge
libspatialite             5.1.0                h1b08085_6    conda-forge
libsqlite                 3.45.3               hcfcfb64_0    conda-forge
libssh2                   1.11.0               h7dfc565_0    conda-forge
libtiff                   4.6.0                hddb2be6_3    conda-forge
libvorbis                 1.3.7                h0e60522_0    conda-forge
libwebp                   1.4.0                h2466b09_0    conda-forge
libwebp-base              1.4.0                hcfcfb64_0    conda-forge
libxml2                   2.12.7               h283a6d9_0    conda-forge
libxslt                   1.1.39               h3df6e99_0    conda-forge
libzip                    1.10.1               h1d365fa_3    conda-forge
libzlib                   1.2.13               hcfcfb64_5    conda-forge
lxml                      5.2.2           py312h56c7e3b_0    conda-forge
lz4-c                     1.9.4                hcfcfb64_0    conda-forge
lzo                       2.10              hcfcfb64_1001    conda-forge
m2w64-gcc-libgfortran     5.3.0                         6    conda-forge
m2w64-gcc-libs            5.3.0                         7    conda-forge
m2w64-gcc-libs-core       5.3.0                         7    conda-forge
m2w64-gmp                 6.1.0                         2    conda-forge
m2w64-libwinpthread-git   5.0.0.4634.697f757               2    conda-forge
markupsafe                2.1.5           py312he70551f_0    conda-forge
matplotlib-inline         0.1.7              pyhd8ed1ab_0    conda-forge
minizip                   4.0.5                h5bed578_0    conda-forge
mkl                       2024.1.0           h66d3029_692    conda-forge
mkl-devel                 2024.1.0           h57928b3_692    conda-forge
mkl-include               2024.1.0           h66d3029_692    conda-forge
mock                      5.1.0              pyhd8ed1ab_0    conda-forge
msys2-conda-epoch         20160418                      1    conda-forge
nest-asyncio              1.6.0              pyhd8ed1ab_0    conda-forge
nitro                     2.7.dev8             h1537add_0    conda-forge
nose2                     0.9.2                      py_0    conda-forge
numpy                     1.26.4          py312h8753938_0    conda-forge
openjpeg                  2.5.2                h3d672ee_0    conda-forge
openpyxl                  3.1.2                    pypi_0    pypi
openssl                   3.3.0                h2466b09_2    conda-forge
owslib                    0.31.0             pyhd8ed1ab_0    conda-forge
packaging                 24.0               pyhd8ed1ab_0    conda-forge
pandas                    2.2.2                    pypi_0    pypi
parso                     0.8.4              pyhd8ed1ab_0    conda-forge
pcre2                     10.43                h17e33f8_0    conda-forge
pdal                      2.7.1                h78909d1_6    conda-forge
pickleshare               0.7.5                   py_1003    conda-forge
pip                       24.0               pyhd8ed1ab_0    conda-forge
pixman                    0.43.4               h63175ca_0    conda-forge
platformdirs              4.2.2              pyhd8ed1ab_0    conda-forge
plotly                    5.22.0             pyhd8ed1ab_0    conda-forge
ply                       3.11               pyhd8ed1ab_2    conda-forge
poppler                   24.04.0              h747fd5a_0    conda-forge
poppler-data              0.4.12               hd8ed1ab_0    conda-forge
postgresql                16.3                 h7f155c9_0    conda-forge
proj                      9.4.0                he13c7e8_1    conda-forge
prompt-toolkit            3.0.42             pyha770c72_0    conda-forge
psutil                    5.9.8           py312he70551f_0    conda-forge
psycopg2                  2.9.9           py312hf50bb3c_0    conda-forge
pthreads-win32            2.9.1                hfa6e2cd_3    conda-forge
pure_eval                 0.2.2              pyhd8ed1ab_0    conda-forge
pyarrow                   16.1.0                   pypi_0    pypi
pygments                  2.18.0             pyhd8ed1ab_0    conda-forge
pyparsing                 3.1.2              pyhd8ed1ab_0    conda-forge
pyproj                    3.6.1           py312h616b599_6    conda-forge
pyqt                      5.15.9          py312he09f080_5    conda-forge
pyqt5-sip                 12.12.2         py312h53d5487_5    conda-forge
pyqtwebkit                5.15.9          py312hca0710b_2    conda-forge
pysocks                   1.7.1              pyh0701188_6    conda-forge
python                    3.12.3          h2628c8c_0_cpython    conda-forge
python-dateutil           2.9.0              pyhd8ed1ab_0    conda-forge
python_abi                3.12                    4_cp312    conda-forge
pytz                      2024.1             pyhd8ed1ab_0    conda-forge
pywin32                   306             py312h53d5487_2    conda-forge
pyyaml                    6.0.1           py312he70551f_1    conda-forge
pyzmq                     26.0.3          py312hd7027bb_0    conda-forge
qca                       2.3.8                h2624d1c_0    conda-forge
qgis                      3.36.3          py312hdab107f_0    conda-forge
qjson                     0.9.0             h04a78d6_1009    conda-forge
qscintilla2               2.14.1          py312hca0710b_0    conda-forge
qt-main                   5.15.8              hcef0176_21    conda-forge
qtkeychain                0.14.3               hf9d22a5_0    conda-forge
qtwebkit                  5.212               h4d8ddc9_16    conda-forge
qwt                       6.3.0                h9417a65_0    conda-forge
rasterio                  1.3.10                   pypi_0    pypi
re2                       2023.09.01           hd3b24a8_2    conda-forge
requests                  2.32.1             pyhd8ed1ab_0    conda-forge
setuptools                69.5.1             pyhd8ed1ab_0    conda-forge
shapely                   2.0.4                    pypi_0    pypi
sip                       6.7.12          py312h53d5487_0    conda-forge
six                       1.16.0             pyh6c4a22f_0    conda-forge
snappy                    1.2.0                hfb803bf_1    conda-forge
snuggs                    1.4.7                    pypi_0    pypi
spdlog                    1.13.0               h64d2f7d_0    conda-forge
sqlalchemy                2.0.30                   pypi_0    pypi
sqlite                    3.45.3               hcfcfb64_0    conda-forge
stack_data                0.6.2              pyhd8ed1ab_0    conda-forge
tbb                       2021.12.0            hc790b64_1    conda-forge
tenacity                  8.3.0              pyhd8ed1ab_0    conda-forge
tiledb                    2.22.0               h13740f5_4    conda-forge
tk                        8.6.13               h5226925_1    conda-forge
toml                      0.10.2             pyhd8ed1ab_0    conda-forge
tomli                     2.0.1              pyhd8ed1ab_0    conda-forge
tornado                   6.4             py312he70551f_0    conda-forge
traitlets                 5.14.3             pyhd8ed1ab_0    conda-forge
typing_extensions         4.11.0             pyha770c72_0    conda-forge
tzdata                    2024.1                   pypi_0    pypi
ucrt                      10.0.22621.0         h57928b3_0    conda-forge
uriparser                 0.9.8                h5a68840_0    conda-forge
urllib3                   2.2.1              pyhd8ed1ab_0    conda-forge
vc                        14.3                ha32ba9b_20    conda-forge
vc14_runtime              14.38.33135         h835141b_20    conda-forge
vs2015_runtime            14.38.33135         h22015db_20    conda-forge
wcwidth                   0.2.13             pyhd8ed1ab_0    conda-forge
wheel                     0.43.0             pyhd8ed1ab_1    conda-forge
win_inet_pton             1.1.0              pyhd8ed1ab_6    conda-forge
xerces-c                  3.2.5                h63175ca_0    conda-forge
xz                        5.2.6                h8d14728_0    conda-forge
yaml                      0.2.5                h8ffe710_2    conda-forge
zeromq                    4.3.5                he1f189c_4    conda-forge
zipp                      3.17.0             pyhd8ed1ab_0    conda-forge
zlib                      1.2.13               hcfcfb64_5    conda-forge
zstd                      1.5.6                h0ea2cb4_0    conda-forge

Environment info

active environment : qgis_dev
    active env location : \anaconda3\envs\qgis_dev
            shell level : 2
       user config file : \.condarc
 populated config files : 
          conda version : 23.10.0
    conda-build version : 3.25.0
         python version : 3.11.5.final.0
       virtual packages : __archspec=1=x86_64
                          __cuda=12.3=0
                          __win=0=0
       base environment : \anaconda3  (writable)
      conda av data dir : \anaconda3\etc\conda
  conda av metadata url : None
           channel URLs : https://repo.anaconda.com/pkgs/main/win-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/win-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/msys2/win-64
                          https://repo.anaconda.com/pkgs/msys2/noarch
          package cache : \anaconda3\pkgs
                          \.conda\pkgs
                          \AppData\Local\conda\conda\pkgs
       envs directories : \anaconda3\envs
                          \.conda\envs
                          \AppData\Local\conda\conda\envs
               platform : win-64
             user-agent : conda/23.10.0 requests/2.31.0 CPython/3.11.5 Windows/10 Windows/10.0.22621 solver/libmamba conda-libmamba-solver/23.11.0 libmambapy/1.5.3 aau/0.4.3 c/8-vPlzrjxjaYu5zf2_lXHw s/RA-SHHiitLZsFiSdZ6iN2g e/U2nIOdCW9NhLdpQpCI0EzQ
          administrator : False
             netrc file : None
           offline mode : False
@gillins
Copy link
Contributor

gillins commented Jun 3, 2024

I see that you've installed rasterio via pypi which is known to cause conflicts with conda (they package their own GDAL library and is not recommended for production - see https://rasterio.readthedocs.io/en/stable/installation.html).
Can you try again in a brand new environment with just qgis? And not install anything from pypi into it...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@gillins @EricSamson-Tract