[Python][C++] pyarrow does not build without parquet #45034
Comments
raulcd
changed the title
|
I was trying to understand if this is something that was added on 18.0.0 but it seems that this is something that wasn't possible in the past either. |
|
I was able to build without Parquest commenting the Parquet line in ArrowDatasetConfig.cmake.in and ArrowAceroConfig.cmake.in. Just an hack, though. |
|
Thanks for the input there. Acero should never require parquet currently. Dataset on the other hand should require it if we've build with PARQUET enabled. Just sharing as with the fix we should remove the related Parquet requirement from Acero entirely |
raulcd
added a commit
to raulcd/arrow
that referenced
this issue
Dec 16, 2024
… from Arrow Dataset when not necessary
kou
pushed a commit
that referenced
this issue
Dec 17, 2024
…Arrow Dataset when not necessary (#45035) ### Rationale for this change Currently we are defining Parquet as a requirement for Acero and for Dataset on `ArrowAceroConfig.cmake.in` and `ArrowDatasetConfig.cmake.in` respectively. Parquet should never be necessary for Acero and could be necessary for Dataset only if it has been built with Parquet support. ### What changes are included in this PR? Remove Parquet from `ArrowAceroConfig.cmake.in` and add it as optional dependency for `ArrowDatasetConfig.cmake.in` ### Are these changes tested? I've tested with a really minimal build of Arrow and Pyarrow. I don't think we want to add a build without Parquet but with Dataset and/or Acero at CI just to test this scenario. Logs for the local build without Parquet: ``` -- Found Cython version: 3.0.11 -- Arrow version: 19.0.0 -- Found the Arrow shared library: /home/raulcd/code/dist/lib/libarrow.so.1900.0.0 -- Found the Arrow import library: ARROW_IMPORT_LIB-NOTFOUND -- Found the Arrow static library: -- Building PyArrow with Dataset -- ArrowAcero version: 19.0.0 -- Found the ArrowAcero shared library: /home/raulcd/code/dist/lib/libarrow_acero.so.1900.0.0 -- Found the ArrowAcero import library: ARROW_ACERO_IMPORT_LIB-NOTFOUND -- Found the ArrowAcero static library: -- ArrowDataset version: 19.0.0 -- Found the ArrowDataset shared library: /home/raulcd/code/dist/lib/libarrow_dataset.so.1900.0.0 -- Found the ArrowDataset import library: ARROW_DATASET_IMPORT_LIB-NOTFOUND -- Found the ArrowDataset static library: -- Building PyArrow with Acero -- Parquet Encryption is NOT Enabled ``` The `IMPORT_LIB-NOTFOUND` has always been present and it's not new. ### Are there any user-facing changes? No as this should have already been the case. * GitHub Issue: #45034 Authored-by: Raúl Cumplido <[email protected]> Signed-off-by: Sutou Kouhei <[email protected]>
|
Issue resolved by pull request 45035 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Describe the bug, including details regarding any error messages, version, and platform.
see https://bugs.gentoo.org/944828
When building apache-arrow without parquet, the ParquetTargets.cmake file is not deployed.
But there are still trace of Parquet in ArrowDatasetConfig.cmake and ArrowAceroConfig.cmake
That causes pyarrow cmake to fail, as ParquetTargets.cmake is searched even if it not needed
This happen in version 18.1.0 and 18.0.0, on linux/gentoo amd64
Component(s)
C++
The text was updated successfully, but these errors were encountered: