Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PyVelox builds fail on CI #10963

Closed
kgpai opened this issue Sep 10, 2024 · 2 comments
Closed

PyVelox builds fail on CI #10963

kgpai opened this issue Sep 10, 2024 · 2 comments
Labels
build triage Newly created issue that needs attention.

Comments

@kgpai
Copy link
Contributor

kgpai commented Sep 10, 2024

Problem description

Seems like this is due to lib stemmer not being built with position independent code ?

PyVelox builds fail with the following error :


 && /opt/rh/gcc-toolset-12/root/bin/g++ -fPIC -mavx2 -mfma -mavx -mf16c -mlzcnt -std=c++17 -mbmi2 -D USE_VELOX_COMMON_BASE -D HAS_UNCAUGHT_EXCEPTIONS -g  -Wl,-export-dynamic -shared  -o /__w/velox/velox/velox_main/_build/lib.linux-x86_64-3.9/pyvelox/pyvelox.cpython-39-x86_64-linux-gnu.so pyvelox/CMakeFiles/pyvelox.dir/complex.cpp.o pyvelox/CMakeFiles/pyvelox.dir/conversion.cpp.o pyvelox/CMakeFiles/pyvelox.dir/pyvelox.cpp.o pyvelox/CMakeFiles/pyvelox.dir/serde.cpp.o pyvelox/CMakeFiles/pyvelox.dir/signatures.cpp.o  -Wl,-rpath,/usr/local/lib:/usr/local/lib64:  velox/type/libvelox_type.a  velox/vector/libvelox_vector.a  velox/core/libvelox_core.a  velox/exec/libvelox_exec.a  velox/parse/libvelox_parse_parser.a  velox/functions/prestosql/registration/libvelox_functions_prestosql.a  velox/functions/sparksql/libvelox_functions_spark.a  velox/functions/prestosql/aggregates/libvelox_aggregates.a  velox/functions/sparksql/aggregates/libvelox_functions_spark_aggregates.a  velox/duckdb/conversion/libvelox_duckdb_parser.a  velox/parse/libvelox_parse_expression.a  velox/parse/libvelox_parse_utils.a  velox/functions/libvelox_function_registry.a  velox/duckdb/conversion/libvelox_duckdb_conversion.a  /usr/local/lib/libduckdb_static.a  /usr/local/lib/libduckdb_fsst.a  /usr/local/lib/libduckdb_fmt.a  /usr/local/lib/libduckdb_pg_query.a  /usr/local/lib/libduckdb_re2.a  /usr/local/lib/libduckdb_miniz.a  /usr/local/lib/libduckdb_utf8proc.a  /usr/local/lib/libduckdb_hyperloglog.a  /usr/local/lib/libduckdb_fastpforlib.a  /usr/local/lib/libduckdb_mbedtls.a  /usr/local/lib/libjemalloc_extension.a  velox/functions/prestosql/libvelox_functions_prestosql_impl.a  velox/external/md5/libvelox_external_md5.a  velox/functions/prestosql/types/libvelox_presto_types.a  velox/functions/prestosql/json/libvelox_functions_json.a  velox/functions/lib/libvelox_functions_lib_date_time_formatter.a  /usr/local/lib/libstemmer.a  velox/functions/lib/libvelox_is_null_functions.a  _deps/simdjson-build/libsimdjson.a  velox/functions/sparksql/specialforms/libvelox_functions_spark_specialforms.a  velox/functions/prestosql/aggregates/libvelox_aggregates.a  velox/functions/lib/libvelox_functions_lib.a  velox/common/hyperloglog/libvelox_common_hyperloglog.a  velox/functions/lib/aggregates/libvelox_functions_aggregates.a  velox/exec/libvelox_exec.a  velox/expression/libvelox_expression.a  velox/functions/lib/libvelox_functions_util.a  /usr/lib64/libdouble-conversion.so.3.1.5  velox/exec/trace/libvelox_query_trace_exec.a  velox/core/libvelox_core.a  velox/vector/arrow/libvelox_arrow_bridge.a  velox/common/caching/libvelox_caching.a  velox/connectors/libvelox_connector.a  velox/common/io/libvelox_common_io.a  velox/serializers/libvelox_presto_serializer.a  velox/row/libvelox_row_fast.a  velox/vector/libvelox_vector.a  velox/expression/libvelox_expression_functions.a  velox/expression/type_calculation/libvelox_type_calculation.a  velox/expression/signature_parser/libvelox_signature_parser.a  velox/type/libvelox_type.a  velox/type/tz/libvelox_type_tz.a  velox/common/base/libvelox_common_base.a  velox/common/base/libvelox_exception.a  velox/common/serialization/libvelox_serialization.a  velox/common/memory/libvelox_memory.a  velox/common/time/libvelox_time.a  velox/buffer/libvelox_buffer.a  velox/common/config/libvelox_common_config.a  velox/common/file/libvelox_file.a  velox/common/testutil/libvelox_test_util.a  velox/common/compression/libvelox_common_compression.a  velox/common/process/libvelox_process.a  velox/type/libvelox_type.a  velox/type/tz/libvelox_type_tz.a  velox/common/base/libvelox_common_base.a  velox/common/base/libvelox_exception.a  velox/common/serialization/libvelox_serialization.a  velox/common/memory/libvelox_memory.a  velox/common/time/libvelox_time.a  velox/buffer/libvelox_buffer.a  velox/common/config/libvelox_common_config.a  velox/common/file/libvelox_file.a  velox/common/testutil/libvelox_test_util.a  velox/common/compression/libvelox_common_compression.a  velox/common/process/libvelox_process.a  velox/common/encode/libvelox_encode.a  velox/common/base/libvelox_status.a  velox/external/date/libvelox_external_date.a  /usr/lib64/libre2.so.9.0.0  /usr/local/lib/libfolly.a  /usr/local/lib/libboost_regex.so.1.84.0  /usr/local/lib/libboost_context.so.1.84.0  /usr/local/lib/libboost_filesystem.so.1.84.0  /usr/local/lib/libboost_program_options.so.1.84.0  /usr/local/lib/libboost_system.so.1.84.0  /usr/local/lib/libboost_thread.so.1.84.0  /usr/local/lib/libboost_atomic.so.1.84.0  /usr/lib64/libdouble-conversion.so  /usr/local/lib64/libgflags.so.2.2.2  /usr/local/lib64/libglog.so  /usr/lib64/libevent.so  /usr/lib64/libz.so  /usr/lib64/libssl.so  /usr/lib64/libcrypto.so  /usr/lib64/liblz4.so  /usr/lib64/libzstd.so  /usr/local/lib64/libsnappy.a  /usr/lib64/libdwarf.so  /usr/lib64/libsodium.so  -ldl  /usr/local/lib64/libglog.so  /usr/local/lib64/libgflags.so.2.2.2  /usr/local/lib64/libfmt.a && :
    /opt/rh/gcc-toolset-12/root/usr/libexec/gcc/x86_64-redhat-linux/12/ld: /usr/local/lib/libstemmer.a(libstemmer.o): relocation R_X86_64_32 against `.data' can not be used when making a shared object; recompile with -fPIC
    /opt/rh/gcc-toolset-12/root/usr/libexec/gcc/x86_64-redhat-linux/12/ld: failed to set dynamic section sizes: bad value
    

System information

https://github.com/facebookincubator/velox/actions/runs/10800144787/job/29957468844?pr=10962

CMake log

No response

@kgpai kgpai added build triage Newly created issue that needs attention. labels Sep 10, 2024
@kgpai
Copy link
Contributor Author

kgpai commented Sep 10, 2024

cc: @assignUser

kgpai pushed a commit to kgpai/velox-1 that referenced this issue Sep 11, 2024
Summary:
Reverting PR facebookincubator#10884 due to it breaking PyVelox builds and Fuzzer .
Fixes issue : facebookincubator#10963

Original commit changeset: dec13d672129

Original Phabricator Diff: D62373662

Reviewed By: amitkdutta

Differential Revision: D62477240
kgpai pushed a commit to kgpai/velox-1 that referenced this issue Sep 11, 2024
…or#10965)

Summary:
Pull Request resolved: facebookincubator#10965

Reverting PR facebookincubator#10884 due to it breaking PyVelox builds and Fuzzer .
Fixes issue : facebookincubator#10963

Original commit changeset: dec13d672129

Original Phabricator Diff: D62373662

Reviewed By: xiaoxmeng, amitkdutta, kagamiori

Differential Revision: D62477240
facebook-github-bot pushed a commit that referenced this issue Sep 11, 2024
Summary:
Pull Request resolved: #10965

Reverting PR #10884 due to it breaking PyVelox builds and Fuzzer .
Fixes issue : #10963

Original commit changeset: dec13d672129

Original Phabricator Diff: D62373662

Reviewed By: xiaoxmeng, amitkdutta, kagamiori

Differential Revision: D62477240

fbshipit-source-id: a98174e1be031864e84dd11ba8eb3475415d5a0e
@assignUser
Copy link
Collaborator

Closed by #10965

facebook-github-bot pushed a commit that referenced this issue Sep 26, 2024
Summary:
The early merged pr (38f9a1f) was reverted by #10965 due to this issue: #10963.

In the original impl., a patch is expected to be applied to add `-fPIC` before building lib
stemmer, but when building docker image (ghcr.io/facebookincubator/velox-dev:centos9),
that patch file is not available to use due to the copied setup script outside velox repo is
executed. See code:
https://github.com/facebookincubator/velox/blob/7f2d7adaa1544c72129fd1c8d0766755ff354455/scripts/centos.dockerfile#L22
This pr proposes the installation of lib stemmer with the above issue fixed.

Pull Request resolved: #10984

Reviewed By: DanielHunte

Differential Revision: D63344087

Pulled By: Yuhta

fbshipit-source-id: a1467f5392ef828efdd219d0b7f1dfe0003d00ed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
build triage Newly created issue that needs attention.
Projects
None yet
Development

No branches or pull requests

2 participants