unable to build due to error in concurrent hash map: "value_type of the container must be the same as its allocator's"

[keyan3]

First, thanks for contributing this awesome package. I just cloned the repository and was following the installation instructions for the python extension (./autobuild --install-python) when I ran into the following error. It seems that the instantiation of TBB's concurrent_hash_map that takes place in src/queue is faulty:

g++ -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot 
/Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -I/usr/local/include -I/usr/local/opt/[email protected]/include -I/usr/local/opt/sqlite/include -I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9 
-c src/dataset.cpp -o build/temp.macosx-11-x86_64-3.9/src/dataset.o -O3 -std=c++11 -I include -stdlib=libc++

In file included from src/dataset.cpp:1:
In file included from src/dataset.hpp:16:
In file included from /usr/local/include/tbb/concurrent_hash_map.h:17:
/usr/local/include/tbb/../oneapi/tbb/concurrent_hash_map.h:625:5: error: static_assert failed due to requirement 
'std::is_same<std::__1::pair<Message *const, bool>, std::__1::pair<Message *, bool>>::value'
"value_type of the container must be the same as its allocator's"

    static_assert(std::is_same<value_type, typename Allocator::value_type>::value,
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/queue.hpp:76:27: note: in instantiation of template class
'tbb::detail::d2::concurrent_hash_map<Message *, bool, MembershipKeyHashCompare, tbb::detail::d1::scalable_allocator<std::__1::pair<Message *, bool>>>' requested here
    membership_table_type membership;
                          ^

This is using clang 12.0.5 on a MacBook Pro. Here is the output of g++ --version for reference:

(base) keyan@MacBook-Pro-2 gosdt % g++ --version
Configured with: --prefix=/Library/Developer/CommandLineTools/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1
Apple clang version 12.0.5 (clang-1205.0.22.9)
Target: x86_64-apple-darwin20.6.0
Thread model: posix
InstalledDir: /Library/Developer/CommandLineTools/usr/bin

[keyan3]

Here is the full output of ./autobuild --install-python for reference:

(gosdt) keyan@MacBook-Pro-2 gosdt % ./autobuild --install-python
 /bin/sh ./config.status
config.status: creating Makefile
config.status: executing depfiles commands
test -z "gosdt " || rm -f gosdt 
rm -f *.o
rm -f src/*.o
rm -f test/*.o
running clean
running build
running build_ext
building 'gosdt' extension
creating build/temp.macosx-11-x86_64-3.9
creating build/temp.macosx-11-x86_64-3.9/src
g++ -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -I/usr/local/include -I/usr/local/opt/[email protected]/include -I/usr/local/opt/sqlite/include -I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/bitmask.cpp -o build/temp.macosx-11-x86_64-3.9/src/bitmask.o -O3 -std=c++11 -I include -stdlib=libc++
src/bitmask.cpp:317:15: warning: comparison of integers of different signs: 'int' and 'const unsigned int' [-Wsign-compare]
    if (begin >= this -> _size) { return false; }
        ~~~~~ ^  ~~~~~~~~~~~~~
src/bitmask.cpp:319:15: warning: comparison of integers of different signs: 'int' and 'const unsigned int' [-Wsign-compare]
    if (begin >= this -> _size) { return false; }
        ~~~~~ ^  ~~~~~~~~~~~~~
src/bitmask.cpp:630:15: warning: comparison of integers of different signs: 'int' and 'unsigned int' [-Wsign-compare]
    if (start >= size()) { return size(); }
        ~~~~~ ^  ~~~~~~
3 warnings generated.
g++ -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -I/usr/local/include -I/usr/local/opt/[email protected]/include -I/usr/local/opt/sqlite/include -I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/configuration.cpp -o build/temp.macosx-11-x86_64-3.9/src/configuration.o -O3 -std=c++11 -I include -stdlib=libc++
g++ -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX11.sdk -I/usr/local/include -I/usr/local/opt/[email protected]/include -I/usr/local/opt/sqlite/include -I/usr/local/opt/[email protected]/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c src/dataset.cpp -o build/temp.macosx-11-x86_64-3.9/src/dataset.o -O3 -std=c++11 -I include -stdlib=libc++
In file included from src/dataset.cpp:1:
In file included from src/dataset.hpp:16:
In file included from /usr/local/include/tbb/concurrent_hash_map.h:17:
/usr/local/include/tbb/../oneapi/tbb/concurrent_hash_map.h:625:5: error: static_assert failed due to requirement 'std::is_same<std::__1::pair<Message *const, bool>, std::__1::pair<Message *, bool>>::value' "value_type of the container must be the same as its allocator's"
    static_assert(std::is_same<value_type, typename Allocator::value_type>::value,
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/queue.hpp:76:27: note: in instantiation of template class 'tbb::detail::d2::concurrent_hash_map<Message *, bool, MembershipKeyHashCompare, tbb::detail::d1::scalable_allocator<std::__1::pair<Message *, bool>>>' requested here
    membership_table_type membership;
                          ^
src/dataset.cpp:52:18: warning: unused variable 'number_of_rows' [-Wunused-variable]
    unsigned int number_of_rows = 0; // Number of samples after compressions
                 ^
1 warning and 1 error generated.
error: command '/usr/bin/g++' failed with exit code 1

[margoseltzer]

I think we have a tbb version problem. Might you be able to try to do this using tbb/2020_U3_1 ?

[keyan3]

Thanks for the response @margoseltzer! I just tried using tbb@2020 and it does work. In addition, with the one-line change to src/queue.hpp in my pull request (#10), I am able to install GOSDT with newer versions of tbb as well.

[yli110-stat697]

Hey @keyan3 Thank you so much for the fix in src/queue.hpp I was able to ./autobuild --install-python with tbb.

Just FYI I tried tbb@2020 but it was deprecated and not working for me. Thanks for your magic solution 👍