Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Segmentation Fault (Core Dumped) #109

Open
hashsiddique opened this issue Jun 16, 2021 · 11 comments
Open

Segmentation Fault (Core Dumped) #109

hashsiddique opened this issue Jun 16, 2021 · 11 comments

Comments

@hashsiddique
Copy link

Hi! I'm Hassan Naeem. I'm facing an issue while executing ID2T code. The code gives "Segmentation Fault (Core Dumped)" at the output, once the pcap file is input. Can anyone help me out in this regard please? Why I'm getting such an error.

Screenshot of error attached please.
Segmentation_fault
@hashsiddique
Copy link
Author

@pepper-jk .. can you help me out please?

@pepper-jk
Copy link
Collaborator

pepper-jk commented Jun 18, 2021 •
edited
Loading

Sorry, I did not get a notification until you mentioned me. I'll take a look at this now.
(also I'm fixing my notification settings.)

@pepper-jk
Copy link
Collaborator

pepper-jk commented Jun 18, 2021 •
edited
Loading

Could you provide a pcap file that produces the error? So that I can reproduce this.
Are there pcap files that work for you? Or in other words does this happen with every pcap you have tried so far?

Also what OS / Linux distribution and version are you running?
Any issues during build?

Possibly information about your python setup:

  • python version
  • are all dependencies installed correctly?

Other notes:

  • .pcap.ng files are not supported and those are sometimes saved with the .pcap extension. It might be possible that you got such a file. Try to convert the file to a normal pcap with a tool like wireshark and try again.
  • do you have enough space left on your home partition / in your home directory to create the database?

@hashsiddique
Copy link
Author

@pepper-jk Thank you so much for getting back, it means a lot as I'm stuck on this issue for some time now.

The discussed pcap file (datagusta.pcap) that is giving errors had pcapng extension, but I converted its extension to pcap through wireshark (save as option). Still it will not work?

Yes, I have tried other pcap files (02 more actually) and they work fine.

Ubuntu 20.04.2 LTS
Python Version 3.8.5
The final attempt of building the package and dependencies installation was successful. (Initially I was facing issues building up and installing dependencies).

Yes, I have enough space on Home partition (35 GBs).

@hashsiddique
Copy link
Author

@pepper-jk I think there is some issue in recognizing pcap file converted (from pcapng) using wireshark and gives error (Segmentation Fault). Can you refer me any other converter (to convert pcapng to pcap) please (and ID2T program also recognizes that file)?

@pepper-jk
Copy link
Collaborator

The discussed pcap file (datagusta.pcap) that is giving errors had pcapng extension, but I converted its extension to pcap through wireshark (save as option). Still it will not work?

That sounds strange.

Are you sure, you changed the file format not just the extension in the file name?

2021-06-19-150640_3640x1920_scrot

@pepper-jk
Copy link
Collaborator

pepper-jk commented Jun 19, 2021 •
edited
Loading

@pepper-jk I think there is some issue in recognizing pcap file converted (from pcapng) using wireshark and gives error (Segmentation Fault). Can you refer me any other converter (to convert pcapng to pcap) please (and ID2T program also recognizes that file)?

I have not worked with pcaps in a while, with a bit of googling these turned up:

  • This free online converter by NETRESEC (Be careful! This is HTTP only, so if you have sensitive data, don't use this! Also if you don't trust them, don't use it.)
  • tsahrk is able to do it as well tshark -F pcap -r {pcapng file} -w {pcap file} (but this should provide the same results as wireshark)
  • Tcpdump version 4.1.1 and later should also handle pcapng files. So you should be able to convert them. tcpdump -r {pcapng file} -w {pcap file}

Hope this helps, but in the future you should just search for tools like this yourself. As this is not directly related to the ID2T project.

@pepper-jk
Copy link
Collaborator

If you could upload the pcapng and pcap file here, I could try to reproduce the issue myself.

@LuccaRoofthooft
Copy link

Hello @pepper-jk
Any update on this? I am facing the same issue as Hassan Naeem. I can create attacks for some files but for some not. The files are all in pcap form, I tried to convert them again to pcap to be sure but it still gives the same error.

System set up information:
MATE 1.26.0 VM
Python version 3.10.6
130GB disk space left

@LuccaRoofthooft
Copy link

LuccaRoofthooft commented Apr 17, 2023 •
edited by pepper-jk
Loading 

$ ./build.sh --full ouput

Updating submodules

Detected OS: Debian
Packages: Checking...
Packages: Found.
Requirement already satisfied: pip in ./.venv/lib/python3.10/site-packages (23.1)
Requirement already satisfied: cairocffi==0.8.1 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 1)) (0.8.1)
Requirement already satisfied: cffi>=1.12.2 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 2)) (1.15.1)
Requirement already satisfied: coverage>=4.5.1 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 3)) (7.2.3)
Requirement already satisfied: cycler>=0.10.0 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 4)) (0.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 5)) (1.4.4)
Requirement already satisfied: lea==2.3.5 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 6)) (2.3.5)
Requirement already satisfied: matplotlib>=2.2.3 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 7)) (3.7.1)
Requirement already satisfied: memory-profiler>=0.54.0 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 8)) (0.61.0)
Requirement already satisfied: numpy>=1.15 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 9)) (1.24.2)
Requirement already satisfied: psutil>=5.4.5 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 10)) (5.9.4)
Requirement already satisfied: pycparser>=2.18 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 11)) (2.21)
Requirement already satisfied: pyparsing>=2.2.0 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 12)) (3.0.9)
Requirement already satisfied: python-dateutil>=2.7.2 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 13)) (2.8.2)
Requirement already satisfied: pytz>=2018.4 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 14)) (2023.3)
Requirement already satisfied: pyxdg>=0.26 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 15)) (0.28)
Requirement already satisfied: scapy==2.4.2 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 16)) (2.4.2)
Requirement already satisfied: scipy>=1.1.0 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 17)) (1.10.1)
Requirement already satisfied: six>=1.11.0 in ./.venv/lib/python3.10/site-packages (from -r resources/requirements.txt (line 18)) (1.16.0)
Requirement already satisfied: contourpy>=1.0.1 in ./.venv/lib/python3.10/site-packages (from matplotlib>=2.2.3->-r resources/requirements.txt (line 7)) (1.0.7)
Requirement already satisfied: fonttools>=4.22.0 in ./.venv/lib/python3.10/site-packages (from matplotlib>=2.2.3->-r resources/requirements.txt (line 7)) (4.39.3)
Requirement already satisfied: packaging>=20.0 in ./.venv/lib/python3.10/site-packages (from matplotlib>=2.2.3->-r resources/requirements.txt (line 7)) (23.1)
Requirement already satisfied: pillow>=6.2.0 in ./.venv/lib/python3.10/site-packages (from matplotlib>=2.2.3->-r resources/requirements.txt (line 7)) (9.5.0)
-- Running Release configuration.
-- CMake version: 3.22.1
-- CMAKE_CXX_COMPILER '/usr/bin/c++' 'GNU' '11.3.0'
-- CMAKE_CXX_FLAGS                ''
-- CMAKE_CXX_FLAGS_RELEASE        '-O3 -DNDEBUG -Wall'
-- SQLITECPP_RUN_CPPLINT OFF
-- Could NOT find cppcheck
-- SQLITECPP_RUN_DOXYGEN OFF
-- SQLITECPP_BUILD_EXAMPLES OFF
-- SQLITECPP_BUILD_TESTS OFF
-- Tins library found in /usr/local/lib/libtins.so
-- Python includes found in: /usr/include/python3.10
-- Python libs found in: /usr/lib/x86_64-linux-gnu/libpython3.10.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/eaton/ID2T/code_boost/src/build
Consolidate compiler generated dependencies of target cpputils
Consolidate compiler generated dependencies of target SQLiteCpp
Consolidate compiler generated dependencies of target sqlite3
Consolidate compiler generated dependencies of target botnetcomm
[ 11%] Built target sqlite3
[ 22%] Building CXX object CMakeFiles/botnetcomm.dir/cxx/botnet_comm_processor.cpp.o
[ 55%] Built target SQLiteCpp
[ 61%] Building CXX object CMakeFiles/cpputils.dir/cxx/utilities.cpp.o
Consolidate compiler generated dependencies of target pcapreader
[ 72%] Building CXX object CMakeFiles/pcapreader.dir/cxx/statistics.cpp.o
[ 72%] Building CXX object CMakeFiles/pcapreader.dir/cxx/pcap_processor.cpp.o
[ 77%] Building CXX object CMakeFiles/pcapreader.dir/cxx/statistics_db.cpp.o
[ 83%] Building CXX object CMakeFiles/pcapreader.dir/cxx/utilities.cpp.o
In file included from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:16,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/botnet_comm_processor.h:10,
                 from /home/eaton/ID2T/code_boost/src/cxx/botnet_comm_processor.cpp:1:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h: In function ‘pybind11::detail::internals& pybind11::detail::get_internals()’:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h:194:27: warning: ‘void PyEval_InitThreads()’ is deprecated [-Wdeprecated-declarations]
  194 |         PyEval_InitThreads();
      |         ~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/python3.10/Python.h:130,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/common.h:112,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pytypes.h:12,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/botnet_comm_processor.h:10,
                 from /home/eaton/ID2T/code_boost/src/cxx/botnet_comm_processor.cpp:1:
/usr/include/python3.10/ceval.h:122:37: note: declared here
  122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
      |                                     ^~~~~~~~~~~~~~~~~~
In file included from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:16,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/utilities.cpp:2:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h: In function ‘pybind11::detail::internals& pybind11::detail::get_internals()’:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h:194:27: warning: ‘void PyEval_InitThreads()’ is deprecated [-Wdeprecated-declarations]
  194 |         PyEval_InitThreads();
      |         ~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/python3.10/Python.h:130,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/common.h:112,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pytypes.h:12,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/utilities.cpp:2:
/usr/include/python3.10/ceval.h:122:37: note: declared here
  122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
      |                                     ^~~~~~~~~~~~~~~~~~
In file included from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:16,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/utilities.cpp:2:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h: In function ‘pybind11::detail::internals& pybind11::detail::get_internals()’:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h:194:27: warning: ‘void PyEval_InitThreads()’ is deprecated [-Wdeprecated-declarations]
  194 |         PyEval_InitThreads();
      |         ~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/python3.10/Python.h:130,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/common.h:112,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pytypes.h:12,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/utilities.cpp:2:
/usr/include/python3.10/ceval.h:122:37: note: declared here
  122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
      |                                     ^~~~~~~~~~~~~~~~~~
In file included from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:16,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/pcap_processor.h:12,
                 from /home/eaton/ID2T/code_boost/src/cxx/pcap_processor.cpp:1:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h: In function ‘pybind11::detail::internals& pybind11::detail::get_internals()’:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h:194:27: warning: ‘void PyEval_InitThreads()’ is deprecated [-Wdeprecated-declarations]
  194 |         PyEval_InitThreads();
      |         ~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/python3.10/Python.h:130,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/common.h:112,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pytypes.h:12,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/pcap_processor.h:12,
                 from /home/eaton/ID2T/code_boost/src/cxx/pcap_processor.cpp:1:
/usr/include/python3.10/ceval.h:122:37: note: declared here
  122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
      |                                     ^~~~~~~~~~~~~~~~~~
In file included from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:16,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/statistics_db.h:13,
                 from /home/eaton/ID2T/code_boost/src/cxx/statistics.cpp:8:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h: In function ‘pybind11::detail::internals& pybind11::detail::get_internals()’:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h:194:27: warning: ‘void PyEval_InitThreads()’ is deprecated [-Wdeprecated-declarations]
  194 |         PyEval_InitThreads();
      |         ~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/python3.10/Python.h:130,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/common.h:112,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pytypes.h:12,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/statistics_db.h:13,
                 from /home/eaton/ID2T/code_boost/src/cxx/statistics.cpp:8:
/usr/include/python3.10/ceval.h:122:37: note: declared here
  122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
      |                                     ^~~~~~~~~~~~~~~~~~
In file included from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:16,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/statistics_db.h:13,
                 from /home/eaton/ID2T/code_boost/src/cxx/statistics_db.cpp:1:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h: In function ‘pybind11::detail::internals& pybind11::detail::get_internals()’:
/home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/internals.h:194:27: warning: ‘void PyEval_InitThreads()’ is deprecated [-Wdeprecated-declarations]
  194 |         PyEval_InitThreads();
      |         ~~~~~~~~~~~~~~~~~~^~
In file included from /usr/include/python3.10/Python.h:130,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/detail/common.h:112,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pytypes.h:12,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/cast.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/attr.h:13,
                 from /home/eaton/ID2T/code_boost/src/pybind11/include/pybind11/pybind11.h:43,
                 from /home/eaton/ID2T/code_boost/src/cxx/statistics_db.h:13,
                 from /home/eaton/ID2T/code_boost/src/cxx/statistics_db.cpp:1:
/usr/include/python3.10/ceval.h:122:37: note: declared here
  122 | Py_DEPRECATED(3.9) PyAPI_FUNC(void) PyEval_InitThreads(void);
      |                                     ^~~~~~~~~~~~~~~~~~
[ 88%] Linking CXX shared library libcpputils.so
[ 88%] Built target cpputils
[ 94%] Linking CXX shared library libbotnetcomm.so
[ 94%] Built target botnetcomm
[100%] Linking CXX shared library libpcapreader.so
[100%] Built target pcapreader


All is set. ID2T is ready.

Run efficiency tests with the command './test_efficiency'
Run unit tests with the command './run_tests'
Run ID2T with the command './id2t'

@pepper-jk
Copy link
Collaborator

@LuccaRoofthooft there is no update on this from me. I'm not sure if @hashsiddique solved the problem themself or if they just lost interest. Without more information I was not able to help.

Your log shows some interesting warnings regarding pybind methods deprecated at or after python 3.9. Maybe this is the reason, you have issues. So I would suggest setting up a python 3.8 and 3.9 environment and see if you can reproduce both the warnings from the build script and your actual problem.

Unfortunately, I currently have little time to investigate issues for ID2T. But I still suggest opening a separate issue, as we can not be sure yours is related to this one. Especially as you both use different python versions. In your issue please elaborate on your actual issue. Are you also getting a Segmentation Fault (Core Dumped)? Could you provide a pcap file that works and one that does not? Also please link to this comment from the new issue. Thank you very much.

@LuccaRoofthooft LuccaRoofthooft mentioned this issue Apr 19, 2023
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pepper-jk @LuccaRoofthooft @hashsiddique