KeyError: 'DW_AT_byte_size' error on Ubuntu-Linux 4.15.0_48-generic system using Python 2.7.12

[dmwharris]

Hello, after creating a volatility profile for an Ubuntu-Linux 4.15.0_48-generic system using version 2.6 and running it against a LiME sample created with

insmod lime-4.15.0-48-generic.ko "path=/home/developer/lime-4.15.0-48-generic.lime format=lime"

I get the following error:

root@nuc2:/home/developer/Downloads/volatility-master# python vol.py --profile=LinuxUbuntu4_15_0-48-genericx64 -f /home/developer/lime-4.15.0-48-generic.lime modules
Volatility Foundation Volatility Framework 2.6
Traceback (most recent call last):
File "vol.py", line 192, in
main()
File "vol.py", line 183, in main
command.execute()
File "/home/developer/Downloads/volatility-master/volatility/commands.py", line 116, in execute
if not self.is_valid_profile(profsself._config.PROFILE):
File "/home/developer/Downloads/volatility-master/volatility/plugins/overlays/linux/linux.py", line 216, in init
obj.Profile.init(self, *args, **kwargs)
File "/home/developer/Downloads/volatility-master/volatility/obj.py", line 862, in init
self.reset()
File "/home/developer/Downloads/volatility-master/volatility/plugins/overlays/linux/linux.py", line 227, in reset
self.load_vtypes()
File "/home/developer/Downloads/volatility-master/volatility/plugins/overlays/linux/linux.py", line 264, in load_vtypes
vtypesvar = dwarf.DWARFParser(dwarfdata).finalize()
File "/home/developer/Downloads/volatility-master/volatility/dwarf.py", line 71, in init
self.feed_line(line)
File "/home/developer/Downloads/volatility-master/volatility/dwarf.py", line 162, in feed_line
self.process_statement(**parsed) #pylint: disable-msg=W0142
File "/home/developer/Downloads/volatility-master/volatility/dwarf.py", line 204, in process_statement
self.vtypes[name] = [ int(data['DW_AT_byte_size'], self.base), {} ]
KeyError: 'DW_AT_byte_size'

Python version is 2.7.12

I was using the dwarfdump associated with Ubuntu 16.01 xenial, which as 20120410-2+deb7u2build0.16.04.1.

Is this because version 2.6 doesn't support Linux 4.15.0-48-generic? Or should I use a later version of dwarfdump?

[pagabuc]

Hey @dmwharris, can you please share the profile? And yes, I suggest to try with another version of dwarfdump!

[shehreyarahmedkohati]

Hi also getting some what same error,
I am using Volatility 2.6 to analyze a memory image from a Ubuntu 16.04 server captured using and in Lime format. I am getting the below error persistently. I have made a several profiles for the OS using the System map details.
Error:
sudo python vol.py -f "ram.lime" --profile=LinuxUbuntu1604-36x64 linux_pslist
Volatility Foundation Volatility Framework 2.6
*** Failed to import volatility.plugins.registry.shutdown (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.getservicesids (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.timeliner (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.malware.apihooks (NameError: name 'distorm3' is not defined)
*** Failed to import volatility.plugins.malware.servicediff (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.userassist (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.getsids (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.shellbags (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.evtlogs (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.tcaudit (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.shimcache (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.dumpregistry (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.lsadump (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.malware.threads (NameError: name 'distorm3' is not defined)
*** Failed to import volatility.plugins.mac.apihooks_kernel (ImportError: No module named distorm3)
*** Failed to import volatility.plugins.registry.amcache (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.mac.check_syscall_shadow (ImportError: No module named distorm3)
*** Failed to import volatility.plugins.malware.svcscan (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.auditpol (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.ssdt (NameError: name 'distorm3' is not defined)
*** Failed to import volatility.plugins.registry.registryapi (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.mac.apihooks (ImportError: No module named distorm3)
*** Failed to import volatility.plugins.envars (ImportError: No module named Crypto.Hash)
Traceback (most recent call last):
File "vol.py", line 192, in
main()
File "vol.py", line 183, in main
command.execute()
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/plugins/linux/common.py", line 64, in execute
commands.Command.execute(self, *args, **kwargs)
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/commands.py", line 116, in execute
if not self.is_valid_profile(profsself._config.PROFILE):
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/plugins/overlays/linux/linux.py", line 216, in init
obj.Profile.init(self, *args, **kwargs)
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/obj.py", line 862, in init
self.reset()
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/plugins/overlays/linux/linux.py", line 227, in reset
self.load_vtypes()
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/plugins/overlays/linux/linux.py", line 264, in load_vtypes
vtypesvar = dwarf.DWARFParser(dwarfdata).finalize()
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/dwarf.py", line 71, in init
self.feed_line(line)
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/dwarf.py", line 162, in feed_line
self.process_statement(**parsed) #pylint: disable-msg=W0142
File "/home/digit/Downloads/lmg-master/LiME-1.9/src/volatility-master/volatility/dwarf.py", line 204, in process_statement
self.vtypes[name] = [ int(data['DW_AT_byte_size'], self.base), {} ]
KeyError: 'DW_AT_byte_size'

Any help would be appreciated!

[atcuno]

Can you please confirm that you are using the latest master checkout of Volatility from here (github)? Line 204 does not match your backtrace: https://github.com/volatilityfoundation/volatility/blob/master/volatility/dwarf.py#L204

[atcuno]

@shehreyarahmedkohati (and others), please git pull and try to run Volatility with your profiles again. I made an update to dwarf.py to hopefully address everyone at once:

7b3f52b

[atcuno]

Closed this by accident.

[atcuno]

@olifre please git pull again and re-try

[olifre]

@atcuno That's interesting! Now it runs for ~5 minutes and uses some GB of memory, so it seems the dwarf parsing went fine.
Then, I end up at a (Pdb) prompt. Typing w, I get:

Volatility Foundation Volatility Framework 2.6.1
> /somewhere/volatility/plugins/overlays/linux/linux.py(262)_merge_anonymous_members()
-> raise exceptions.VolatilityException("Inconsistent linux profile - unable to look up " + str(e))
(Pdb) w
  /somewhere/vol.py(192)<module>()
-> main()
  /somewhere/vol.py(183)main()
-> command.execute()
  /somewhere/volatility/plugins/linux/common.py(67)execute()
-> commands.Command.execute(self, *args, **kwargs)
  /somewhere/volatility/commands.py(116)execute()
-> if not self.is_valid_profile(profs[self._config.PROFILE]()):
  /somewhere/volatility/plugins/overlays/linux/linux.py(218)__init__()
-> obj.Profile.__init__(self, *args, **kwargs)
  /somewhere/volatility/obj.py(862)__init__()
-> self.reset()
  /somewhere/volatility/plugins/overlays/linux/linux.py(232)reset()
-> self.load_vtypes()
  /somewhere/volatility/plugins/overlays/linux/linux.py(270)load_vtypes()
-> self._merge_anonymous_members(vtypesvar)
> /somewhere/volatility/plugins/overlays/linux/linux.py(262)_merge_anonymous_members()
-> raise exceptions.VolatilityException("Inconsistent linux profile - unable to look up " + str(e))

[olifre]

@atcuno Do I interpret correctly that this (likely) means this issue is solved and we now have volatilityfoundation/volatility3#222 reproduced with Volatility 2?

[atcuno]

It is not solved. I am working on it, but getting those new types to parse is being difficult.

[olifre]

Thanks! No worries, I was just interpreting this as being back to the original issue now (for which we are still unsure whether the dump has some issues), but since that's not the case, for sure take your time implementing those new types.

[lain3d]

is there a minimum commit id where this isn't a problem

[lain3d]

any news on this?

[canDry]

I'm assuming this is the same issue...

python vol.py --info
Volatility Foundation Volatility Framework 2.6.1
*** Failed to import volatility.plugins.registry.shutdown (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.getservicesids (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.timeliner (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.malware.apihooks (NameError: name 'distorm3' is not defined)
*** Failed to import volatility.plugins.malware.servicediff (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.userassist (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.getsids (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.shellbags (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.evtlogs (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.tcaudit (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.dumpregistry (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.registry.lsadump (ImportError: No module named Crypto.Hash)
*** Failed to import volatility.plugins.malware.threads (NameError: name 'distorm3' is not defined)
*** Failed to import volatility.plugins.mac.apihooks_kernel (ImportError: No module named distorm3)

Ubuntu 18LTS... Python 2.7.17

Can someone plz confirm whether this is the same (or different) issue?

---

Update: hadn't installed them ie:

apt install python-crypto python-distorm3

Works now.

[vincentroberge]

On a Ubuntu 22.04 image, I get the following error. It seems to be related to this thread. Any help would be appreciated. I use the latest version of Volatility 2.6.1 just pulled from github yesterday. Thank you.

$ vol.py -f ubuntu.vmem --profile=Linuxubuntu-22.04-desktop-amd64_5.15.0-33-genericx64 linux_pslist
Volatility Foundation Volatility Framework 2.6.1
Traceback (most recent call last):
  File "/usr/local/bin/vol.py", line 192, in <module>
    main()
  File "/usr/local/bin/vol.py", line 183, in main
    command.execute()
  File "/usr/local/lib/python2.7/dist-packages/volatility/plugins/linux/common.py", line 67, in execute
    commands.Command.execute(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/volatility/commands.py", line 116, in execute
    if not self.is_valid_profile(profs[self._config.PROFILE]()):
  File "/usr/local/lib/python2.7/dist-packages/volatility/plugins/overlays/linux/linux.py", line 218, in __init__
    obj.Profile.__init__(self, *args, **kwargs)
  File "/usr/local/lib/python2.7/dist-packages/volatility/obj.py", line 862, in __init__
    self.reset()
  File "/usr/local/lib/python2.7/dist-packages/volatility/plugins/overlays/linux/linux.py", line 232, in reset
    self.load_vtypes()
  File "/usr/local/lib/python2.7/dist-packages/volatility/plugins/overlays/linux/linux.py", line 269, in load_vtypes
    vtypesvar = dwarf.DWARFParser(dwarfdata).finalize()
  File "/usr/local/lib/python2.7/dist-packages/volatility/dwarf.py", line 72, in __init__
    self.feed_line(line)
  File "/usr/local/lib/python2.7/dist-packages/volatility/dwarf.py", line 161, in feed_line
    self.process_variable(parsed['data'])
  File "/usr/local/lib/python2.7/dist-packages/volatility/dwarf.py", line 330, in process_variable
    data['DW_AT_decl_file'].split()[1], data['DW_AT_type']))
IndexError: list index out of range

[Gathub22]

Just want to leave here there is a pending PR with a patch that adds DWARFv5 and fixes the same KeyError. The patch works for me. Maybe this could help to you :)