Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tweaks, basic DWARF-based info #69

Merged
merged 6 commits into from
Feb 14, 2024
Merged

tweaks, basic DWARF-based info #69

merged 6 commits into from
Feb 14, 2024

Conversation

monoidic
Copy link
Contributor

@monoidic monoidic commented Sep 28, 2023

This PR sets up the groundwork for getting DWARF-based info from ELF files, and adds fetching the built-in goroot and build version using it as examples, which currently otherwise require hacks like disassembly of code. The addresses and values of various other values (e.g runtime.firstmoduledata for the moduledata, or various type definitions) could be located as well.
This, of course, has the drawback of only working on ELF and requiring DWARF to not be stripped, though it should be portable across all Go platforms which use ELF, both by OS and CPU architecture. And DWARF is, by default, included.

This PR also includes some housekeeping and tweaks:

  • bumping dependencies
  • simplifying typeOffset()
  • renaming getSectionDataFromOffset to getSectionDataFromAddress for clarity (the "offset" passed as an argument is a memory address, not a file offset)
  • replace uses of the deprecated io/ioutil module with equivalent code
  • replace some f.Seek(x, 0) calls with f.Seek(x, io.SeekStart) to remove the magic value
  • run go generate to update gopkg_gen.go and stdpkg_gen.go
  • fix some typoes
  • do not silently ignore some error values
  • update tests to handle go.12.0 correctly and add said version to testdata/build.go (todo: PR to /goretk/testdata)

@monoidic
Copy link
Contributor Author

And autogenerating the code for the moduledata{version}{machine_word_size} structs + converting them to moduledata without using any (runtime) reflection.
I also took the liberty of trimming the definitions down to the last actually used field. Go 1.21 changed the definition of the moduledata struct, but until inittasks itself (which could be interesting) or any of the fields succeeding it are used, it won't make any difference here.

@TcM1911
Copy link
Member

TcM1911 commented Oct 3, 2023

Thank you for contributing. I will make time to review this by the end of this week.

elf.go Outdated Show resolved Hide resolved
elf.go Outdated Show resolved Hide resolved
elf.go Outdated Show resolved Hide resolved
elf.go Outdated Show resolved Hide resolved
file_test.go Outdated Show resolved Hide resolved
gen_moduledata.go Outdated Show resolved Hide resolved
gen_moduledata.go Outdated Show resolved Hide resolved
goroot.go Outdated Show resolved Hide resolved
goversion.go Outdated Show resolved Hide resolved
slow_test.go Outdated Show resolved Hide resolved
@Zxilly
Copy link
Contributor

Zxilly commented Jan 22, 2024

Any further work on this patch? I can help to merge this.

@TcM1911
Copy link
Member

TcM1911 commented Jan 29, 2024

The conflicts must be fixed. It would probably be best also to break out the moduledata and the DWARF extraction into separate PRs.

@monoidic
Copy link
Contributor Author

monoidic commented Feb 9, 2024

Oh, I'd forgotten about this PR.

The conflicts must be fixed. It would probably be best also to break out the moduledata and the DWARF extraction into separate PRs.

Very well, I will wait for #77 to be accepted and then rework this PR.

@TcM1911
Copy link
Member

TcM1911 commented Feb 9, 2024

#77 has been merged.

@monoidic monoidic force-pushed the develop branch 2 times, most recently from 6e4f070 to 62fb40e Compare February 9, 2024 22:36
@monoidic monoidic changed the title version updates, tweaks, basic DWARF-based info tweaks, basic DWARF-based info Feb 9, 2024
@codecov-commenter
Copy link

codecov-commenter commented Feb 9, 2024

Codecov Report

Attention: 37 lines in your changes are missing coverage. Please review.

Comparison is base (6d39e92) 78.87% compared to head (76855ef) 78.57%.
Report is 1 commits behind head on develop.

Files Patch % Lines
dwarf.go 74.28% 19 Missing and 8 partials ⚠️
goversion.go 20.00% 3 Missing and 1 partial ⚠️
goroot.go 25.00% 2 Missing and 1 partial ⚠️
type.go 85.00% 3 Missing ⚠️

❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop      #69      +/-   ##
===========================================
- Coverage    78.87%   78.57%   -0.30%     
===========================================
  Files           15       16       +1     
  Lines         3474     3589     +115     
===========================================
+ Hits          2740     2820      +80     
- Misses         518      543      +25     
- Partials       216      226      +10     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@monoidic monoidic requested a review from TcM1911 February 9, 2024 23:08
testdata/gold Outdated Show resolved Hide resolved
moduledata.go Outdated Show resolved Hide resolved
@Zxilly
Copy link
Contributor

Zxilly commented Feb 10, 2024

Maybe we can add some sample with debug info to gold.

@Zxilly
Copy link
Contributor

Zxilly commented Feb 10, 2024

I noticed macho.File also has a https://pkg.go.dev/debug/macho#File.DWARF method, can this logic also applied to macho?

@Zxilly
Copy link
Contributor

Zxilly commented Feb 10, 2024

Even debug/pe also has a https://pkg.go.dev/debug/pe#File.DWARF. And all of them use the debug/dwarf.
This made me curious, as I always thought pe files used a separate PDB debug info file

@monoidic
Copy link
Contributor Author

Huh, you're right, @Zxilly . Somewhat surprised to see that PE is on that list.

I gave the fileHandler interface a getDwarf method to get the DWARF data and just made the DWARF data extraction code use fileHandler, so it works across all binary formats.

Copy link
Contributor

@Zxilly Zxilly left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@Zxilly
Copy link
Contributor

Zxilly commented Feb 11, 2024

hope this PR get merged soon as it may conflicted with #86, blocking the further development

- rename `getSectionDataFromOffset` to `getSectionDataFromAddress` for clarity
- use `io.SeekStart` instead of magic value 0
- fix some typoes
@monoidic
Copy link
Contributor Author

Rebased to fix conflicts.

Copy link
Member

@TcM1911 TcM1911 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a lot of places where it results in a panic. Since this is a library, I think we should try to not bring down the main application if we can. Please switch to errors instead.

It could be good to add some tests too. Either by using "golden" binaries or building some on the fly like it's done in https://github.com/goretk/gore/blob/develop/slow_test.go.

@monoidic
Copy link
Contributor Author

Those panics were definitely not intended to be there, whoops.
I thought I'd have to have a chicken-or-egg issue with pushing unstripped test binaries to the testdata. But slow_test works.
I have to use a different source file if I want getGoRootFromDwarf to succeed as runtime.defaultGOROOT is not included unless needed.

Also, the tmpDirs slice in TestMain in slow_test was not previously collecting all temporary directories and a bunch of build files were getting left behind. I fixed that.

Copy link
Member

@TcM1911 TcM1911 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor fix needed and this is ready to be merged.

dwarf.go Show resolved Hide resolved
@Zxilly
Copy link
Contributor

Zxilly commented Feb 13, 2024

The CI error was caused by go 1.22, which moduledata currently not exist in the main branch. After #88 merged it gets resolved.

@TcM1911
Copy link
Member

TcM1911 commented Feb 13, 2024

#88 has been merged. Please try again.

@Zxilly
Copy link
Contributor

Zxilly commented Feb 13, 2024

Seems works fine now.

@TcM1911 TcM1911 merged commit a8ca096 into goretk:develop Feb 14, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants