Clean up the table API

[gnosek]

What type of PR is this?

Uncomment one (or more) /kind <> lines:

/kind bug

/kind cleanup

/kind design

/kind documentation

/kind failing-test

/kind feature

Any specific area of the project related to this PR?

Uncomment one (or more) /area <> lines:

/area API-version

/area build

/area CI

/area driver-kmod

/area driver-bpf

/area driver-modern-bpf

/area libscap-engine-bpf

/area libscap-engine-gvisor

/area libscap-engine-kmod

/area libscap-engine-modern-bpf

/area libscap-engine-nodriver

/area libscap-engine-noop

/area libscap-engine-source-plugin

/area libscap-engine-savefile

/area libscap

/area libpman

/area libsinsp

/area tests

/area proposals

Does this PR require a change in the driver versions?

/version driver-API-version-major

/version driver-API-version-minor

/version driver-API-version-patch

/version driver-SCHEMA-version-major

/version driver-SCHEMA-version-minor

/version driver-SCHEMA-version-patch

What this PR does / why we need it:

Today we have two APIs to access tables from sinsp (core or plugin) code:

• the plugin API (C-style, implemented by plugin tables directly and exposed back to them with a fairly thin wrapper)
• the C++ API, implemented directly by the built in tables and retrofitted to plugin-provided tables

Unfortunately, adapting plugin tables to the C++ API is awkward and complex, which makes extending either API harder than it could be.

The goal of this PR is to separate the C++ API as implemented by the built in tables from the C++ API as used by table consumers, so that the consumer API is based only on the C-style plugin API.

Which issue(s) this PR fixes:

Fixes #

Special notes for your reviewer:

There's quite a bit of churn here, so it's probably best to review commit by commit. Some of the later commits are semi-related cleanups (like the typeinfo changes) that let us remove some code duplication. I understand we wanted to keep state support sort of separate from plugin support, but in the end the two are intertwined enough that any real separation between them is fictional.

In the grand tradition of libs, the C++ api is left undocumented ;) We might want to pull it out to a more separate place too (the C++ SDK would be perfect IMHO, except it would introduce a circular dependency with sinsp)

Does this PR introduce a user-facing change?:

new: new API to access tables from C++ code

[github-actions]

Perf diff from master - unit tests

     9.30%     +0.89%  [.] sinsp::next
     1.42%     -0.82%  [.] sinsp_evt::get_direction
     7.38%     +0.71%  [.] sinsp_evt::get_type
     3.46%     -0.62%  [.] sinsp_thread_manager::get_thread_ref
     1.80%     -0.42%  [.] sinsp_evt_filter::sinsp_evt_filter
     2.61%     +0.35%  [.] sinsp_parser::process_event
     1.53%     -0.33%  [.] sinsp_evt::get_ts
     0.34%     +0.32%  [.] libsinsp::state::stl_container_table_adapter<std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >, libsinsp::state::value_table_entry_adapter<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, libsinsp::state::value_table_entry_adapter<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >::dynamic_fields_t>::stl_container_table_adapter
     0.68%     +0.27%  [.] sinsp_parser::parse_context_switch
     0.79%     +0.27%  [.] std::_Hashtable<long, std::pair<long const, std::shared_ptr<sinsp_threadinfo> >, std::allocator<std::pair<long const, std::shared_ptr<sinsp_threadinfo> > >, std::__detail::_Select1st, std::equal_to<long>, std::hash<long>, std::__detail::_Mod_range_hashing, std::__detail::_Default_ranged_hash, std::__detail::_Prime_rehash_policy, std::__detail::_Hashtable_traits<false, false, true> >::find

Heap diff from master - unit tests

peak heap memory consumption: -3.20M
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Heap diff from master - scap file

peak heap memory consumption: -80.92K
peak RSS (including heaptrack overhead): 0B
total memory leaked: 0B

Benchmarks diff from master

Comparing gbench_data.json to /root/actions-runner/_work/libs/libs/build/gbench_data.json
Benchmark                                                         Time             CPU      Time Old      Time New       CPU Old       CPU New
----------------------------------------------------------------------------------------------------------------------------------------------
BM_sinsp_split_mean                                            +0.0332         +0.0330           142           146           142           146
BM_sinsp_split_median                                          +0.0352         +0.0349           141           146           141           146
BM_sinsp_split_stddev                                          -0.3731         -0.3707             2             1             2             1
BM_sinsp_split_cv                                              -0.3932         -0.3907             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_mean                  +0.0912         +0.0910            57            62            57            62
BM_sinsp_concatenate_paths_relative_path_median                +0.0910         +0.0907            56            62            56            62
BM_sinsp_concatenate_paths_relative_path_stddev                -0.2864         -0.2984             0             0             0             0
BM_sinsp_concatenate_paths_relative_path_cv                    -0.3460         -0.3569             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_mean                     +0.0042         +0.0040            24            24            24            24
BM_sinsp_concatenate_paths_empty_path_median                   +0.0041         +0.0038            24            24            24            24
BM_sinsp_concatenate_paths_empty_path_stddev                   +0.4147         +0.4242             0             0             0             0
BM_sinsp_concatenate_paths_empty_path_cv                       +0.4088         +0.4184             0             0             0             0
BM_sinsp_concatenate_paths_absolute_path_mean                  +0.1659         +0.1657            56            65            56            65
BM_sinsp_concatenate_paths_absolute_path_median                +0.1618         +0.1615            56            65            56            65
BM_sinsp_concatenate_paths_absolute_path_stddev                +0.1025         +0.0990             1             1             1             1
BM_sinsp_concatenate_paths_absolute_path_cv                    -0.0544         -0.0572             0             0             0             0
BM_sinsp_split_container_image_mean                            +0.0026         +0.0024           388           389           388           389
BM_sinsp_split_container_image_median                          +0.0027         +0.0025           388           389           388           389
BM_sinsp_split_container_image_stddev                          +0.0897         +0.0876             2             3             2             3
BM_sinsp_split_container_image_cv                              +0.0869         +0.0850             0             0             0             0

[codecov]

Codecov Report

Attention: Patch coverage is 88.25911% with 58 lines in your changes missing coverage. Please review.

Project coverage is 75.37%. Comparing base (07a16e9) to head (7989c78).
Report is 6 commits behind head on master.

Files with missing lines	Patch %	Lines
userspace/libsinsp/state/table.cpp	88.10%	22 Missing ⚠️
userspace/libsinsp/plugin_table_api.cpp	78.57%	15 Missing ⚠️
userspace/libsinsp/plugin.h	93.40%	6 Missing ⚠️
userspace/libsinsp/state/table_adapters.h	40.00%	6 Missing ⚠️
userspace/libsinsp/state/type_info.h	73.68%	5 Missing ⚠️
userspace/libsinsp/state/table.h	92.15%	4 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #2193      +/-   ##
==========================================
+ Coverage   75.19%   75.37%   +0.18%     
==========================================
  Files         261      262       +1     
  Lines       33878    33774     -104     
  Branches     5801     5752      -49     
==========================================
- Hits        25475    25458      -17     
+ Misses       8403     8316      -87     

Flag	Coverage Δ
libsinsp	75.37% <88.25%> (+0.18%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[FedeDP]

/approve

I went commit by commit as suggested, everything makes sense and LGTM!
Great job @gnosek !

[poiana]

LGTM label has been added.

Git tree hash: 3498cc14f4ef2e91e04cb145cfed7caa7e5da69f

[FedeDP]

/milestone 0.20.0

[poiana]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: FedeDP, gnosek

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [FedeDP,gnosek]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment