feat(ingest): migrate Cassandra source to new SDK

[hsheth2]

• Includes some enhancements to the SDK
• Refactors some minor bits in the cassandra data models + removes some unnecessary code
• Moves to using Dataset and Container for everything in the cassandra source except profiling
• Functionality of the source remains largely unchanged

Checklist

• The PR conforms to DataHub's Contributing Guideline (particularly Commit Message Format)
• Links to related issues (if applicable)
• Tests for the changes have been added/updated (if applicable)
• Docs related to the changes have been added/updated (if applicable). If a new feature has been added a Usage Guide has been added for the same.
• For any breaking change/potential downtime/deprecation/big changes an entry has been made in Updating DataHub

[codecov]

Codecov Report

Attention: Patch coverage is 85.86957% with 13 lines in your changes missing coverage. Please review.

✅ All tests successful. No failed tests found.

Files with missing lines	Patch %	Lines
...rc/datahub/ingestion/source/cassandra/cassandra.py	84.74%	9 Missing ⚠️
metadata-ingestion/src/datahub/sdk/__init__.py	66.66%	4 Missing ⚠️

❌ Your patch status has failed because the patch coverage (85.86%) is below the target coverage (90.00%). You can increase the patch coverage or adjust the target coverage.

Files with missing lines	Coverage Δ
...ta-ingestion/src/datahub/ingestion/run/pipeline.py	85.20% <100.00%> (+0.20%)	⬆️
...atahub/ingestion/source/cassandra/cassandra_api.py	67.30% <100.00%> (+0.64%)	⬆️
metadata-ingestion/src/datahub/sdk/_entity.py	95.00% <100.00%> (+0.66%)	⬆️
metadata-ingestion/src/datahub/sdk/container.py	97.08% <100.00%> (+1.00%)	⬆️
metadata-ingestion/src/datahub/sdk/dataset.py	83.54% <100.00%> (+1.34%)	⬆️
metadata-ingestion/src/datahub/sdk/__init__.py	77.77% <66.66%> (-22.23%)	⬇️
...rc/datahub/ingestion/source/cassandra/cassandra.py	87.65% <84.74%> (+0.98%)	⬆️

... and 8 files with indirect coverage changes

---

Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 407b188...f238286. Read the comment docs.

[sgomezvillamor]

LGTM

[sgomezvillamor]

In which cases should we allow/restrict the usage of these private modules?

[hsheth2]

I'm probably going to make Entity a public interface in the future - will do that in a follow-up

In general, we should avoid usages of private modules

[sgomezvillamor]

given we are calling _set_extra_aspects for both container and dataset, should we move that responsibility to the parent: super().__init__(urn, extra_aspects) ?

[hsheth2]

tbh I'm not super happy with the interface setup right now

Specifically, I'm not sure if set_extra_aspects should be called immediately after the super.init, or at the end of the Dataset/Container init method

Once I have a stronger opinion on it, I'll refactor this stuff