Dangling Parquet files in delta table

[jorritsandbrink]

dlt version

0.5.4a0

Describe the problem

Dangling Parquet files in Delta tables:

The top 3 Parquet files are not managed by the Delta table. The Delta log does not reference them. They are completely ignored. They should not be there.

Issue exists since #1494.

Expected behavior

No response

Steps to reproduce

Run test_delta_table_core test (deactive drop_active_pipeline_data() to inspect Delta table folder).

Operating system

Windows

Runtime environment

Local

Python version

3.8

dlt data source

No response

dlt destination

No response

Other deployment details

No response

Additional information

No response