[BUGFIX] Resolve partition deleting bug in insert_overwrite mode

[roslovets]

Bug

Currently boto3 is able to accidentally skip deleting of some partitions during incremental build in insert_overwrite mode.

Why it's important

Adapter can skip deleting of some partitions before inserting from rows temporary table. New data will appended for undeleted partitions but not overwrited.

This bug prevents me from using this adapter in my pipelines.

What wrong with code

Logic of receiving partition details is poor. get_partitions() function should take into account NextToken parameter (a continuation token, if this is not the first call to retrieve these partitions).

How I fixed this bug

I added loop to be sure that boto3 will find and delete target partition. I got inspiration from awswrangler library, example of get_partitions usage.

[roslovets]

I tested this patch, it works.

[aut0clave]

Duplicate of #82

[nicor88]

@roslovets this issue was fixed in the community fork dbt-labs/dbt-athena#2 and released yesterday, available in pypi dbt-athena-community==1.0.3, here

Note that the solution is a bit leaner, as it uses glue pagination with build_full_result() method to avoid not necessary looping (it loops under the odd).

[roslovets]

@nicor88 That's what I need, thanks a lot!