[HWORKS-1565][APPEND] Make sure the spark program exits after completion #435

Job	Run time
Lint and Stylecheck	13s
Unit Tests (3.8)	2m 23s
Unit Tests (Pandas 1.x)	1m 57s
Unit Tests (Local TZ)	2m 22s
Unit Tests (3.9)	2m 3s
Unit Tests (No Optional Dependencies)	1m 56s
Unit Tests (3.10)	2m 37s
Unit Tests (Typechecked)	2m 12s
Unit Tests (Windows)	8m 6s
Unit Tests (Windows) (Local TZ)	7m 6s
	30m 55s

Provide feedback