[HWORKS-1565] Make sure the spark program exits after completion #239

Job	Run time
Lint and Stylecheck	9s
Unit Tests (No Optional Dependencies)	2m 10s
Unit Tests (Pandas 1.x)	4m 24s
Unit Tests (3.8)	2m 37s
Unit Tests (Local TZ)	2m 44s
Unit Tests (3.9)	2m 36s
Unit Tests (Typechecked)	2m 30s
Unit Tests (3.10)	2m 50s
Unit Tests (Windows)	9m 41s
Unit Tests (Windows) (Local TZ)	9m 1s
	38m 42s

Provide feedback