Use str for SparklyTest options (#52)

* `spark.sql.shuffle.partitions` in `SparklyTest` should be set to string, because `int` value breaks integration testing in Spark 2.0.2.
tubular · Aug 3, 2017 · caa398c · caa398c
1 parent 3d2bdfa
commit caa398c
Show file tree

Hide file tree

Showing 3 changed files with 6 additions and 2 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -1,3 +1,7 @@
+# 2.2.1
+* `spark.sql.shuffle.partitions` in `SparklyTest` should be set to string,
+because `int` value breaks integration testing in Spark 2.0.2. 
+
 # 2.2.0
 * Add instant iterative development mode. `sparkly-testing --help` for more details.
 * Use in-memory db for Hive Metastore in `SparklyTest` (faster tests).

diff --git a/sparkly/__init__.py b/sparkly/__init__.py
@@ -19,4 +19,4 @@
 assert SparklySession
 
 
-__version__ = '2.2.0'
+__version__ = '2.2.1'
diff --git a/sparkly/testing.py b/sparkly/testing.py
@@ -91,7 +91,7 @@ def setup_session(cls):
             'spark.sql.warehouse.dir': tempfile.mkdtemp(suffix='sparkly'),
 
             # Reduce number of shuffle partitions (faster tests).
-            'spark.sql.shuffle.partitions': 4,
+            'spark.sql.shuffle.partitions': '4',
         })
 
     @classmethod