Getting NullPointerException when running creatIndex

[TakeshiKovacs]

Hi Team

I'm trying to create and index using guidance from the example notebook.
I'm creating a data frame by doing a spark.read.format("delta").load("abfss://somepathinAzure/")

I have an index config like this:

mat_flat_IndexConfig = IndexConfig("mat_flatIndex1", ["someColumn"], ["someOtherColumn"])

hs.createIndex(df, mat_flat_IndexConfig)

The error I get is
Py4JJavaError: An error occurred while calling o577.createIndex.
: java.lang.NullPointerException

I'm not sure what the issue is, possibly where it's trying to store the index?
I did also try setting index llocations to dbfs to try and rule out storage permissions to somethink like this
spark.conf.set("spark.hyperspace.system.path", "/dbfs/FileStore/mat")

But still no dice.

I'm using the library for scala 2.12
com.microsoft.hyperspace:hyperspace-core_2.12:0.4.0

And I'm using databricks runtime 8.3 which has scala 2.12 in it.

Does anyone have any ideas?

[sezruby]

@TakeshiKovacs Hyperspace 0.4.0 does not support Spark3.
The current master branch supports Spark3 and it will be included 0.5.0.
Thanks!