Aggregated data into index #506
-
As i see hyperspace supports index creation only if query scans hdfs file. Is there plan to support for below kind of query. val df = spark.sql("select dim1,dim2,sum(measure1) as measure1 from table group by dim1,dim2") hs.createIndex(df, IndexConfig("dim1dim2", indexedColumns = Seq("dim1","dim2"), includedColumns = Seq("measure1"))) |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment
-
There is an issue related to the feature: #186 Since the spark plan of the query can be different depending on spark versions, it's somewhat difficult to support the feature. Though it's not a simple item/implementation and not planned yet. |
Beta Was this translation helpful? Give feedback.
There is an issue related to the feature: #186
Since the spark plan of the query can be different depending on spark versions, it's somewhat difficult to support the feature.
Maybe we could try to implement it with a SQL string as a source, like
hs.createIndex(IndexConfig("name", "sql"))
Though it's not a simple item/implementation and not planned yet.