You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Currently FTA only counts uniqueness where the cardinality of the set is less than maxCardinality (12,000 by default). So assuming the number of distinct elements in the set is less than 12,000 which is typically highly likely then the uniqueness percentage (and hence the uniqueness count) should be perfect. Once the number of distinct elements exceeds the defined maximum then FTA simply reports -1 to indicate it does not know the answer.
The obvious solution to this problem is to use something like Hyperloglog (see https://en.wikipedia.org/wiki/HyperLogLog) - note: need to use a variant that supports merging. This would then allow us to implement an approx_count_distinct and hence generate this uniqueness metric even if the cardinality of the set is high. This would be extremely close but not perfect.
The text was updated successfully, but these errors were encountered:
Currently FTA only counts uniqueness where the cardinality of the set is less than maxCardinality (12,000 by default). So assuming the number of distinct elements in the set is less than 12,000 which is typically highly likely then the uniqueness percentage (and hence the uniqueness count) should be perfect. Once the number of distinct elements exceeds the defined maximum then FTA simply reports -1 to indicate it does not know the answer.
The obvious solution to this problem is to use something like Hyperloglog (see https://en.wikipedia.org/wiki/HyperLogLog) - note: need to use a variant that supports merging. This would then allow us to implement an approx_count_distinct and hence generate this uniqueness metric even if the cardinality of the set is high. This would be extremely close but not perfect.
The text was updated successfully, but these errors were encountered: