Evaluating embedding based on how good the embedding explains a continuous/binary column

[hoangthienan95]

First off thank you so much for the great package and manuscript. It's awesome to see such important work done in dimensionality reduction/single cell unifying and comparing the DR methods.

My question is about evaluating embedding. In the documentation, you showed how to compare the embeddings based on PCA loss and geodesic spearman R. If I cluster the data and know the cluster labels I can also use metrics like Adjusted Rand Index and Adjusted Mutual Information to evaluate the clustering (and the embedding indirectly).

However, if I only have a binary or continuous column (like expression of a gene) and wants to see how good the embedding "explains" my column (the relationship can be highly non-linear), and evaluate the different embedding methods/parameters based on that, what should I do? I can train an XGBoost model to predict my target and get R-squared/AUC using only the embedding, but then I'll also have the model's hyperparameters to tune. Do you have any suggestions for this problem?