try assess on star incase it crashes.

[fkgruber]

No description provided.

[rimorob]

It definitely crashes. There are no safeguards for computing metrics that require there to be ground truth positives available. The true dag for the star cases has no examples of true edges that don't connect to the latent space

Separate question: how can we evaluate edges to the latent space? We'd have to match the inferred nodes to the latent nodes in some way.

[fkgruber]

the problem is if the true vs estimated latent dimensionality is different.
Only if we treat any edge between any latent and an observable as 1 edge in both true or estimated network

[fkgruber]

It definitely crashes. There are no safeguards for computing metrics that require there to be ground truth positives available. The true dag for the star cases has no examples of true edges that don't connect to the latent space

Separate question: how can we evaluate edges to the latent space? We'd have to match the inferred nodes to the latent nodes in some way.

It definitely crashes. There are no safeguards for computing metrics that require there to be ground truth positives available. The true dag for the star cases has no examples of true edges that don't connect to the latent space

not sure how you want to define the performance when there are no positive cases. We could do something different in those cases.

[rimorob]

I think the solution is to only report metrics that don't crash, e.g. lift (for FNR, for example). This can be easily done by wrapping each metric inside try/catch

[fkgruber]

also skip if there is no true network provided.
if there is true latent it should plot that only.
maybe some plot mse vs iteration
correplation plot vs iteration

[rimorob]

See separate issue that I've just closed - assess() will plot corrplot if true and inferred latent spaces are available, and will exit after that if the true network is not available.

[fkgruber]

I think the solution is to only report metrics that don't crash, e.g. lift (for FNR, for example). This can be easily done by wrapping each metric inside try/catch

lift is defined (in performance functions) as P(Yhat = + | Y = +)/P(Yhat = +). but the conditional is only defined if P(Y=+) >0. So in the star case it will be undefined. Same with TPR so AUC or prAUC can't be estimated either.

But I can show the other stuff instead.