Can't reproduce MCSL main results

[MateuszOlko]

Hi, I tried reproducing results from the MCSL paper https://arxiv.org/pdf/1910.08527.pdf unsuccessfully.

I tried the code below. I aimed to use the parameters described in the paper and in the appendix.
I also tried variants with minor tweaks to this one.

from castle.algorithms import MCSL
from castle.common import GraphDAG
from castle.datasets import DAG, IIDSimulation
from castle.metrics import MetricsDAG

weighted_random_dag = DAG.erdos_renyi(n_nodes=10, n_edges=10, weight_range=(0.5, 2.0), seed=123)
dataset = IIDSimulation(W=weighted_random_dag, n=3000, method='nonlinear', sem_type='gp')
true_dag, X = dataset.B, dataset.X

# mcsl learn
mc = MCSL(model_type='nn',
          iter_step=1000,
          rho_thresh=1e20,
          init_rho=1e-3,
          rho_multiply=5,
          graph_thresh=0.5,
          l1_graph_penalty=2e-3,
          temperature=0.2,
          learning_rate=3e-2,
          h_thresh=0.25,
          h_tol=1e-8,
          max_iter=25,
          )
mc.learn(X)

# plot est_dag and true_dag
GraphDAG(mc.causal_matrix_weight, true_dag)

# calculate accuracy
met = MetricsDAG(mc.causal_matrix, true_dag)
print(met.metrics)

I always get results with SHD > 20. Have I chosen the parameters badly? Could you provide me with an example script that reproduces a result from the paper (any graph size and data generation method)?

[shaido987]

Hello,

For better support for this issue, I suggest you email the paper's corresponding author (email: [email protected]).