"HYPHYMPI" Failing but "hyphy" working with 2.5.70

[lrschaff]

Dear HyPhy Team,

It seems the bug that has to do with something not being properly exported to compute nodes by MPI still persists with 2.5.70. I am running with:

$ srun HYPHYMPI RELAX --alignment ./gene_600.fa --tree ./536mammals_ML_tree.nwk --test Foreground --models All --srv Branch-site --multiple-hit "Double+Triple" --keepident true

And getting the error:

[...]

Fitting the general descriptive (separate k per branch) model

Grid result 7 = -213986.1435143871 (best = -192163.4429078236)Error:

Master node received an error:Operation '^' is not implemented/defined for a AssociativeList
While computing: utility.Extend(^(mpi.gJHXkMzL.arguments[2]),mpi.gJHXkMzL.result)

Function call stack
1 : [namespace = mpi.gJHXkMzL] utility.Extend(^(arguments[2]),result);

Keyword arguments:
    {
     "keepident":"true",
     "multiple-hit":"Double+Triple"
    }

---

2 : [namespace = mpi.aux.EaU_Gajr] Call((queue[from])[utility.getGlobalValue("terms.mpi.callback")],from,Eval(result),(queue[from])[utility.getGlobalValue("terms.mpi.arguments")]);

Keyword arguments:
    {
     "keepident":"true",
     "multiple-hit":"Double+Triple"
    }

---

3 : [namespace = mpi.uWUXdajl] node=aux._handle_receieve(queue);

Keyword arguments:
    {
     "keepident":"true",
     "multiple-hit":"Double+Triple"
    }

---

4 : [namespace = mpi.jfvuBvnJ] mpi.QueueComplete(queue);

Keyword arguments:
    {
     "keepident":"true",
     "multiple-hit":"Double+Triple"
    }

---

5 : [namespace = DhYdAqBM] grid_results=mpi.ComputeOnGridSetValues(&likelihoodFunction,run_options[utility.getGlobalValue("terms.search_grid")],{0:model_objects,1:initial_values,2:run_options[utility.getGlobalValue("terms.run_options.proportional_branch_length_scaler")]},"mpi.ComputeOnGrid.SimpleEvaluatorWithValues","mpi.ComputeOnGrid.ResultHandler");

Keyword arguments:
    {
     "keepident":"true",
     "multiple-hit":"Double+Triple"
    }

---

6 : relax.grid_search.results=estimators.FitLF(relax.filter_names,relax.trees,relax.ge_model_map,relax.final_partitioned_mg_results,relax.model_object_map,{"retain-lf-object":TRUE,terms.run_options.apply_user_constraints:"relax.init.k",terms.run_options.proportional_branch_length_scaler:relax.nm.bl_scalers,terms.run_options.optimization_settings:{"OPTIMIZATION_METHOD":"nedler-mead","MAXIMUM_OPTIMIZATION_ITERATIONS":500,"OPTIMIZATION_PRECISION":relax.nm.precision},terms.search_grid:relax.initial_grid,terms.search_restarts:relax.N.initial_guesses,terms.run_options.optimization_log:relax.optimization_log_file(".GE-1-log.json")});

Keyword arguments:
    {
     "keepident":"true",
     "multiple-hit":"Double+Triple"
    }

---

[...]

Is there anything you guys need to help resolve this issue?

Thank you!

Lillie

[spond]

Dear @lrschaff,

Sorry for the delay in responding. Let me check into it. One thing to keep in mind, RELAX does not make good use of MPI. It's more of a "task-farming" approach (separate process for separate genes).

Best,
Sergei

[lrschaff]

Hi Sergei,

No worries! Thank you for checking into it. Do you recommend another way to automate genes, other than using HYPHYMPI?

Thank you,

Lillie

[spond]

Dear @lrschaff,

Something like snakemake is a good general way to automate workload distribution, i.e. where you have many jobs to run on many inputs.

Best,
Sergei