can we clean unneeded modules out of the RTX/code/reasoningtool code directory?

[saramsey]

No description provided.

[saramsey]

From a quick audit of the code, it looks like

RTX/code/UI/OpenAPI/python-flask-server/openapi_server/controllers/pubmed_mesh_ngd_controller.py

references

RTX/code/reasoningtool/kg-construction/NormGoogleDistance

So clearly that /pubmed_mesh_ngd function seems to depend on code in the reasoningtool directory tree.

Further, a simple audit of code in the RTX/code/ARAX directory shows a bunch of potential dependencies on the RTX/code/reasoningtool directory:

sramsey-laptop:ARAX sramsey$ find . -name \*.py -exec grep 'reasoningtool' {} /dev/null \;
./ARAXQuery/ARAX_query.py:sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../reasoningtool/QuestionAnswering")
./ARAXQuery/Filter_KG/remove_edges.py:sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../../reasoningtool/kg-construction/")
./ARAXQuery/Overlay/ngd/estimate_coverage.py:sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../../../reasoningtool/kg-construction/")
./ARAXQuery/Overlay/predict_drug_treats_disease.py:sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../reasoningtool/kg-construction/")
./ARAXQuery/Overlay/GraphSage_train/py_scripts/LogReg_edited.py:sys.path.append(os.path.sep.join([*pathlist[:(RTXindex + 1)], 'code', 'reasoningtool', 'kg-construction']))
./ARAXQuery/Overlay/GraphSage_train/py_scripts/LogReg_edited.py:sys.path.append(os.path.sep.join([*pathlist[:(RTXindex + 1)], 'code', 'reasoningtool', 'QuestionAnswering']))
./ARAXQuery/Overlay/GraphSage_train/py_scripts/generate_training_data.py:    drugs.to_csv('/home/grads/cqm5886/work/RTX/code/reasoningtool/MLDrugRepurposing/Test_graphsage/kg2_8_0/raw_training_data/drugs.txt',sep='\t',index=False)
./ARAXQuery/Overlay/GraphSage_train/py_scripts/generate_training_data.py:    dataGenerator.generate_MyChemData(drugs=drugs, output_path='/home/grads/cqm5886/work/RTX/code/reasoningtool/MLDrugRepurposing/Test_graphsage/kg2_8_0/raw_training_data',dist=2, batchsize=50)
./ARAXQuery/Overlay/GraphSage_train/py_scripts/generate_training_data.py:#     dataGenerator.generate_SemmedData(mysqldump_path='/home/cqm5886/work/RTX/code/reasoningtool/MLDrugRepurposing/Test_graphsage/semmedVER43_2020_R_PREDICATION.sql.gz', output_path='/home/cqm5886/work/RTX/code/reasoningtool/MLDrugRepurposing/Test_graphsage/kg2_5_1/raw_training_data/')
./ARAXQuery/Overlay/GraphSage_train/py_scripts/pull_canonicalized_KG2C.py:parser.add_argument('--OutFolder', type=str, help="The path of output folder", default='~/RTX/code/reasoningtool/MLDrugRepurposing/kg2_5_0/kg2canonicalized_data/')
./ARAXQuery/Overlay/add_node_pmids.py:sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../../reasoningtool/kg-construction/")
./ARAXQuery/Filter_Results/sort_results.py:sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../../reasoningtool/kg-construction/")
./KnowledgeSources/COHD_local/scripts/COHDIndex.py:sys.path.append(os.path.sep.join([*pathlist[:(RTXindex + 1)], 'code', 'reasoningtool', 'QuestionAnswering']))
./KnowledgeSources/Generate_all_KP_apis.py:sys.path.append(os.path.dirname(os.path.abspath(__file__))+"/../../reasoningtool/kg-construction/")
./KnowledgeSources/Generate_all_KP_apis.py:mypath = os.path.dirname(os.path.abspath(__file__))+"/../../reasoningtool/kg-construction/"
./KnowledgeSources/Generate_all_KP_apis.py:            fid.write(f"""          - parser: ../../reasoningtool/kg-construction/{module_name}.py\n""")

[saramsey]

So it looks like we can't just delete the code in RTX/code/reasoningtool... the above evidence strongly suggests that we depend on some modules in that directory tree. But probably only a few modules out of many that are deprecated. But perhaps we could start by pruning modules in that directory tree that are not used anywhere else in the code-base. Does anyone have an IDE that can map out those dependencies? Because we dynamically adjust module search paths in pretty much every python module in this code-base, that kind of static analysis seems (to me) not so straightforward.

[saramsey]

I'm going to make an issue-2122 branch to work on this

[saramsey]

I'm working this issue in the issue-2122 branch.

[saramsey]

I've cleaned out maybe half the python modules in RTX/code/reasoningtool, focusing on the ones that were obviously deprecated. There are still 70 python modules left in there, but a lot of them are in QuestionAnswering or MLDrugRepurposing where PSU should take the lead on deciding what to keep or eliminate.

[saramsey]

All unit tests passing; now going to test on /kg2test and on /test

[saramsey]

OK, I've merged the code for this branch into master (see #2131)

[saramsey]

I've deleted the branch issue-2122