DiffuPath is an analytic tool for biological networks that connects the generic label propagation algorithms from DiffuPy to biological networks encoded in several formats such as Simple Interaction Format (SIF) or Biological Expression Language (BEL). For example, in the application scenario presented in the paper, we use three pathway databases (i.e., KEGG, Reactome and WikiPathways) and their integrated network retrieved from PathMe [1] to analyze three multi-omics datasets. However, other biological networks can be imported from the Bio2BEL ecosystem [2].
If you use DiffuPy in your work, please consider citing:
Marín-Llaó, J., et al. (2020). MultiPaths: a python framework for analyzing multi-layer biological networks using diffusion algorithms. Bioinformatics, 37(1), 137-139.
The latest stable code can be installed from PyPI with:
$ python3 -m pip install diffupathThe most recent code can be installed from the source on GitHub with:
$ python3 -m pip install git+https://github.com/multipaths/diffupath.gitRequired to install the latest PathMe version directly from GitHub:
$ python3 -m pip install git+https://github.com/PathwayMerger/PathMe.gitFor developers, the repository can be cloned from GitHub and installed in editable mode with:
$ git clone https://github.com/multipaths/diffupath.git
$ cd diffupath
$ python3 -m pip install -e .diffupath requires the following libraries:
networkx (>=2.1) pybel (0.13.2) biokeen (0.0.14) click (7.0) tqdm (4.31.1) numpy (1.16.3) scipy (1.2.1) scikit-learn (0.21.3) pandas (0.24.2) openpyxl (3.0.2) plotly (4.5.3) matplotlib (3.1.2) matplotlib_venn (0.11.5) bio2bel (0.2.1) pathme diffupy
The main required input to run diffusion using DiffuPath is:
1) A dataset of scores/ponderations, which will be propagated over the integrated PathMe background network. (see Input Formatting below)
For its usability, you can either:
- Use the Command Line Interface (see down).
- Use pythonicaly the functions provided in diffupath.diffuse:
from diffupath.diffuse import run_diffusion
# DATA INPUT and GRAPH as PATHs -> returned as *PandasDataFrame*
diffusion_scores = run_diffusion(~/data/input_scores.csv, ~/data/network.csv).as_pd_dataframe()
# DATA INPUT and GRAPH as Python OBJECTS -> exported *as_csv*
diffusion_scores = run_diffusion(input_scores, network).as_csv('~/output/diffusion_results.csv')You can customize the PathMe integrated background network:
- Constructing it by selecting among the available Biological Network Databases (see database).
- Filtering the default network either by database or by omic.
diffusion_scores = run_diffusion(input_scores, database = 'pathme_drugbank', filter_network_omic = ['gene', 'mirna'])If you wish to use your own network, we recommend you to check the supported network formats in DiffuPy and directly use DiffuPy, since DiffuPath wraps it to offer diffusion with the PathMe environment networks.
The diffusion method by default is z, which statistical normalization has previously shown outperformance over raw diffusion[1]. Further parameters to adapt the propagation procedure can be provided, such as choosing among the available diffusion methods or providing a custom method function. See the diffusion Methods and/or Method modularity.
diffusion_scores_select_method = run(input_scores, method = 'raw')
from networkx import page_rank # Custom method function
diffusion_scores_custom_method = run(input_scores, method = page_rank)You can also provide your own kernel method or select among the provided in kernels.py, you can provide it as kernel_method argument. By default regularised_laplacian_kernel is used.
from diffupath.kernels import p_step_kernel # Custom kernel calculation function
diffusion_scores_custom_kernel_method = run(input_scores, method = 'raw', kernel_method = p_step_kernel)So method stands for the diffusion process method, and kernel_method for the kernel calculation method.
The following commands can be used directly from your terminal:
- Download a database for network analysis.
The following command generates a BEL file representing the network of the given database.
$ python3 -m diffupath database get-database --database=<database-name>To check the available databases, run the following command:
$ python3 -m diffupath database ls- Run a diffusion analysis
The following command will run a diffusion method on a given network with the given data
$ python3 -m diffupath diffusion diffuse --network=<path-to-network-file> --data=<path-to-data-file> --method=<method>- Run a diffusion analysis
$ python3 -m diffupath diffusion evaluate -i=<input_data> -n=<path_network>The input is preprocessed and further mapped before the diffusion. See input mapping or or see process_input docs in DiffuPy for further details. Here are exposed the covered input formats for its preprocessing.
You can submit your dataset in any of the following formats:
- CSV (.csv)
- TSV (.tsv)
Please ensure that the dataset minimally has a column 'Node' containing node IDs. You can also optionally add the following columns to your dataset:
- NodeType
- LogFC [*]
- p-value
| [*] | Log2 fold change |
DiffuPath accepts several input formats which can be codified in different ways. See the diffusion scores summary for more details.
- You can provide a dataset with a column 'Node' containing node IDs.
| Node |
|---|
| A |
| B |
| C |
| D |
2. You can also provide a dataset with a column 'Node' containing node IDs as well as a column 'NodeType', indicating the entity type of the node to run diffusion by entity type.
| Node | NodeType |
|---|---|
| A | Gene |
| B | Gene |
| C | Metabolite |
| D | Gene |
3. You can also choose to provide a dataset with a column 'Node' containing node IDs as well as a column 'logFC' with their LogFC. You may also add a 'NodeType' column to run diffusion by entity type.
| Node | LogFC |
|---|---|
| A | 4 |
| B | -1 |
| C | 1.5 |
| D | 3 |
4. Finally, you can provide a dataset with a column 'Node' containing node IDs, a column 'logFC' with their logFC and a column 'p-value' with adjusted p-values. You may also add a 'NodeType' column to run diffusion by entity type.
| Node | LogFC | p-value |
|---|---|---|
| A | 4 | 0.03 |
| B | -1 | 0.05 |
| C | 1.5 | 0.001 |
| D | 3 | 0.07 |
You can also take a look at our sample datasets folder for some examples files.
In this section, we describe the types of networks you can select to run diffusion methods over. These include the following and are described in detail in this section [†]:
- Select a network representing an individual biological database
- Select multiple databases to generate a harmonized network
- Select from one of four predefined collections of biological databases representing a harmonized network
- Submit your own network [‡] from one of the accepted formats
| [†] | Please note that all networks available through DiffuPath have been generated using PyBEL v.0.13.2 [12]. |
| [‡] | If there are duplicated nodes in your network, please take a look at this Jupyter Notebook to address the issue. |
Because of the high computational cost of generating the kernel, we provide links to pre-calculated kernels for a set of networks representing biological databases.
| Database | Description | Reference | Download |
|---|---|---|---|
| DDR | Disease-disease associations | [3] | ddr.json |
| DrugBank | Drug and drug target interactions | [4] | drugbank.json |
| Gene Ontology | Hierarchy of tens of thousands of biological processes | [5] | go.json |
| HSDN | Associations between diseases and symptoms | [6] | hsdn.json |
| KEGG | Multi-omics interactions in biological pathways | [7] | kegg.json |
| miRTarBase | Interactions between miRNA and their targets | [8] | mirtarbase.json |
| Reactome | Multi-omics interactions in biological pathways | [9] | reactome.json |
| SIDER | Associations between drugs and side effects | [10] | sider.json |
| WikiPathways | Multi-omics interactions in biological pathways | [11] | wikipathways.json |
If you would like to use one of our predefined collections, you can similarly download pre-calculated kernels for sets of networks representing integrated biological databases.
| Collection | Database | Description | Download |
|---|---|---|---|
| #1 | KEGG, Reactome and WikiPathways | -omics and biological processes/pathways | pathme.json |
| #2 | KEGG, Reactome, WikiPathways and DrugBank | -omics and biological processes/pathways with a strong focus on drug/chemical interactions | pathme_drugbank.json |
| #3 | KEGG, Reactome, WikiPathways and MirTarBase | -omics and biological processes/ pathways enriched with miRNAs | pathme_mirtarbase.json |
You can also submit your own networks in any of the following formats:
- BEL (.bel)
- CSV (.csv)
- Edge list (.lst)
- GML (.gml or .xml)
- GraphML (.graphml or .xml)
- Pickle (.pickle)
- TSV (.tsv)
- TXT (.txt)
Minimally, please ensure each of the following columns are included in the network file you submit:
- Source
- Target
Optionally, you can choose to add a third column, "Relation" in your network (as in the example below). If the relation between the Source and Target nodes is omitted, and/or if the directionality is ambiguous, either node can be assigned as the Source or Target.
| Source | Target | Relation |
|---|---|---|
| A | B | Increase |
| B | C | Association |
| A | D | Association |
You can also take a look at our sample networks folder for some examples.
Eventhough it is not relevant for the input user usage, it is relevant for the diffusion process assessment taking into account the input mapped entities over the background network, since the coverage of the input implies the actual entities-scores that are being diffused. In other words, only will be further processed for diffusion, the entities which label matches an entity in the network.
The diffusion running will report the mapping as follows:
Mapping descriptive statistics
wikipathways:
gene_nodes (474 mapped entities, 15.38% input coverage)
mirna_nodes (2 mapped entities, 4.65% input coverage)
metabolite_nodes (12 mapped entities, 75.0% input coverage)
bp_nodes (1 mapped entities, 0.45% input coverage)
total (489 mapped entities, 14.54% input coverage)
kegg:
gene_nodes (1041 mapped entities, 33.80% input coverage)
mirna_nodes (3 mapped entities, 6.98% input coverage)
metabolite_nodes (6 mapped entities, 0.375% input coverage)
bp_nodes (12 mapped entities, 5.36% input coverage)
total (1062 mapped entities, 31.58% input coverage)
reactome:
gene_nodes (709 mapped entities, 23.02% input coverage)
mirna_nodes (1 mapped entities, 2.33% input coverage)
metabolite_nodes (6 mapped entities, 37.5% input coverage)
total (716 mapped entities, 22.8% input coverage)
total:
gene_nodes (1461 mapped entities, 43.44% input coverage)
mirna_nodes (4 mapped entities, 0.12% input coverage)
metabolite_nodes (13 mapped entities, 0.38% input coverage)
bp_nodes (13 mapped entities, 0.39% input coverage)
total (1491 mapped entities, 44.34% input coverage)To graphically see the mapping coverage, you can also plot a heatmap view of the mapping (see views). To see how the mapping is performed over a input pipeline preprocessing, take a look at this JupyterNotebook or see process_input docs in DiffuPy.
The returned format is a custom Matrix type, with node labels as rows and a column with the diffusion score, which can be exported into the following formats:
diffusion_scores.to_dict()
diffusion_scores.as_pd_dataframe()
diffusion_scores.as_csv()
diffusion_scores.to_nx_graph()DiffuPath is a scientific software that has been developed in an academic capacity, and thus comes with no warranty or guarantee of maintenance, support, or back-up of data.
| [1] | Domingo-Fernandez, D., Mubeen, S., Marin-Llao, J., Hoyt, C., et al. Hofmann-Apitius, M. (2019). PathMe: Merging and exploring mechanistic pathway knowledge. BMC Bioinformatics, 20:243. |
| [2] | Hoyt, C. T., et al. (2019). Integration of Structured Biological Data Sources using Biological Expression Language. bioRxiv, 631812. |
| [3] | Menche, J., et al. (2015). Disease networks. Uncovering disease-disease relationships through the incomplete interactome. Science, 347(6224), 1257601. |
| [4] | Wishart, D. S., et al. (2018). DrugBank 5.0: a major update to the DrugBank database for 2018. Nucleic Acids Research, 46(D1), D1074–D1082. |
| [5] | Ashburner, M., et al. (2000). Gene ontology: tool for the unification of biology. The Gene Ontology Consortium. Nature Genetics, 25(1), 25–9. |
| [6] | Zhou, X., Menche, J., Barabási, A. L., & Sharma, A. (2014). Human symptoms–disease network. Nature communications, 5(1), 1-10. |
| [7] | Kanehisa, et al. (2017). KEGG: new perspectives on genomes, pathways, diseases and drugs.. Nucleic Acids Res. 45,D353-D361. |
| [8] | Huang, H. Y., et al. (2020). miRTarBase 2020: updates to the experimentally validated microRNA–target interaction database. Nucleic acids research, 48(D1), D148-D154. |
| [9] | Fabregat, A et al. (2016). The Reactome Pathway Knowledgebase. Nucleic Acids Research 44. Database issue: D481–D487. |
| [10] | Kuhn, M., et al. (2016). The SIDER database of drugs and side effects. Nucleic Acids Research, 44(D1), D1075–D1079. |
| [11] | Slenter, D.N., et al. (2017). WikiPathways: a multifaceted pathway database bridging metabolomics to other omics research. Nucleic Acids Research, 46(D1):D661-D667. |
| [12] | Hoyt, C. T., et al. (2017). PyBEL: a Computational Framework for Biological Expression Language. Bioinformatics, 34(December), 1–2. |