HOPE2Net (HistOlogy and Position Embedding Network) is a multilayer perceptron that properly learns the balance between histology features and positions embeddings for prediction tasks such as gene expressions and pathway activities. HOPE2Net is a useful computational tool for analyses of spatial transcriptomics data, which builds the bridge between H&E-stained histology images and gene expressions. HOPE2Net can be applied to multiple tissue sections with or without images.
We described and introduced HOPE2Net in our methodological paper. To find code to reproduce the results we generated in that paper, please visit the result folder github repository, which provides all code (including that for other methods) necessary to reproduce our results.
A tutorial jupyter notebook, together with a dataset, is publicly downloadable.
import torch
from MLP_model import plNet
Hope2Net = plNet(n_feature = 1000, n_hidden1 = 128, n_hidden2 = 64, n_output=1000)
x1 = torch.Tensor(200, 1000)
x2 = torch.Tensor(200, 1000)
Hope2Net(x1, x2)This is an illustration of simple feedforward for Hope2Net. X1 mimics the image feature matrix and X2 represents the position embeddings. When computing these two modalities of features, we enforce them with same dimension for the purpose of allocating weights. The weights will further be learned during the training process.
We use transfer learning approach to extract deep image features for each tile where the barcoded spot is centered. Three state-of-art DL architectures are considered: VGG19, ResNet50, and ViT16. The pre-trained weights on imageNet1k dataset can be accessed by keras.applicaiton or pytorch_pretrained_vit libraries. Free free to utilize the implementation at Tile_features.py.
- Python >= 3.8
- Torch >= 2.0.1
- scanpy >= 1.5.1
- pytorch_lightning