Add TokenGT model

[michailmelonas]

PyG implementation of the Tokenized Graph Transformer following "Pure Transformers are Powerful Graph Learners" (https://arxiv.org/pdf/2207.02505). Includes support for both Laplacian eigenvectors and ORF node identifiers (implemented via a simple data Transform object). A graph regression example is also included.

For a detailed blog post about the implementation, see https://medium.com/stanford-cs224w/pyg-implementation-tokengt-e4aa74dc867b.

[michailmelonas]

@wsad1 @EdisonLeeeee @akihironitta any thoughts on when this contribution will get reviewed? :)

[puririshi98]

@michailmelonas this is cool, ill review and help merge soon as my time allows,

[puririshi98]

this looks good, will do a deep review soon

[puririshi98]

this is good at a high level. however i want to see how it compares to existing work. Can you please update this example:
https://github.com/pyg-team/pytorch_geometric/blob/master/examples/ogbn_train.py#L31
to have a "--gnn-choice" arg parse option, with choices ["sage, gat, tokengt_graph_transformer"]. and run all 3 in your environment to see how they compare. Please make the highest test acc the default. I can review a little closer once that initial test is done

[michailmelonas]

@puririshi98 sure thing, will do asap.

[puririshi98]

@michailmelonas lmk when ready for further review

[michailmelonas]

@puririshi98 apologies for only getting back to you now - have been swamped at work.

TokenGT requires specifying n_nodes orthogonal vectors ("node identifiers"). This is infeasible for the ogbn-papers100M dataset which has over 100M nodes. Therefore, rather than amending ogbn_train.py, I instead added token_gt_ogbn.py: a script that makes it easy to benchmark TokenGT against GCN on the ogbg-molhiv dataset (ideally, I'd like to run the model on PCQM4Mv2 as in the paper, but given my computational resources this was the best I could do). Running said script, I get slightly worse (but comparable) results for TokenGT vs GCN: the former has a validation ROC-AUC of 0.774 and the latter has 0.819.