Skip to content

Commit

Permalink
Add the WuKong model (#106)
Browse files Browse the repository at this point in the history 
* the implementation of WuKong by XiaoLongtao
  • Loading branch information
@XiaoLongtaoo @xpai
XiaoLongtaoo authored and xpai committed Nov 5, 2024
1 parent e0d5495 commit 0996f7e
Show file tree
Hide file tree
Showing 13 changed files with 398 additions and 69 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
[Doing] Add support of multi-gpu training

**FuxiCTR v2.3.4, 2024-11-05**
+ [Feature] Add WuKong model
+ [Fix] Fix OOV token update ([#119](https://github.com/reczoo/FuxiCTR/issues/119))
+ [Fix] Speed up parquet dataset reading ([#121](https://github.com/reczoo/FuxiCTR/issues/121))
+ [Fix] Fix add_loss() isue that does not work after renaming to compute_loss() ([#122](https://github.com/reczoo/FuxiCTR/issues/122))
Expand Down
31 changes: 16 additions & 15 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -73,25 +73,26 @@ Click-through rate (CTR) prediction is a critical task for various industrial ap
| 37 | SIGIR'23 | [FinalNet](./model_zoo/FinalNet) | [FINAL: Factorized Interaction Layer for CTR Prediction](https://dl.acm.org/doi/10.1145/3539618.3591988) :triangular_flag_on_post:**Huawei** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/FinalNet) | `torch` |
| 38 | SIGIR'23 | [EulerNet](./model_zoo/EulerNet) | [EulerNet: Adaptive Feature Interaction Learning via Euler's Formula for CTR Prediction](https://dl.acm.org/doi/10.1145/3539618.3591681) :triangular_flag_on_post:**Huawei** | [:arrow_upper_right:](https://github.com/Ethan-TZ/EulerNet/tree/main/%23Code4FuxiCTR%23) | `torch` |
| 39 | CIKM'23 | [GDCN](./model_zoo/GDCN) | [Towards Deeper, Lighter and Interpretable Cross Network for CTR Prediction](https://dl.acm.org/doi/pdf/10.1145/3583780.3615089) :triangular_flag_on_post:**Microsoft** | | `torch` |
| 40 | Arxiv'24 | [DCNv3](./model_zoo/DCNv3) | [DCNv3: Towards Next Generation Deep Cross Network for Click-Through Rate Prediction](https://arxiv.org/abs/2407.13349) :triangular_flag_on_post:**AHU, Huawei** | [:arrow_upper_right:](https://github.com/salmon1802/DCNv3/tree/master/checkpoints) | `torch` |
| 40 | ICML'24 | [WuKong](./model_zoo/WuKong) | [Wukong: Towards a Scaling Law for Large-Scale Recommendation](https://arxiv.org/abs/2403.02545) :triangular_flag_on_post:**Meta** | | `torch` |
| 41 | Arxiv'24 | [DCNv3](./model_zoo/DCNv3) | [DCNv3: Towards Next Generation Deep Cross Network for Click-Through Rate Prediction](https://arxiv.org/abs/2407.13349) | [:arrow_upper_right:](https://github.com/salmon1802/DCNv3/tree/master/checkpoints) | `torch` |
|<tr><th colspan=6 align="center">:open_file_folder: **Behavior Sequence Modeling**</th></tr>|
| 41 | KDD'18 | [DIN](./model_zoo/DIN) | [Deep Interest Network for Click-Through Rate Prediction](https://www.kdd.org/kdd2018/accepted-papers/view/deep-interest-network-for-click-through-rate-prediction) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/DIN) | `torch` |
| 42 | AAAI'19 | [DIEN](./model_zoo/DIEN) | [Deep Interest Evolution Network for Click-Through Rate Prediction](https://arxiv.org/abs/1809.03672) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/DIEN) | `torch` |
| 43 | DLP-KDD'19 | [BST](./model_zoo/BST) | [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://arxiv.org/abs/1905.06874) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/BST) | `torch` |
| 44 | CIKM'20 | [DMIN](./model_zoo/DMIN) | [Deep Multi-Interest Network for Click-through Rate Prediction](https://dl.acm.org/doi/10.1145/3340531.3412092) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/DMIN) | `torch` |
| 45 | AAAI'20 | [DMR](./model_zoo/DMR) | [Deep Match to Rank Model for Personalized Click-Through Rate Prediction](https://ojs.aaai.org/index.php/AAAI/article/view/5346) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/DMR) | `torch` |
| 46 | DLP-KDD'22 | [ETA](./model_zoo/ETA) | [Efficient Long Sequential User Data Modeling for Click-Through Rate Prediction](https://arxiv.org/abs/2209.12212) :triangular_flag_on_post:**Alibaba** | | `torch` |
| 47 | CIKM'22 | [SDIM](./model_zoo/SDIM) | [Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction](https://arxiv.org/abs/2205.10249) :triangular_flag_on_post:**Meituan** | | `torch` |
| 48 | KDD'23 | [TransAct](./model_zoo/TransAct) | [TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest](https://arxiv.org/abs/2306.00248) :triangular_flag_on_post:**Pinterest** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/TransAct) | `torch` |
| 42 | KDD'18 | [DIN](./model_zoo/DIN) | [Deep Interest Network for Click-Through Rate Prediction](https://www.kdd.org/kdd2018/accepted-papers/view/deep-interest-network-for-click-through-rate-prediction) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/DIN) | `torch` |
| 43 | AAAI'19 | [DIEN](./model_zoo/DIEN) | [Deep Interest Evolution Network for Click-Through Rate Prediction](https://arxiv.org/abs/1809.03672) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/DIEN) | `torch` |
| 44 | DLP-KDD'19 | [BST](./model_zoo/BST) | [Behavior Sequence Transformer for E-commerce Recommendation in Alibaba](https://arxiv.org/abs/1905.06874) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/BST) | `torch` |
| 45 | CIKM'20 | [DMIN](./model_zoo/DMIN) | [Deep Multi-Interest Network for Click-through Rate Prediction](https://dl.acm.org/doi/10.1145/3340531.3412092) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/DMIN) | `torch` |
| 46 | AAAI'20 | [DMR](./model_zoo/DMR) | [Deep Match to Rank Model for Personalized Click-Through Rate Prediction](https://ojs.aaai.org/index.php/AAAI/article/view/5346) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/DMR) | `torch` |
| 47 | DLP-KDD'22 | [ETA](./model_zoo/ETA) | [Efficient Long Sequential User Data Modeling for Click-Through Rate Prediction](https://arxiv.org/abs/2209.12212) :triangular_flag_on_post:**Alibaba** | | `torch` |
| 48 | CIKM'22 | [SDIM](./model_zoo/SDIM) | [Sampling Is All You Need on Modeling Long-Term User Behaviors for CTR Prediction](https://arxiv.org/abs/2205.10249) :triangular_flag_on_post:**Meituan** | | `torch` |
| 49 | KDD'23 | [TransAct](./model_zoo/TransAct) | [TransAct: Transformer-based Realtime User Action Model for Recommendation at Pinterest](https://arxiv.org/abs/2306.00248) :triangular_flag_on_post:**Pinterest** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/TransAct) | `torch` |
|<tr><th colspan=6 align="center">:open_file_folder: **Dynamic Weight Network**</th></tr>|
| 49 | NeurIPS'22 | [APG](./model_zoo/APG) | [APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction](https://arxiv.org/abs/2203.16218) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/APG) | `torch` |
| 50 | KDD'23 | [PPNet](./model_zoo/PEPNet) | [PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information](https://arxiv.org/abs/2302.01115) :triangular_flag_on_post:**KuaiShou** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/PPNet) | `torch` |
| 50 | NeurIPS'22 | [APG](./model_zoo/APG) | [APG: Adaptive Parameter Generation Network for Click-Through Rate Prediction](https://arxiv.org/abs/2203.16218) :triangular_flag_on_post:**Alibaba** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/APG) | `torch` |
| 51 | KDD'23 | [PPNet](./model_zoo/PEPNet) | [PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information](https://arxiv.org/abs/2302.01115) :triangular_flag_on_post:**KuaiShou** | [:arrow_upper_right:](https://github.com/reczoo/BARS/tree/main/ranking/ctr/PPNet) | `torch` |
|<tr><th colspan=6 align="center">:open_file_folder: **Multi-Task Modeling**</th></tr>|
| 51 | MachineLearn'97 | [SharedBottom](./model_zoo/multitask/SharedBottom) | [Multitask Learning](https://link.springer.com/article/10.1023/A:1007379606734) | | `torch` |
| 52 | KDD'18 | [MMoE](./model_zoo/multitask/MMOE) | [Modeling Task Relationships in Multi-task Learning with Multi-Gate Mixture-of-Experts](https://dl.acm.org/doi/pdf/10.1145/3219819.3220007) :triangular_flag_on_post:**Google** | | `torch` |
| 53 | KDD'18 | [PLE](./model_zoo/multitask/PLE) | [Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations](https://dl.acm.org/doi/10.1145/3383313.3412236) :triangular_flag_on_post:**Tencent** | | `torch` |
| 52 | MachineLearn'97 | [SharedBottom](./model_zoo/multitask/SharedBottom) | [Multitask Learning](https://link.springer.com/article/10.1023/A:1007379606734) | | `torch` |
| 53 | KDD'18 | [MMoE](./model_zoo/multitask/MMOE) | [Modeling Task Relationships in Multi-task Learning with Multi-Gate Mixture-of-Experts](https://dl.acm.org/doi/pdf/10.1145/3219819.3220007) :triangular_flag_on_post:**Google** | | `torch` |
| 54 | KDD'18 | [PLE](./model_zoo/multitask/PLE) | [Progressive Layered Extraction (PLE): A Novel Multi-Task Learning (MTL) Model for Personalized Recommendations](https://dl.acm.org/doi/10.1145/3383313.3412236) :triangular_flag_on_post:**Tencent** | | `torch` |
|<tr><th colspan=6 align="center">:open_file_folder: **Multi-Domain Modeling**</th></tr>|
| 54 | KDD'23 | PEPNet | [PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information](https://arxiv.org/abs/2302.01115) :triangular_flag_on_post:**KuaiShou** | | `torch` |
| 55 | KDD'23 | PEPNet | [PEPNet: Parameter and Embedding Personalized Network for Infusing with Personalized Prior Information](https://arxiv.org/abs/2302.01115) :triangular_flag_on_post:**KuaiShou** | | `torch` |

## Benchmarking

Expand Down
24 changes: 12 additions & 12 deletions model_zoo/EulerNet/config/model_config.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,16 +12,16 @@ Base:
feature_specs: null
feature_config: null

EulerNet_test:
EulerNet_default: # This is a config template
model: EulerNet
dataset_id: tiny_npz
dataset_id: TBD
loss: 'binary_crossentropy'
metrics: ['logloss', 'AUC']
task: binary_classification
optimizer: adam
shape: [52]
net_ex_dropout: 0.1
net_im_dropout: 0.1
shape: [20]
net_ex_dropout: 0.0
net_im_dropout: 0.0
layer_norm: true
learning_rate: 1.0e-3
embedding_regularizer: 0
Expand All @@ -34,24 +34,24 @@ EulerNet_test:
monitor: {'AUC': 1, 'logloss': -1}
monitor_mode: 'max'

EulerNet_default: # This is a config template
EulerNet_test:
model: EulerNet
dataset_id: TBD
dataset_id: tiny_npz
loss: 'binary_crossentropy'
metrics: ['logloss', 'AUC']
task: binary_classification
optimizer: adam
shape: [20]
net_ex_dropout: 0.0
net_im_dropout: 0.0
shape: [52]
net_ex_dropout: 0.1
net_im_dropout: 0.1
layer_norm: true
learning_rate: 1.0e-3
embedding_regularizer: 0
net_regularizer: 0
batch_size: 10000
embedding_dim: 10
epochs: 100
epochs: 1
shuffle: True
seed: 2021
monitor: {'AUC': 1, 'logloss': -1}
monitor_mode: 'max'
monitor_mode: 'max'
Loading

0 comments on commit 0996f7e

Please sign in to comment.