Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add NaryDisEmbedding & Transformer Component #485

Merged
merged 34 commits into from
Sep 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
e0a8a1d
add custom op demo
yangxudong Oct 17, 2023
bc9f2f5
add custom op demo
yangxudong Oct 20, 2023
1fb8ac0
add custom op demo
yangxudong Oct 20, 2023
43e17c6
add custom op demo
yangxudong Oct 20, 2023
efa5683
add custom op demo
yangxudong Oct 20, 2023
03b9868
add custom op demo
yangxudong Oct 21, 2023
fdaa9a8
Merge branch 'master' of https://github.com/alibaba/EasyRec into cust…
yangxudong Jan 15, 2024
7efcdf9
add custom_op
yangxudong Jan 29, 2024
ffda1da
Merge branch 'master' into custom_op
yangxudong Jul 1, 2024
f2f5b74
add custom op
yangxudong Jul 26, 2024
1b1511b
add custom op
yangxudong Jul 27, 2024
5e9f7a4
add custom op & kd loss
yangxudong Aug 14, 2024
07b5ee3
add custom op & kd loss
yangxudong Aug 19, 2024
645b0dd
add custom op & kd loss
yangxudong Aug 20, 2024
262e2ca
add custom op & kd loss
yangxudong Aug 21, 2024
de96ccf
add support for rank distillation
yangxudong Sep 5, 2024
f015215
Merge branch 'master' into custom_op
yangxudong Sep 9, 2024
88adf24
add support for rank distillation
yangxudong Sep 9, 2024
34039f1
add support for rank distillation
yangxudong Sep 9, 2024
3c8182e
fix name of custom sequence augment op
yangxudong Sep 9, 2024
a7f4b9c
modify custom sequence augment op
yangxudong Sep 9, 2024
696c68d
modify deploy_ext.sh
yangxudong Sep 9, 2024
5c1303e
modify deploy_ext.sh
yangxudong Sep 10, 2024
2ea4b2a
fix bug of SeqAugmentOps
yangxudong Sep 10, 2024
01f3b16
fix bug of SeqAugmentOps
yangxudong Sep 10, 2024
fc0bd57
fix bug of SeqAugmentOps
yangxudong Sep 10, 2024
016c19f
modify of SeqAugmentOps
yangxudong Sep 10, 2024
7b075c1
modify pairwise loss
yangxudong Sep 12, 2024
a2ab568
Merge branch 'master' into custom_op
yangxudong Sep 12, 2024
c648b38
add narydis embedding
yangxudong Sep 14, 2024
dfe285e
add transformer layer
yangxudong Sep 15, 2024
1659947
add transformer layer
yangxudong Sep 18, 2024
491e7ea
add transformer layer
yangxudong Sep 21, 2024
a261dbb
add transformer layer
yangxudong Sep 23, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 0 additions & 57 deletions docs/source/component/backbone.md
Original file line number Diff line number Diff line change
Expand Up @@ -1094,63 +1094,6 @@ MovieLens-1M数据集效果:
- Cross Decoupling Network: [CDN](../models/cdn.md)
- DLRM+SENet: [dlrm_senet_on_criteo.config](https://github.com/alibaba/EasyRec/tree/master/examples/configs/dlrm_senet_on_criteo.config)

# 组件库介绍

## 1.基础组件

| 类名 | 功能 | 说明 | 示例 |
| ----------------- | ------ | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------------------------- |
| MLP | 多层感知机 | 可定制激活函数、initializer、Dropout、BN等 | [案例1](#wide-deep) |
| Highway | 类似残差链接 | 可用来对预训练embedding做增量微调 | [highway network](../models/highway.html) |
| Gate | 门控 | 多个输入的加权求和 | [Cross Decoupling Network](../models/cdn.html#id2) |
| PeriodicEmbedding | 周期激活函数 | 数值特征Embedding | [案例5](#dlrm-embedding) |
| AutoDisEmbedding | 自动离散化 | 数值特征Embedding | [dlrm_on_criteo_with_autodis.config](https://github.com/alibaba/EasyRec/tree/master/examples/configs/dlrm_on_criteo_with_autodis.config) |
| TextCNN | 文本卷积 | 提取文本序列的特征 | [text_cnn_on_movielens.config](https://github.com/alibaba/EasyRec/tree/master/examples/configs/text_cnn_on_movielens.config) |

**备注**:Gate组件的第一个输入是权重向量,后面的输入拼凑成一个列表,权重向量的长度应等于列表的长度

## 2.特征交叉组件

| 类名 | 功能 | 说明 | 示例 |
| -------------- | --------------------- | ---------------- | -------------------------------------------------------------------------------------------------------------------------- |
| FM | 二阶交叉 | DeepFM模型的组件 | [案例2](#deepfm) |
| DotInteraction | 二阶内积交叉 | DLRM模型的组件 | [案例4](#dlrm) |
| Cross | bit-wise交叉 | DCN v2模型的组件 | [案例3](#dcn) |
| BiLinear | 双线性 | FiBiNet模型的组件 | [fibinet_on_movielens.config](https://github.com/alibaba/EasyRec/tree/master/examples/configs/fibinet_on_movielens.config) |
| FiBiNet | SENet & BiLinear | FiBiNet模型 | [fibinet_on_movielens.config](https://github.com/alibaba/EasyRec/tree/master/examples/configs/fibinet_on_movielens.config) |
| Attention | Dot-product attention | Transformer模型的组件 | |

## 3.特征重要度学习组件

| 类名 | 功能 | 说明 | 示例 |
| --------- | ----------------- | ------------ | ----------------------------------------------------- |
| SENet | 建模特征重要度 | FiBiNet模型的组件 | [MMoE](../models/mmoe.html#id4) |
| MaskBlock | 建模特征重要度 | MaskNet模型的组件 | [Cross Decoupling Network](../models/cdn.html#id2) |
| MaskNet | 多个串行或并行的MaskBlock | MaskNet模型 | [DBMTL](../models/dbmtl.html#dbmtl-based-on-backbone) |
| PPNet | 参数个性化网络 | PPNet模型 | [PPNet](../models/ppnet.html#id2) |

## 4. 序列特征编码组件

| 类名 | 功能 | 说明 | 示例 |
| ---------- | ---------------- | ------------------- | ------------------------------------------------------------------------------------------------------------------------ |
| DIN | target attention | DIN模型的组件 | [DIN_backbone.config](https://github.com/alibaba/EasyRec/blob/master/samples/model_config/din_backbone_on_taobao.config) |
| BST | transformer | BST模型的组件 | [BST_backbone.config](https://github.com/alibaba/EasyRec/blob/master/samples/model_config/bst_backbone_on_taobao.config) |
| SeqAugment | 序列数据增强 | crop, mask, reorder | [CL4SRec](../models/cl4srec.html#id2) |

## 5. 多目标学习组件

| 类名 | 功能 | 说明 | 示例 |
| ---- | --------------------------- | --------- | ------------ |
| MMoE | Multiple Mixture of Experts | MMoE模型的组件 | [案例8](#mmoe) |

## 6. 辅助损失函数组件

| 类名 | 功能 | 说明 | 示例 |
| ------------- | ---------- | --------- | ----------- |
| AuxiliaryLoss | 用来计算辅助损失函数 | 常用在自监督学习中 | [案例7](#id7) |

各组件的详细参数请查看"[组件详细参数](component.md)"。

# 如何自定义组件

在 `easy_rec/python/layers/keras` 目录下新建一个`py`文件,也可直接添加到一个已有的文件中。我们建议目标类似的组件定义在同一个文件中,减少文件数量;比如特征交叉的组件都放在`interaction.py`里。
Expand Down
Loading
Loading