Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[feature]: add embedding parallel #438

Merged
merged 69 commits into from
Jan 3, 2024
Merged

[feature]: add embedding parallel #438

merged 69 commits into from
Jan 3, 2024

Commits on Oct 18, 2023

  1. add support for keep model

    杨熙 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    54c9e9e View commit details
    Browse the repository at this point in the history

  2. add adam_s

    @chengmengli06
    chengmengli06 committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    bd5bd02 View commit details
    Browse the repository at this point in the history

Commits on Oct 19, 2023

  1. add sok debug case2

    @chengmengli06
    chengmengli06 committed Oct 19, 2023
    Configuration menu
    Copy the full SHA
    ae74f18 View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. add drop_remainder option as required by sok

    @chengmengli06
    chengmengli06 committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    bbfe530 View commit details
    Browse the repository at this point in the history

  2. add adam sparse

    @chengmengli06
    chengmengli06 committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    5477cb2 View commit details
    Browse the repository at this point in the history

Commits on Oct 21, 2023

  1. fix parquet input bugs

    @chengmengli06
    chengmengli06 committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    23a9fba View commit details
    Browse the repository at this point in the history

  2. add all2all impl

    @chengmengli06
    chengmengli06 committed Oct 21, 2023
    Configuration menu
    Copy the full SHA
    4ce5229 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. add saver for sok

    @chengmengli06
    chengmengli06 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    628674a View commit details
    Browse the repository at this point in the history

  2. add missing files

    @chengmengli06
    chengmengli06 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    a1032da View commit details
    Browse the repository at this point in the history

  3. add support for eval

    @chengmengli06
    chengmengli06 committed Oct 26, 2023
    Configuration menu
    Copy the full SHA
    7f33ddb View commit details
    Browse the repository at this point in the history

Commits on Oct 28, 2023

  1. add load kv embed op

    @chengmengli06
    chengmengli06 committed Oct 28, 2023
    Configuration menu
    Copy the full SHA
    5d858be View commit details
    Browse the repository at this point in the history

Commits on Oct 29, 2023

  1. fix bug

    @chengmengli06
    chengmengli06 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    b5a143e View commit details
    Browse the repository at this point in the history

  2. add export sok embed

    @chengmengli06
    chengmengli06 committed Oct 29, 2023
    Configuration menu
    Copy the full SHA
    d0910e3 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. fix sok optimizer wrapper call _finish two times bug

    @chengmengli06
    chengmengli06 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    e844169 View commit details
    Browse the repository at this point in the history

  2. no need to switch non slot

    @chengmengli06
    chengmengli06 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    b60e3ba View commit details
    Browse the repository at this point in the history

  3. add support for lookup only

    @chengmengli06
    chengmengli06 committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    4124a16 View commit details
    Browse the repository at this point in the history

Commits on Oct 31, 2023

  1. remove debug code

    @chengmengli06
    chengmengli06 committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    40fd1a7 View commit details
    Browse the repository at this point in the history

  2. fix dynamic partition bug

    @chengmengli06
    chengmengli06 committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    592f7bd View commit details
    Browse the repository at this point in the history

  3. add load embed to whl package

    @chengmengli06
    chengmengli06 committed Oct 31, 2023
    Configuration menu
    Copy the full SHA
    a1909cb View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2023

  1. optimize segment_ids performance

    @chengmengli06
    chengmengli06 committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    dc3e45b View commit details
    Browse the repository at this point in the history

  2. add predictor files

    @chengmengli06
    chengmengli06 committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    528fa3b View commit details
    Browse the repository at this point in the history

  3. fix predict bug

    @chengmengli06
    chengmengli06 committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    34938de View commit details
    Browse the repository at this point in the history

  4. fix bug

    @chengmengli06
    chengmengli06 committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    03fb4d9 View commit details
    Browse the repository at this point in the history

Commits on Nov 2, 2023

  1. fix eval bug and dense grad bug

    @chengmengli06
    chengmengli06 committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    e3613f3 View commit details
    Browse the repository at this point in the history

  2. add support for dense lookup in distributed mode

    @chengmengli06
    chengmengli06 committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    3292707 View commit details
    Browse the repository at this point in the history

  3. refactor code style

    @chengmengli06
    chengmengli06 committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    bf0edb0 View commit details
    Browse the repository at this point in the history

  4. add support for finetune

    @chengmengli06
    chengmengli06 committed Nov 2, 2023
    Configuration menu
    Copy the full SHA
    76937dc View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2023

  1. add support for ALL_COLUMNS in parquet predictor

    @chengmengli06
    chengmengli06 committed Nov 6, 2023
    Configuration menu
    Copy the full SHA
    e3b91a5 View commit details
    Browse the repository at this point in the history

Commits on Nov 10, 2023

  1. fix predict bug

    @chengmengli06
    chengmengli06 committed Nov 10, 2023
    Configuration menu
    Copy the full SHA
    62ef27d View commit details
    Browse the repository at this point in the history

  2. fix numpy version

    @chengmengli06
    chengmengli06 committed Nov 10, 2023
    Configuration menu
    Copy the full SHA
    7c55ff7 View commit details
    Browse the repository at this point in the history

  3. make batch_size work for parquet predictor

    @chengmengli06
    chengmengli06 committed Nov 10, 2023
    Configuration menu
    Copy the full SHA
    77efe8a View commit details
    Browse the repository at this point in the history

  4. fix multiproc bug

    @chengmengli06
    chengmengli06 committed Nov 10, 2023
    Configuration menu
    Copy the full SHA
    2b34fda View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2023

  1. make read parquet fast

    @chengmengli06
    chengmengli06 committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    f606c01 View commit details
    Browse the repository at this point in the history

Commits on Nov 19, 2023

  1. fix dataset miss padding data bug

    @chengmengli06
    chengmengli06 committed Nov 19, 2023
    Configuration menu
    Copy the full SHA
    fab2dad View commit details
    Browse the repository at this point in the history

  2. add loss weight update

    @chengmengli06
    chengmengli06 committed Nov 19, 2023
    Configuration menu
    Copy the full SHA
    9f903e7 View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2023

  1. update load_embed to support large embedding tables

    @chengmengli06
    chengmengli06 committed Nov 21, 2023
    Configuration menu
    Copy the full SHA
    cb1a931 View commit details
    Browse the repository at this point in the history

Commits on Nov 22, 2023

  1. support old training path

    @chengmengli06
    chengmengli06 committed Nov 22, 2023
    Configuration menu
    Copy the full SHA
    3ac7494 View commit details
    Browse the repository at this point in the history

Commits on Nov 27, 2023

  1. support rebuild queue

    @chengmengli06
    chengmengli06 committed Nov 27, 2023
    Configuration menu
    Copy the full SHA
    4fd07b9 View commit details
    Browse the repository at this point in the history

Commits on Nov 28, 2023

  1. drop old code

    @chengmengli06
    chengmengli06 committed Nov 28, 2023
    Configuration menu
    Copy the full SHA
    9eab301 View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2023

  1. add fast auc calculate

    @chengmengli06
    chengmengli06 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    41df7ce View commit details
    Browse the repository at this point in the history

  2. add debug info

    @chengmengli06
    chengmengli06 committed Nov 29, 2023
    Configuration menu
    Copy the full SHA
    51b306c View commit details
    Browse the repository at this point in the history

Commits on Dec 8, 2023

  1. merge master and remove extra files

    @chengmengli06
    chengmengli06 committed Dec 8, 2023
    Configuration menu
    Copy the full SHA
    cf1159b View commit details
    Browse the repository at this point in the history

Commits on Dec 9, 2023

  1. merge master and remove extra files

    @chengmengli06
    chengmengli06 committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    23a379f View commit details
    Browse the repository at this point in the history

  2. add embedding parallel support

    @chengmengli06
    chengmengli06 committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    b15eec9 View commit details
    Browse the repository at this point in the history

  3. drop useless files

    @chengmengli06
    chengmengli06 committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    9a768fb View commit details
    Browse the repository at this point in the history

  4. update embedding parallel test

    @chengmengli06
    chengmengli06 committed Dec 9, 2023
    Configuration menu
    Copy the full SHA
    4a51066 View commit details
    Browse the repository at this point in the history

Commits on Dec 10, 2023

  1. remove nsys profile code

    @chengmengli06
    chengmengli06 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    e1a3670 View commit details
    Browse the repository at this point in the history

  2. remove debug code

    @chengmengli06
    chengmengli06 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    c349303 View commit details
    Browse the repository at this point in the history

  3. remove queue multiprocessing extra params

    @chengmengli06
    chengmengli06 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    9ac2f84 View commit details
    Browse the repository at this point in the history

  4. remove queue multiprocessing extra params

    @chengmengli06
    chengmengli06 committed Dec 10, 2023
    Configuration menu
    Copy the full SHA
    7e4b06b View commit details
    Browse the repository at this point in the history

Commits on Dec 11, 2023

  1. fix unit test bug

    @chengmengli06
    chengmengli06 committed Dec 11, 2023
    Configuration menu
    Copy the full SHA
    aab9cab View commit details
    Browse the repository at this point in the history

Commits on Dec 18, 2023

  1. fix bug

    @chengmengli06
    chengmengli06 committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    e7d94d4 View commit details
    Browse the repository at this point in the history

  2. fix bug

    @chengmengli06
    chengmengli06 committed Dec 18, 2023
    Configuration menu
    Copy the full SHA
    ce0eb4f View commit details
    Browse the repository at this point in the history

Commits on Dec 19, 2023

  1. fix ut bug

    @chengmengli06
    chengmengli06 committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    8ee2981 View commit details
    Browse the repository at this point in the history

  2. add support for non gpu case

    @chengmengli06
    chengmengli06 committed Dec 19, 2023
    Configuration menu
    Copy the full SHA
    c39c744 View commit details
    Browse the repository at this point in the history

Commits on Dec 22, 2023

  1. add custom op src and build scripts

    @chengmengli06
    chengmengli06 committed Dec 22, 2023
    Configuration menu
    Copy the full SHA
    cbb8596 View commit details
    Browse the repository at this point in the history

Commits on Dec 27, 2023

  1. remove useless code

    @chengmengli06
    chengmengli06 committed Dec 27, 2023
    Configuration menu
    Copy the full SHA
    cdf6ccc View commit details
    Browse the repository at this point in the history

Commits on Dec 29, 2023

  1. add support for scalar data when loading parquet_files

    @chengmengli06
    chengmengli06 committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    97b2f08 View commit details
    Browse the repository at this point in the history

  2. fix merge conflicts

    @chengmengli06
    chengmengli06 committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    091476b View commit details
    Browse the repository at this point in the history

  3. fix bug

    @chengmengli06
    chengmengli06 committed Dec 29, 2023
    Configuration menu
    Copy the full SHA
    c0813a9 View commit details
    Browse the repository at this point in the history

Commits on Jan 1, 2024

  1. add comments on parquet input

    @chengmengli06
    chengmengli06 committed Jan 1, 2024
    Configuration menu
    Copy the full SHA
    cc3f7d0 View commit details
    Browse the repository at this point in the history

Commits on Jan 2, 2024

  1. add test cases on embedding parallel and parquet dataset

    @chengmengli06
    chengmengli06 committed Jan 2, 2024
    Configuration menu
    Copy the full SHA
    6203f24 View commit details
    Browse the repository at this point in the history

Commits on Jan 3, 2024

  1. limit parquet case to run on python3 and tf2.x

    @chengmengli06
    chengmengli06 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    22d54cb View commit details
    Browse the repository at this point in the history

  2. limit parquet case to run on python3 and tf2.x

    @chengmengli06
    chengmengli06 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    b58e246 View commit details
    Browse the repository at this point in the history

  3. fix export typo

    @chengmengli06
    chengmengli06 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    1987ce7 View commit details
    Browse the repository at this point in the history

  4. fix export typo

    @chengmengli06
    chengmengli06 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    c10b951 View commit details
    Browse the repository at this point in the history

  5. reset init var settings, fix fast auc bug

    @chengmengli06
    chengmengli06 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    66e1ebd View commit details
    Browse the repository at this point in the history

  6. fix merge conflicts

    @chengmengli06
    chengmengli06 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    bb72631 View commit details
    Browse the repository at this point in the history

  7. add script for custom ops build

    @chengmengli06
    chengmengli06 committed Jan 3, 2024
    Configuration menu
    Copy the full SHA
    8de576e View commit details
    Browse the repository at this point in the history