Code

Torch dp support (WIP)#3207

Code

Draft

grimoire wants to merge 89 commits intoInternLM:mainInternLM/lmdeploy:mainfrom grimoire:torch-dp-supportgrimoire/lmdeploy:torch-dp-supportCopy head branch name to clipboard

+1,357-694

Commits

Commits on Jan 15, 2025

• better dist context

  grimoire

  committed
  0d006e2

  Copy full SHA for 0d006e2

Commits on Jan 16, 2025

• can not exit

  grimoire

  committed
  e5baae6

  Copy full SHA for e5baae6

• Merge branch 'main' into torch-multinode

  grimoire

  committed
  e561e6d

  Copy full SHA for e561e6d

Commits on Jan 17, 2025

• multinode support

  grimoire

  committed
  8a61faf

  Copy full SHA for 8a61faf

• better exception

  grimoire

  committed
  ae7a742

  Copy full SHA for ae7a742

Commits on Jan 21, 2025

• Merge branch 'main' into torch-multinode

  grimoire

  committed
  2f75ee9

  Copy full SHA for 2f75ee9

• merge main

  grimoire

  committed
  709d293

  Copy full SHA for 709d293

Commits on Jan 22, 2025

• refactor

  grimoire

  committed
  cefbf98

  Copy full SHA for cefbf98

Commits on Jan 23, 2025

• fix local rank

  grimoire

  committed
  ada2cc3

  Copy full SHA for ada2cc3

Commits on Jan 24, 2025

• replace group

  grimoire

  committed
  713aad5

  Copy full SHA for 713aad5

Commits on Feb 6, 2025

• merge main

  grimoire

  committed
  2ec93c8

  Copy full SHA for 2ec93c8

• fix dist

  grimoire

  committed
  374b52a

  Copy full SHA for 374b52a

• remove useless code

  grimoire

  committed
  727ea86

  Copy full SHA for 727ea86

Commits on Feb 7, 2025

• remove finish flag

  grimoire

  committed
  b454c39

  Copy full SHA for b454c39

Commits on Feb 10, 2025

• Merge branch 'main' into torch-multinode

  grimoire

  committed
  3baab2d

  Copy full SHA for 3baab2d

• refactor engine and model agent

  grimoire

  committed
  742af3b

  Copy full SHA for 742af3b

• uni executor

  grimoire

  committed
  420ab0f

  Copy full SHA for 420ab0f

Commits on Feb 11, 2025

• wip

  grimoire

  committed
  e2b82a7

  Copy full SHA for e2b82a7

• tp

  grimoire

  committed
  40251d5

  Copy full SHA for 40251d5

• fix

  grimoire

  committed
  300263e

  Copy full SHA for 300263e

Commits on Feb 12, 2025

• less async

  grimoire

  committed
  ec10731

  Copy full SHA for ec10731

• Merge branch 'main' into torch-multinode-v2

  grimoire

  committed
  2ee3ca8

  Copy full SHA for 2ee3ca8

• circle buf

  grimoire

  committed
  ece1313

  Copy full SHA for ece1313

• event per block

  grimoire

  committed
  911d9ab

  Copy full SHA for 911d9ab

• fast mp

  grimoire

  committed
  3b6fa54

  Copy full SHA for 3b6fa54

• fix error handler

  grimoire

  committed
  a560cd9

  Copy full SHA for a560cd9

• remove safe wait

  grimoire

  committed
  6071e55

  Copy full SHA for 6071e55

• context in model agent

  grimoire

  committed
  13d1187

  Copy full SHA for 13d1187

Commits on Feb 13, 2025

• fix on stop

  grimoire

  committed
  894edb8

  Copy full SHA for 894edb8

• check before init

  grimoire

  committed
  1b8d35e

  Copy full SHA for 1b8d35e

• support close

  grimoire

  committed
  6db53ab

  Copy full SHA for 6db53ab

Commits on Feb 14, 2025

• fix tp close

  grimoire

  committed
  c3bf202

  Copy full SHA for c3bf202

• ray ver0

  grimoire

  committed
  7ab7ff3

  Copy full SHA for 7ab7ff3

Commits on Feb 15, 2025

• fix close

  grimoire

  committed
  8bfd5cb

  Copy full SHA for 8bfd5cb

Commits on Feb 16, 2025

• fix remote code

  grimoire

  committed
  5596b53

  Copy full SHA for 5596b53

• optimize ray

  grimoire

  committed
  343eb78

  Copy full SHA for 343eb78

• better checker and logger

  grimoire

  committed
  f3444ce

  Copy full SHA for f3444ce

• pack tensor

  grimoire

  committed
  a35dc75

  Copy full SHA for a35dc75

Commits on Feb 17, 2025

• auto check dist

  grimoire

  committed
  f47996a

  Copy full SHA for f47996a

• fix mp gloo

  grimoire

  committed
  da27d28

  Copy full SHA for da27d28

• Merge branch 'main' into torch-multinode-v2

  grimoire

  committed
  fc02b67

  Copy full SHA for fc02b67

• add timer tools

  grimoire

  committed
  5366f5f

  Copy full SHA for 5366f5f

Commits on Feb 18, 2025

• better scheduler

  grimoire

  committed
  230228d

  Copy full SHA for 230228d

• fix mp hang

  grimoire

  committed
  14bb0ae

  Copy full SHA for 14bb0ae

• fix mp

  grimoire

  committed
  f66510a

  Copy full SHA for f66510a

• fix chat

  grimoire

  committed
  8ee9e86

  Copy full SHA for 8ee9e86

Commits on Feb 19, 2025

• less output

  grimoire

  committed
  f293e64

  Copy full SHA for f293e64

• Merge branch 'main' into torch-multinode-v2

  grimoire

  committed
  f41bc0b

  Copy full SHA for f41bc0b

• merge main

  grimoire

  committed
  b74e649

  Copy full SHA for b74e649

• optimize ray get output

  grimoire

  committed
  e8d1606

  Copy full SHA for e8d1606

• remove nsight runtime env

  grimoire

  committed
  4ce9343

  Copy full SHA for 4ce9343

• dag

  grimoire

  committed
  fbdffd2

  Copy full SHA for fbdffd2

Commits on Feb 20, 2025

• optimize mp & lint

  grimoire

  committed
  5871c27

  Copy full SHA for 5871c27

• merge main

  grimoire

  committed
  5c46f97

  Copy full SHA for 5c46f97

• optimize mp

  grimoire

  committed
  adfa3f3

  Copy full SHA for adfa3f3

Commits on Feb 21, 2025

• add base workerwrapper

  grimoire

  committed
  0f63336

  Copy full SHA for 0f63336

• fix gather, update flags

  grimoire

  committed
  ff64f11

  Copy full SHA for ff64f11

• better return mask

  grimoire

  committed
  4ba2561

  Copy full SHA for 4ba2561

• add choice

  grimoire

  committed
  026ebe4

  Copy full SHA for 026ebe4

• enable mp,ray with worldsize=1

  grimoire

  committed
  a4639e9

  Copy full SHA for a4639e9

• fix mp exit

  grimoire

  committed
  c76dcfc

  Copy full SHA for c76dcfc

• fix mp vlm

  grimoire

  committed
  c5661ed

  Copy full SHA for c5661ed

• chat exit

  grimoire

  committed
  af83ff2

  Copy full SHA for af83ff2

• add docs

  grimoire

  committed
  f1a8a08

  Copy full SHA for f1a8a08

Commits on Feb 24, 2025

• lint

  grimoire

  committed
  730c9e9

  Copy full SHA for 730c9e9

• doc

  grimoire

  committed
  ef2811b

  Copy full SHA for ef2811b

• dp check

  grimoire

  committed
  82f0f21

  Copy full SHA for 82f0f21

• fix blocked fp8 moe

  grimoire

  committed
  d28d690

  Copy full SHA for d28d690

• remove mask

  grimoire

  committed
  95b2249

  Copy full SHA for 95b2249

Commits on Feb 25, 2025

• support dp, async

  grimoire

  committed
  efcb5df

  Copy full SHA for efcb5df

• remove debug line

  grimoire

  committed
  d34145d

  Copy full SHA for d34145d

• fix model tp

  grimoire

  committed
  8960d29

  Copy full SHA for 8960d29

Commits on Feb 27, 2025

• Merge branch 'main' into torch-multinode-v2

  grimoire

  committed
  411d12e

  Copy full SHA for 411d12e

• support sync execute

  grimoire

  committed
  3481f7a

  Copy full SHA for 3481f7a

• Merge branch 'torch-multinode-v2' into torch-dp-support

  grimoire

  committed
  8f8e708

  Copy full SHA for 8f8e708

Commits on Feb 28, 2025

• fix chat stopwords

  grimoire

  committed
  3647cae

  Copy full SHA for 3647cae

• refactor chat

  grimoire

  committed
  e371647

  Copy full SHA for e371647

Commits on Mar 3, 2025

• merge main

  grimoire

  committed
  fae62ad

  Copy full SHA for fae62ad

• add warmup

  grimoire

  committed
  ade7bce

  Copy full SHA for ade7bce

• Merge branch 'torch-multinode-v2' into torch-dp-support

  grimoire

  committed
  8d65226

  Copy full SHA for 8d65226

Commits on Mar 4, 2025

• merge main

  grimoire

  committed
  49e7f4c

  Copy full SHA for 49e7f4c

• disable warmup

  grimoire

  committed
  e4339e7

  Copy full SHA for e4339e7

Commits on Mar 6, 2025

• dp support

  grimoire

  committed
  f7515e4

  Copy full SHA for f7515e4

• Merge branch 'main' into torch-dp-support

  grimoire

  committed
  1f10133

  Copy full SHA for 1f10133

• fix ut, merge main, force eager

  grimoire

  committed
  32cd198

  Copy full SHA for 32cd198

• support qwen2/internlm2/internlm3

  grimoire

  committed
  d4a42ee

  Copy full SHA for d4a42ee

Commits on Mar 9, 2025

• merge main

  grimoire

  committed
  db8b965

  Copy full SHA for db8b965

• support blocked fp8 all gather

  grimoire

  committed
  4ab0ce2

  Copy full SHA for 4ab0ce2

• add more model support

  grimoire

  committed
  c0cd790

  Copy full SHA for c0cd790