17 Pull requests merged by 9 people

• [shardformer] fix import

  #5788 merged Jun 6, 2024

• [misc] update requirements

  #5787 merged Jun 6, 2024

• [install]fix setup

  #5786 merged Jun 6, 2024

• [misc] fix dist logger

  #5782 merged Jun 5, 2024

• Allow building cuda extension without a device.

  #5535 merged Jun 5, 2024

• [gemini] optimize reduce scatter d2h copy

  #5760 merged Jun 5, 2024

• [hotfix] fix testcase in test_fx/test_tracer

  #5779 merged Jun 5, 2024

• [Test/CI] remove test cases to reduce CI duration

  #5753 merged Jun 5, 2024

• [misc] Accelerate CI for zero and dist optim

  #5758 merged Jun 5, 2024

• [hotfix] fix llama flash attention forward.

  #5777 merged Jun 5, 2024

• [Inference]Add Streaming LLM

  #5745 merged Jun 5, 2024

• [devops] fix docker ci

  #5780 merged Jun 4, 2024

• [misc] update dockerfile

  #5776 merged Jun 4, 2024

• [CI/tests] simplify some test case to reduce testing time

  #5755 merged Jun 4, 2024

• [Hotfix] Add missing init file in inference.executor

  #5774 merged Jun 3, 2024

• Fix/fix testcase

  #5770 merged Jun 3, 2024

• [shardformre] fix llama policy

  #5765 merged Jun 3, 2024

7 Pull requests opened by 6 people

• [Inference] Refactor inference modeling

  #5771 opened Jun 3, 2024

• Multi round dialogue branch

  #5772 opened Jun 3, 2024

• [Feauture] MoE refactor

  #5775 opened Jun 4, 2024

• [Gemini] Use async stream to prefetch and h2d data moving

  #5781 opened Jun 5, 2024

• [Inference]Lazy Init Support

  #5785 opened Jun 6, 2024

• Support 4D parallel + Flash Attention

  #5789 opened Jun 6, 2024

• fix Llama rotary embedding api change for transformers 4.39.3

  #5790 opened Jun 7, 2024

4 Issues closed by 2 people

• [PROPOSAL]: Refactor inference engine by selecting backend during init of modules

  #5773 closed Jun 7, 2024

• [BUG]: Cannot build extensions when no gpu device exists

  #5534 closed Jun 5, 2024

• [BUG]:Report some errors in test_fx/test_tracer

  #5778 closed Jun 5, 2024

• [BUG]: Report some errors in test

  #5768 closed Jun 3, 2024

7 Unresolved conversations

Sometimes conversations happen on old items that aren’t yet closed. Here is a list of all the Issues and Pull Requests with unresolved conversations.

• [Feature] optimize PP overlap

  #5735 commented on Jun 7, 2024 • 12 new comments

• [ColossalChat] Colossalchat upgrade

  #5759 commented on Jun 7, 2024 • 6 new comments

• [FEATURE]: SP with FlashAttention

  #5762 commented on Jun 3, 2024 • 1 new comment

• [BUG]: docker build cuda extension error

  #5732 commented on Jun 6, 2024 • 1 new comment

• [lora] support lora for Gemini

  #5001 commented on Jun 1, 2024 • 1 new comment

• [WIP][Infer] Inference Distributed RPC Framework Optimization

  #5756 commented on Jun 5, 2024 • 0 new comments

• [moe/zero] refactor low level optimizer

  #5767 commented on Jun 7, 2024 • 0 new comments