Discussions

• 1

  🙏

  Saving checkpoints in deepspeed

  base-y asked May 1, 2022 in Q&A · Unanswered

  

  

  1
• 1

  🙏

  ZeRO and model parallelism

  base-y asked Apr 25, 2022 in Q&A · Unanswered

  

  

  9
• 1

  🙏

  Can PLD technology be used on the Megatron-LM?

  chinoll asked May 4, 2022 in Q&A · Unanswered

  

  

  2
• 1

  🙏

  Optimizing efficiency using deepspeed

  base-y asked May 6, 2022 in Q&A · Answered

  

  

  1
• 1

  💬

  Difference with torch.distributed

  base-y started Apr 27, 2022 in General

  

  

  2
• 1

  🙏

  Difference between allgather and and allreduce

  base-y asked Apr 25, 2022 in Q&A · Answered

  

  

  1
• 1

  💡

  resume training with different numbers of nodes.

  tangjiasheng started Apr 18, 2022 in Ideas

  

  

  1
• 1

  🙏

  What is is right way to share weights between multiple stages in pipeline parallelism?

  zarzen asked Mar 22, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  Any suggestion about gradient accumulation and BatchNorm?

  tangjiasheng asked Mar 21, 2022 in Q&A · Unanswered

  

  

  0
• 1

  💬

  autotuning failed to run: pdsh@GCRSANDBOX324: localhost: ssh exited with exit code 255

  dunalduck0 started Feb 24, 2022 in General

  

  

  1
• 2

  🙏

  CUDA error with INT 8 inference

  gsujankumar asked Feb 23, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  About Torch Extension ERROR.

  chenjunyu66 asked Feb 21, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  Model checkpointing in training

  zarzen asked Feb 9, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  Fastai with Deepspeed

  vishalghor asked Jan 31, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  Experience with Triton

  sarvghotra asked Jan 13, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  torch.roll in DeepSpeed

  sarvghotra asked Jan 12, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  Positional Embedding in DeepSpeed Transformer Kernel

  sarvghotra asked Jan 12, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  Cannot see effectiveness of Zero 3. Any help is much appreciated.

  hpourmodheji asked Jan 6, 2022 in Q&A · Unanswered

  

  

  5
• 1

  🙏

  Overriding DeepSpeed autocasting

  gahdritz asked Jan 10, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  Same code snippet are written twice

  Neleon asked Jan 6, 2022 in Q&A · Unanswered

  

  

  0
• 1

  🙏

  Is there anything wrong with this usage?

  Neleon asked Jan 5, 2022 in Q&A · Answered

  

  

  1
• 1

  🙏

  resume checkpoint w/o deepspeed

  tangjiasheng asked Dec 27, 2021 in Q&A · Unanswered

  

  

  2
• 3

  🙏

  DeepSpeed inference for online serving

  vdantu asked Jan 3, 2022 in Q&A · Unanswered

  

  

  1
• 1

  🙏

  Clarification on 1-bit Adam speed comparison for SQuAD

  sarvghotra asked Dec 18, 2021 in Q&A · Unanswered

  

  

  1
• 1

  💬

  Cannot See Effective Memory Reduction with ZeRO 3

  hpourmodheji started Dec 17, 2021 in General

  

  

  0