An open source library released by Microsoft; called DeepSpeed, greatly advances model training for very large models consisting of more than 100 Billion Parameters.

Deep Speed improves four aspects:

• Scale: up to 10x bigger
• Speed: Up to 5x faster
• Cost: 5x reduction in cost
• Usability: Low code impact.

Please follow the link at the Microsoft Research Blog.