Cisco Training Courses

Insoft has been serving IT community with official Cisco training offering since 2010. Find all the relevant information on Cisco training on this page.

View More

Cisco Certifications

Experience a blended learning approach that combines the best of instructor-led training and self-paced e-learning to help you prepare for your certification exam.

View More

Cisco Learning Credits

Cisco Learning Credits (CLCs) are prepaid training vouchers redeemed directly with Cisco that make planning for your success easier when purchasing Cisco products and services.

Have CLCs and want to redeem them?

Cisco Continuing Education

The Cisco Continuing Education Program offers all active certification holders flexible options to recertify by completing a variety of eligible training items.

View More

Cisco Digital Learning

Certified employees are VALUED assets. Explore Cisco official Digital Learning Library to educate yourself through recorded sessions.

Browse CDLL Catalogue

Cisco Business Enablement

The Cisco Business Enablement Partner Program focuses on sharpening the business skills of Cisco Channel Partners and customers.

View More

Fortinet Technical Certifications

The Fortinet Network Security Expert (NSE) program is an eight-level training and certification program to teach engineers of their network security for Fortinet FW skills and experience.

View More

Fortinet Technical Courses

Insoft is recognised as Fortinet Authorized Training Center in selected locations across EMEA.

View More

Official ATC Status

Check our ATC Status across selected countries in Europe.

View More

Fortinet Services Packages

Insoft Services has developed a specific solution to streamline and simplify the process of installing or migrating to Fortinet Products.

Browse Packages

Prepforce Bootcamp

The only comprehensive source available today to prepare for Fortinet NSE 8 certification globally.

View More

Microsoft Training

Insoft Services provides Microsoft training in EMEAR. We offer Microsoft technical training and certification courses that are led by world-class instructors.

View More

Technical Training

The evolution of Extreme Networks Technical Training provides a comprehensive progressive pathway from Associate to Professional accreditation.

View More

ATP Accreditation

As an authorised training partner (ATP), Insoft Services ensures that you receive the highest standards of education available.

View More

 

In a world where technologies are evolving rapidly, every company - business needs a partner to rely on and trust for the smooth and secure operation of its network infrastructure.

View More

 

Our Mission: Provide an expert set of modern & leading edge Network Automation skills to the market through professional services.

View More

 

In a world where technologies are evolving rapidly, every company - business needs a partner to rely on and trust for the smooth and secure operation of its network infrastructure.

View More

 

In a world where technologies are evolving rapidly, every company - business needs a partner to rely on and trust for the smooth and secure operation of its network infrastructure.

View More

 

In a world where technologies are evolving rapidly, every company - business needs a partner to rely on and trust for the smooth and secure operation of its network infrastructure.

View More

 

In a world where technologies are evolving rapidly, every company - business needs a partner to rely on and trust for the smooth and secure operation of its network infrastructure.

View More

 

We help organisations to deploy Software-Defined Networking (SDN) solutions, such as Cisco DNA. Besides, our team has extensive experience in integrating Cisco DNA Center with third-party systems.

View More

 

In a world where technologies are evolving rapidly, every company - business needs a partner to rely on and trust for the smooth and secure operation of its network infrastructure.

View More

About Us

Our training portfolio includes a wide range of IT training from IP providers, including Cisco, Extreme Networks, Fortinet, Microsoft, to name a few, in EMEA.

View More

Azure-powered hyperscale model training on DeepSpeed

Azure-powered hyperscale model training on DeepSpeed

08 December 2022

In recent years, many new products and features have been developed as a result of large-scale deep learning models being trained on large data sets. There is an increasing demand on an unprecedented scale to train and optimize even more complex models.

Microsoft already trains some of the largest models including the 530 billion-parameter Megatron-Turing (MT-NLG 530B) model and GPT-3 on Azure. Usually, it requires several manual, error-prone steps that users need to set up in order to train these models.

However, owing to a new tech stack breakthrough, we can now use DeepSpeed with Azure, in straightforward training pipelines that make use of either the suggested AzureML recipes or bash scripts for VMSS-based settings.

Why DeepSpeed 

DeepSpeed is a PyTorch deep learning optimization toolkit. The library is intended to reduce computational power and memory consumption while training large distributed models with improved parallelism on existing computer hardware. 

Designed for training with low latency and high throughput, DeepSpeed features the Zero Redundancy Optimizer (ZeRO), which is useful for training models with one trillion or more parameters.

Full stack optimization

A full stack optimization approach is employed by Microsoft to achieve excellent performance and scalability avoiding unnecessary complexity. In other words, all necessary components have been optimized, integrated, and thoroughly tested, including the hardware, the OS, the VM image, the Docker image (containing optimized Python packages such as PyTorch, DeepSpeed, and ONNX Runtime), and the Azure ML APIs for user interaction.

Large models may be trained well at scale with DeepSpeed on Azure, thanks to this newly improved architecture. 

​​To handle extensive AI training, Azure Machine Learning (AzureML) deploys sizable fleets of the newest GPUs propelled by the InfiniBand network. The upgraded tech stack increased performance outcomes, scalable to twice as many GPUs (1024 vs. 512), twice as large model sizes, and up to 1.8 times greater compute throughput per GPU (150 TFLOPs vs. 81 TFLOPs).

The Microsoft engineers were able to maintain an efficient throughput/GPU (>157 TFLOPs) in a near-linear fashion as the model size increased from 175 billion parameters to 2 trillion parameters using the DeepSpeed ZeRO-3, its novel CPU offloading capabilities, and a high-performance Azure stack powered by InfiniBand interconnects and A100 GPUs.

Key Takeaway

By effectively breaking the GPU memory barrier, Azure and DeepSpeed enable the users to quickly and effectively train trillion-parameter models at scale. From the evidence, this alliance can easily be said as the future of hyperscale model training.

Tags:
Subject Matter Expert
Insoft Services
Martin is an experienced IT consultant with 15+ years of experience from working at world leading companies like HP, IBM and Microsoft. He was involved as a consultant and subject matter expert in numerous Cloud migration and implementation projects in the banking and public sector. Martin Graduated as an Information Science engineer but also holds a Masters degree in Intelectual Property Law and is a Microsoft Certified Trainer.

  • Recent Blogs

  • Relevant Blogs

  • No Comments

    Comments are closed.