distributed-llm-pretraining-torchtitan - AI Research Skills
by Orchestra Research
Cross-platformSkillskillai-researchAI-Research-SKILLs-main
Description
Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.
Type
Skill
Ecosystem
Cross-platform
Trust Score
85%
Related Skills
Stable Diffusion WebUI
Feature-rich web interface for Stable Diffusion image generation by AUTOMATIC1111.
Cross-platformai-tool
LangChain
Comprehensive framework for building LLM-powered applications with chains, agents, and retrieval.
Cross-platformframework
LobeChat
Modern, extensible AI chat framework with plugin ecosystem and multi-model support.
Cross-platformopen-source
Open WebUI
Self-hosted web UI for LLMs with multi-model support, RAG, and plugin system.
Cross-platformopen-source