HaiScale Distributed Data Parallel (DDP): Parallel training library that implements numerous types of parallelism similar to Data Parallelism (DP), Pipeline Parallelism (PP), Tensor Parallelism (TP), Experts Parallelism (EP), Fully Sharded Data Parallel (FSDP) and Zero Redundancy Optimizer (ZeRO). In 2023, in-nation access was blocked to Hugging Face, an organization that maintains libraries containing training knowledge sets commonly used for large language models.