Awesome LLM Frameworks
收集 LLM 全链路有关的框架和工具,供自己学习和参考
[TOC]
训练
- OpenRL
- alibaba/ROLL
- DeepSpeed
- 微调
- VLA
调度
Ckpt
KV 管理
推理
- vLLM
- SGlang
- TensorRT-LLM
- LMdeploy
- ktransformers
- ggml-org/llama.cpp
- thu-pacman/chitu
- alibaba/rtp-llm
- Nano vLLM
- accelerate
- PD-disaggregation
- Speculative-Decoding
算子与通信
- Triton
- NIXL
- RDMA
- NVLimk
- PCIE
- Ascend HIXL
可观测
云原生
基础设施
- Hardware
- CPU
- GPU NVIDIA
- NPU
- Ascend
- Cloud
- Aliyun
- huoshan
- tencent
- AWS