Gpu Optimization

vllmPython

A high-throughput, memory-efficient inference and serving engine for large language models (LLMs).

Colossal-AI - An Integrated Large-scale Model Training System with Efficient Parallelization TechniquesPython

A unified deep learning system for efficient large-scale model training and inference with advanced parallelism strategies.

triton-inference-serverPython

An open-source inference serving platform for deploying AI models from multiple frameworks across cloud, data center, and edge devices.

BigDLPython

An LLM acceleration library for Intel XPU (GPU, NPU, CPU) to speed up local inference and finetuning of popular models.

BentoMLPython

A Python library for building production-ready model inference APIs, job queues, and multi-model serving systems for AI applications.

Vulkan Best Practice for Mobile Developers TutorialsC++

A collection of best practices and guidelines for optimizing Vulkan applications on mobile devices with Arm GPUs.

TResNet: High Performance GPU-Dedicated ArchitecturePython

A PyTorch implementation of TResNet, a high-performance convolutional neural network architecture optimized for GPU training and inference.