Huggingface Integration

vllmPython

A high-throughput, memory-efficient inference and serving engine for large language models (LLMs).

sglangPython

A high-performance serving framework for large language models and multimodal models, delivering low-latency and high-throughput inference.

shainetCrystal

A pure Crystal machine learning library for building and training neural networks with CPU/GPU support and PyTorch compatibility.