An open-source pipeline for training medical domain GPT models using PT, SFT, RLHF, DPO, ORPO, and GRPO methods.
MedicalGPT is an open-source training pipeline for developing medical domain large language models. It implements the full ChatGPT training methodology—including incremental pretraining, supervised fine-tuning, RLHF, DPO, ORPO, and GRPO—to create specialized models that understand and generate medical text. The project solves the problem of adapting general-purpose LLMs to the nuanced, high-stakes domain of healthcare.
AI researchers, machine learning engineers, and healthcare technology developers who need to train or fine-tune LLMs for medical applications such as clinical dialogue, medical QA, and health informatics.
Developers choose MedicalGPT because it provides a complete, production-ready pipeline for medical LLM training with support for the latest alignment techniques (DPO, ORPO, GRPO), extensive model compatibility, and practical tooling for deployment—all in a single open-source repository.
MedicalGPT: Training Your Own Medical GPT Model with ChatGPT Training Pipeline. 训练医疗大模型,实现了包括增量预训练(PT)、有监督微调(SFT)、RLHF、DPO、ORPO、GRPO。
Open-Awesome is built by the community, for the community. Submit a project, suggest an awesome list, or help improve the catalog on GitHub.
Supports a wide range of open-source LLMs like LLaMA, Qwen, and Baichuan, with frequent updates for new architectures such as Qwen3.5 and Llama-3, ensuring flexibility.
Implements the full ChatGPT methodology, including PT, SFT, RLHF, DPO, ORPO, and GRPO, providing end-to-end tools for medical LLM development from scratch.
Curates and links to medical datasets like shibing624/medical and general ones like sharegpt_gpt4, simplifying data preparation for domain adaptation.
Includes Gradio for demos, FastAPI for servers, and vLLM for multi-GPU inference, plus tools for model merging and quantization to ease production use.
VRAM requirements are high (e.g., 60GB+ for full-parameter 7B models), making it inaccessible for teams without substantial GPU resources, as detailed in the hardware table.
Involves managing multiple shell scripts (e.g., run_sft.sh) and DeepSpeed settings, which can be daunting and error-prone for users unfamiliar with distributed training.
Documentation is split between Chinese and English wikis, potentially leading to inconsistencies or missing details, especially for non-Chinese speakers.