PyTorch LoHA adapter for LLM fine‑tuning with CLI and full pipeline
The project implements the Low‑Rank Hadamard Product (LoHA) adapter from scratch in pure PyTorch, letting users fine‑tune large language models without relying on external PEFT libraries. It provides a functional CLI, Python API, and scripts for training, evaluation, merging, and serving, plus extensive documentation and tests. Targeted at ML engineers and researchers who need a lightweight, customizable fine‑tuning solution, it offers a clear alternative to LoRA with comparable performance and more rank flexibility. Its end‑to‑end design and self‑contained implementation make it a practical, ready‑to‑use tool in the LLM fine‑tuning ecosystem.
View on GitHub →zeta4lab/lora-loha-finetuning