Reproducible FP32‑to‑INT8 quantization and benchmark for MiniLM on Arm64
ArmBench MiniLM lets you turn a pinned FP32 sentence‑embedding model into an INT8 version and instantly benchmark it on native Arm64 hardware. It quantizes the model with ONNX Runtime, measures latency, size, and embedding fidelity, and outputs JSON, Markdown, and HTML reports. The tool is useful for developers deploying NLP inference services on CPUs where speed and memory matter. Compared to generic quantization scripts, it provides a fully auditable, reproducible workflow with built‑in validation.
View on GitHub →yhay81/armbench-minilm