AIPerf

AIPerf is an end-to-end benchmark suite utilizing automated machine learning (AutoML). It represents real AI scenarios, and scales auto-adaptively to various scales of machines.

The automl pipeline (including NAS, HPO, etc.) is implemented in a highly parallel and flexible way to ensure the efficiency and optimization potential on diverse systems with customizable configurations. We utilize operations per second (OPS), which is measured in an analytical and systematic approach, as the major metric to quantify the AI performance.With flexible workload and single metric, our benchmark can scale and rank AI-HPC easily.