BenchPRO - Benchmark Performance & Reproducibility Orchestrator
BenchPRO is a Python based framework that simplifies and standarizes the process of building applications, executing benchmarks and collection performace results on HPC systems.
This project is under active development.
BenchPRO is a benchmarking framework that enforces a standardized approach to compiling and running benchmarks. Additionally, the framework automatically records significant provenance and performance data for reference. The framework allows domain experts to share their well optimized applications and representative benchmark datasets in a reproducible manner. This way, an individual with a limited scientific or application background can run benchmarks through the framework, compare performance to previous results and examine provenance data to help identify the root cause of any discrepancies. This framework significantly enhances the reproducibility of benchmarking efforts and reduces the labor required to maintain a benchmark suite. The utility was designed to meet the following set of goals:
Automate the process of building applications, running benchmarks and storing result data.
Structure the framework to promote the standardization of techniques and workflows as a step towards improving benchmark reproducibility.
Accommodate a number of benchmarking activities like comparative performance assessments, regression testing, and scalability studies.
Store as much provenance data as possible for future reference.
Provide an intuitive way of exploring and comparing benchmark results.
- New User Guide
- Adding an Application Profile
- Adding a Benchmark Profile
- Available test cases