Add a StatisticsManager for averaging the results of repeated profilings.#7
Add a StatisticsManager for averaging the results of repeated profilings.#7
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request adds a StatisticsManager class to support repeated profiling and averaging of results for more stable benchmarking. The PR also adds CLI support via a new stats command and the --repeat flag on the profile command.
Changes:
- Added
StatisticsManagerclass for computing averages and standard deviations of profiling results - Added
statsCLI command for post-processing multiple profile JSON files - Added
--repeat,--in-place,--quiet, and--output-directoryflags to theprofilecommand - Updated the
coloursdependency (v0.1.1 → v0.2.0) and addedgitpython(v3.1.46) - Added git metadata tracking for benchmarking (repo, branch, commit, dirty files, UTC timestamp)
- Updated various
Colour.print()calls to use more specific logging methods (error,warning,info) - Added
walltime,user_time,system_time,total_timefields toProfileDetails - Modified profiler initialization to support
in_placeand customoutput_dirparameters - Changed log file name from
logs.logtoprofile_logs.log
Reviewed changes
Copilot reviewed 19 out of 20 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| uv.lock | Updated colours dependency and added gitpython with dependencies |
| pyproject.toml | Updated version to 0.3.0, added gitpython dependency, updated colours revision |
| fixingahole/profiler/stats_manager.py | New file implementing statistics aggregation and metadata tracking |
| fixingahole/profiler/profiler.py | Added support for repeated runs, in-place profiling, custom output directories |
| fixingahole/profiler/profile_summary.py | Added walltime to ProfileDetails, added time calculation properties |
| fixingahole/profiler/utils.py | Added subfolder_only parameter to find_path, updated error handling |
| fixingahole/cli/main.py | Added stats command, added new CLI flags for profile command |
| fixingahole/init.py | Exported StatisticsManager class |
| fixingahole/config.py | Updated error handling to use new Colour methods |
| tests/* | Comprehensive test coverage for new features, updated existing tests |
| README.md | Documented new features and benchmarking workflow |
| .pre-commit-config.yaml | Added end-of-file-fixer and trailing-whitespace hooks |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 20 out of 21 changed files in this pull request and generated 6 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 23 out of 24 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Sometimes profiling results differ slightly between runs from noise or other factors. These changes support more consistent and stable benchmarking endeavours by adding conveniences for profiling scripts repeatedly and averaging over the results.
We add a new
StatisticsManagerclass to accomplish this internally and provide CLI access via thestatscommand.We also refactor the config to be more flexible and robust.