Fix timezone inconsistencies in data shaping

[grgmiller]

The pipeline was raising warnings that certain plant-level and BA-level outputs had slightly more or less than 8760 timestamps in a year. It turns out that this was caused by a pervasive bug related to timezone mismatches in various hourly shaping and data aggregation processes.

Closes #326
Closes DAT-521

Plant-level data issue: Monthly EIA-923 data was being shaped to hourly using the hourly fleet profile for the appropriate fleet in the BA. However, the BA-fleet profiles used the BA's official reporting timezone: in the case that a plant was not located the same timezone as the official plant BA, this meant that the plant data was getting shifted an hour relative to its local time, which tended to drop an hour of data from the year. Example: SOCO spans both Eastern and Central time, but reports in Central, so a plant located in eastern time would have the wrong profile assigned to it.

• Fix: For each BA, we now create profiles for all timezones with plants in them for that BA, and match the correct local timezone profile to each plant when doing shaping. To deal with year edge hours, we just ffill/bfill the adjacent hour to ensure we have a complete profile.
• Result: plant output files may have plant timeseries with multiple UTC year ranges in them, rather than having all plant timeseries match the BA's official timezone.

Power-sector data issue: Again, because a BA can have plants from multiple time zones in it, when aggregating all the plant data to the fleet level, we end up with timeseries with more than 8760 hours, because it spans 8760 hours in two timezones.

• Fix: filter the aggregated results to the BA's official reporting timezone to keep 8760 hours.
• Result: edge hours may be missing data for certain plants.

National-average wind/solar imputation: When calculating profiles to shape wind and solar data, we first try to use that BA's profile; if that doesn't exist, we take an average profile from all neighboring BAs; if that doesn't exist, we take a national average. Because this data spans many timezones, our approach has been to create a tz-naive local time, then average together - this best reflects the same profile in each timezones' relative local time to get a pattern. However, there were two bugs in this process: 1) we didn't handle the DST transition, so there were some data gaps, and also a discontinous jump in profile at the boundary and 2) we were calculating these profiles on a per-month basis, resulting in some month-edge hourly data issues.

• The fix: Calculate the national average once for the whole year, then slice that as needed for each month, and when calculating the profile, first convert local prevailing time to local standard time before making it tz-naive.

Testing

Used Claude to write a number of tests related to these bugs.

Where to look

• Plant-level: src/oge/impute_hourly_profiles.py — expand_hourly_profiles_to_plant_timezones(), plus the merge-key changes in shape_monthly_eia_data_as_hourly() and combine_and_export_hourly_plant_data().
• Plant-level completion: src/oge/data_cleaning.py — complete_hourly_timeseries(). The diff reads larger than it is; most of it is the same logic un-indented once the count-based early return is gone.
• Power-sector: src/oge/data_cleaning.py's new filter_to_ba_local_year(), called from src/oge/data_pipeline.py right before write_power_sector_results.
• National-average imputation: src/oge/impute_hourly_profiles.py — calculate_national_average_wind_solar_profiles(), get_national_average_profile_for_fuel_month(), and the corresponding updates to average_diba_wind_solar_profiles() and the three validate_* functions.
• Shared helper: src/oge/helpers.py — create_local_year_timestamps().
• Docs: Shaping Using Fleet-Specific Profiles.md, updated to describe the filter-after-aggregation approach.
  filepaths.py and load_data.py have two trivial ruff format line-wraps — no logic changes, safe to skip.

Review estimate

30 minutes

Future work

None

Checklist

• Update the documentation to reflect changes made in this PR
• Format all updated python files using ruff
• Clear outputs from all notebooks modified
• Add docstrings and type hints to any new functions created