Consumption Aggregation & Effective Consumption

Business Context

Business-Level Definition

A metering point that is read remotely, read by hand and billed produces three versions of the same month, and all three are true. Someone still has to answer the plain question: how much did this place use in January.

This feature answers it twice over. It keeps every version available for comparison, and it produces one figure — the effective consumption — by taking, for each quarter hour, the version the configuration trusts most. It also provides the summaries everything else reads: by hour, day, month and year, always derived from the same quarter-hours, so a chart, a report and an invoice check never disagree about the same period.

Requirements Definition

Without this step each consumer would pick its own way to choose between sources and its own way to add figures up, which is how a product ends up with a monthly total in a report that does not match the same month in a chart. That happened in the previous system and cost more trust than any single wrong number.

Two properties matter more than they first appear. The choice between sources must happen per quarter hour, before anything is added up: remote data covering three weeks of a month and a manual reading covering all of it are not alternatives to pick between at month level — the honest answer uses the remote data where it exists and falls back for the rest. And a correction anywhere in the chain has to reach every summary built over it, or yesterday's wrong number survives in the yearly figure long after the reading behind it was fixed.

Technical Context

User Stories / Use Cases

• As an energy manager, I want one consumption figure per period without choosing a source every time, so I can read a chart at a glance.
• As an energy manager, I want to see the remote, manual, billed and effective series side by side, so I can see where they disagree and why.
• As an energy manager, I want billed consumption marked as billed, so I do not read a modelled shape as a measurement.
• As an energy manager, I want a corrected reading to change the day, the month and the year figures, so nothing stale survives the correction.
• As a facility manager, I want yearly and monthly figures to add up from the same data as the quarter-hourly view, so the totals reconcile.

UI/UX Design

N/A — the feature has no interface of its own. Its results are presented by Chart Engine and by the reporting features; the labelling requirement for billed series is stated under Functional Requirements and honoured by those surfaces.

Functional Requirements

Effective consumption

• Choose, for each 15-minute interval, the value of the highest-priority source available for that interval.
• Resolve the priority from the gauge, then the client default, then the system default of remote before manual before invoice.
• Apply the choice before any addition: a daily, monthly or yearly effective figure is the sum of the values already chosen per interval.
• Return effective consumption as a read-time view; store no effective series and introduce no source value for it.
• Leave virtual-gauge output outside the priority ordering, resolved by its own rule.
• Leave an interval empty in the effective series where no source covers it.

Aggregation

• Maintain hourly, daily and monthly summaries of every stored series, derived from the 15-minute values.
• Provide yearly summaries from the same derivation.
• Preserve the value kind, measured quantity, flow direction, tariff band and source in every summary; never add two sources together.
• Offer a granularity only where the underlying summary supports it; never interpolate or resample to fake a finer one.
• Serve charts, reports and downstream calculations from these same summaries.

Serving a period total

• Total an arbitrary period, not only whole calendar buckets: a billing period running from the fourth of one month to the fourth of the next is answered as one figure.
• Group a total by tariff, source, gauge, medium or usage type on request, and always keep medium, quantity and unit apart so nothing incomparable is added.
• Group by usage type from the purpose of the gauge a value came from, so a caller that normalises or prices only the heating share can ask for it here rather than deriving it.
• Answer a sizing question with the largest quarter-hour value of a period as readily as with its total, so a maximum is never re-derived by each caller that needs one.
• Accept any window the caller computes, so a rolling year or a rolling ninety days needs no separate way of asking.
• Default to effective consumption, and total a named source series instead when the caller asks for one.
• Total money the same way as a quantity, so an invoice's cost and its consumption come from one place.
• Return the coverage behind every total: gauges asked for, gauges with data, intervals covered, and buildings with no recognised total.
• Fail a request naming a gauge that does not exist rather than returning a smaller total, which is indistinguishable from a fall in consumption.
• Mark a total provisional while a recalculation is pending for any gauge and interval inside the period.

Keeping summaries true

• Refresh the summaries covering any window whose series changed.
• Refresh the summaries of a cost series when the invoice behind it changes.
• Leave stored series untouched when a source priority changes; it changes only the effective result.
• Recompute stored results derived from effective consumption — virtual-gauge series among them — when a source priority changes.
• Keep every 15-minute series and every summary indefinitely; apply no retention to computed data.
• Mark a series derived from an invoice as billed wherever it is returned, so it is never mistaken for measurement.

Aggregation across gauges

Time aggregation answers "this gauge, that period". The other question a portfolio asks — these five buildings, a campus, a sector, a client, everything on one medium — is a sum across gauges. It is computed when it is read, from stored series, and is never materialised, so it cannot go stale.

• Resolve the selection to a set of gauges; which buildings a filter selects is Chart Engine's, how that set becomes one number is this feature's.
• Deduplicate by gauge: a gauge reachable through two paths in the selection counts once.
• Apply each gauge's sign once, from its output mode and direction; excluded and report-only gauges do not enter the sum.
• Never mix a building total with its own members in one sum, and do not build a portfolio figure by adding building totals together.
• Group by medium, measured quantity and unit; sum across media only after converting to a common energy unit, and say so in the result.
• Return coverage with the figure: how many gauges the selection holds, how many have data for the period, and how many buildings have no recognised total.

For a selection whose buildings all have a recognised total and share no sub-gauge across a building boundary, summing the members and summing the building totals give the same number. Where the two differ, the difference is a configuration fault — a sub-gauge counted in two places, or a building with no formula — and the coverage figures are what expose it rather than the number quietly absorbing it.

Internationalization & Localization

N/A — the feature returns numbers and period boundaries; labels and formats belong to the presenting surfaces.

Non-Functional Requirements

• Consistency. A period's figure is identical whichever granularity or consumer it is read through.
• Availability of history. Summaries stay available for periods whose raw inputs have already been thinned.
• Scalability. Summary maintenance is incremental — a changed window refreshes that window, not the whole series.

Performance

Reads are served from the coarsest summary that satisfies the requested granularity, so a five-year monthly chart never scans quarter-hourly data, and a portfolio figure over hundreds of gauges reads monthly summaries rather than their intervals. The cost that has to be bounded is the refresh after a correction: it is proportional to the changed window, not to the length of history, which is what makes a bulk correction survivable.

Transactional Operations

N/A — the read path takes no locks and writes nothing. Summary refreshes are maintained by the database as an incremental operation over the changed window; they are not part of any user transaction, and a refresh that fails leaves the previous summary in place to be recomputed.

Processes & Related Systems / Components

Trigger	Owner of the trigger	What this feature does
Series computed or recomputed	Consumption Calculation	Refresh the summaries covering the changed window
Cost series written	Cost Calculation	Refresh the money summaries for the same window
Source priority changed on a gauge or client	Gauge Management, Client Management	Change nothing stored; trigger recomputation of results derived from effective consumption
Chart, report or downstream calculation requests a period	Chart Engine, reporting features	Serve source series, effective series or summaries at the requested granularity
Invoice screen or tolerance check requests a period total	Invoice Management	Total the period from readings, grouped by tariff, with its coverage

Diagrams & Models

Answering a request for one month of effective consumption:

sequenceDiagram
    participant Q as Chart Engine
    participant E as Effective consumption
    participant G as Gauge and client settings
    participant S as Stored series and summaries
    Q->>E: Consumption for a gauge, a month, a granularity
    E->>G: Source priority for this gauge
    G-->>E: Gauge setting, else client default, else system default
    E->>S: Series of each source over the month
    S-->>E: Values per interval
    E->>E: Pick the best available source per interval
    E->>E: Add the picked values up to the requested granularity
    E-->>Q: One series, with the source named per value

Why the choice happens before the sum:

flowchart LR
    A[Interval 1: remote present] --> D[Pick remote]
    B[Interval 2: remote missing, manual present] --> E[Pick manual]
    C[Interval 3: nothing] --> F[Leave empty]
    D --> G[Sum the picked values]
    E --> G
    F --> G
    G --> H[Effective total for the period]

API Analysis

See api-a.md — one endpoint, the period total. Series at a granularity remain Chart Engine's, and the invoice-side figures Invoice Management's.

Domain Model (ER diagram) & Data Attribute Table

This feature introduces no entity. It reads and summarises the series that Consumption Calculation and Cost Calculation write, and reads two settings that decide the source choice.

erDiagram
    CONSUMPTION ||--o{ CONSUMPTION_SUMMARY : "is summarised as"
    GAUGE ||--o{ CONSUMPTION : "measures"
    GAUGE }o--|| CLIENT : "defaults from"

• consumption — the 15-minute series this feature summarises and selects from.
• gauge — the per-gauge source priority.
• client — the client default source priority.
• Enum - GaugeSourcePriority — the six orderings the choice can follow.

The summaries themselves are derived database objects over the series, not entities: they carry the same columns as the series with the interval widened to an hour, a day or a month, and they hold no fact that is not in the series. Documenting them as entities would create a second description of the same columns, free to drift from the first.

Data

No configuration data of its own. The only inputs that change behaviour are the two source-priority settings, which are gauge and client data.

Test Data

Use the project's Test Data location named in the overlay. The case worth seeding explicitly is a gauge whose remote coverage stops part-way through a month while a manual reading spans the whole of it — the partial-coverage case, where the effective total equals neither source series.

Logging

The project has no shared logging-conventions document yet, so this feature's events are stated in full here.

• .info — summaries refreshed for a window: gauge, granularity, interval count, trigger, requestId.
• .info — effective consumption served: gauge, period, granularity, which sources contributed.
• .warn — a requested granularity is not available for the period and the request was answered at the nearest supported one.
• .error — a summary refresh failed and the previous summary was kept: gauge, window, cause.

Monitoring

Two feature-level needs: the age of the oldest summary still waiting to be refreshed, which is how a stale chart is detected before a user reports it; and the failure count of summary refreshes.

Caching

The summaries are themselves the cache — a materialised, incrementally maintained copy of what the series would produce. No second caching layer sits in front of them, because a cached figure that survives a correction is exactly the failure this feature exists to prevent.

Backward Compatibility and Migration

Nothing is migrated: summaries are built from the recomputed series. The behaviour change users will notice is the source choice itself — the previous system filled gaps from a secondary source only where the primary had nothing for a whole day, while the choice is now made per quarter hour, so a month with partial remote coverage produces a different, more faithful total than the legacy figure for the same month.

Legal Context

N/A — the feature summarises measurement data and makes no claim subject to a specific regulation; the legal weight on money figures is covered by Cost Calculation.

Cybersecurity Considerations

Summaries inherit the isolation of the series they are built from: the tenant predicate is applied by the same application-level guard, and a summary is queried only through it. A derived object that dropped the tenant column would be a cross-tenant leak, so the tenant stays part of every summary's grouping.

Data Privacy. No personal data.

Risk Assessment

Business Risks

The risk is a figure that is stale rather than wrong: a summary that was not refreshed after a correction looks exactly like a current one. It is mitigated by refreshing on every change to a series and by monitoring the age of pending refreshes, and it is bounded by the fact that summaries can always be rebuilt from the series.

A second, smaller risk is misreading: billed consumption spread over a period looks like measurement at a fine granularity. It is labelled as billed wherever it is returned for that reason.

Technical Risks

Risk	Consequence	Mitigation
Refresh backlog after a bulk correction	Summaries lag behind the series	Refresh is incremental per window; the backlog age is monitored
Effective selection is computed at the wrong point	A month's figure silently disagrees with the sum of its days	Selection happens per interval before any addition, and the rule is stated once here
A summary is queried without the tenant predicate	Cross-tenant leak	The guard applies the predicate to every statement, including summaries
Priority changed but derived results not recomputed	Virtual-gauge figures disagree with the effective series they came from	A priority change triggers recomputation of stored results built on effective consumption
Cascade — this feature stops	Charts, reports, benchmarking, savings and emissions lose their data source	Series remain intact and summaries are rebuildable from them

Auditing, Reporting & Measurement

Summaries carry no audit trail of their own: they are derived, rebuildable, and traceable through the series to the inputs, which are audited where they are entered. Every value returned names the source it came from, which is what makes a disputed figure answerable — a user asking why a month looks wrong can be shown which source covered which part of it.

The feature's own measurement is the pair named under Monitoring: refresh backlog age, and refresh failures.