Documentation

Appearance

Updated Sep 11, 2026 by Barča Dvořáková · Owner: analysisactivefeatureconfluence-migration Edit on GitHub

Data Aggregator Administration ​

Business Context ​

Business-Level Definition ​

Automated data collection from external remote-reading and invoice systems is handled by a dedicated data aggregator service. It connects to 18 distinct external source types and stores what it collects — connectors and their individual data channels — in its own database schema, owned exclusively by that service.

This feature defines the administration surface for that data: how connectors and their channels are made visible to energy managers, and how a channel gets paired to the gauge it measures. It does not define the aggregator's collection logic itself, and it does not let anyone create a new connector — connector provisioning is a deployment-time operation, not a UI operation (see Functional Requirements).

Requirements Definition ​

  • Give energy managers visibility into the health of each connector (is it delivering data, when did it last succeed) without requiring direct database access to a schema owned by another service.
  • Give a Porsenna system admin a combined view of every connector and endpoint across all tenants (not filtered to a single client).
  • Let a regular user pair an unmatched data channel to the gauge it belongs to from that gauge's own detail screen:
    • one gauge at a time
    • scoped to endpoints in their own tenant
    • re-pair a channel that is already paired with a confirmation step
    • Porsenna system admin can perform the same one-at-a-time pairing action from the connector's endpoint list, across tenants (this entry point already existed prior to this feature and is retained)
    • a connector-level panel for bulk (multi-endpoint) pairing is planned for later (v2)
  • Let a Porsenna system admin view a connector's connection details, including the login/account identifier used against the external system.
  • Let an admin adjust the outage threshold used to compute a connector's health status.
  • Keep the aggregator's own schema exclusively owned by the aggregator service. This feature reads a projection of that data, not the schema itself.
  • Keep tenant isolation intact: the aggregator's own storage predates and does not implement per-tenant scoping; this feature's own tables must.

Acceptance Criteria ​

  • A connector list is available per client, showing computed status (operational / warning / error, derived from lastSuccessfulRequest against a configurable outage threshold), and endpoint counts by category (active, deactivated, unmatched).
    • Porsenna system admin can additionally view a single combined list of all connectors and endpoints across every tenant, unfiltered by client, with the same status and count fields.
  • For v1, an unmatched endpoint can be paired to a gauge one at a time:
    • either from that gauge's own detail screen (regular user, scoped to their own tenant)
    • or from the connector's endpoint list (Porsenna system admin, cross-tenant)
    • both entry points perform the same single pairing action. Bulk (multi-endpoint) pairing from the connector-level panel is deferred to v2.
  • Re-pairing an endpoint that already has a gauge requires an explicit confirmation step, regardless of entry point.
  • The outage threshold is editable per connector.
  • A connector's connection details — including the login/account identifier used to authenticate against the external system (e.g. MERV-PORS-043b) — are visible only to a Porsenna system admin (a cross-tenant administrative role), not to a client's own admin or to regular users.
  • No screen or endpoint in this feature can create a connector, edit its credentials, or edit its non-secret configuration. These remain deployment-time operations outside this feature's scope.
  • Connector and endpoint data shown in the UI is sourced from this feature's own tenant-scoped tables, never by a live read against the aggregator's schema.
  • A pairing decision made in the UI reaches the aggregator service so that data collection is routed to the correct gauge going forward.
  • Every non-secret key present in a connector's freeform configuration is checked against a denylist (password, secret, token, key, and similar) before it is surfaced anywhere in the UI or projection.

N/A — not available in the source material.

Technical Context ​

User Stories / Use Cases ​

  • As an energy manager, I open a client's connector list and see, for each connector, its status, how many endpoints are active/deactivated/unmatched, and when it last delivered data.
  • As a Porsenna system admin, I open a single combined list of all connectors and endpoints across every tenant, so I can find and pair an unmatched endpoint without first having to know which client it belongs to.
  • As a regular user, I open a gauge's detail screen and pair it to an unmatched endpoint from an external system, one gauge at a time.
  • As a regular user, I re-pair an endpoint that is already paired to a different gauge and confirm the change before it takes effect.
  • As a Porsenna system admin, I view a connector's connection details, including the login/account identifier used against the external system, to check which account a client's data is flowing through.
  • As an admin, I adjust a connector's outage threshold so that the status calculation matches how tolerant this client's operations are of a delayed feed.

UI/UX Design ​

Screens are the API Connectors tab already described under Client Management (retained, with its existing one-at-a-time pairing action available to a Porsenna system admin), plus a new pairing action on the gauge detail screen (Gauge Management) for regular users.

Both entry points perform the same single-endpoint pairing (FR3); only the connector-level bulk (multi-endpoint) pairing panel is deferred to v2.

Functional Requirements ​

FR1. Connector and endpoint data displayed to users is read from this feature's own tenant-scoped tables, kept current by a scheduled synchronisation process that reads the aggregator's connector and endpoint records. Synchronisation latency of a few minutes is acceptable; the UI is not expected to reflect the aggregator's state instantly.

FR2. A connector's freeform configuration data is projected into this feature's tables only after every key is checked against a denylist (password, secret, token, key, and case-insensitive variants/substrings thereof). Any key matching the denylist is dropped before projection, never stored or displayed. [MISSING — needs clarification: confirm the denylist wording and whether it should be a fixed list or a configurable one, with the business owner.]

FR3. Pairing an endpoint to a gauge is written immediately to this feature's own tables (tenant-scoped, permission-checked) and is then propagated to the aggregator service so that future data collection for that channel is routed to the paired gauge. The propagation mechanism is a call the synchronisation process makes to the aggregator service — not a direct write into the aggregator's schema. For v1, pairing is initiated one gauge and one endpoint at a time, either from the gauge's own detail screen or from the connector's endpoint list. The connector-level bulk (multi-endpoint) pairing panel is deferred to v2.

FR4. Because the aggregator's own reference to a gauge is not yet expressed in the same identifier format this system uses elsewhere, the synchronisation process maintains a mapping between the two identifier forms for as long as that difference exists. This is a known, deliberately deferred piece of technical debt, not a permanent design choice — see Backward Compatibility and Migration.

FR5. No endpoint or screen in this feature creates a connector or edits its credentials or non-secret configuration. Connector provisioning remains a deployment-time operation on the aggregator side.

FR6. A connector's connection details — including the login/account identifier used to authenticate against the external system (e.g. MERV-PORS-043b) — are visible only to a Porsenna system admin (a cross-tenant administrative role). They are not exposed to a client's own admin or to regular users.

FR7. The list of recognised connector source types must reflect every source type the aggregator actively collects from, not a partial list. [MISSING — needs clarification: the authoritative source-type list should be confirmed against the aggregator's active configuration at implementation time, since it has previously drifted out of sync with what's documented.]

FR8. Automatic matching of an endpoint to a gauge (for example, based on a shared identifier such as EAN/EIC) is out of scope for v1. All pairing in v1 is manual, initiated by a user from the gauge detail screen. Automatic matching is deferred to v2.

FR9. A Porsenna system admin can query connector and endpoint data across all tenants in one unscoped view, in addition to the per-client view available to regular users and client admins. This is a separate read path against the same tenant-scoped tables (FR1), not a new schema.

Non-Functional Requirements ​

  • Reading the connector/endpoint list must not depend on the aggregator service being available — it is served entirely from this feature's own tenant-scoped projection.
  • Pairing a gauge must be rejected if the gauge does not belong to the same tenant as the connector being administered.
  • The synchronisation process must not write tenant business data into the aggregator's own schema, and must not read or write any schema other than the aggregator's connector/endpoint records and this feature's own tables.

API Analysis ​

GET /v1/clients/:id/connectors
GET /v1/connectors                    (Porsenna system admin only — unscoped, cross-tenant)
PATCH /v1/clients/:id/connectors/:connId
POST /v1/clients/:id/connectors/:connId/pair

Connector and endpoint data returned by these endpoints is sourced from this feature's own projection (FR1), not from a live call to the aggregator service.

Domain Model (ER diagram) & Data Attribute Table ​

No ER diagram in source material. This feature does not introduce new entities of its own. It relies on two existing entities, both of which need attribute-level updates as a follow-up to this design (tracked separately, per the project's convention of keeping DAT changes on the entity's own page):

  • remoteSource — needs a non-secret configuration attribute added (a denylist-filtered projection of the aggregator's connector configuration, see FR2). Its many-to-many relationship to client is retained: one connector can genuinely serve more than one client (for example, a single regional API connection shared across several clients). A 1:1 connector-to-client model would be preferable in principle, but is not being pursued — real cases already require M:N.
  • remoteEndPoint — the gauge reference carried by this entity's projection is expressed as this system's own gauge identifier; the aggregator's own copy of that reference uses a different identifier form for a connector until the aggregator's own tables converge (FR4). This should be documented explicitly on the entity page once the mapping mechanism is implemented, so nobody mistakes the two identifier forms for interchangeable.

Data ​

No new seed data. The synchronisation process backfills this feature's tables from the aggregator's existing connectors and endpoints on first run, for every tenant that has a connector assigned.

Test Data ​

N/A — not covered in source material.

Logging ​

.info: connector paired to gauge; connector re-paired (with previous and new gauge); outage threshold changed; synchronisation run completed (connectors/endpoints created or updated).

.warn: a configuration key was dropped by the denylist during synchronisation; a pairing write could not be propagated to the aggregator service after retry.

Monitoring ​

N/A — not covered in source material beyond the logging entries above.

Caching ​

N/A — not covered in source material.

Backward Compatibility and Migration ​

The aggregator's own reference to a gauge does not yet use this system's identifier format. Converting it is intentionally deferred: the aggregator's data is still read directly by another live system that depends on the current identifier form, and changing it now would break that system before its own migration off the aggregator's tables is complete. Until that migration happens (tracked as separate, epic-level work), this feature's synchronisation process carries a mapping layer between the two identifier forms (FR4). Once that migration completes, the aggregator's own identifier can be converted directly and the mapping layer removed.

N/A — not addressed in the source Confluence page; migrated as a reference copy without new legal analysis.

Cybersecurity Considerations ​

The denylist-filtering mechanism (FR2) and the admin-only visibility of connection credentials (FR6) are the source material's security-relevant content; both are documented under Functional Requirements above rather than restated here, per the source's structure. No dedicated data-privacy assessment is present in the source. N/A beyond the above — migrated as a reference copy without new security analysis.

Risk Assessment ​

N/A — not addressed in the source Confluence page; migrated as a reference copy without new risk analysis.

Auditing, Reporting & Measurement ​

N/A — not addressed in the source Confluence page beyond the Logging entries above.

Migrated as a reference copy from Confluence page "Data Aggregator Administration" (id 689897484). Content reorganized to fit this repo's feature-doc template; not re-analyzed against the current codebase. Two items are flagged [MISSING — needs clarification] in the source and are preserved here as open items.