Swap to private solarweb access

AGENTS.md

@@ -0,0 +1,9 @@
+# Private configuration
+
+`config.toml` is private and must remain secret from coding agents.
+
+- Never open, read, search, copy, print, diff, or otherwise inspect that file's contents with agent tools or helper scripts.
+- Exclude it explicitly from repository searches, even when searching ignored files.
+- The application may load its private configuration internally during user-authorized runs; this does not authorize displaying or inspecting the configuration or credentials. Use isolated tests with synthetic configuration and credentials for unit verification.
+- Make configuration changes only in `config.example.toml`; the user maintains the private file.
+- Do not inspect or log credentials, runtime session cookies, or authentication tokens. The user permits inspecting API/chart response data for debugging.

README.md

@@ -28,10 +28,10 @@ observations never expire, even if their measurement dates are much older.
 ```bash
 python3 -m venv .venv
 source .venv/bin/activate
-python -m pip install --only-binary=pyarrow -r requirements.txt
+python3 -m pip install --only-binary=pyarrow -r requirements.txt
 cp config.example.toml config.toml
-python -m solar_relay --config config.toml status
-python -m solar_relay --config config.toml run
+python3 -m solar_relay --config config.toml status
+python3 -m solar_relay --config config.toml run
 ```
 
 Use `.venv/bin/python` directly if you prefer not to activate the environment.
@@ -96,7 +96,8 @@ uploader, status request or CLI replay from overwriting another operation. HTTP
 requests happen outside the archive lock.
 
 The upload request is a JSON object with `schema_version: 1` and a `samples` array,
-authenticated with the bearer token named by `upload.token_env`. The website must
+authenticated with `upload.token` from private `config.toml`, or the environment
+variable named by `upload.token_env` when that setting is empty. The website must
 commit its data before responding:
 
 ```json
@@ -130,6 +131,11 @@ stop the service and other CLI writers before copying the archive directory as a
 
 ## Recovering history after Wi-Fi loss
 
+The default mode always collects live data from the inverter's local Fronius API.
+The optional backup (`[history].enabled = true`) fills historical energy gaps from
+Solar.web in a separate worker. It does not replace or change live collection;
+when there are no eligible gaps, it makes no Solar.web requests.
+
 GEN24 records historical data for transfer to Solar.web after reconnecting. This is
 separate from its local Solar API: Fronius explicitly lists archive access for GEN24
 as unavailable, and this inverter's `GetArchiveData.cgi` returns 404. Inspecting its
@@ -137,36 +143,88 @@ current web interface also found no energy-history retrieval endpoint. Its SDP s
 download is an encrypted `.cry` diagnostic file, not a documented energy-history API.
 
 See the official [Solar API manual, section 5.1.1](https://www.fronius.com/~/downloads/Solar%20Energy/Operating%20Instructions/42%2C0410%2C2012.pdf)
-and [Solar.web Query API manual, section 6.5](https://www.fronius.com/~/downloads/Solar%20Energy/User%20Information/SE_UI_API_InterfaceDocumentation_EN.pdf).
+for the local API's archive limitations.
 
-Optional recovery reads missing five-minute **measured energy** through the separate,
-authenticated Solar.web Query API, after the inverter uploads its backlog. This uses
-no public guest page. Obtain Query API access and keys from Fronius; its commercial
-terms are separate from normal Solar.web access. No cloud calls occur while disabled.
+Optional recovery signs in to Solar.web with a real Fronius account and reads its
+private daily charts after the inverter uploads its backlog, using the same login
+flow and chart series as the sibling `solar` website. No cloud calls occur while
+disabled. A public guest link and Query API subscription/keys are not needed.
 
-Set `[history].enabled = true` and `pv_system_id` in the configuration, and put these
-in the service's root-readable `/etc/solar-relay/solar-relay.env`:
+Create a dedicated Solar.web / Fronius account and grant it permission to view this
+installation's metrics. Keep public sharing disabled. The account can have guest
+permissions on the installation, but it signs in privately through the normal account
+login; the deactivated public guest interface is never used.
 
-```text
-SOLARWEB_ACCESS_KEY_ID=your-access-key-id
-SOLARWEB_ACCESS_KEY_VALUE=your-access-key-value
+Set the backup account credentials directly in your private `config.toml`:
+
+```toml
+[history]
+enabled = true
+pv_system_id = "your-pv-system-uuid"
+username = "your-account-email"
+password = "your-account-password"
 ```
 
-The recovery worker checks hourly, looking back seven days by default and allowing
-a two-hour delay for the inverter's hourly uploads. It fetches only UTC days with
-missing energy extractions, requests five energy channels, limits each request to
-24 hours, and skips intervals already present. Requests use HTTPS, headers for
-credentials, timeouts and no redirects. Missing/partial cloud data stays missing and
-is tried again later. The configured lookback cannot exceed the retention period.
+No extra environment file is needed for this login. Set both `username` and `password`;
+an incomplete pair is rejected. Direct credentials take precedence over the environment.
+Use TOML literal strings (`'...'`) if a password contains backslashes or double quotes.
+
+Alternatively, leave both credentials empty and set the environment variables named
+by `username_env` and `password_env`. For the systemd service, these can go in its
+root-readable `/etc/solar-relay/solar-relay.env`:
+
+```text
+SOLARWEB_USERNAME=your-account-email
+SOLARWEB_PASSWORD=your-account-password
+```
+
+While live collection runs, the separate recovery worker checks at startup and hourly
+by default. It checks every five-minute energy interval back to
+`collector.retention_days` (30 days by default), including entirely empty days and
+dates before the first local observation. A two-hour delay allows the inverter to
+upload its recent history. Only local dates containing gaps are fetched, and existing
+live or recovered energy is preserved. When all eligible intervals are present, no
+Solar.web requests are made. Unavailable chart days remain missing for retry and do
+not prevent other dates from being recovered. Login and network failures stop that
+attempt and retry on the next scheduled check.
+
+Set `collector.timezone` to the installation's Solar.web timezone; local chart dates
+include 23/25-hour DST days. The old `history.lookback_days` setting is accepted but
+ignored; `collector.retention_days` controls the automatic recovery window.
+
+Recovery pairs the `FromGen` Production series from the consumption view with the
+`ToConsumer` Consumption series from the production view, regardless of the display
+language. As in `solar`, each five-minute W value is treated as constant over that
+interval and divided by 12 to obtain Wh. Self-consumption is the smaller of production
+and consumption; purchase/export are their nonnegative differences. These are chart
+estimates, rather than the former Query API's measured Wh channels. The original
+paired power values and conversion method are retained in each recovered record.
+Null or unpaired readings stay missing and are tried again later.
+
+Login uses HTTPS and in-memory cookies. Credentials are submitted only to the Fronius
+password endpoint; SSO callbacks and redirects are restricted to the Solar.web and
+Fronius login hosts. Credentials are excluded from configuration reprs and never
+logged. Session cookies and authentication responses are never logged or written to
+disk. An expired session is renewed once. Rejected credentials,
+MFA, CAPTCHA, consent, or a changed sign-in form stop that attempt with a redacted
+error. Complete any required interactive account setup in a browser.
+
+When migrating from Query API recovery, replace `access_key_id_env` and
+`access_key_value_env` with `username` and `password` in `[history]`, or use the
+environment fallback above. Obsolete configuration keys are accepted and ignored
+for migration. Restart the service after updating its configuration or environment.
+Recovery requires the account credentials even if API keys remain set.
 
 ```bash
 .venv/bin/python -m solar_relay --config config.toml recover \
   --from 2026-10-29 --until 2026-10-31
 ```
 
-Manual recovery accepts at most 30 days per invocation and can query dates older than
-the automatic lookback. Only aligned 300-second energy records with all required Wh
-channels are accepted; longer aggregates are not split into invented five-minute values.
+Manual recovery with explicit date bounds accepts at most 30 days per invocation and
+can query dates older than the automatic retention window. Unbounded recovery checks
+the whole retention window, even when longer than 30 days. Only aligned five-minute
+chart samples in W are accepted;
+longer aggregates and unexpected units are rejected.
 Recovery stores stable UUIDs and `history_interval` records, which the main website
 uses directly without integrating them as live power. Install the corresponding
 `solar` receiver update before enabling cloud recovery.
@@ -186,13 +244,14 @@ fresh installation, create it from the example only when it does not exist:
 ```bash
 cd /var/www/html/solar-relay
 test -e config.toml || cp config.example.toml config.toml
-chmod 0644 config.toml
+sudo chgrp www-data config.toml
+chmod 0640 config.toml
 ```
 
-The `www-data` service user needs read access to this configuration. It contains
-setting values and environment variable names; keep credentials in the root-readable
-environment file below. Ensure the proxy serves the status endpoint rather than
-repository files.
+The `www-data` service user needs read access to this private configuration, which
+may contain the upload token and Solar.web login credentials. Keep real credentials
+out of `config.example.toml` and Git. Ensure the proxy serves the status endpoint
+rather than repository files.
 
 Keep the archive under `/var/www/html/solar-relay/data/archive`. The relative setting
 `data/archive` resolves to that location because paths are relative to `config.toml`.
@@ -222,12 +281,20 @@ exist before starting the service; otherwise systemd exits with `226/NAMESPACE`
 before Python can create it. Existing data must be writable by `www-data`. Use
 `sudo -u www-data` for archive-writing CLI commands to preserve ownership.
 
-Set `[upload].url` to the main website's `https://your-solar-host/index.php?a=relay`.
-Put `SOLAR_RELAY_TOKEN=your-shared-token` in `/etc/solar-relay/solar-relay.env` with
-permissions 0600, readable by root. Configure the same token and source ID on the
-main website as described in its README. Tokens and API keys stay out of TOML/Git.
-Create `/etc/solar-relay` with `sudo install -d -m 0755 /etc/solar-relay` when adding
-that optional environment file.
+Set the upload URL and shared bearer token in private `config.toml`:
+
+```toml
+[upload]
+url = "https://your-solar-host/index.php?a=relay"
+token = "your-shared-token"
+```
+
+Configure the same token and source ID on the main website as described in its README.
+No extra environment file is needed. Alternatively, leave `token` empty and provide
+the variable named by `token_env` (default `SOLAR_RELAY_TOKEN`). For systemd, put
+`SOLAR_RELAY_TOKEN=your-shared-token` in `/etc/solar-relay/solar-relay.env` with
+permissions 0600, readable by root. Create `/etc/solar-relay` with
+`sudo install -d -m 0755 /etc/solar-relay` when adding that optional environment file.
 
 ```bash
 sudo cp deploy/solar-relay.service /etc/systemd/system/
@@ -322,4 +389,4 @@ node --check solar_relay/web/status.js
 Tests cover normalization, real local HTTP delivery/retries, Parquet round-trips,
 concurrent writers, interrupted writes, retention, DST completeness,
 mocked authenticated history recovery, status routes and the deployment script.
-Live Solar.web recovery requires real Query API credentials and is not exercised by tests.
+Live Solar.web recovery requires real account credentials and is not exercised by tests.

config.example.toml

@@ -14,7 +14,7 @@ min_request_interval_seconds = 4
 
 [upload]
 url = "" # Set to https://your-solar-host/index.php?a=relay to enable uploads.
-token_env = "SOLAR_RELAY_TOKEN"
+token = "" # Shared bearer token; enter the real value only in private config.toml.
 batch_size = 100
 request_timeout_seconds = 10
 retry_initial_seconds = 5
@@ -28,14 +28,10 @@ base_path = "/solar-relay"
 history_days = 14
 
 [history]
-# Optional, authenticated Solar.web Query API. No public guest link is used.
-# GEN24's local Solar API has no historical-data endpoint.
 enabled = false
 pv_system_id = "" # Your Solar.web PV system UUID; required when enabled.
-access_key_id_env = "SOLARWEB_ACCESS_KEY_ID"
-access_key_value_env = "SOLARWEB_ACCESS_KEY_VALUE"
-lookback_days = 7
+username = "" # Account email; enter real credentials only in private config.toml.
+password = ""
 lag_seconds = 7200 # Allow the inverter to upload its hourly historical data.
 check_interval_seconds = 3600
-request_timeout_seconds = 10
-
+request_timeout_seconds = 45

solar_relay/cli.py

@@ -43,7 +43,8 @@ def _run(config, store):
     stop = threading.Event()
     previous = {}
     uploader = Uploader(config.upload, store) if config.upload.url else None
-    recovery = HistoryRecovery(config.history, config.collector.source_id, store) if config.history.enabled else None
+    recovery = HistoryRecovery(config.history, config.collector.source_id, store,
+                               timezone_name=config.collector.timezone) if config.history.enabled else None
     worker = None
     history_worker = None
     status_server = None
@@ -137,8 +138,9 @@ def main(argv=None):
             print(json.dumps({"deleted": store.prune()}))
         elif args.command == "recover":
             if not config.history.enabled:
-                raise ValueError("Enable history and configure Solar.web API credentials before recovery")
-            recovery = HistoryRecovery(config.history, config.collector.source_id, store)
+                raise ValueError("Enable history and configure Solar.web account credentials before recovery")
+            recovery = HistoryRecovery(config.history, config.collector.source_id, store,
+                                       timezone_name=config.collector.timezone)
             print(json.dumps({"recovered": recovery.recover(start, until)}))
         elif args.command == "export":
             for sample in store.records(start, until):

solar_relay/config.py

@@ -1,6 +1,6 @@
-"""Configuration contains environment variable names, never upload credentials."""
+"""Private runtime configuration; credential fields are excluded from reprs."""
 
-from dataclasses import dataclass
+from dataclasses import dataclass, field
 from datetime import datetime, timezone
 import math
 from pathlib import Path
@@ -9,6 +9,8 @@ import tomllib
 from urllib.parse import urlsplit
 from zoneinfo import ZoneInfo
 
+LEGACY_HISTORY_SETTINGS = {"access_key_id_env", "access_key_value_env", "lookback_days"}
+
 
 def utc_text(value: datetime) -> str:
     if value.tzinfo is None:
@@ -52,6 +54,7 @@ class UploadConfig:
     request_timeout_seconds: float = 10
     retry_initial_seconds: float = 5
     retry_max_seconds: float = 300
+    token: str = field(default="", repr=False)
 
 
 @dataclass(frozen=True)
@@ -67,12 +70,13 @@ class HttpConfig:
 class HistoryConfig:
     enabled: bool = False
     pv_system_id: str = ""
-    access_key_id_env: str = "SOLARWEB_ACCESS_KEY_ID"
-    access_key_value_env: str = "SOLARWEB_ACCESS_KEY_VALUE"
-    lookback_days: int = 7
+    username_env: str = "SOLARWEB_USERNAME"
+    password_env: str = "SOLARWEB_PASSWORD"
     lag_seconds: int = 7200
     check_interval_seconds: int = 3600
-    request_timeout_seconds: float = 10
+    request_timeout_seconds: float = 45
+    username: str = field(default="", repr=False)
+    password: str = field(default="", repr=False)
 
 
 @dataclass(frozen=True)
@@ -118,7 +122,8 @@ def load_config(path: Path) -> Config:
         "inverter": set(InverterConfig.__dataclass_fields__),
         "upload": set(UploadConfig.__dataclass_fields__),
         "http": set(HttpConfig.__dataclass_fields__),
-        "history": set(HistoryConfig.__dataclass_fields__),
+        # Keep existing private configurations valid while using the full retention window.
+        "history": set(HistoryConfig.__dataclass_fields__) | LEGACY_HISTORY_SETTINGS,
     }.items():
         if not isinstance(data.get(section, {}), dict):
             raise ValueError(f"{section} must be a table")
@@ -157,6 +162,10 @@ def load_config(path: Path) -> Config:
         raise ValueError("upload.url must be a string")
     if upload.url:
         _url(upload.url, "upload.url", upload=True)
+    if not isinstance(upload.token, str):
+        raise ValueError("upload.token must be a string")
+    if upload.token and any(ord(c) < 33 or ord(c) > 126 for c in upload.token):
+        raise ValueError("upload.token must be a bearer token without whitespace")
     if not isinstance(upload.token_env, str) or not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", upload.token_env):
         raise ValueError("upload.token_env must be an environment variable name")
     if isinstance(upload.batch_size, bool) or not isinstance(upload.batch_size, int) or not 1 <= upload.batch_size <= 500:
@@ -175,21 +184,25 @@ def load_config(path: Path) -> Config:
         raise ValueError("http.base_path must be empty or a URL path without a trailing slash")
     if isinstance(http.history_days, bool) or not isinstance(http.history_days, int) or not 1 <= http.history_days <= 90:
         raise ValueError("http.history_days must be an integer between 1 and 90")
-    history = HistoryConfig(**data.get("history", {}))
+    history = HistoryConfig(**{key: value for key, value in data.get("history", {}).items()
+                              if key not in LEGACY_HISTORY_SETTINGS})
     if not isinstance(history.enabled, bool):
         raise ValueError("history.enabled must be true or false")
     if not isinstance(history.pv_system_id, str) or (history.pv_system_id and not re.fullmatch(r"[0-9a-fA-F]{8}(?:-[0-9a-fA-F]{4}){3}-[0-9a-fA-F]{12}", history.pv_system_id)):
         raise ValueError("history.pv_system_id must be a UUID")
     if history.enabled and not history.pv_system_id:
         raise ValueError("Set history.pv_system_id before enabling Solar.web recovery")
-    for name in ("access_key_id_env", "access_key_value_env"):
+    for name in ("username", "password"):
+        if not isinstance(getattr(history, name), str):
+            raise ValueError(f"history.{name} must be a string")
+    if (history.username or history.password) and (not history.username.strip() or not history.password):
+        raise ValueError("Set both history.username and history.password, or leave both empty to use environment variables")
+    for name in ("username_env", "password_env"):
         if not isinstance(getattr(history, name), str) or not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", getattr(history, name)):
             raise ValueError(f"history.{name} must name an environment variable")
-    for name, minimum in [("lookback_days", 1), ("lag_seconds", 3600), ("check_interval_seconds", 3600)]:
+    for name, minimum in [("lag_seconds", 3600), ("check_interval_seconds", 3600)]:
         value = getattr(history, name)
         if isinstance(value, bool) or not isinstance(value, int) or value < minimum:
             raise ValueError(f"history.{name} must be an integer of at least {minimum}")
-    if history.lookback_days > 30 or (history.enabled and history.lookback_days > retention):
-        raise ValueError("history.lookback_days must be at most 30 and no longer than retention_days when enabled")
     _number(history.request_timeout_seconds, "history.request_timeout_seconds", 0.1)
     return Config(collector, inverter, upload, http, history)

solar_relay/coverage.py

@@ -25,7 +25,7 @@ def _finite(value):
 def sample_coverage(sample):
     observed = milliseconds(sample["observed_at"])
     if "history_interval" in sample:
-        # Cloud history has measured interval energy, but no cumulative meter or relay history.
+        # Cloud history has interval energy, but no cumulative meter or relay history.
         return (sample["id"], sample["source_id"], observed, observed, 1, observed, 0, observed, 0)
     readings = sample.get("readings", {})
     result = [sample["id"], sample["source_id"], observed]

solar_relay/history.py

@@ -1,19 +1,20 @@
-"""Optional private Solar.web Query API recovery of missing five-minute energy.
+"""Optional signed-in Solar.web chart recovery of missing five-minute energy.
 
-GEN24's local Solar API exposes no archive. This reads the independent cloud API
-after the inverter has uploaded its internal backlog; it never requests a guest page.
+GEN24's local Solar API exposes no archive. Use the same private account login,
+chart series and five-minute W-to-Wh conversion as the sibling solar website.
 """
 
 from datetime import datetime, timedelta, timezone
 import logging
 import math
 import os
-from urllib.parse import urlencode
 from uuid import NAMESPACE_URL, uuid5
+from zoneinfo import ZoneInfo
 
 from .config import utc_text
 from .coverage import milliseconds
-from .http import JsonHttp, TransportError
+from .http import TransportError
+from .solarweb import ChartDataError, SolarWebClient
 
 LOG = logging.getLogger(__name__)
 CHANNELS = {
@@ -40,6 +41,7 @@ def history_sample(entry, source_id, system_id):
     when = utc_text(datetime.fromtimestamp(stamp / 1000, timezone.utc))
     return {
         "schema_version": 1,
+        # Preserve IDs of intervals already recovered with the former Query API.
         "id": str(uuid5(NAMESPACE_URL, f"https://swqapi.solarweb.com/{system_id}/{source_id}/{when}")),
         "source_id": source_id, "observed_at": when,
         "readings": dict.fromkeys([
@@ -54,68 +56,80 @@ def history_sample(entry, source_id, system_id):
     }
 
 
+def chart_entries(production, consumption):
+    """Pair real chart samples; assume constant W over five minutes, as solar does."""
+    for stamp in sorted(production.keys() & consumption.keys()):
+        generated, consumed = production[stamp], consumption[stamp]
+        values = {"production": generated / 12, "consumption": consumed / 12,
+                  "self_consumption": min(generated, consumed) / 12,
+                  "bought": max(0, consumed - generated) / 12, "sold": max(0, generated - consumed) / 12}
+        yield {"logDateTime": utc_text(datetime.fromtimestamp(stamp / 1000, timezone.utc)), "logDuration": 300,
+               "channels": [{"channelName": name, "channelType": "Energy", "unit": "Wh", "value": values[field]}
+                            for field, name in CHANNELS.items()],
+               "chart_power_w": {"production": generated, "consumption": consumed},
+               "energy_method": "constant_power_over_five_minutes"}
+
+
 class HistoryRecovery:
-    def __init__(self, config, source_id, store, *, http=None, credentials=None):
+    def __init__(self, config, source_id, store, *, timezone_name="Europe/Zurich", client=None, credentials=None):
         self.config, self.source_id, self.store = config, source_id, store
-        names = (config.access_key_id_env, config.access_key_value_env)
-        values = credentials if credentials is not None else [os.environ.get(name, "") for name in names]
-        if len(values) != 2 or any(not isinstance(value, str) or not value or any(ord(c) < 33 or ord(c) > 126 for c in value) for value in values):
-            raise ValueError(f"Set {names[0]} and {names[1]} before enabling Solar.web recovery")
-        self.headers = dict(zip(("AccessKeyId", "AccessKeyValue"), values))
-        self.http = http or JsonHttp(config.request_timeout_seconds)
+        self.zone = ZoneInfo(timezone_name)
+        if client is None:
+            names = (config.username_env, config.password_env)
+            if credentials is not None:
+                values = credentials
+            elif config.username or config.password:
+                values = (config.username, config.password)
+            else:
+                values = [os.environ.get(name, "") for name in names]
+            if (len(values) != 2 or any(not isinstance(value, str) or not value for value in values)
+                    or not values[0].strip()):
+                raise ValueError("Set history.username and history.password in config.toml, "
+                                 f"or {names[0]} and {names[1]} before enabling Solar.web recovery")
+            client = SolarWebClient(*values, timeout=config.request_timeout_seconds)
+        self.client = client
 
     def recover(self, start=None, until=None, *, now=None, stop=None):
         now = now or datetime.now(timezone.utc)
         latest = int((now - timedelta(seconds=self.config.lag_seconds)).timestamp()) // 300 * 300
-        first = self.store.status()["first_observation"]
-        earliest = max(int((now - timedelta(days=self.config.lookback_days)).timestamp()) // 300 * 300,
-                       milliseconds(first) // 300_000 * 300 if first else latest)
+        # Check the whole retained window, including days before any live observation.
+        # Round up so recovery never requests an interval older than the cutoff.
+        earliest = math.ceil((now - timedelta(days=self.store.retention_days)).timestamp() / 300) * 300
         lower = milliseconds(start) // 300_000 * 300 if start else earliest
         upper = min(milliseconds(until) // 300_000 * 300, latest) if until else latest
         if lower >= upper:
             return 0
-        if upper - lower > 30 * 86400:
+        if (start is not None or until is not None) and upper - lower > 30 * 86400:
             raise ValueError("Recover at most 30 days at once")
         stamp = lambda epoch: utc_text(datetime.fromtimestamp(epoch, timezone.utc))
         present = self.store.energy_slots(self.source_id, stamp(lower), stamp(upper))
         missing = set(range(lower, upper, 300)) - present
-        # Query only UTC days containing gaps. No request spans the API's 24-hour limit.
-        days = sorted({epoch // 86400 * 86400 for epoch in missing})
+        LOG.info("Checking %d historical energy intervals; %d are missing", (upper - lower) // 300, len(missing))
+        # Chart dates use the installation's local timezone, including 23/25-hour DST days.
+        days = sorted({datetime.fromtimestamp(epoch, self.zone).date() for epoch in missing})
         recovered = 0
         for day in days:
             if stop and stop.is_set():
                 break
-            begin, end = max(day, lower), min(day + 86400, upper)
-            page = 0
-            while True:
+            LOG.info("Recovering Solar.web energy history for %s", day)
+            try:
+                production = self.client.get_chart(self.config.pv_system_id, "production", day)
                 if stop and stop.is_set():
                     break
-                query = urlencode({"from": stamp(begin), "to": stamp(end), "timezone": "zulu",
-                                   "channel": ",".join(CHANNELS.values()), "offset": page * 300, "limit": 300})
-                url = f"https://swqapi.solarweb.com/pvsystems/{self.config.pv_system_id}/histdata?{query}"
-                response = self.http.request(url, extra_headers=self.headers)
-                system_id = response.get("pvSystemId")
-                if not isinstance(system_id, str) or system_id.lower() != self.config.pv_system_id.lower() or not isinstance(response.get("data"), list):
-                    raise ValueError("Unexpected Solar.web history response")
-                entries = response["data"]
-                if len(entries) > 300:
-                    raise ValueError("Solar.web history response exceeds the requested page size")
-                for entry in entries:
-                    try:
-                        sample = history_sample(entry, self.source_id, self.config.pv_system_id)
-                        epoch = milliseconds(sample["observed_at"]) // 1000
-                    except (KeyError, TypeError, ValueError, AttributeError):
-                        # Partial/unavailable cloud records remain missing and are tried again later.
+                consumption = self.client.get_chart(self.config.pv_system_id, "consumption", day)
+            except ChartDataError:
+                # A day with no usable cloud data must not block recovery of newer gaps.
+                LOG.warning("Solar.web chart data unavailable for %s; leaving its gaps for the next check", day)
                 continue
-                    if begin <= epoch < end and epoch in missing:
+            for entry in chart_entries(production, consumption):
+                epoch = milliseconds(entry["logDateTime"]) // 1000
+                if epoch not in missing or datetime.fromtimestamp(epoch, self.zone).date() != day:
+                    continue
+                sample = history_sample(entry, self.source_id, self.config.pv_system_id)
                 self.store.add(sample)
                 missing.remove(epoch)
                 recovered += 1
-                if len(entries) < 300:
-                    break
-                page += 1
-                if page > 2:
-                    raise ValueError("Solar.web history pagination did not finish")
+        LOG.info("History check finished: %d intervals recovered, %d still missing", recovered, len(missing))
         return recovered
 
     def run(self, stop):

solar_relay/solarweb.py

@@ -0,0 +1,254 @@
+"""Private, cookie-based Solar.web chart access, matching solar's SolarWebClient."""
+
+from dataclasses import dataclass, field
+from html.parser import HTMLParser
+from http.client import HTTPException
+from http.cookiejar import CookieJar
+import json
+import math
+import re
+from urllib.error import HTTPError, URLError
+from urllib.parse import unquote, urlencode, urljoin, urlsplit
+from urllib.request import HTTPCookieProcessor, Request, build_opener
+
+from .http import NoRedirects, TransportError, _invalid_constant
+
+API_SERVER = "https://www.solarweb.com"
+LOGIN_SERVER = "https://login.fronius.com"
+LOGIN_ERROR = ("Solar.web login failed. Check the configured username and password. "
+               "MFA, CAPTCHA or consent may require an interactive login.")
+DATASETS = {"production": ("consumption", "FromGen"), "consumption": ("production", "ToConsumer")}
+
+
+class ChartDataError(TransportError):
+    """A chart day is unavailable or cannot be safely decoded; login errors are separate."""
+
+
+@dataclass
+class Response:
+    status: int
+    url: str = field(repr=False)
+    body: bytes = field(default=b"", repr=False)
+    location: str = field(default="", repr=False)
+
+
+class Forms(HTMLParser):
+    def __init__(self, body):
+        super().__init__()
+        self.forms, self.current = [], None
+        try:
+            self.feed(body.decode("utf-8"))
+            self.close()
+        except (UnicodeError, ValueError):
+            raise TransportError("Solar.web did not return a valid sign-in page") from None
+
+    def handle_starttag(self, tag, attrs):
+        attrs = dict(attrs)
+        if tag == "form":
+            self.current = {"action": attrs.get("action") or "", "method": (attrs.get("method") or "get").lower(),
+                            "fields": {}, "password": False, "user_input": False, "visible": False, "captcha": False}
+            self.forms.append(self.current)
+        form = self.current
+        if form is None:
+            return
+        if "data-sitekey" in attrs or "frc-captcha" in (attrs.get("class") or ""):
+            form["captcha"] = True
+        if tag != "input" or "disabled" in attrs:
+            return
+        name, kind = attrs.get("name"), (attrs.get("type") or "text").lower()
+        form["password"] |= kind == "password" and name == "password"
+        form["user_input"] |= name == "usernameUserInput"
+        form["visible"] |= kind not in ("hidden", "submit", "button")
+        if kind == "hidden" and name:
+            form["fields"][name] = attrs.get("value") or ""
+
+    def handle_endtag(self, tag):
+        if tag == "form":
+            self.current = None
+
+
+def origin(url):
+    parsed = urlsplit(url)
+    return f"{parsed.scheme}://{parsed.hostname}"
+
+
+def validate_url(url):
+    try:
+        parsed = urlsplit(url)
+        valid = (parsed.scheme == "https" and origin(url) in (API_SERVER, LOGIN_SERVER)
+                 and parsed.username is None and parsed.password is None and parsed.port in (None, 443)
+                 and not any(ord(c) <= 32 or ord(c) == 127 or c == "\\" for c in url)
+                 and not re.search(r"GuestLogOn|DemoLogOn", unquote(parsed.path), re.I))
+    except ValueError:
+        valid = False
+    if not valid:
+        raise TransportError("Solar.web attempted an untrusted URL or guest access. Request blocked.")
+
+
+class SolarWebClient:
+    MAX_RESPONSE_BYTES = 8 * 1024 * 1024
+
+    def __init__(self, username, password, timeout=45):
+        if not isinstance(username, str) or not username.strip() or not isinstance(password, str) or not password:
+            raise ValueError("Set the Solar.web account username and password before recovery")
+        self._username, self._password = username.strip(), password
+        self.timeout = timeout
+        self.cookies = CookieJar()  # In memory only; sessions are never written to disk.
+        self.opener = build_opener(NoRedirects(), HTTPCookieProcessor(self.cookies))
+        self.authenticated = False
+        self.chart_systems = set()
+
+    def _send(self, url, form, referer, as_json):
+        headers = {"Accept": "application/json" if as_json else "text/html",
+                   "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 "
+                                 "(KHTML, like Gecko) Chrome/156.0.0.0 Safari/537.36"}
+        if as_json:
+            headers["X-Requested-With"] = "XMLHttpRequest"
+        if referer:
+            # SSO query parameters must never appear in a cross-origin Referer.
+            headers["Referer"] = origin(referer) + "/"
+        body = None
+        if form is not None:
+            body = urlencode(form).encode("utf-8")
+            headers.update({"Content-Type": "application/x-www-form-urlencoded", "Origin": origin(referer)})
+        request = Request(url, data=body, headers=headers)
+        try:
+            try:
+                response = self.opener.open(request, timeout=self.timeout)
+            except HTTPError as error:
+                response = error
+            with response:
+                raw = response.read(self.MAX_RESPONSE_BYTES + 1)
+                result = Response(response.code, url, raw, response.headers.get("Location", ""))
+        except (URLError, OSError, HTTPException, ValueError) as error:
+            raise TransportError(f"Solar.web network request failed ({type(error).__name__})") from None
+        if len(raw) > self.MAX_RESPONSE_BYTES:
+            raise TransportError("Solar.web response exceeds 8 MiB")
+        return result
+
+    def _request(self, url, form=None, referer="", as_json=False):
+        for _ in range(12):
+            validate_url(url)
+            response = self._send(url, form, referer, as_json)
+            if response.status not in (301, 302, 303, 307, 308):
+                return response
+            if not response.location:
+                raise TransportError("Solar.web returned a redirect without a destination")
+            if form is not None and response.status in (307, 308):
+                raise TransportError("Solar.web attempted to redirect a sensitive POST. Request blocked.")
+            referer, url = url, urljoin(url, response.location)
+            form = None
+        raise TransportError("Solar.web exceeded the redirect limit")
+
+    @staticmethod
+    def _signed_out(response):
+        path = urlsplit(response.url).path.lower()
+        return (response.status in (401, 403) or origin(response.url) != API_SERVER
+                or path.startswith("/account/externallogin")
+                or bool(re.search(rb'''(?:type\s*=\s*["']password["']|href\s*=\s*["']/Account/ExternalLogin(?:[?"']))''',
+                                  response.body, re.I)))
+
+    @staticmethod
+    def _check_status(response):
+        if response.status != 200:
+            raise TransportError(f"Solar.web returned HTTP {response.status}. No response content was logged.")
+
+    def _login(self):
+        response = self._request(API_SERVER + "/Account/ExternalLogin")
+        submitted = callback = False
+        for _ in range(6):
+            self._check_status(response)
+            if callback and not self._signed_out(response):
+                self.authenticated = True
+                return
+            next_form = None
+            for form in Forms(response.body).forms:
+                if form["password"]:
+                    if submitted or form["captcha"]:
+                        raise TransportError(LOGIN_ERROR)
+                    action = urljoin(response.url, form["action"])
+                    validate_url(action)
+                    if (origin(response.url) != LOGIN_SERVER or origin(action) != LOGIN_SERVER
+                            or urlsplit(action).path != "/commonauth" or form["method"] != "post"
+                            or not form["fields"].get("sessionDataKey")):
+                        raise TransportError("Solar.web sign-in form changed; credentials were not submitted.")
+                    fields = dict(form["fields"], username=self._username, password=self._password)
+                    if form["user_input"]:
+                        fields["usernameUserInput"] = self._username
+                    next_form = action, fields
+                    submitted = True
+                    break
+                if (submitted and not form["visible"] and "state" in form["fields"]
+                        and ("id_token" in form["fields"] or "code" in form["fields"])):
+                    action = urljoin(response.url, form["action"])
+                    validate_url(action)
+                    if (origin(response.url) != LOGIN_SERVER or origin(action) != API_SERVER
+                            or urlsplit(action).path != "/Account/ExternalLoginCallback" or form["method"] != "post"):
+                        raise TransportError("Solar.web sign-in callback changed; authentication was stopped.")
+                    next_form = action, form["fields"]
+                    callback = True
+                    break
+            if next_form is None:
+                raise TransportError(LOGIN_ERROR)
+            response = self._request(*next_form, referer=response.url)
+        raise TransportError(LOGIN_ERROR)
+
+    def _reset(self):
+        self.authenticated = False
+        self.chart_systems.clear()
+        self.cookies.clear()
+
+    def get_chart(self, system_id, dataset, date):
+        view, series_id = DATASETS[dataset]
+        chart_url = API_SERVER + "/Chart/Chart?" + urlencode({"pvSystemId": system_id})
+        query = urlencode({"pvSystemId": system_id, "year": date.year, "month": date.month,
+                           "day": date.day, "interval": "day", "view": view})
+        for _ in range(2):  # Reauthenticate an expired session once; never retry rejected credentials here.
+            if not self.authenticated:
+                self._login()
+            if system_id not in self.chart_systems:
+                page = self._request(chart_url)
+                if self._signed_out(page):
+                    self._reset()
+                    continue
+                self._check_status(page)
+                self.chart_systems.add(system_id)
+            response = self._request(API_SERVER + "/Chart/GetChartNew?" + query, referer=chart_url, as_json=True)
+            if self._signed_out(response):
+                self._reset()
+                continue
+            self._check_status(response)
+            return decode_chart(response.body, series_id)
+        raise TransportError("Solar.web session expired or access to this PV system was denied. No guest access was attempted.")
+
+
+def decode_chart(body, series_id):
+    try:
+        data = json.loads(body, parse_constant=_invalid_constant)
+        series = data["settings"]["series"]
+        if not isinstance(series, list):
+            raise ValueError
+        matches = [item for item in series if isinstance(item, dict) and item.get("id") == series_id]
+        if len(matches) != 1:
+            raise ValueError
+        item = matches[0]
+        if item["tooltip"]["valueSuffix"].strip() != "W" or not isinstance(item["data"], list):
+            raise ValueError
+        points, seen = {}, set()
+        for point in item["data"]:
+            if not isinstance(point, list) or len(point) != 2:
+                raise ValueError
+            stamp, watts = point
+            if (isinstance(stamp, bool) or not isinstance(stamp, (int, float))
+                    or not 0 <= stamp <= 253402300799000 or stamp % 300_000 or stamp in seen):
+                raise ValueError
+            seen.add(stamp)
+            if watts is None:
+                continue
+            if (isinstance(watts, bool) or not isinstance(watts, (int, float))
+                    or not 0 <= watts <= 1e18 or not math.isfinite(watts)):
+                raise ValueError
+            points[int(stamp)] = watts
+        return points
+    except (KeyError, TypeError, ValueError, AttributeError, UnicodeError, RecursionError):
+        raise ChartDataError("Solar.web chart series, samples or units changed. Import stopped; no response content was logged.") from None

solar_relay/status.py

@@ -121,7 +121,7 @@ class StatusServer:
                               "poll_interval_seconds": self.config.collector.poll_interval_seconds},
                 "archive": {"samples": archive["samples"], "pending": archive["pending"], "last_upload": archive["last_upload"],
                             "upload_enabled": bool(self.config.upload.url), "format": "parquet", "retention_days": self.config.collector.retention_days},
-                "history": {"enabled": self.config.history.enabled, "provider": "solarweb_query_api",
+                "history": {"enabled": self.config.history.enabled, "provider": "solarweb_account_charts",
                             "boiler_history_available": False},
                 "days": self.store.coverage(self.config.collector.source_id, self.config.collector.timezone, days, now),
             }

solar_relay/upload.py

@@ -20,9 +20,9 @@ class Uploader:
             raise ValueError("Set upload.url to the website's ingestion endpoint first")
         self.config = config
         self.store = store
-        self.token = token if token is not None else os.environ.get(config.token_env, "")
-        if not self.token or any(ord(c) < 33 or ord(c) > 126 for c in self.token):
-            raise ValueError(f"Set {config.token_env} to a nonempty bearer token without whitespace")
+        self.token = token if token is not None else config.token or os.environ.get(config.token_env, "")
+        if not isinstance(self.token, str) or not self.token or any(ord(c) < 33 or ord(c) > 126 for c in self.token):
+            raise ValueError(f"Set upload.token in config.toml or {config.token_env} to a nonempty bearer token without whitespace")
         self.http = http or JsonHttp(config.request_timeout_seconds)
 
     def upload_once(self) -> int:

tests/test_history.py

@@ -1,14 +1,16 @@
-from copy import deepcopy
+from dataclasses import replace
 from datetime import datetime, timedelta, timezone
 from pathlib import Path
 import tempfile
 import unittest
+from unittest.mock import Mock, patch
 import pyarrow.parquet as pq
-from urllib.parse import parse_qs, urlsplit
+from zoneinfo import ZoneInfo
 
 from solar_relay.config import HistoryConfig, utc_text
-from solar_relay.history import CHANNELS, HistoryRecovery, history_sample
+from solar_relay.history import CHANNELS, HistoryRecovery, chart_entries, history_sample
 from solar_relay.http import TransportError
+from solar_relay.solarweb import ChartDataError
 from solar_relay.storage import Store
 from tests.test_status import observed
 
@@ -22,13 +24,16 @@ def entry(when):
                          for field, channel in CHANNELS.items()]}
 
 
-class FakeHistoryHttp:
-    def __init__(self, entries):
-        self.entries, self.requests = entries, []
+class FakeCharts:
+    def __init__(self, when):
+        self.points = {dataset: {int((when + timedelta(minutes=offset)).timestamp() * 1000): watts
+                                 for offset in [0, 5, 10]}
+                       for dataset, watts in [("production", 1200), ("consumption", 1440)]}
+        self.requests = []
 
-    def request(self, url, **kwargs):
-        self.requests.append((url, kwargs))
-        return {"pvSystemId": SYSTEM, "data": deepcopy(self.entries)}
+    def get_chart(self, system_id, dataset, date):
+        self.requests.append((system_id, dataset, date))
+        return dict(self.points[dataset])
 
 
 class HistoryTests(unittest.TestCase):
@@ -38,9 +43,9 @@ class HistoryTests(unittest.TestCase):
         self.store = Store(Path(self.directory.name) / "archive")
         self.when = datetime(2026, 10, 30, 10, tzinfo=timezone.utc)
         self.source = "test-house"
-        self.http = FakeHistoryHttp([entry(self.when + timedelta(minutes=offset)) for offset in [0, 5, 10]])
+        self.client = FakeCharts(self.when)
         self.config = HistoryConfig(enabled=True, pv_system_id=SYSTEM)
-        self.recovery = HistoryRecovery(self.config, self.source, self.store, http=self.http, credentials=("id-test", "value-test"))
+        self.recovery = HistoryRecovery(self.config, self.source, self.store, client=self.client)
 
     def recover(self):
         return self.recovery.recover(utc_text(self.when), utc_text(self.when + timedelta(minutes=15)), now=self.when + timedelta(hours=4))
@@ -56,12 +61,12 @@ class HistoryTests(unittest.TestCase):
         self.assertIsNone(recovered[0]["readings"]["boiler_enabled"])
         self.assertNotIn("boiler", recovered[0]["source_timestamps"])
         self.assertIn("boiler", recovered[0]["errors"])
-        url, kwargs = self.http.requests[0]
-        self.assertEqual(urlsplit(url).hostname, "swqapi.solarweb.com")
-        self.assertEqual(kwargs["extra_headers"], {"AccessKeyId": "id-test", "AccessKeyValue": "value-test"})
-        self.assertNotIn("value-test", url)
+        self.assertEqual(self.client.requests, [(SYSTEM, dataset, self.when.date())
+                                               for dataset in ("production", "consumption")])
+        self.assertEqual(recovered[0]["raw"]["solarweb_history"]["chart_power_w"],
+                         {"production": 1200, "consumption": 1440})
         self.assertEqual(self.recover(), 0)
-        self.assertEqual(len(self.http.requests), 1)
+        self.assertEqual(len(self.client.requests), 2)
 
     def test_cloud_energy_does_not_mark_boiler_or_meter_history_complete(self):
         self.assertEqual(self.recover(), 3)
@@ -76,10 +81,11 @@ class HistoryTests(unittest.TestCase):
         self.assertEqual(row["interval_energy_wh"]["production"], 100)
 
     def test_partial_cloud_records_remain_missing_for_retry(self):
-        self.http.entries[1]["channels"][0]["value"] = None
+        stamp = int((self.when + timedelta(minutes=5)).timestamp() * 1000)
+        del self.client.points["production"][stamp]
         self.assertEqual(self.recover(), 2)
         self.assertEqual(self.recover(), 0)
-        self.http.entries[1] = entry(self.when + timedelta(minutes=5))
+        self.client.points["production"][stamp] = 1200
         self.assertEqual(self.recover(), 1)
 
     def test_recovered_identifiers_are_stable_for_duplicate_delivery(self):
@@ -99,23 +105,129 @@ class HistoryTests(unittest.TestCase):
             history_sample(value, self.source, SYSTEM)
 
     def test_missing_private_credentials_fail_before_requests(self):
-        with self.assertRaisesRegex(ValueError, "SOLARWEB_ACCESS_KEY"):
+        with self.assertRaisesRegex(ValueError, "SOLARWEB_USERNAME and SOLARWEB_PASSWORD"):
             HistoryRecovery(self.config, self.source, self.store, credentials=("", ""))
 
-    def test_requests_are_split_at_utc_midnight_even_across_dst(self):
-        self.http.entries = []
+    def test_config_credentials_take_precedence_over_environment(self):
+        config = replace(self.config, username="private@example.test", password="synthetic password é &")
+        with patch("solar_relay.history.os.environ", {"SOLARWEB_USERNAME": "env@example.test", "SOLARWEB_PASSWORD": "env-password"}), \
+                patch("solar_relay.history.SolarWebClient", return_value=self.client) as client:
+            recovery = HistoryRecovery(config, self.source, self.store)
+            self.assertEqual(recovery.recover(utc_text(self.when), utc_text(self.when + timedelta(minutes=15)),
+                                              now=self.when + timedelta(hours=4)), 3)
+            client.assert_called_once_with("private@example.test", "synthetic password é &", timeout=config.request_timeout_seconds)
+
+    def test_empty_config_credentials_use_environment_fallback(self):
+        config = replace(self.config, username_env="BACKUP_USER", password_env="BACKUP_PASSWORD")
+        with patch("solar_relay.history.os.environ", {"BACKUP_USER": "env@example.test", "BACKUP_PASSWORD": "env-password"}), \
+                patch("solar_relay.history.SolarWebClient", return_value=self.client) as client:
+            HistoryRecovery(config, self.source, self.store)
+            client.assert_called_once_with("env@example.test", "env-password", timeout=config.request_timeout_seconds)
+
+    def test_partial_config_credentials_never_mix_with_environment(self):
+        for changes in [{"username": "private@example.test"}, {"password": "private-password"}]:
+            with self.subTest(changes=changes), \
+                    patch("solar_relay.history.os.environ", {"SOLARWEB_USERNAME": "env@example.test", "SOLARWEB_PASSWORD": "env-password"}), \
+                    patch("solar_relay.history.SolarWebClient") as client:
+                with self.assertRaisesRegex(ValueError, "history.username and history.password"):
+                    HistoryRecovery(replace(self.config, **changes), self.source, self.store)
+                client.assert_not_called()
+
+    def test_requests_use_local_dates_across_dst(self):
         self.recovery.recover("2026-10-24T22:00:00Z", "2026-10-26T00:00:00Z", now=datetime(2026, 10, 27, tzinfo=timezone.utc))
-        self.assertEqual(len(self.http.requests), 2)
-        for url, _ in self.http.requests:
-            query = parse_qs(urlsplit(url).query)
-            start, end = [datetime.fromisoformat(query[name][0]) for name in ("from", "to")]
-            self.assertLessEqual((end - start).total_seconds(), 86400)
+        self.assertEqual([date.isoformat() for _, dataset, date in self.client.requests if dataset == "production"],
+                         ["2026-10-25", "2026-10-26"])
 
     def test_recent_intervals_wait_for_the_hourly_cloud_upload(self):
-        self.http.entries = []
-        self.recovery.recover(utc_text(self.when), utc_text(self.when + timedelta(hours=4)), now=self.when + timedelta(hours=4))
-        query = parse_qs(urlsplit(self.http.requests[0][0]).query)
-        self.assertEqual(query["to"], [utc_text(self.when + timedelta(hours=2))])
+        for points in self.client.points.values():
+            points.clear()
+            for offset in [0, 115, 120, 235]:
+                points[int((self.when + timedelta(minutes=offset)).timestamp() * 1000)] = 1200
+        self.assertEqual(self.recovery.recover(utc_text(self.when), utc_text(self.when + timedelta(hours=4)),
+                                              now=self.when + timedelta(hours=4)), 2)
+
+    def test_automatic_recovery_checks_retention_before_first_observation_and_respects_cutoff(self):
+        now = datetime(2026, 11, 2, 12, 2, 17, tzinfo=timezone.utc)
+        cutoff = datetime(2026, 10, 3, 12, 5, tzinfo=timezone.utc)
+        latest = datetime(2026, 11, 2, 10, tzinfo=timezone.utc)
+        recoverable = [cutoff, now.replace(minute=0, second=0) - timedelta(days=25), latest - timedelta(minutes=5)]
+        points = {int(when.timestamp() * 1000): 1200 for when in
+                  [cutoff - timedelta(minutes=5), *recoverable, latest]}
+        self.client.points = {"production": points, "consumption": points}
+        self.store.add(observed(now.replace(minute=0, second=0)))
+        self.assertEqual(self.recovery.recover(now=now), 3)
+        recovered = [record for record in self.store.records() if "history_interval" in record]
+        self.assertEqual([record["observed_at"] for record in recovered], [utc_text(when) for when in recoverable])
+        self.assertEqual(min(day for _, _, day in self.client.requests), cutoff.date())
+
+    def test_automatic_recovery_checks_entirely_empty_days_and_archives(self):
+        self.store.retention_days = 2
+        self.assertEqual(self.recovery.recover(now=self.when + timedelta(days=2)), 3)
+        self.assertEqual({day for _, _, day in self.client.requests},
+                         {(self.when + timedelta(days=offset)).date() for offset in range(3)})
+
+    def test_automatic_recovery_supports_retention_longer_than_manual_limit(self):
+        self.store.retention_days = 45
+        now = self.when + timedelta(days=40)
+        self.assertEqual(self.recovery.recover(now=now), 3)
+        self.assertEqual(min(day for _, _, day in self.client.requests), (now - timedelta(days=45)).date())
+        with self.assertRaisesRegex(ValueError, "at most 30 days"):
+            self.recovery.recover(utc_text(now - timedelta(days=45)), utc_text(now), now=now)
+
+    def test_complete_retention_window_does_not_request_solarweb(self):
+        now = self.when + timedelta(days=1)
+        lower = int(self.when.timestamp())
+        upper = int((now - timedelta(seconds=self.config.lag_seconds)).timestamp())
+        store = Mock(retention_days=1)
+        store.energy_slots.return_value = set(range(lower, upper, 300))
+        recovery = HistoryRecovery(self.config, self.source, store, client=self.client)
+        self.assertEqual(recovery.recover(now=now), 0)
+        self.assertEqual(self.client.requests, [])
+        store.add.assert_not_called()
+
+    def test_unavailable_cloud_day_does_not_block_newer_missing_days(self):
+        self.store.retention_days = 2
+        available = self.when + timedelta(days=1)
+        self.client.points = {dataset: {int(available.timestamp() * 1000): 1200}
+                              for dataset in ("production", "consumption")}
+        get_chart = self.client.get_chart
+
+        def charts(system, dataset, day):
+            if day == self.when.date():
+                raise ChartDataError("synthetic secret response must not be logged")
+            return get_chart(system, dataset, day)
+
+        with patch.object(self.client, "get_chart", side_effect=charts), self.assertLogs("solar_relay.history", level="WARNING") as logs:
+            self.assertEqual(self.recovery.recover(now=self.when + timedelta(days=2)), 1)
+        self.assertNotIn("synthetic secret response", " ".join(logs.output))
+        self.assertEqual(next(self.store.records())["observed_at"], utc_text(available))
+
+    def test_login_or_transport_failure_stops_the_scan_without_repeated_attempts(self):
+        with patch.object(self.client, "get_chart", side_effect=TransportError("Login failed")) as client:
+            with self.assertRaisesRegex(TransportError, "Login failed"):
+                self.recovery.recover(now=self.when + timedelta(days=2))
+        self.assertEqual(client.call_count, 1)
+
+    def test_chart_conversion_matches_solar_and_retains_provenance(self):
+        for generated, consumed in [(1200, 1440), (2400, 1200), (0, 0)]:
+            with self.subTest(generated=generated, consumed=consumed):
+                stamp = int(self.when.timestamp() * 1000)
+                record = next(chart_entries({stamp: generated}, {stamp: consumed}))
+                values = history_sample(record, self.source, SYSTEM)["history_interval"]["energy_wh"]
+                self.assertEqual(values, {"production": generated / 12, "consumption": consumed / 12,
+                                          "self_consumption": min(generated, consumed) / 12,
+                                          "bought": max(0, consumed - generated) / 12,
+                                          "sold": max(0, generated - consumed) / 12})
+                self.assertEqual(record["energy_method"], "constant_power_over_five_minutes")
+
+    def test_dst_days_recover_every_actual_five_minute_slot(self):
+        for day, expected in [("2026-03-29", 276), ("2026-10-25", 300)]:
+            with self.subTest(day=day):
+                start = datetime.fromisoformat(day).replace(tzinfo=ZoneInfo("Europe/Zurich"))
+                end = start + timedelta(days=1)
+                points = {stamp: 1200 for stamp in range(int(start.timestamp() * 1000), int(end.timestamp() * 1000), 300_000)}
+                self.client.points = {"production": points, "consumption": points}
+                self.assertEqual(self.recovery.recover(utc_text(start), utc_text(end), now=end + timedelta(days=1)), expected)
 
 
 if __name__ == "__main__":

tests/test_relay.py

@@ -10,7 +10,7 @@ import threading
 import unittest
 import subprocess
 import sys
-from unittest.mock import patch
+from unittest.mock import Mock, patch
 from urllib.parse import urlsplit
 
 from solar_relay.config import InverterConfig, UploadConfig, date_bound, load_config
@@ -244,7 +244,7 @@ class DeliveryTests(unittest.TestCase):
             reading = client.collect("test")
             self.assertEqual(reading["errors"], {})
             self.store.add(reading)
-            uploader = Uploader(UploadConfig(url=base + "/ingest"), self.store, token="test-token")
+            uploader = Uploader(UploadConfig(url=base + "/ingest", token="test-token"), self.store)
             httpd.fail_once = True
             with self.assertRaises(UploadError):
                 uploader.upload_once()
@@ -323,6 +323,27 @@ class DeliveryTests(unittest.TestCase):
             with self.assertRaisesRegex(ValueError, "SOLAR_RELAY_TOKEN"):
                 Uploader(UploadConfig(url="https://example.test/ingest"), self.store)
 
+    def test_config_upload_token_takes_precedence_over_environment(self):
+        reading = sample()
+        self.store.add(reading)
+        http = Mock()
+        http.request.return_value = {"accepted_ids": [reading["id"]]}
+        with patch("solar_relay.upload.os.environ", {"SOLAR_RELAY_TOKEN": "env-token"}):
+            uploader = Uploader(UploadConfig(url="https://example.test/ingest", token="config-token"), self.store, http=http)
+            self.assertEqual(uploader.upload_once(), 1)
+        self.assertEqual(http.request.call_args.kwargs["token"], "config-token")
+
+    def test_empty_config_token_uses_environment_fallback(self):
+        with patch("solar_relay.upload.os.environ", {"BACKUP_TOKEN": "env-token"}):
+            uploader = Uploader(UploadConfig(url="https://example.test/ingest", token_env="BACKUP_TOKEN"), self.store)
+        self.assertEqual(uploader.token, "env-token")
+
+    def test_invalid_config_token_does_not_fall_back_or_leak_its_value(self):
+        with patch("solar_relay.upload.os.environ", {"SOLAR_RELAY_TOKEN": "env-token"}):
+            with self.assertRaises(ValueError) as raised:
+                Uploader(UploadConfig(url="https://example.test/ingest", token="invalid secret token"), self.store)
+        self.assertNotIn("invalid secret token", str(raised.exception))
+
     def test_cli_sample_and_export_use_the_persistent_archive(self):
         with server() as (_, base):
             config = Path(self.directory.name) / "config.toml"
@@ -381,6 +402,51 @@ class ConfigTests(unittest.TestCase):
         with self.assertRaises(ValueError):
             self.load('\n[http]\nbase_path="/../config"\n')
 
+    def test_history_account_settings_and_migration_from_query_api_keys(self):
+        config, _ = self.load('\n[history]\nusername_env="BACKUP_USER"\npassword_env="BACKUP_PASSWORD"\n')
+        self.assertEqual(config.history.username_env, "BACKUP_USER")
+        self.assertEqual(config.history.password_env, "BACKUP_PASSWORD")
+        self.assertFalse(config.history.enabled)
+        config, _ = self.load('\n[history]\naccess_key_id_env="OLD_ID"\naccess_key_value_env="OLD_KEY"\n')
+        self.assertEqual(config.history.username_env, "SOLARWEB_USERNAME")
+        self.assertEqual(config.history.password_env, "SOLARWEB_PASSWORD")
+        for name in ("username_env", "password_env"):
+            with self.subTest(name=name), self.assertRaisesRegex(ValueError, "environment variable"):
+                self.load(f'\n[history]\n{name}="invalid name"\n')
+
+    def test_history_uses_retention_instead_of_legacy_lookback_setting(self):
+        with tempfile.TemporaryDirectory() as directory:
+            path = Path(directory) / "synthetic-test.toml"
+            path.write_text('[collector]\nretention_days=2\n[history]\nenabled=true\n'
+                            'pv_system_id="73733052-6f10-47a5-9746-7f7e76ebcb8c"\nlookback_days=7\n')
+            config = load_config(path)
+        self.assertEqual(config.collector.retention_days, 2)
+        self.assertTrue(config.history.enabled)
+        self.assertFalse(hasattr(config.history, "lookback_days"))
+
+    def test_private_config_accepts_credentials_and_redacts_them_from_repr(self):
+        config, _ = self.load('\n[upload]\ntoken="synthetic-shared-token"\n'
+                              '\n[history]\nusername="synthetic@example.test"\npassword="synthetic password é &"\n')
+        self.assertEqual(config.upload.token, "synthetic-shared-token")
+        self.assertEqual(config.history.username, "synthetic@example.test")
+        self.assertEqual(config.history.password, "synthetic password é &")
+        for secret in (config.upload.token, config.history.username, config.history.password):
+            self.assertNotIn(secret, repr(config))
+
+    def test_partial_or_mistyped_private_credentials_are_rejected_without_values(self):
+        for fields in ['username="synthetic@example.test"', 'password="synthetic-secret"',
+                       'username="   "\npassword="synthetic-secret"', 'username=42', 'password=42']:
+            with self.subTest(fields=fields):
+                with self.assertRaises(ValueError) as raised:
+                    self.load(f'\n[history]\n{fields}\n')
+                self.assertNotIn("synthetic-secret", str(raised.exception))
+                self.assertNotIn("synthetic@example.test", str(raised.exception))
+        for value in ['42', '"invalid secret token"']:
+            with self.subTest(value=value):
+                with self.assertRaisesRegex(ValueError, "upload.token") as raised:
+                    self.load(f'\n[upload]\ntoken={value}\n')
+                self.assertNotIn("invalid secret token", str(raised.exception))
+
 
 if __name__ == "__main__":
     unittest.main()

tests/test_schedule.py

@@ -63,6 +63,11 @@ class ScheduleTests(unittest.TestCase):
         self.assertEqual(clock.waits[0], (301, 299))
         self.assertTrue(all(stamp % 300 == 0 for stamp in recorded))
 
+    def test_default_live_collection_never_initializes_cloud_recovery(self):
+        with patch("solar_relay.cli.HistoryRecovery", side_effect=AssertionError("Unexpected cloud recovery")):
+            recorded, _ = self.run_collector(301)
+        self.assertEqual(recorded, [600, 900, 1200])
+
     def test_restart_in_the_same_interval_adds_no_extra_observation(self):
         before, _ = self.run_collector(590, count=1)
         after, _ = self.run_collector(615, count=2)

tests/test_solarweb.py

@@ -0,0 +1,203 @@
+"""Exercise browser authentication with synthetic credentials and responses only."""
+
+from datetime import date
+from io import BytesIO
+import json
+import unittest
+from urllib.error import URLError
+from urllib.parse import parse_qs, urlsplit
+from unittest.mock import Mock
+
+from solar_relay.http import TransportError
+from solar_relay.solarweb import API_SERVER, LOGIN_SERVER, Response, SolarWebClient, decode_chart, validate_url
+
+SYSTEM = "73733052-6f10-47a5-9746-7f7e76ebcb8c"
+DAY = date(2026, 10, 30)
+STAMP = 1793354400000
+PASSWORD = "synthetic password & Unicode é"
+LOGIN_FORM = b'''<form action="/commonauth" method="post">
+<input type="hidden" name="sessionDataKey" value="synthetic-session">
+<input name="usernameUserInput"><input type="password" name="password">
+<input type="hidden" name="disabled" value="unused" disabled></form>'''
+CALLBACK_FORM = b'''<form action="https://www.solarweb.com/Account/ExternalLoginCallback" method="post">
+<input type="hidden" name="state" value="synthetic-state">
+<input type="hidden" name="id_token" value="synthetic-token"></form>'''
+
+
+def chart(series_id="FromGen", points=None, suffix=" W"):
+    return json.dumps({"settings": {"series": [{"id": series_id, "name": "Localized display name",
+                                                "tooltip": {"valueSuffix": suffix},
+                                                "data": points if points is not None else [[STAMP, 1200], [STAMP + 300_000, None]]}]}}).encode()
+
+
+def login_steps():
+    return [(302, b"", LOGIN_SERVER + "/authenticationendpoint/login.do?sessionDataKey=synthetic-session"),
+            (200, LOGIN_FORM, ""), (200, CALLBACK_FORM, ""),
+            (302, b"", API_SERVER + "/"), (200, b"<html>Account</html>", "")]
+
+
+class ScriptedClient(SolarWebClient):
+    def __init__(self, steps):
+        super().__init__("account@example.test", PASSWORD)
+        self.steps, self.requests = list(steps), []
+
+    def _send(self, url, form, referer, as_json):
+        self.requests.append((url, form, referer, as_json))
+        if not self.steps:
+            raise AssertionError("Unexpected request")
+        status, body, location = self.steps.pop(0)
+        return Response(status, url, body, location)
+
+
+class LoginTests(unittest.TestCase):
+    def test_real_account_flow_and_authenticated_charts_reuse_session(self):
+        client = ScriptedClient(login_steps() + [(200, b"Chart", ""), (200, chart(), ""),
+                                                 (200, chart("ToConsumer"), "")])
+        self.assertEqual(client.get_chart(SYSTEM, "production", DAY), {STAMP: 1200})
+        self.assertEqual(client.get_chart(SYSTEM, "consumption", DAY), {STAMP: 1200})
+        self.assertTrue(client.authenticated)
+        self.assertFalse(client.steps)
+        posts = [(url, form) for url, form, _, _ in client.requests if form is not None]
+        self.assertEqual(posts, [(LOGIN_SERVER + "/commonauth", {"sessionDataKey": "synthetic-session",
+                                 "username": "account@example.test", "usernameUserInput": "account@example.test",
+                                 "password": PASSWORD}),
+                                (API_SERVER + "/Account/ExternalLoginCallback",
+                                 {"state": "synthetic-state", "id_token": "synthetic-token"})])
+        for request, view in zip(client.requests[-2:], ["consumption", "production"]):
+            url, form, referer, as_json = request
+            query = parse_qs(urlsplit(url).query)
+            self.assertEqual(query, {"pvSystemId": [SYSTEM], "year": ["2026"], "month": ["10"],
+                                     "day": ["30"], "interval": ["day"], "view": [view]})
+            self.assertIsNone(form)
+            self.assertTrue(as_json)
+            self.assertEqual(urlsplit(referer).path, "/Chart/Chart")
+        for url, _, _, _ in client.requests:
+            self.assertNotIn(PASSWORD, url)
+            self.assertNotIn("GuestLogOn", url)
+            self.assertNotIn("swqapi", url)
+
+    def test_expired_chart_session_is_renewed_once(self):
+        client = ScriptedClient(login_steps() + [(200, b"Chart", ""), (401, b"", "")]
+                                + login_steps() + [(200, b"Chart", ""), (200, chart(), "")])
+        self.assertEqual(client.get_chart(SYSTEM, "production", DAY), {STAMP: 1200})
+        self.assertEqual(sum(form is not None and "password" in form for _, form, _, _ in client.requests), 2)
+
+    def test_persistent_denial_stops_after_one_session_renewal(self):
+        client = ScriptedClient(login_steps() + [(403, b"", "")] + login_steps() + [(403, b"", "")])
+        with self.assertRaisesRegex(TransportError, "No guest access"):
+            client.get_chart(SYSTEM, "production", DAY)
+        self.assertFalse(client.steps)
+
+    def test_rejected_credentials_are_submitted_only_once(self):
+        client = ScriptedClient(login_steps()[:2] + [(200, LOGIN_FORM, "")])
+        with self.assertRaisesRegex(TransportError, "login failed"):
+            client.get_chart(SYSTEM, "production", DAY)
+        self.assertEqual(sum(form is not None for _, form, _, _ in client.requests), 1)
+
+    def test_captcha_or_mfa_does_not_trigger_password_retries(self):
+        for body in [LOGIN_FORM.replace(b"</form>", b'<div data-sitekey="captcha"></div></form>'),
+                     b'<form method="post"><input name="otp"></form>']:
+            with self.subTest(body=body):
+                client = ScriptedClient([login_steps()[0], (200, body, "")])
+                with self.assertRaisesRegex(TransportError, "login failed"):
+                    client.get_chart(SYSTEM, "production", DAY)
+                self.assertFalse(any(form is not None for _, form, _, _ in client.requests))
+
+    def test_changed_or_untrusted_password_form_is_not_submitted(self):
+        bodies = [LOGIN_FORM.replace(b"/commonauth", b"https://evil.example/commonauth"),
+                  LOGIN_FORM.replace(b"/commonauth", b"/other"), LOGIN_FORM.replace(b'"post"', b'"get"'),
+                  LOGIN_FORM.replace(b"sessionDataKey", b"differentKey")]
+        for body in bodies:
+            with self.subTest(body=body):
+                client = ScriptedClient([login_steps()[0], (200, body, "")])
+                with self.assertRaises(TransportError):
+                    client.get_chart(SYSTEM, "production", DAY)
+                self.assertFalse(any(form is not None for _, form, _, _ in client.requests))
+
+    def test_untrusted_callback_is_never_sent_tokens(self):
+        for target in [b"https://evil.example/callback", b"https://login.fronius.com/commonauth",
+                       b"https://www.solarweb.com/other"]:
+            with self.subTest(target=target):
+                body = CALLBACK_FORM.replace(b"https://www.solarweb.com/Account/ExternalLoginCallback", target)
+                client = ScriptedClient(login_steps()[:2] + [(200, body, "")])
+                with self.assertRaises(TransportError):
+                    client.get_chart(SYSTEM, "production", DAY)
+                self.assertEqual(sum(form is not None for _, form, _, _ in client.requests), 1)
+
+    def test_guest_and_untrusted_redirects_are_blocked_before_network(self):
+        for target in [API_SERVER + "/Account/GuestLogOn?pvSystemId=x", API_SERVER + "/Account/DemoLogOn",
+                       "https://evil.example/", "http://www.solarweb.com/"]:
+            with self.subTest(target=target):
+                client = ScriptedClient([(302, b"", target)])
+                with self.assertRaisesRegex(TransportError, "Request blocked"):
+                    client.get_chart(SYSTEM, "production", DAY)
+                self.assertEqual(len(client.requests), 1)
+
+    def test_post_redirect_cannot_forward_credentials(self):
+        for status in [307, 308]:
+            with self.subTest(status=status):
+                client = ScriptedClient(login_steps()[:2] + [(status, b"", API_SERVER + "/")])
+                with self.assertRaisesRegex(TransportError, "sensitive POST"):
+                    client.get_chart(SYSTEM, "production", DAY)
+                self.assertEqual(len(client.requests), 3)
+
+
+class TransportTests(unittest.TestCase):
+    def test_url_allowlist(self):
+        for url in [API_SERVER + "/", LOGIN_SERVER + "/commonauth", API_SERVER + ":443/Chart/Chart"]:
+            validate_url(url)
+        for url in ["https://www.solarweb.com.evil.example/", "https://user:pass@www.solarweb.com/",
+                    API_SERVER + ":444/", "http://www.solarweb.com/", API_SERVER + "/%47uestLogOn",
+                    API_SERVER + "/DemoLogOn", API_SERVER + "/bad\npath", API_SERVER + "\\@evil.example/"]:
+            with self.subTest(url=url), self.assertRaises(TransportError):
+                validate_url(url)
+
+    def test_transport_redacts_network_errors_and_response_repr(self):
+        client = SolarWebClient("account@example.test", PASSWORD)
+        client.opener = Mock()
+        client.opener.open.side_effect = URLError("synthetic-token " + PASSWORD)
+        with self.assertRaises(TransportError) as raised:
+            client._request(API_SERVER + "/")
+        self.assertNotIn(PASSWORD, str(raised.exception))
+        self.assertNotIn("synthetic-token", str(raised.exception))
+        self.assertEqual(repr(Response(200, API_SERVER + "/?token=synthetic-token", b"secret", "secret")), "Response(status=200)")
+
+    def test_transport_keeps_sso_queries_out_of_referer_and_encodes_password(self):
+        client = SolarWebClient("account@example.test", PASSWORD)
+        response = BytesIO(b"success")
+        response.code, response.headers = 200, {}
+        client.opener = Mock()
+        client.opener.open.return_value = response
+        client._request(LOGIN_SERVER + "/commonauth", {"password": PASSWORD}, LOGIN_SERVER + "/?state=synthetic-token")
+        request = client.opener.open.call_args.args[0]
+        self.assertEqual(request.get_header("Referer"), LOGIN_SERVER + "/")
+        self.assertEqual(request.get_header("Origin"), LOGIN_SERVER)
+        self.assertEqual(parse_qs(request.data.decode())["password"], [PASSWORD])
+
+    def test_responses_are_bounded(self):
+        client = SolarWebClient("account@example.test", PASSWORD)
+        response = BytesIO(b"x" * (client.MAX_RESPONSE_BYTES + 1))
+        response.code, response.headers = 200, {}
+        client.opener = Mock()
+        client.opener.open.return_value = response
+        with self.assertRaisesRegex(TransportError, "exceeds 8 MiB"):
+            client._request(API_SERVER + "/")
+
+
+class ChartTests(unittest.TestCase):
+    def test_series_uses_id_and_skips_null_instead_of_zeroing(self):
+        self.assertEqual(decode_chart(chart(), "FromGen"), {STAMP: 1200})
+
+    def test_bad_units_missing_series_and_invalid_points_stop_import(self):
+        bodies = [b"not JSON", b'{"settings":{"series":[]}}', chart("Other"), chart(suffix="kW"),
+                  chart(points=[[STAMP + 1, 1200]]), chart(points=[[STAMP, True]]),
+                  chart(points=[[STAMP, -1]]), chart(points=[[STAMP, float("nan")]]),
+                  chart(points=[[STAMP, 1], [STAMP, 2]]), chart(points=[[True, 1]]),
+                  chart(points=[[STAMP, "1200"]]), chart(points=[[STAMP]])]
+        for body in bodies:
+            with self.subTest(body=body), self.assertRaises(TransportError):
+                decode_chart(body, "FromGen")
+
+
+if __name__ == "__main__":
+    unittest.main()