Alby96andClaude Opus 5
e14308bbb6
Scanned the two public ARPA Piemonte APIs and recorded what they actually contain, so this question does not get re-litigated from the marketing pages later. Neither needs a key, a token or a registration. This corrects something I got wrong. The docs said ARPA states no licence for its data, based on the open-data page saying only "gratuiti". The site's legal notice does state one — CC BY 4.0, commercial use explicitly permitted, credit "Fonte: Arpa Piemonte - www.arpa.piemonte.it" — and both APIs link that notice from their OpenAPI description, so anything they serve is covered. Only the radar volumes remain ambiguous, because their own page does not repeat the licence and the access link is still issued by email; both questions belong in the same message to ARPA. The survey's operative findings: - api_realtime carries 374 stations with coordinates, of which 286 have a rain gauge, spanning 74 m to 2820 m of elevation. - Its observations lag by about 4.5 hours on an hourly cadence, measured uniformly across every station sampled. That is the property that decides how it can be used: it is an observation archive with a publishing delay, not a real-time feed, and it must never sit next to 5-minute radar looking current. - Only 4 of 143 hydrometer stations publish guard and danger thresholds, and 5 publish a river name. A river-level warning built on this would be empty for 97% of stations, so that idea is recorded as rejected rather than pending. - Meteoweb is a 3.27-million-record historical archive plus a seismic catalogue: a research dataset, not app content. One thing is worth adopting: rain-gauge accumulations as ground truth for the radar. Radar infers rainfall from reflectivity aloft, gauges measure what reached the ground, and the network is densest exactly where Alpine terrain blocks the beam. It belongs behind the worker, clearly timestamped, once the worker exists. Also notes that the credit ARPA asks for is the full "Fonte: Arpa Piemonte - www.arpa.piemonte.it", not the bare name currently in the region config, to be adopted the moment any ARPA-sourced data is displayed. No code changes: nothing is integrated yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
328 lines
14 KiB
Markdown
328 lines
14 KiB
Markdown
# Data sources
|
||
|
||
Every endpoint below was verified on 2026-09-10. Anything not verified is marked
|
||
**TO VERIFY** and must not be relied on until checked.
|
||
|
||
## 1. Radar — Radar-DPC (the active source)
|
||
|
||
Public platform of the Italian Dipartimento della Protezione Civile.
|
||
|
||
- Base URL: `https://radar-api.protezionecivile.it/`
|
||
- Transitional test base URL: `https://wagiqofvnk.execute-api.eu-south-1.amazonaws.com/prod`
|
||
|
||
### `GET /findLastProductByType?type=<PRODUCT>`
|
||
|
||
Returns the timestamp of the most recent sample of a product.
|
||
|
||
```json
|
||
{
|
||
"total": 1,
|
||
"lastProducts": [
|
||
{ "productType": "VMI", "time": 1758706200000, "period": "PT5M" }
|
||
]
|
||
}
|
||
```
|
||
|
||
- `time` is epoch milliseconds, UTC.
|
||
- `period` is the ISO-8601 cadence of the product.
|
||
- `400` when `type` is missing, `404` when no product is available.
|
||
|
||
### `POST /downloadProduct`
|
||
|
||
Returns a pre-signed S3 URL for the raw raster file.
|
||
|
||
```json
|
||
{ "productType": "VMI", "productDate": 1758706200000 }
|
||
```
|
||
|
||
```json
|
||
{
|
||
"bucket": "dpc-radar",
|
||
"key": "VMI/22-09-2025-14-20.tif",
|
||
"url": "https://dpc-radar.s3.eu-south-1.amazonaws.com/...",
|
||
"expiresSeconds": 900
|
||
}
|
||
```
|
||
|
||
- `productDate` must be **rounded down to the product step**, otherwise `404`.
|
||
- Pre-signed URLs expire after 300–900 s: download immediately, never cache the URL.
|
||
- `400` when `productDate` is missing or non-numeric.
|
||
|
||
### Mandatory header
|
||
|
||
```
|
||
origin: https://radar.protezionecivile.it
|
||
```
|
||
|
||
Required on **every** request to the platform for security reasons; requests without it
|
||
are rejected. `Content-Type: application/json` is also required on `downloadProduct`.
|
||
|
||
### Products and cadence
|
||
|
||
| Cadence | Products |
|
||
|---|---|
|
||
| 5 min | `VMI`, `SRI`, `SRT1`, `IR_108` |
|
||
| 30–60 min | `TEMP`, `CUM3`, `CUM6`, `CUM12`, `CUM24`, `CAPPI_1`…`CAPPI_10`, `VIL`, `ETM`, `POH` |
|
||
| — | `SITES` (GeoJSON, fetched directly from S3 with `GET`) |
|
||
|
||
The MVP uses **`VMI`** (Vertical Maximum Intensity) as the precipitation layer.
|
||
History retention is 14 days. Files are GeoTIFF; CRS is read from the file metadata
|
||
rather than assumed — the worker reprojects to EPSG:3857 in all cases.
|
||
|
||
### Raster format (read from a live VMI file, 2026-09-10)
|
||
|
||
| Property | Value |
|
||
|---|---|
|
||
| Size | 1200 × 1400 px |
|
||
| Pixel size | 1000 × 1000 m — a **1 km grid** |
|
||
| Bands / type | 1 band, **Float32**, LZW compressed |
|
||
| Tie point | raster (0,0) → model (-600000, 650000) |
|
||
| Model type | **projected**, datum WGS84 (GeographicTypeGeoKey 4326) |
|
||
| Projection centre | 12.5° E, 42.0° N (`ProjCenterLong`/`ProjCenterLat`) |
|
||
| Linear units | metre |
|
||
| Nodata tag | absent — expect NaN |
|
||
|
||
**The rasters are not in EPSG:4326 and not in EPSG:3857.** They are on a custom
|
||
projection centred on Italy, so reprojection to Web Mercator is mandatory, not an
|
||
optimisation. The extent works out to 1200 km east-west and 1400 km north-south around
|
||
that centre, which covers the whole country — the worker crops to the region bounding
|
||
box before doing anything else, so it never carries the full mosaic through the
|
||
pipeline.
|
||
|
||
The GeoKeys as written are internally inconsistent (`ProjCoordTransGeoKey` = 1, which
|
||
is transverse Mercator, combined with the centre keys an azimuthal projection uses).
|
||
**Do not hardcode a source CRS.** The worker reads the CRS from the file with rasterio
|
||
and warps from whatever it finds, and logs the detected CRS on every run so a change at
|
||
the source is visible instead of silently shifting the imagery.
|
||
|
||
Values are physical units (dBZ for VMI), not palette indices — the colour mapping is
|
||
ours, and the legend that goes with it is published in the manifest.
|
||
|
||
### Push channel
|
||
|
||
`wss://radar-wss.protezionecivile.it` pushes a notification whenever a new sample is
|
||
published. The worker uses this as its **primary trigger**, with a 5-minute cron as a
|
||
fallback, so we never poll DPC aggressively.
|
||
|
||
### License
|
||
|
||
**CC BY-SA.** Attribution to "Radar-DPC" is mandatory and derived products must be
|
||
released under the same license. The rendered PNG frames we publish are a derived
|
||
product and inherit CC BY-SA.
|
||
|
||
Docs: <https://dpc-radar.readthedocs.io/it/latest/>
|
||
|
||
## 2. Radar — ARPA Piemonte (future adapter, disabled)
|
||
|
||
C-band polarimetric radars at Bric della Croce and Monte Settepani, scanning every
|
||
5 minutes. Real-time volumes of reflectivity, Doppler velocity and differential
|
||
reflectivity are published in **OPERA HDF5 (ODIM)** format, covering the last hour.
|
||
|
||
Listed as open data at <https://www.arpa.piemonte.it/dato/open-data>, described there as
|
||
constantly updated, machine-readable and free of charge.
|
||
|
||
**Two things block adoption, and both belong to the project owner, not to the code:**
|
||
|
||
1. **The access link is not public.** The page states that to obtain the real-time link
|
||
for radar and radiosonde data *"è necessario inviare una e-mail all'indirizzo
|
||
info.meteo@arpa.piemonte.it"*. There is no documented endpoint to call without it.
|
||
2. **The licence is stated site-wide, not on that page.** The open-data page itself says
|
||
only that the data is *gratuiti*. ARPA's legal notice at
|
||
<https://www.arpa.piemonte.it/note-legali> does state a licence for environmental and
|
||
open data: **CC BY 4.0, commercial use permitted**, crediting
|
||
*"Fonte: Arpa Piemonte - www.arpa.piemonte.it"*. Logos, emblems and third-party
|
||
content are excluded from it.
|
||
|
||
Both ARPA REST APIs link that same notice from their OpenAPI `description`, so for
|
||
them the licence is unambiguous. The radar page does not repeat it, so the safe move
|
||
is to ask ARPA to confirm the terms apply to the radar volumes in the same email that
|
||
requests the access link.
|
||
|
||
Until the link exists, `ArpaRadarSource` stays a stub whose every method throws. It is
|
||
named by the region config as unavailable so the seam is visible, not because it is
|
||
nearly ready.
|
||
|
||
## 3. ARPA Piemonte REST APIs — surveyed 2026-09-10, not yet adopted
|
||
|
||
Two public APIs, **no authentication, no key, no registration**. Both link
|
||
<https://www.arpa.piemonte.it/note-legali> from their OpenAPI `description`, so both are
|
||
CC BY 4.0 with commercial use permitted.
|
||
|
||
### `api_realtime` — <https://utility.arpa.piemonte.it/api_realtime>
|
||
|
||
Spec at `/api_realtime/openapi.json` (the `/docs` page is JS-rendered and cannot be
|
||
scraped; fetch the spec instead).
|
||
|
||
| Endpoint | Content |
|
||
|---|---|
|
||
| `GET /pie_anag` | Station registry: **374 stations**, `lat`/`lng`, elevation, municipality, province, basin, sensor letters |
|
||
| `GET /data_pie` | Hourly observations, last 3 days: `air_temperature`, `humidity`, `cum_rain_1h/3h/6h/12h/24h`, `wind`, `gust_of_wind` + directions, `snow_height`, `hydrometric_level`. Paginated |
|
||
| `GET /pie_neve` | Daily validated snow height, 08:00 |
|
||
| `GET /ggd` | Heating and cooling degree days |
|
||
| `GET /status` | Health check |
|
||
|
||
Sensor coverage across the 374 stations: **286 with a rain gauge (P)**, 277 temperature,
|
||
143 hydrometer (H), 90 wind, 79 snow. Rain gauges span 74 m to 2820 m elevation.
|
||
|
||
**Measured latency: ~4.5 hours**, uniform across every station sampled at 16:30 on
|
||
2026-09-10, on an hourly cadence. This is the single most important property of the feed:
|
||
it is an observation archive with a publishing delay, **not** a real-time source, and it
|
||
cannot be presented alongside 5-minute radar as if it were current.
|
||
|
||
**Hydrometric thresholds are effectively absent**: only 4 of 143 hydrometer stations
|
||
carry `guard_threshold` / `danger_threshold`, and only 5 carry a river name. A river-level
|
||
warning feature built on this would be blank for 97% of stations.
|
||
|
||
### Meteoweb — <https://utility.arpa.piemonte.it/docs/>
|
||
|
||
Spec at `/schema/` (YAML). Django REST Framework, hyperlinked, paginated.
|
||
|
||
Historical archive under `/meteoidro/`: daily and monthly meteo (**3.27 M** daily
|
||
records), hydrological (**931 k**), snow, century-long series, annual rainfall maxima,
|
||
flow-duration curves and rating curves, plus the station and measurement-point
|
||
registries. Latest daily data was one day behind (`2026-09-09`).
|
||
|
||
Also `/sismica/`: **35 910** seismic events since 2000 with position, depth and
|
||
magnitude, and the seismic classification of municipalities.
|
||
|
||
### Verdict
|
||
|
||
Only one thing here is on-scope: **rain-gauge accumulations as ground truth for the
|
||
radar**. Radar infers rainfall from reflectivity aloft; gauges measure what actually
|
||
reached the ground, and the network is dense exactly where Alpine terrain blocks the
|
||
radar beam. Worth adding as a clearly-timestamped observations layer once the worker
|
||
exists — never as a live reading.
|
||
|
||
Everything else is out of scope for this app: snow depth, degree days, the historical
|
||
archive and the seismic catalogue all belong to a different product. Hydrometric levels
|
||
would be valuable during heavy rain but the thresholds that would make them meaningful
|
||
are not published.
|
||
|
||
## 4. Forecast — out of scope
|
||
|
||
The app shows radar, not forecasts. No forecast provider is integrated, and MET Norway,
|
||
ItaliaMeteo ICON-2I and Open-Meteo are all out of scope. See CLAUDE.md for the current
|
||
scope boundary.
|
||
|
||
## 5. Alerts — ARPA Piemonte XML-CAP
|
||
|
||
- Feed: `https://www.arpa.piemonte.it/export/xmlcap/allerta.xml`
|
||
- Landing page: <https://www.arpa.piemonte.it/rischi_naturali/snippets_arpa/allerta/index.html>
|
||
- Official PDF bulletin: `https://www.arpa.piemonte.it/rischi_naturali/boll/bollettino_allerta.pdf`
|
||
|
||
Issued daily by 13:00 by the Centro Funzionale Regionale, 36 hours of validity.
|
||
|
||
### Alert zones (11)
|
||
|
||
`Piem-A` … `Piem-M` (A, B, C, D, E, F, G, H, I, L, M), covering: Toce; Val Sesia/Cervo/
|
||
Chiusella; Valli Orco/Lanzo/bassa val Susa/Sangone; Alta val Susa/Chisone/Pellice/Po;
|
||
Valli Varaita/Maira/Stura; Valle Tanaro; Belbo e Bormida; Scrivia; Pianura
|
||
settentrionale; Pianura e colline torinesi; Pianura cuneese.
|
||
The exact code-to-name mapping is stored in `app/assets/regions/piemonte.json`.
|
||
|
||
### CAP document structure (verified against the live feed, 2026-09-10)
|
||
|
||
One `<alert>` contains one `<info>` block **per zone**; the zone is in
|
||
`info/area/areaDesc` and holds the bare code (`Piem-A`), not a name.
|
||
|
||
```xml
|
||
<alert xmlns="urn:oasis:names:tc:emergency:cap:1.2">
|
||
<identifier>261/2026</identifier> <!-- bulletin number / year -->
|
||
<sender>centro.funzionale@arpa.piemonte.it</sender>
|
||
<sent>2026-09-09T10:00:02+00:00</sent>
|
||
<status>Actual</status><msgType>Alert</msgType><scope>Public</scope>
|
||
<note>AVVISO DI CONDIZIONI METEOROLOGICHE AVVERSE ...</note>
|
||
<info>
|
||
<event>GIALLO</event> <!-- overall level for this zone -->
|
||
<onset>...</onset><expires>...</expires>
|
||
<parameter><valueName>EFFETTI SUL TERRITORIO</valueName><value>...</value></parameter>
|
||
<parameter><valueName>TEMPORALI_1224</valueName><value>GIALLO</value></parameter>
|
||
<parameter><valueName>TEMPORALI_2436</valueName><value>VERDE</value></parameter>
|
||
...
|
||
<area><areaDesc>Piem-A</areaDesc></area>
|
||
</info>
|
||
<!-- 10 more info blocks, one per zone -->
|
||
</alert>
|
||
```
|
||
|
||
Parameters are named `<CATEGORY>_<WINDOW>`:
|
||
|
||
- **Categories** — `IDRAULICO`, `IDROGEOLOGICO`, `TEMPORALI`, `NEVE`, `VALANGHE`.
|
||
- **Windows** — `1224` (hours 12–24) and `2436` (hours 24–36), matching the 36 hours
|
||
of validity.
|
||
|
||
So each zone carries 10 level values plus one overall `<event>` level and a free-text
|
||
`EFFETTI SUL TERRITORIO`.
|
||
|
||
### Levels — six values, not four
|
||
|
||
| Value | Meaning |
|
||
|---|---|
|
||
| `VERDE` | no significant phenomena |
|
||
| `GIALLO` | localised phenomena |
|
||
| `ARANCIONE` | widespread phenomena |
|
||
| `ROSSO` | numerous or extensive phenomena |
|
||
| `BIANCO` | **avalanche scale only** — outside the season / not assessed |
|
||
| `-` | no value published for that category and window |
|
||
|
||
`BIANCO` and `-` are not "green with a different name" and must not be collapsed into
|
||
it. The parser models the level as an enum with explicit `bianco` and `notAvailable`
|
||
members, and the UI renders them as their own state rather than as "no alert".
|
||
|
||
Levels are republished **verbatim, never reinterpreted**, always alongside a link to
|
||
the official bulletin. No license is stated on the ARPA page, so we credit ARPA
|
||
Piemonte and link the source rather than claiming any reuse right.
|
||
|
||
Official bulletin PDF, linked from every alert view:
|
||
`https://www.arpa.piemonte.it/rischi_naturali/boll/bollettino_allerta.pdf`
|
||
|
||
### Zone codes
|
||
|
||
`Piem-A` … `Piem-M`, eleven zones: **the Italian alphabet is used, so J and K do not
|
||
exist** — the sequence is A B C D E F G H I L M. The code-to-name mapping lives in
|
||
`app/assets/regions/piemonte.json`; the human-readable names are not in the feed and
|
||
come from the ARPA zone documentation.
|
||
|
||
A captured copy of the live feed is kept as a test fixture so the parser is verified
|
||
against the real document rather than a hand-written approximation.
|
||
|
||
## 6. Base map — OpenFreeMap
|
||
|
||
OpenStreetMap vector tiles served by **OpenFreeMap** (<https://openfreemap.org>).
|
||
|
||
- Style in use: `https://tiles.openfreemap.org/styles/positron`
|
||
- Tiles: `https://tiles.openfreemap.org/planet`
|
||
- Glyphs: `https://tiles.openfreemap.org/fonts/{fontstack}/{range}.pbf`
|
||
|
||
**No API key, no registration, no request limits, commercial use permitted.** That is
|
||
what makes it the right starting point: every other free tier — MapTiler, Stadia, Jawg,
|
||
Thunderforest — requires a key, which means a secret to manage and a quota to outgrow.
|
||
|
||
Positron rather than Liberty or Bright: a radar overlay has to be the loudest thing on
|
||
screen, and Positron is a desaturated grey base designed to sit under data. On Liberty
|
||
the precipitation colours compete with road casings and landuse fills.
|
||
|
||
### Mandatory credits
|
||
|
||
| Credit | Required? |
|
||
|---|---|
|
||
| `© OpenStreetMap contributors` (ODbL) | **yes** |
|
||
| `© OpenMapTiles` | **yes** |
|
||
| `OpenFreeMap` | optional, and appreciated |
|
||
|
||
The style JSON carries **no `attribution` field**, so MapLibre will not display these on
|
||
its own. The app renders them itself from the region config: the two mandatory credits
|
||
go in the always-visible attribution bar, and all three are listed on the Sources screen.
|
||
|
||
### When there is no network
|
||
|
||
With `MAP_STYLE_URL=offline` the app generates a self-contained style from the region
|
||
bounding box — flat background plus the extent outline, no network sources at all. It is
|
||
deliberately plain so it is never mistaken for a finished map, and it is what the widget
|
||
tests run against.
|
||
|
||
## 7. Lightning — out of scope
|
||
|
||
Not part of the app. If it is ever revisited: **Blitzortung prohibits commercial use**,
|
||
and the DPC `LTG` product is the source to reach for instead.
|