Reading data and building on it
The platform's screens read it through one interface, the read API. Your program may use the same interface. This page takes you through it in the order you will need it: sign in, list the series, read values, follow changes, call an indicator, call a model. Each step has a recipe you can run as it stands.
The guide for readers of data: 1. How the platform thinks · 2. Reading data and building on it · 3. What stays the same · 4. What changed
Read How the platform thinks first if the words
valid time, known time and as_of are new to you. The
read API's reference lists every request and field;
this page teaches the ones a reader needs.
Before you start
You need three things.
- An account that may read. A person reads the
platform's data with the role
traderoranalyst. An operator grants roles. If you can open a chart in the platform's screens, you have one. - The read API's address. This platform's is below; the reference names it too.
- Python 3.11 or later with
httpx, for the recipes in Python;curl, for the short ones.
The recipes take everything from the environment and keep nothing in their code. For this platform, set:
export BFF_URL=http://localhost:8620
export BFF_ISSUER=http://localhost:8180/realms/bff
BFF_URL is the read API. BFF_ISSUER is the
sign-in service, which the next step uses. Then install
httpx in a virtual environment:
python3 -m venv .venv
.venv/bin/pip install httpx
. .venv/bin/activate
Save each recipe under the name its heading gives, in one folder.
1. Sign in
Every request but the health check carries a token: a signed note from the platform's sign-in service that says who you are and which roles you hold. You send it in a header:
Authorization: Bearer <the token>
On a screen, the browser gets the token for you when you sign in. A program has no sign-in page to show, so it uses the device flow, the way a television signs in to a film service: the program prints an address and a short code, you open the address in any browser and approve the code, and the sign-in service hands the program a token. Your password never passes through the program.
signin.py
"""Sign in as yourself, and print a token for the other recipes.
The device flow of OAuth 2: this prints an address and a code, you approve
the code in a browser, and the sign-in service answers with a token. No
password and no secret passes through this script.
Settings, from the environment:
BFF_ISSUER the sign-in service, such as http://localhost:8180/realms/bff
(on a server, https://auth.<domain>/realms/bff)
BFF_REFRESH_TOKEN optional: the refresh token of an earlier run; with it
the token is renewed, and no browser is needed
It prints two shell commands, so run it like this:
eval "$(python signin.py)" # sets BFF_TOKEN and BFF_REFRESH_TOKEN
Needs Python 3.11 or later, and httpx.
"""
import base64
import hashlib
import os
import secrets
import sys
import time
import httpx
ISSUER = os.environ["BFF_ISSUER"].rstrip("/")
CLIENT = "bff-sdk" # the platform's public client for programs a person runs
with httpx.Client(timeout=30.0) as http:
found = http.get(f"{ISSUER}/.well-known/openid-configuration").raise_for_status().json()
refresh = os.environ.get("BFF_REFRESH_TOKEN")
answer = {}
if refresh:
answer = http.post(
found["token_endpoint"],
data={"grant_type": "refresh_token", "refresh_token": refresh, "client_id": CLIENT},
).json()
if "access_token" not in answer:
# A proof key (RFC 7636): only this run can redeem the code it asked for.
verifier = secrets.token_urlsafe(48)
digest = hashlib.sha256(verifier.encode()).digest()
challenge = base64.urlsafe_b64encode(digest).rstrip(b"=").decode()
started = (
http.post(
found["device_authorization_endpoint"],
data={
"client_id": CLIENT,
"scope": "openid",
"code_challenge": challenge,
"code_challenge_method": "S256",
},
)
.raise_for_status()
.json()
)
where = started.get("verification_uri_complete") or started["verification_uri"]
print(f"Open {where} and approve the code {started['user_code']}.", file=sys.stderr)
wait = float(started.get("interval", 5))
deadline = time.monotonic() + float(started.get("expires_in", 600))
while True:
if time.monotonic() > deadline:
sys.exit("The code expired before it was approved.")
time.sleep(wait)
answer = http.post(
found["token_endpoint"],
data={
"grant_type": "urn:ietf:params:oauth:grant-type:device_code",
"device_code": started["device_code"],
"client_id": CLIENT,
"code_verifier": verifier,
},
).json()
if "access_token" in answer:
break
if answer.get("error") == "slow_down":
wait += 5
elif answer.get("error") != "authorization_pending":
sys.exit(f"The sign-in was refused: {answer.get('error')}.")
print(f"Signed in. The token lasts {answer.get('expires_in', '?')} seconds.", file=sys.stderr)
print(f"export BFF_TOKEN='{answer['access_token']}'")
print(f"export BFF_REFRESH_TOKEN='{answer.get('refresh_token') or refresh or ''}'")
Run it, approve the code in the browser, and your shell holds a token:
eval "$(python signin.py)"
What to know about the token:
- It lasts five minutes. That is short on purpose: a
token that leaks is soon worthless. Run the same command again to renew
it. The second time,
BFF_REFRESH_TOKENis set, so no browser is needed. - A sign-in ends after 30 minutes without a renewal, and after 10 hours whatever you do. Then the recipe asks you to approve a code again.
- The token is you. Whoever holds it reads what you read, for its five minutes. Keep it out of files and logs; the refresh token even more so.
- This recipe reads as a person at a keyboard. It
signs in through
bff-sdk, the platform's public client for programs a person runs. It has no secret, so there is none to keep. But someone must approve a code to start it, and again when the sign-in ends. A program that runs with nobody there signs in by itself, with a key: see Sign in as a program.
If you write models or indicators you may have the SDK already. Its
bff-sdk login signs in the same way and keeps the token in a
file only you can read: see Models and
indicators.
2. Ask who you are
Before reading data, make the smallest request there is. It proves the address and the token, and it tells you what you may do.
whoami.sh
#!/bin/sh
# Ask the read API who you are and what you may do.
#
# Settings, from the environment:
#
# BFF_URL the read API: http://localhost:8620 on a development stack,
# https://api.<domain> on a server
# BFF_TOKEN an access token: eval "$(python signin.py)"
#
# sh whoami.sh
set -eu
curl --fail-with-body -sS -H "Authorization: Bearer $BFF_TOKEN" "$BFF_URL/v1/me"
echo
The answer, for a trader who also writes models:
{
"identity": "tara",
"caller": "person",
"name": "Tara Test",
"roles": ["author", "trader"],
"scopes": ["data:read", "ext:deploy"],
"reads": {"registered_feeders": true, "forecasts": true, "own": "ext.tara"}
}
scopes is what the read API lets you do. It comes from your
roles:
| Role | Scope | Lets you |
|---|---|---|
trader, analyst | data:read |
Read series, follow the stream, compute indicators, call the functions granted to you. Everything on this page. |
author | ext:deploy |
Deploy models and indicators of your own. |
operator | ext:admin |
Look after people's functions. |
reads says which series you read: every feeder's, every
forecast's, and your own under ext.tara. Anyone signed in may
ask this, even with no right to read; reads then says they read
nothing.
If the request fails with status 401, the token is missing, malformed or expired: sign in again.
3. List the series
GET /v1/series answers the catalog: every series you may
read, in the order of their names. You need the scope
data:read.
The catalog grows, so it comes by pages. An
answer holds a page of series and one thing more,
next_cursor:
{
"series": [{"name": "ecb.eur_usd", "...": "..."}, {"name": "entsoe.de_lu_load", "...": "..."}],
"next_cursor": "eyJrIjpbImVudHNvZS5kZV9sdV9sb2FkIl0sImwiOiJzZXJpZXMiLCJxIjoiMmRhMjI5ZTBiZDQ1MDZlYiIsInYiOjF9"
}
To read the next page, ask again with the same filters and
cursor set to that value. On the last page
next_cursor is null: that is how you know you have
them all. Think of a cursor as a bookmark the platform hands you: it says
where the page you hold ended, and you hand it back to go on from
there.
| Parameter | Means |
|---|---|
limit | The most series in a page. Absent, 500. At most 1,000: a larger number is taken as 1,000, not refused. It must be a whole number from 1. |
cursor | The next_cursor of the page before, exactly as you were given it. Absent, the first page. |
Four things to know about a cursor:
- Do not read it, and do not make one. It is text for the platform, and its form may change with a release. Do not keep one from one day to the next either: start again from the first page.
- It belongs to the request that gave it. Send it with
the same filters. A cursor of another list, or of the same list asked
with other filters, is refused: status 400,
invalid_cursor. - The list may change while you read it. A series that was there from your first page to your last is on exactly one page. One added in between is on a later page or on none, by where its name falls.
- The answer does not count the list. There is no total: count what you read.
The other lists that grow are read the same way: the models and the
indicators you may call (step 7), with 200 in a page unless you ask
otherwise and 500 at most. A program that reads only the first page of a
list will be right until the day the list outgrows a page, and then wrong
without a sign. So read until next_cursor is
null, as this recipe does:
list_series.py
"""List the series you may read.
The catalog comes by pages. A page says `next_cursor`: send it back as
`cursor`, with the same filter, for the page after. It is null on the last
page. A page holds 500 series unless `limit` asks otherwise, and 1,000 at most.
`view=full` gives each series every field, its times among them; `filter`
keeps those it names, as in `market = "DE-LU" AND origin = "fetched"`.
Settings, from the environment:
BFF_URL the read API: http://localhost:8620 on a development stack,
https://api.<domain> on a server
BFF_TOKEN an access token: eval "$(python signin.py)"
BFF_FILTER optional: which series, such as market = "DE-LU"
python list_series.py
Needs Python 3.11 or later, and httpx.
"""
import os
import sys
import httpx
URL = os.environ["BFF_URL"].rstrip("/")
AUTH = {"Authorization": f"Bearer {os.environ['BFF_TOKEN']}"}
query = {"view": "full"}
if os.environ.get("BFF_FILTER"):
query["filter"] = os.environ["BFF_FILTER"]
count = 0
while True:
answer = httpx.get(f"{URL}/v1/series", params=query, headers=AUTH, timeout=30.0)
if answer.status_code != 200:
sys.exit(f"{answer.status_code}: {answer.text}")
page = answer.json()
for series in page["series"]:
print(
f"{series['name']:<44} {series['unit']:<10} "
f"last value for {series['last_valid_at']}, known {series['latest_known_at']}"
)
count += len(page["series"])
if page["next_cursor"] is None:
break
# The page after this one: the same filter, and the cursor.
query["cursor"] = page["next_cursor"]
print(f"{count} series.")
Each series comes in one of two views.
view=basic, the default, gives the seven fields every series
has: enough to choose one.
{
"name": "entsoe.de_lu_price",
"kind": "number",
"unit": "EUR/MWh",
"currency": "EUR",
"market": "DE-LU",
"product": "day-ahead",
"origin": "fetched"
}
view=full gives every field, as the recipe asks:
{
"name": "entsoe.de_lu_price",
"kind": "number",
"unit": "EUR/MWh",
"currency": "EUR",
"market": "DE-LU",
"product": "day-ahead",
"origin": "fetched",
"lineage": "RAW-01",
"description": "Day-ahead price, Germany and Luxembourg",
"source": {"kind": "feeder", "id": "feeder.entsoe",
"description": "ENTSO-E Transparency Platform", "status": "active"},
"definition": null,
"card": {"valid_from": "2025-09-30T22:00:00+00:00", "set_at": "2026-11-02T09:14:07.000000Z",
"grid": "PT15M", "zone": "Europe/Berlin", "publishes": {"day_before": "13:00"},
"revises": false, "origin": "fetched", "lineage": "RAW-01"},
"first_valid_at": "2018-09-30T22:00:00+00:00",
"last_valid_at": "2026-11-18T22:45:00+00:00",
"latest_known_at": "2026-11-17T12:00:00+00:00"
}
| Field | Means |
|---|---|
currency | The currency of a price: the unit's first part where it is three capitals, as in EUR/MWh; null where the unit is no price. |
market, product | Empty where the source names none. A forecast kept as data takes its target's. |
origin | Where the values come from: fetched from a source, computed by a definition, forecast by a model, imported by a person. The card's, where it says; otherwise its source's. |
lineage | The trading requirement the series answers, such as RAW-01; null where its card names none. |
source | Its kind is feeder, forecast or person. A feeder's status is active, suspended or retired, and a retired feeder's series stay readable. |
definition | What makes a forecast or a computed series, and its version; null for one fetched or imported. |
first_valid_at, last_valid_at, latest_known_at | How far the series reaches, as of the moment you read at. |
card is what the series' source says of it that the values
do not. Every field is optional, and the whole card is null
where the source states none, as for a person's series and a forecast kept
as data.
| Field | Means |
|---|---|
valid_from | From which valid time this card holds. Absent, from the beginning. A series can have several cards, each from its own time: the DE-LU day-ahead price had hours before Oct. 1, 2025, and quarter-hours since. |
set_at | When this card was stored. A card corrected later has a later one. |
grid | The time between values, as a duration in the calendar of the card's zone: PT15M, PT1H, P1D. A P1D series keeps its local hour when the clocks change, and a day of PT15M has 92, 96 or 100 quarter-hours. Absent for a series with no regular step. |
zone | The time zone its days and clock times are in. |
publishes | When a value becomes known. {"day_before": "13:00"}: a day's values, on the day before, at that clock time in the card's zone. {"same_day": "10:00"}: a day's values, on that day, at that clock time. {"after": "PT1H"}: that long after the moment a value is for. |
revises | Whether the source ever corrects a value it has published. |
grace | How long after its due time a value counts as late, as a duration. Absent, two hours. |
covers | The parts of each day the source gives, in the card's zone, such as ["12:00-24:00"] for an auction of the afternoon. A time outside them has no value by design. Absent, the whole day. |
calendar | The publication calendar the source keeps, such as TARGET for the European Central Bank's rates. |
origin | Where the values come from: fetched, computed, forecast or imported. |
lineage | The trading requirement the series answers, such as RAW-01. |
The card answers before you read a single value the questions the first
guide taught you to ask: how many values make a day, in which zone the day
lies, and whether a value you read today may be revised tomorrow. The
catalog gives the card in force for the series' last value. Every card a
series had, each from its own valid time, is at
GET /v1/series/{name}/cards, by pages; and one series, every
field, at GET /v1/series/{name}, whose
valid_at asks for the card in force at another time.
Before 3.2.15 the catalog came whole, in one answer, without
next_cursor. A client written then reads the first 500 series
and stops: give it the loop above.
The catalog takes three parameters more. Each holds for every page: send it again with each cursor.
| Parameter | Means | Example |
|---|---|---|
filter | Which series, by name, description, namespace, market, product, origin and lineage: comparisons joined by AND, OR and NOT (Which series are late shows the whole grammar). To search, : keeps a text that holds the word, its case as written. | market = "DE-LU" AND origin = fetched; name:de_lu OR description:"day-ahead" |
view | basic or full. Absent, basic. | view=full |
as_of | The catalog as known at a moment: a series first known after it is left out, and the times are as known then. Absent, now. | as_of=2026-11-01T00:00:00Z |
Before 3.2.18 the catalog took market, product,
source and q. Each is now refused with status 400,
invalid_request, and a detail that says what
replaces it: filter for a market or a product; the full view's
source, or a filter by namespace, for a source; and for a text
the name or description holds, filter=name:"…" OR description:"…".
4. Read values as of a moment
GET /v1/values answers the values of one or more series over
a period, as known at a moment.
| Parameter | Needed | Means |
|---|---|---|
series | Yes | The series: separated by commas, or the parameter repeated. At most 20; 500 asked as columns. |
start | Yes | The first valid time, included. |
end | Yes | The last valid time, excluded. It must be after start. |
as_of | No | The moment to read as of, by when each value became known. Absent, now. |
recorded_as_of | No | A second cut, by when the platform stored each value: what it held by then. |
fill | No | step: every step of each series' grid, each missing one with why. Absent, none: the times the platform holds. |
view | No | basic, the default, or full: every column. |
limit | No | The most rows in a page: 10,000 unless given, and at most 200,000. |
cursor | No | The next_cursor of the page before. |
timeline | No | Absent, live. |
Each time carries its offset: 2026-11-17T23:00:00Z. If you
write the offset as +00:00, encode the plus sign in the
address as %2B; an HTTP library does it for you, and so does
curl with --data-urlencode.
The short recipe reads whatever you name:
read_as_of.sh
#!/bin/sh
# Read series' values over a period, as known at a moment: each with its
# status, and with fill=step every step of its grid, each missing one with why.
#
# Settings, from the environment:
#
# BFF_URL the read API: http://localhost:8620 on a development stack,
# https://api.<domain> on a server
# BFF_TOKEN an access token: eval "$(python signin.py)"
# BFF_SERIES the series, separated by commas: entsoe.de_lu_price
# BFF_START the first valid time, included: 2026-11-17T23:00:00Z
# BFF_END the last valid time, excluded: 2026-11-18T23:00:00Z
# BFF_AS_OF optional: the moment to read as of; now if absent
# BFF_FILL optional: step, for every step of each series' grid
#
# sh read_as_of.sh
set -eu
curl --fail-with-body -sS -G "$BFF_URL/v1/values" \
-H "Authorization: Bearer $BFF_TOKEN" \
--data-urlencode "series=$BFF_SERIES" \
--data-urlencode "start=$BFF_START" \
--data-urlencode "end=$BFF_END" \
${BFF_AS_OF:+--data-urlencode "as_of=$BFF_AS_OF"} \
${BFF_FILL:+--data-urlencode "fill=$BFF_FILL"}
echo
The answer names the moment it read at, then gives each series by name, column by column:
{
"as_of": "2026-11-17T12:30:00+00:00",
"recorded_as_of": null,
"series": {
"entsoe.de_lu_price": {
"kind": "number",
"unit": "EUR/MWh",
"valid_at": ["2026-11-17T23:00:00+00:00", "2026-11-17T23:15:00+00:00"],
"known_at": ["2026-11-17T12:00:00+00:00", "2026-11-17T12:00:00+00:00"],
"value": [71.2, 69.85],
"status": [null, null]
}
},
"next_cursor": null
}
(A real day holds 96 values; two are shown.) The arrays are equally long
and go together by position, in the order of valid time. Without
fill, a valid time with nothing known at as_of is
absent, so a series may answer fewer values than the period has steps, or
none: an empty read is not an error.
status says where each value is in its life, or why there is
none. Beside a value: null where the source says nothing more,
or preliminary, provisional or final.
Where value is null, the source said so itself:
no_data, cancelled (an auction that did not take
place), withdrawn (a value taken back), partial (a
computation over part of its inputs), or unstated for a row
stored before 3.2.14. A null value is never zero.
The longer recipe shows what as_of is for. It reads the same
24 hours twice: as known now, and as known at the moment those hours began.
Predict the result before you run it. For a day-ahead price, how many of the
day's values were known before the day began? For a measured load?
read_as_of.py
"""Read one series twice: as known now, and as known before the period began.
It takes the 24 hours that end with the series' latest value. The first read
asks what is known now. The second asks what was known at the moment those
24 hours began. A day-ahead price is all there in both; a measurement is
there only in the first.
Settings, from the environment:
BFF_URL the read API: http://localhost:8620 on a development stack,
https://api.<domain> on a server
BFF_TOKEN an access token: eval "$(python signin.py)"
BFF_SERIES optional: the series to read; otherwise the catalog's first
series of numbers that holds a value
python read_as_of.py
Needs Python 3.11 or later, and httpx.
"""
import os
import sys
from datetime import datetime, timedelta
import httpx
URL = os.environ["BFF_URL"].rstrip("/")
AUTH = {"Authorization": f"Bearer {os.environ['BFF_TOKEN']}"}
def get(path: str, **query: str) -> dict:
answer = httpx.get(f"{URL}{path}", params=query, headers=AUTH, timeout=30.0)
if answer.status_code != 200:
sys.exit(f"{answer.status_code}: {answer.text}")
return answer.json()
def listed(path: str, key: str, **query: str):
"""Each item of a list, read page by page: `next_cursor` asks for the next."""
while True:
page = get(path, **query)
yield from page[key]
if page["next_cursor"] is None:
return
query["cursor"] = page["next_cursor"]
wanted = os.environ.get("BFF_SERIES")
series = next(
(
s
for s in listed("/v1/series", "series", view="full")
if s["last_valid_at"] and (s["name"] == wanted if wanted else s["kind"] == "number")
),
None,
)
if series is None:
sys.exit(f"No series {wanted or 'of numbers'} with a value to read.")
last = datetime.fromisoformat(series["last_valid_at"])
start = last - timedelta(hours=24)
end = last + timedelta(seconds=1) # end is excluded: a second more keeps the last value
def read(**as_of: str) -> dict:
found = get(
"/v1/values",
series=series["name"],
start=start.isoformat(),
end=end.isoformat(),
**as_of,
)
return found["series"][series["name"]]
now = read()
before = read(as_of=start.isoformat())
print(f"{series['name']} ({now['unit']}), {start.isoformat()} to {last.isoformat()}")
print(f" known now: {len(now['value'])} values")
print(f" known when the period began: {len(before['value'])} values")
if now["value"]:
print(
f" the latest: {now['value'][-1]} for {now['valid_at'][-1]}, "
f"known since {now['known_at'][-1]}"
)
A day-ahead price answers the same count twice: the auction published the whole day beforehand. A measurement answers none the second time: nobody knew it yet.
Every step, and why one is missing
A hole in a series is easy to miss. A chart joins the values on either
side; a rolling mean of 24 values takes the 24 it was given, whatever hours
they are for. fill=step shows the hole. The answer then holds
every step of each series' grid in the period, by its card, and a step
without a value is there with value null and the
reason in status. The platform works the reason out as of
your as_of:
| Status | Means |
|---|---|
| The source's own | no_data, cancelled, withdrawn, partial, unstated, as above. |
not_covered | The source gives no value for this time by design: outside the card's covers, or on a day its publication calendar publishes nothing, such as a weekend of the ECB's. |
not_due | Not late yet: its due time, by the card's publishes, and its grace after it, have not both passed. A card that states no schedule owes a value from the moment it is for. |
source_down | Due, and the last read of the source that covered it failed or was refused. |
rejected | Due, and the platform refused the data that held it. |
late | Due, and the source was read without it, or not read at all. |
A filled step has no known_at: no version speaks of it. A
series whose card states no grid has its stored times alone, and a value
stored off the grid is never left out. Asked as of yesterday, the same step
may say not_due that today says source_down: the
reason is the platform's, as it stood at your moment.
How much of a period has values
GET /v1/coverage counts what fill=step lists,
for up to 500 series in one request, with series,
start, end, as_of and
timeline as above:
{
"as_of": "2026-11-19T14:00:00+00:00",
"series": {
"entsoe.de_lu_load_forecast": {"steps": 96, "has_value": 80,
"missing": {"cancelled": 4, "source_down": 12}}
}
}
That is one day of quarter-hours: 80 values, 4 cancelled, 12 missing because the source was down. The two reads agree at every moment.
Two cuts
as_of cuts by when a value became known. That is the
question a backtest asks: what could a trader have known at 10:40? But a
value can be known at 10:30 and reach the platform at 10:50, after a
source's outage. recorded_as_of asks the stricter question:
what did the platform hold at that moment? It reaches at most a little
behind the clock, twice the longest a delivery may take, so that the same
cut answers the same later; a later one is refused with status 400,
invalid_time.
By pages, and as columns
A read comes by pages, as the catalog does: in the order of the series
you named, then of valid time, limit rows a page, a filled
step among them. A page names each series its rows reach; a series whose
rows run over two pages is on both, and you join its columns page after
page. The cursor carries the moment the first page was read at, so a read
of years agrees with itself however long it runs.
Asked with Accept: application/vnd.apache.arrow.stream, a
page is one Apache Arrow stream: the same fields, by the same names, a
column each, a row a value or a filled step, the series' name in the column
series, and the page's moment and cursor in the stream's
metadata, under bff. pandas reads it with
pyarrow.ipc.open_stream. As columns a read takes up to 500
series of numbers; a series of text is read as JSON.
Every column, and every version
view=full adds what a value says beyond itself:
recorded_at, when the platform stored it;
published_at, when the source says it published it;
known_by, how its known time was set (stated by the
source, due by its schedule, first_seen by the
feeder, revision, computed); flags
and flag_reason, what may be said of its quality
(estimated, held, outlier, and more);
valid_until, the end of its period where it is not one step;
run_id, the run that made a forecast; and
estimates, each error estimate a series declares, such as a
forecast's standard deviation, as a column of its own.
A read gives the latest version of each valid time.
GET /v1/series/{name}/versions, with start,
end and as_of, gives every version of one series
in a period, oldest first within a valid time, every column: a forecast's
earlier runs, a price's preliminary and final, a value and its
withdrawal.
A market's days
GET /v1/reference/calendars/{zone}/{market}/{product}, such
as /v1/reference/calendars/DE-LU/DA/auction?start=2026-03-29&end=2026-03-30,
answers each delivery day, from local midnight to the next, with every
period: its times, its local clock time and the desk's label
(H1 to H25, H3 Q2). March 29, 2026 is
a day of 23 hours in Berlin, and Oct. 25 one of 25, with 02:00 twice. At
most 62 days a read, as known at as_of: Oct. 26, 2025 of DE-LU's
day-ahead auction was hourly until its 15-minute periods were confirmed in
September.
5. Follow the stream
To keep a screen or a model current you could read every few seconds. Most reads would find nothing new. The stream turns that around: you open one connection, and the platform tells you when a series you may read has new values.
GET /v1/events answers a stream of server-sent events: lines
of text that keep coming on one open connection. Each event has a name, most
have an id, and each has one line of JSON.
event: ready
id: 1795089600123-15.7.0.0.3.0.0.0.0.0.0.2
data: {"specversion":"1.0","id":"5d1f0a52-6f0e-4d0c-9a51-0f2a8f5a7c11","source":"api/3.0","type":"bff.stream.ready","time":"2026-11-17T12:00:00.250000+00:00", ... ,"data":{"is_following":true}}
event: changed
id: 1795089601480-15.8.0.0.3.0.0.0.0.0.0.2
data: {"specversion":"1.0","id":"a3c1b6de-1b0c-4a55-8a0e-52b7c6b0d9e4","source":"api/3.0","type":"bff.stream.changed","time":"2026-11-17T12:00:02.000000+00:00", ... ,"data":{"series":[{"name":"entsoe.de_lu_price","from":"2026-11-17T23:00:00+00:00","to":"2026-11-18T22:45:00+00:00","known_at":"2026-11-17T12:00:00+00:00"}]}}
(The three dots stand for attributes left out here to keep the lines
short; the reference lists them all, under
StreamEvent.)
The line of JSON is a CloudEvent: an envelope in the
form of CloudEvents 1.0, the open standard for events, so that tools which
know the standard can read it. The envelope says what kind of event it is
(type, which is bff.stream. and the event's
name), who sent it and when. What the event tells you is inside, under the
envelope's own data. The table shows that inner
data.
| Event | Its data | Means | Do |
|---|---|---|---|
ready | {"is_following": true} |
First, and again after a pause. is_following says whether the platform is telling changes now. |
On a stream you opened without an id, read once what you show, so you start from the present. |
changed | {"series": [...]} |
These series have new values. For each: its name, the earliest and latest valid time of the new values, from and to, and the latest of their known times, known_at. After a long outage of a source, a whole range filled at once is told as one change. |
Read each series from from to to, both included. |
paused | {} |
The platform cannot tell changes for now. | Wait for the next ready. |
missed | {"since": "2026-11-17T12:00:02+00:00"} |
Changes were not told, and the platform cannot tell them now. since is when the last change you were told was announced; it is null when the platform cannot say. |
Read again everything you show. |
expired | {} |
Always last. The token the stream was opened with has expired, and the stream ends. | Renew the token and open a new stream, from the last id you saw. |
Picking up where you left off
A connection breaks: a laptop sleeps, a network drops, the platform is
released. Without help you could not know what changed while you were
away, and would have to read everything again. The id is the
help.
Think of the platform's announcements as a tape that only grows. An
event's id is a mark on that tape: everything before the mark
that you may see has been told to you. So:
- Keep the last
idthe stream sent. Treat it as text; do not take it apart. - When you open the stream again, send it in the header
Last-Event-ID. - The platform tells you, in one or more
changedevents, what was announced after your mark. Nothing is told twice, and nothing you may see is left out.
The platform keeps the last five minutes of the tape ready for this. If
your mark is older, or is not a mark at all, the stream says
missed instead, and you read again what you show, as you would
have without the id. Either way you end up current; the id saves the
reading when the break was short.
A stream opened without Last-Event-ID begins now: it tells
nothing from before, and its ready carries the mark it begins
at. The platform runs as several copies; a mark means the same to each, so
it does not matter which one answers.
Five things about the stream decide how you write a follower.
- It carries no values. A change says which series and which valid times. You read the values yourself, with the request of step 4. So the stream can never show you a value you may not read.
- The id belongs to the event that carries it. Move your mark only after you have acted on that event. Then a crash between the two costs you nothing: you are told the event again.
- It gathers. The platform collects a second's changes and tells them together, one entry a series. After a reconnect it tells all you missed the same way.
- What you may see is decided when it is told. A resumed stream tells you the changes of the series you may read, and of no others, whatever id you send.
- It is a hint, not a ledger. It tells changes on the
timeline
live. Values a source delivers in bulk to catch up on its history are stored without an announcement. A program that must hold everything also reads again from time to time.
The platform may add events. Ignore an event whose name you do not know.
A line that begins with a colon, :ping, comes every 15
seconds. It is a comment that keeps the connection open. It may be followed
by an id line with no data: your mark has moved,
though nothing was told, because what was announced was not yours to see.
Keep that id like any other. If nothing at all arrives for much longer than
15 seconds, the connection is lost: open the stream again, from your
mark.
follow.py
"""Follow the stream of changes, and read what each change names.
The stream says which series have new values; it does not carry the values.
So the loop is: hear a change, read that series over the valid times the
change names.
Each event carries an `id:`, the place the stream has reached. Keep the last
one. When the connection is lost, open the stream again and send that id as
`Last-Event-ID`: the platform then tells what you missed, once. If it cannot,
it says `missed`, and you read again everything you show. On `expired`, sign
in again and open a new stream, with the same id.
Settings, from the environment:
BFF_URL the read API: http://localhost:8620 on a development stack,
https://api.<domain> on a server
BFF_TOKEN an access token: eval "$(python signin.py)"
BFF_SECONDS optional: how long to listen (60)
python follow.py
It ends after BFF_SECONDS, or when the token expires, whichever comes first.
Needs Python 3.11 or later, and httpx.
"""
import json
import os
import sys
import time
from datetime import datetime, timedelta
import httpx
URL = os.environ["BFF_URL"].rstrip("/")
AUTH = {"Authorization": f"Bearer {os.environ['BFF_TOKEN']}"}
until = time.monotonic() + float(os.environ.get("BFF_SECONDS", "60"))
def read(change: dict) -> None:
"""The new values of one series: its valid times from `from` to `to`, both included."""
end = datetime.fromisoformat(change["to"]) + timedelta(seconds=1)
answer = httpx.get(
f"{URL}/v1/values",
headers=AUTH,
timeout=30.0,
params={"series": change["name"], "start": change["from"], "end": end.isoformat()},
)
if answer.status_code != 200:
print(f" could not read {change['name']}: {answer.status_code} {answer.text}")
return
values = answer.json()["series"][change["name"]]
print(
f" {change['name']}: {len(values['value'])} values from {change['from']} "
f"to {change['to']}, known {change['known_at']}"
)
def told(name: str, data: dict) -> bool:
"""Act on one event; False when the stream is over."""
if name == "ready":
print(
"ready: the platform",
"tells" if data["is_following"] else "does not tell",
"changes now",
)
elif name == "changed":
print(f"changed: {len(data['series'])} series")
for change in data["series"]:
read(change)
elif name == "missed":
since = data["since"] or "a time the platform cannot say"
print(f"missed: changes since {since} were not told; read again what you show")
elif name == "paused":
print("paused: changes are not told until the next ready")
elif name == "expired":
print("expired: sign in again, and open a new stream")
return False
return True
def follow(last_id: str | None) -> tuple[str | None, int, bool]:
"""Listen to one stream, from `last_id` on if there is one. Returns the
last id it sent, how many events it told, and whether to open another."""
headers = dict(AUTH)
if last_id is not None:
headers["Last-Event-ID"] = last_id
heard = 0
# The stream sends a comment every 15 seconds; 40 seconds of silence is a lost connection.
with httpx.stream(
"GET", f"{URL}/v1/events", headers=headers, timeout=httpx.Timeout(10.0, read=40.0)
) as stream:
if stream.status_code != 200:
sys.exit(f"{stream.status_code}: {stream.read().decode()}")
name, placed, data = "", None, None
for line in stream.iter_lines():
if time.monotonic() > until:
return last_id, heard, False
if line.startswith("event:"):
name = line.removeprefix("event:").strip()
elif line.startswith("id:"):
placed = line.removeprefix("id:").strip()
elif line.startswith("data:"):
data = line.removeprefix("data:")
elif line == "":
# An event ends at an empty line. Its data is a CloudEvent:
# what the event says is in the CloudEvent's own `data`.
if data is not None:
heard += 1
if not told(name, json.loads(data)["data"]):
return placed or last_id, heard, False
# The place moves once the event is acted on. A comment may
# carry an id too, when the place moved with nothing to tell.
last_id = placed or last_id
name, placed, data = "", None, None
return last_id, heard, True
heard, last_id, again = 0, None, True
while again and time.monotonic() < until:
try:
last_id, events, again = follow(last_id)
heard += events
except httpx.TransportError as error:
# The connection was lost: open the stream again, from the last id.
print(f"lost the stream ({type(error).__name__}); opening it again")
time.sleep(1)
if not heard:
sys.exit("The stream said nothing.")
print(f"Heard {heard} events.")
Because a token lasts five minutes, a stream lasts five minutes at most.
The recipe stops there. A program that runs all day renews its token and
opens the next stream from the last id it saw, in a loop; it reads again
only when the stream says missed.
A person may hold 8 streams open at once. The ninth is refused with
status 429, rate_limited; close one. A platform set up without
the stream answers status 503, events_unavailable: read on a
timer.
A stream also ends when the copy of the read API that serves it stops, as it does when the platform is updated. Open it again from the last id: another copy goes on from there, and tells what was missed, once.
6. Call an indicator
An indicator is a quantity computed from series: a rolling mean, a difference of two prices. The platform computes it when you ask, on the series you name, as known at the moment you name, and returns it. Nothing is stored.
GET /v1/indicators lists the platform's indicators. Each
entry gives its id, its version and every version
it computes, the inputs it takes, its outputs,
and its parameters, each with its type, default and range.
GET /v1/indicators/{id} describes one.
POST /v1/indicators/{id}/compute computes one. Its body is
JSON:
| Field | Needed | Means |
|---|---|---|
inputs | Yes | A series' name for each input the indicator declares: {"series": "entsoe.de_lu_price"}. |
start, end | Yes | The period, as in a read. |
version | No | The version to compute, as text: "2". Absent, the newest. Name it, and the same call answers the same. |
parameters | No | The parameters you set; the rest take their defaults. |
as_of | No | The moment to compute as of. Absent, now. |
step | No | A duration, as text: "PT15M", "PT1H", "P1D". The inputs are averaged over each step first. It must divide a day or be a day. Absent, the inputs' own step. |
zone | No | The market's time zone. Absent, Europe/Berlin. |
timeline | No | Absent, live. |
A field the compute does not take is refused, with
invalid_request. So is one of the wrong type: a
step sent as the number 3600. The reason is the same for both:
a misspelt parameters that was passed over would compute on
the defaults, and you would trust an answer you never asked for.
indicator.py
"""Compute one of the platform's indicators: a rolling mean of 24 values.
It computes over the two days that end with the series' latest value, as
known now. Nothing is stored: the answer is yours.
Settings, from the environment:
BFF_URL the read API: http://localhost:8620 on a development stack,
https://api.<domain> on a server
BFF_TOKEN an access token: eval "$(python signin.py)"
BFF_SERIES optional: the series to compute on; otherwise the catalog's
first series of numbers that holds a value
python indicator.py
Needs Python 3.11 or later, and httpx.
"""
import os
import sys
from datetime import datetime, timedelta
import httpx
URL = os.environ["BFF_URL"].rstrip("/")
AUTH = {"Authorization": f"Bearer {os.environ['BFF_TOKEN']}"}
def said(answer: httpx.Response) -> dict:
if answer.status_code != 200:
sys.exit(f"{answer.status_code}: {answer.text}")
return answer.json()
def listed(path: str, key: str, **query: str):
"""Each item of a list, read page by page: `next_cursor` asks for the next."""
while True:
page = said(httpx.get(f"{URL}{path}", params=query, headers=AUTH, timeout=30.0))
yield from page[key]
if page["next_cursor"] is None:
return
query["cursor"] = page["next_cursor"]
wanted = os.environ.get("BFF_SERIES")
series = next(
(
s
for s in listed("/v1/series", "series", view="full")
if s["last_valid_at"] and (s["name"] == wanted if wanted else s["kind"] == "number")
),
None,
)
if series is None:
sys.exit(f"No series {wanted or 'of numbers'} with a value to compute on.")
last = datetime.fromisoformat(series["last_valid_at"])
call = {
"version": "2", # named, so the same call answers the same
"inputs": {"series": series["name"]},
"parameters": {"window": 24},
"start": (last - timedelta(days=2)).isoformat(),
"end": (last + timedelta(seconds=1)).isoformat(),
}
answer = said(
httpx.post(f"{URL}/v1/indicators/rolling-mean/compute", json=call, headers=AUTH, timeout=60.0)
)
print(
f"{answer['indicator']} version {answer['version']} of {series['name']}, "
f"as known at {answer['as_of']}"
)
print(
f" read from {answer['read_from']}, in steps of {answer['steps']['series']}; "
f"{len(answer['gaps']['series']['start'])} gaps"
)
for name, output in answer["result"].items():
latest = (
f"; the latest, {output['value'][-1]} {output['unit']} for {output['valid_at'][-1]}"
if output["value"]
else ""
)
print(f" {name}: {len(output['value'])} values{latest}")
The answer says what was done as well as the result: which
version computed; the parameters, with the
defaults filled in; read_from, the earliest valid time it read,
which is before start because a rolling mean needs history;
steps, each input's step; and gaps, where an
input misses values. Each output in result has its
unit, its valid_at and its value.
Look at gaps before you trust a mean: a mean across a hole is
a mean of less than you think.
7. Call a model
People deploy forecasting models and indicators of their own on the platform, and grant them to colleagues. You call one as you compute an indicator: over a period, as of a cut-off. The platform runs its code and gives it your rights: it reads only what you may read, and only as known at the cut-off.
| Request | Answers |
|---|---|
GET /v1/functions | The models and indicators you may call, each at its newest released version, in the order of their owners and names. filter keeps some, by kind and owner: kind = "model" AND owner = "tara". Read by pages, as the catalog is. |
GET /v1/functions/{owner}/{name} | One function: its kind, its parameters, each with its type, default and range, and its outputs. ?version= names a version. |
POST /v1/functions/{owner}/{name}/versions/{version}/compute | A call's answer, at the version the path names. |
GET /v1/runs/{id} | One run of a call of yours: what it read, how long it took, how it ended. |
The body of a call is JSON, with these fields and no others:
| Field | Needed | Means |
|---|---|---|
start, end | Yes | The period to answer. |
as_of | No | The cut-off. Absent, now. A cut-off later than now is taken as now. |
parameters | No | The parameters you set; the rest take their defaults. |
model.py
"""Call a model someone deployed and granted to you.
It lists the models you may call, then calls one over a period, as known at
a cut-off, at the version it names: so the same call answers the same. The
model runs on the platform and reads with your rights.
Settings, from the environment:
BFF_URL the read API: http://localhost:8620 on a development stack,
https://api.<domain> on a server
BFF_TOKEN an access token: eval "$(python signin.py)"
BFF_MODEL optional: owner/name, such as examples/naive; otherwise the
first model listed
BFF_START, BFF_END optional: the period; otherwise the 24 hours from
the start of this hour
BFF_AS_OF optional: the cut-off; now if absent
python model.py
On a platform that holds only old data, name a period it holds:
BFF_START=2017-06-01T22:00:00Z BFF_END=2017-06-02T22:00:00Z \
BFF_AS_OF=2017-06-01T08:00:00Z python model.py
Needs Python 3.11 or later, and httpx.
"""
import os
import sys
from datetime import UTC, datetime, timedelta
import httpx
URL = os.environ["BFF_URL"].rstrip("/")
AUTH = {"Authorization": f"Bearer {os.environ['BFF_TOKEN']}"}
def said(answer: httpx.Response) -> dict:
if answer.status_code != 200:
sys.exit(f"{answer.status_code}: {answer.text}")
return answer.json()
def listed(path: str, key: str, **query: str):
"""Each item of a list, read page by page: `next_cursor` asks for the next."""
while True:
page = said(httpx.get(f"{URL}{path}", params=query, headers=AUTH, timeout=30.0))
yield from page[key]
if page["next_cursor"] is None:
return
query["cursor"] = page["next_cursor"]
models = list(listed("/v1/functions", "functions", filter='kind = "model"'))
for model in models:
print(
f"{model['owner']['username']}/{model['name']} version {model['version']}: {model['title']}"
)
if not models:
print("No model is granted to you yet: ask its owner, or deploy one of your own.")
sys.exit(0)
wanted = os.environ.get("BFF_MODEL") or f"{models[0]['owner']['username']}/{models[0]['name']}"
owner, _, name = wanted.partition("/")
# The newest version its owner released, which the call names.
version = said(httpx.get(f"{URL}/v1/functions/{owner}/{name}", headers=AUTH, timeout=30.0))[
"version"
]
hour = datetime.now(UTC).replace(minute=0, second=0, microsecond=0)
call = {
"start": os.environ.get("BFF_START") or hour.isoformat(),
"end": os.environ.get("BFF_END") or (hour + timedelta(hours=24)).isoformat(),
}
if os.environ.get("BFF_AS_OF"):
call["as_of"] = os.environ["BFF_AS_OF"]
# A model may take its declared time, 60 seconds at most, after it has loaded.
answer = said(
httpx.post(
f"{URL}/v1/functions/{owner}/{name}/versions/{version}/compute",
json=call,
headers=AUTH,
timeout=100.0,
)
)
print(
f"{wanted} version {answer['version']}, {answer['start']} to {answer['end']}, "
f"as known at {answer['as_of']}"
)
for output, values in answer["result"].items():
given = [v for v in values["value"] if v is not None]
print(f" {output}: {len(given)} values in {values['unit']}")
print(f" run {answer['run']['id']} took {answer['run']['took_ms']} ms")
The answer repeats the call (start, end,
as_of, the parameters with defaults filled in),
names the version that ran and its owner, and
gives each output in result with its unit,
valid_at and value. A value may be
null: a model leaves empty what it cannot know. A forecast's
quantiles come as columns beside the values, each named
q and its level: q10 for 0.1. run
describes the run: its id, how long it took, and what it
read.
Four things differ from the platform's own indicators.
- You call only what you were granted. A function you
may not call is answered as one that does not exist:
unknown_function. - It is another person's code, and it can fail. A
failure is status 500,
model_failed, with the run's id in the error's details. Give that id to the owner: they read the run's log. - It takes time. A model may take up to 60 seconds, an indicator 5, and a first call waits while the code loads. Give your request 100 seconds before you give up.
- No limit counts your calls by the minute. What
bounds them is the workers that run people's functions: when none is
free within 10 seconds, the answer is status 503,
runner_busy, withRetry-After. Wait that long and call again. (An operator may set a limit of calls a minute; then the answer over it is status 429,rate_limited.)
To write a model of your own, see Models and indicators.
Which series are late
Take the day-ahead price. Its auction's results come out at 13:00 on the day before delivery. If at 15:30 the store still has no price for tomorrow's first quarter-hour, something is wrong: the price is late. You would like to know that before a model uses yesterday's numbers as if they were today's.
GET /v1/freshness tells you, for every series a feeder
keeps feeding and whose card says how often it has a value and when it is
published. You need the scope data:read, and you see the
series you may read.
{
"checked_at": "2026-11-19T12:10:00Z",
"series": [
{"name": "entsoe.de_lu_load", "feeder": "feeder.entsoe",
"last_valid_at": "2026-11-19T10:45:00Z",
"next_valid_at": "2026-11-19T07:00:00Z", "due_at": "2026-11-19T08:15:00Z",
"is_late": true, "status": "source_down"},
{"name": "entsoe.de_lu_price", "feeder": "feeder.entsoe",
"last_valid_at": "2026-11-20T22:45:00Z",
"next_valid_at": "2026-11-20T23:00:00Z", "due_at": "2026-11-20T12:00:00Z",
"is_late": false, "status": null}
],
"next_cursor": null
}
Look at the load. Its latest value is for 10:45, and yet it is late from 07:00. That is a hole: the quarter-hours from 07:00 are missing, though later ones arrived. The platform walks each series along its steps, a week back, and finds a hole inside a series as well as at its end.
| Field | Means |
|---|---|
last_valid_at | The latest time with a value, or with an absence its source stated. |
next_valid_at | The time the series owes first: where it is late, the first time missing past its due time; else the next one to come. |
due_at | When that time's value was, or is, due. |
is_late | Whether a time is missing past its due time by more than the series' grace, two hours unless its card says otherwise. |
status | Why, where it is late: source_down, the last time the platform read the source for it, the source failed; rejected, the platform refused what was read; late, the source was read and did not have it yet. |
Three things are never late:
- What the source said is not there. A cancelled
auction is stored as
nullwith the reasoncancelled. That is an answer, not a gap. A value the source withdrew is gone, though: its time counts as missing. - What the source never gives. An intraday auction that covers noon to midnight owes nothing for the morning.
- What a calendar says is not published. A reference rate fixed on working days owes nothing on a Saturday.
status tells an outage from a slow source. During an outage
the values stored before stay as they were: nothing is written over them,
and a read gives them as before. Once the source answers again, the
missing values are stored with the times the source published them. If
the outage was long, the stream (step 5) tells you once that a whole range
was filled, as one changed from its first time to its last,
rather than a row at a time.
The list comes by pages, as the catalog does, every page judged at the
moment of the first, checked_at. Ask only for what you want
with is_late=true, or with a filter:
GET /v1/freshness?is_late=true
GET /v1/freshness?filter=namespace = entsoe AND status = source_down
A filter compares fields with =, !=,
<, <=, >,
>=, or : for a text that holds a word, and
joins comparisons with AND, OR and
NOT. Put text with spaces in double quotes. This list filters
by name, namespace, feeder,
market, product, is_late,
status, last_valid_at,
next_valid_at and due_at. Note that
OR binds tighter than AND: a AND b OR
c means a AND (b OR c). A filter that does not read is
refused: status 400, invalid_filter.
The sources, and the data catalogue
Freshness speaks of series. Often you want the same answer one level
up: is ENTSO-E on time today? GET /v1/sources answers it for
every provider of data the platform knows, in one read:
{
"checked_at": "2026-11-21T12:10:00Z",
"sources": [
{"id": "entsoe", "name": "ENTSO-E Transparency Platform",
"description": "The European transmission system operators' platform for market and grid data: prices, load, generation, balancing, outages.",
"url": "https://transparency.entsoe.eu", "licence": "entsoe_tp",
"credit": "ENTSO-E Transparency Platform",
"terms": "Name the ENTSO-E Transparency Platform wherever the data is shown. A registered user's personal token; 400 calls a minute.",
"status": "late", "series": 168, "watched": 160, "late": 3}
]
}
Read the counts from the outside in. The platform reads 168 of the provider's series; freshness watches 160 of them, those whose cards say when they are published; 3 of those are late. The platform works out the status, so a page that shows it counts nothing. It is the first of these that holds:
| Status | When |
|---|---|
not_on_platform | The platform reads none of the provider's series yet. |
not_watched | It reads some, and none says when it is published, so none can be late. |
source_down | A late series' last read failed: the source did not answer, or refused. |
rejected | The platform refused what was read for a late series. |
late | A series is late, read without its value. |
on_time | None is late. |
Each source carries what the data catalogue says of it,
and the credit to show wherever its data is shown. The catalogue itself is
GET /v1/catalogue: every provider, its datasets, how each
reaches the platform, and the names its series have here. A dataset is a
family of series with one meaning, one unit and one step, such as
ENTSO-E's day-ahead price by zone; each zone is one series of it. The
catalogue changes only with a release. Both reads need
data:read, and neither comes by pages: the providers are few,
and the same for everyone.
Sign in as a program
The device flow needs you: someone approves a code, and does so again when the sign-in ends. A report that runs at night has nobody to ask. For that, make a program: a client of your own that signs in by itself.
A program is you, without the browser. It reads what you
read, your own series too, and calls the models and indicators you may
call. It keeps no charts and deploys nothing: those you do yourself, signed
in. Its name carries yours, such as
program.tara.risk_report, so a token always says whose program
called. You may keep ten.
A program signs in with a key pair, not a password. The private key stays in a file on the program's machine. The platform holds only the public half, which can check a signature and cannot make one. So a copy of everything the platform stores gives nobody a way in.
Make one
The SDK makes a program in one step. Install it with its extra
program, which adds the library that signs (see
Models and indicators for the SDK), sign in
as yourself, and name the program:
bff-sdk login
bff-sdk program create risk_report --key-file risk_report.pem
This makes the key pair on your machine, writes the private key to
risk_report.pem, which only you can read, and sends the public
half to the platform. It never overwrites a file. Beside the key it writes
risk_report.pem.json, a note of the program's client id, the
key's id and the sign-in service. Then it prints what to set where the
program runs:
export BFF_ISSUER=http://localhost:8180/realms/bff
export BFF_PROGRAM=program.tara.risk_report
export BFF_PROGRAM_KEY_FILE=risk_report.pem
Those three settings are all a program needs: where to sign in, who it is, and the key that proves it. With them set, the SDK's own commands and helpers sign in as the program too.
program.py
To sign in, the program writes a short note that says who it is, signs
the note with its private key, and sends it to the sign-in service. The
note is good for one minute and one use, so a copy of it is worth nothing.
The recipe needs cryptography beside httpx:
pip install cryptography.
"""Sign in as a program, and print a token for the other recipes.
A program is a client of your own that signs in by itself: no browser, no
code to approve. It holds a private key. It writes a short note that says
"I am this program", signs the note with the key, and the sign-in service,
which holds the public half, answers with a token. The key itself is never
sent.
Settings, from the environment:
BFF_ISSUER the sign-in service, such as http://localhost:8180/realms/bff
(on a server, https://auth.<domain>/realms/bff)
BFF_PROGRAM the program's client id, such as program.tara.risk_report
BFF_PROGRAM_KEY_FILE the path of the program's private key, in PEM
BFF_PROGRAM_KEY_ID optional: the key's id, if you gave it one of your own;
without it, the key's thumbprint, which is the id
that bff-sdk program create gives
It prints a shell command, so run it like this:
eval "$(python program.py)" # sets BFF_TOKEN
Needs Python 3.11 or later, httpx and cryptography.
"""
import base64
import hashlib
import json
import os
import sys
import time
import uuid
import httpx
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import padding
ISSUER = os.environ["BFF_ISSUER"].rstrip("/")
PROGRAM = os.environ["BFF_PROGRAM"]
def b64(data):
"""Base64url without padding, as a signed note writes bytes."""
return base64.urlsafe_b64encode(data).rstrip(b"=").decode()
def number(value):
return b64(value.to_bytes((value.bit_length() + 7) // 8, "big"))
with open(os.environ["BFF_PROGRAM_KEY_FILE"], "rb") as file:
key = serialization.load_pem_private_key(file.read(), password=None)
# The key's id. The thumbprint (RFC 7638) is a digest of the public half, so
# the same key always has the same id and nobody has to remember one. The
# half is written in one form: its members in order, and no spaces.
public = key.public_key().public_numbers()
half = {"e": number(public.e), "kty": "RSA", "n": number(public.n)}
canonical = json.dumps(half, separators=(",", ":"), sort_keys=True)
key_id = os.environ.get("BFF_PROGRAM_KEY_ID") or b64(hashlib.sha256(canonical.encode()).digest())
with httpx.Client(timeout=30.0) as http:
found = http.get(f"{ISSUER}/.well-known/openid-configuration").raise_for_status().json()
# The note (RFC 7523): who signs in, to whom, and for how long. It is good
# for one minute and one use, so a copy of it is worth nothing.
now = int(time.time())
header = {"alg": "RS256", "typ": "JWT", "kid": key_id}
claims = {
"iss": PROGRAM,
"sub": PROGRAM,
"aud": found["issuer"],
"jti": str(uuid.uuid4()),
"iat": now,
"exp": now + 60,
}
note = ".".join(b64(json.dumps(part).encode()) for part in (header, claims))
signature = key.sign(note.encode(), padding.PKCS1v15(), hashes.SHA256())
answer = http.post(
found["token_endpoint"],
data={
"grant_type": "client_credentials",
"client_id": PROGRAM,
"client_assertion_type": "urn:ietf:params:oauth:client-assertion-type:jwt-bearer",
"client_assertion": f"{note}.{b64(signature)}",
},
).json()
if "access_token" not in answer:
sys.exit(f"The sign-in was refused: {answer.get('error_description') or answer.get('error')}.")
print(
f"Signed in as {PROGRAM}. The token lasts {answer.get('expires_in', '?')} seconds.",
file=sys.stderr,
)
print(f"export BFF_TOKEN='{answer['access_token']}'")
Run it, and your shell holds a token for the other recipes:
eval "$(python program.py)"
What to know about a program:
- Its token lasts five minutes, and there is no refresh token. A program needs none: it holds its key, so it signs in again whenever it likes. Ask for a new token shortly before the old one ends.
- The read API tells a program what it is. Ask who you
are with the program's token: the answer is yours, with
"program": "program.tara.risk_report"added. - The private key never leaves the machine. Nothing sends it: not the SDK, not the recipe. Keep it readable by the program alone, and out of repositories, images and logs.
- If the key may have been copied, renew it.
bff-sdk program renew risk_report --key-file risk_report.pemmakes a new key pair and registers its public half. From that moment the old key signs in nowhere. The file is replaced only after the platform took the new key, so a failed renewal leaves a program that still works. - An operator may suspend a program. Its next request
fails with status 403 and the code
program_refused, and itsdetailgives the operator's reason.bff-sdk program listshows it too. - A program stands or falls with you. If you lose the
right to read, your programs stop, with the same
program_refused. They read as you, so they cannot read when you cannot. - Remove a program you no longer run.
bff-sdk program remove risk_reportdeletes it, and frees its name and its place among your ten.
Errors
Every failure answers in one shape, whatever the request: a problem, as
the standard for errors over HTTP writes one (RFC 9457). Its
Content-Type is application/problem+json:
{
"type": "/problems/unknown-series",
"title": "Unknown series",
"status": 400,
"detail": "No series [\"entsoe.de_price\"] to read.",
"instance": "/v1/values",
"code": "unknown_series",
"class": "InvalidRequest",
"details": {"unknown": ["entsoe.de_price"]}
}
code is for your program: it is stable, so branch on it.
detail is for a person, and its words may change.
details carries what a program can use, such as the names that
were unknown. class names who is at fault:
InvalidRequest and AccessDenied, the caller;
ModelError, a person's function; the others, the platform.
The other four members are the standard's. status repeats
the answer's status. title is the code in words, the same
every time. instance is the path you asked, without its query.
type names the kind of problem: /problems/ and
the code, with dashes for its underscores, and leads to the code's entry
on the page of every code, Problems: the read
API's address sends /problems/unknown-series there. Its last part
is also the code's row in the table below, so
/problems/unknown-series is
#unknown-series on this page.
Before 3.2.15 an error had four members: error,
code, message and details. A client
written then reads detail where it read message,
and class where it read error;
code and details are as they were.
The errors a reader meets:
| Status | Code | Means | Do |
|---|---|---|---|
| 400 | invalid_request | The request broke a rule the detail names: a parameter missing, a field unknown or of the wrong type, start not before end, over 20 series, a parameter of the catalog before 3.2.18. details.field names the field where there is one. | Correct the request. |
| 400 | invalid_cursor | A cursor is not one this list gave for this request. details.reason says which: malformed, not a cursor, or one cut short or changed; outdated, one kept from before a release; other_list, another list's; other_request, this list's, asked with other filters. | Send the next_cursor of the page before, as it was, with the same filters; or start again from the first page. |
| 400 | invalid_filter | A filter does not read: a field the list does not filter by, a value not of the field's kind, a quote or a parenthesis left open. details.position says where, counting from 0. | Correct the filter; the list's section names its fields. |
| 400 | invalid_time | A time is not written with its offset, or a day not as a date; or a recorded_as_of is closer to the clock than a cut by it may be. details.field names it. | Write it as 2026-11-17T23:00:00Z, a day as 2026-11-17; cut a little earlier. |
| 400 | invalid_duration | A step is not a duration such as PT15M, PT1H or P1D. | Correct it. |
| 400 | unknown_series | A series does not exist, or you may not read it. details.unknown lists the names. | Check the name against the catalog. |
| 400 | unknown_calendar, outside_calendar | No calendar for that zone, market and product; or none of its rules holds a day you asked for. | Check the names, and the days. |
| 400 | unknown_indicator, unknown_function | No such indicator of the platform's; no such function of a person's, or one you may not call. | Check the list; ask the owner for a grant. |
| 400 | unknown_version | No such version. details.versions lists those there are. | Name one of them. |
| 400 | invalid_parameter | A parameter is outside its declared range, or not of its type. | Read the declaration. |
| 400 | invalid_input | An indicator's input is not a series of numbers, or its inputs' units differ. | Choose other series. |
| 401 | unauthenticated | No token, or one that is malformed or expired. | Sign in again. |
| 403 | insufficient_scope | Your roles do not allow this. details.scope names the scope you lack. | Ask an operator for the role. |
| 403 | timeline_not_allowed | The timeline is another person's. | Read live, or a simulation of your own. |
| 403 | access_denied | A person's function was called with a token that is neither a person's nor a person's program's; or a program asked for what only its owner does, such as keeping charts. | Sign in as a person, or as a person's program; do yourself what a program may not. |
| 403 | program_refused | The token is a program's, and the program is suspended or removed, or its owner may no longer read. The detail says which. | Ask the program's owner, or the operator who suspended it. |
| 403 | version_suspended | An operator stopped this version of a function. details.reason says why. | Call another version, or ask the owner. |
| 413 | payload_too_large | An indicator reads over 200,000 values, its history included; or a person's function reads more in one read, or in one call, than it may. details.limit gives the limit. | Ask for a shorter period or fewer series. |
| 429 | rate_limited | Too many open streams; or more calls of people's functions in a minute than an operator has set, if one has. | Wait as long as Retry-After says; close a stream. |
| 500 | model_failed, invalid_answer | A person's function failed, or answered what it did not declare. details.run is the run's id. | Tell the owner, with the id. |
| 500 | internal | A fault of the platform's. | Tell an operator. |
| 503 | store_unavailable | The platform cannot reach its database. | Try again after Retry-After. |
| 503 | events_unavailable | This platform tells no changes. | Read on a timer. |
| 503 | runner_busy, runner_unavailable | No worker is free to run a person's function, or none answered. | Try again after Retry-After. |
| 504 | model_timeout | A person's function took longer than its declared time. | Ask for a shorter period; tell the owner. |
A status 429 or 503 comes with a Retry-After header, in
seconds: 5 for 429 and 1 for 503. These are the failures worth trying
again as they are. A 400 will fail the same way however often you send
it.
The platform may add codes. Decide by the code where you know it, and by the status where you do not: 4xx, correct the request; 5xx, try later or tell someone.
Limits
| What | Limit | Over it |
|---|---|---|
| Series in one read or one compute | 20; 500 in a read as columns, and in a coverage | 400, invalid_request |
| Rows in one page of values | 200,000; 10,000 unless you ask | A larger limit is taken as 200,000 |
| Values an indicator reads, its history included | 200,000 | 413, payload_too_large |
| Days in one read of a calendar | 62 | 400, invalid_request |
| Streams one person holds open | 8 | 429, rate_limited |
| Streams open on the platform, in all | 2,000 | 429, rate_limited |
| Calls of people's functions, by one caller | No limit, unless an operator sets one | 503, runner_busy, when no worker is free; 429, rate_limited, over a limit that is set |
| Series in one page of the catalog | 1,000; 500 unless you ask | A larger limit is taken as 1,000 |
| Models or indicators in one page of their lists | 500; 200 unless you ask | A larger limit is taken as 500 |
| A model's time for one call | 60 s, or less if it declares less | 504, model_timeout |
| A person's indicator's time for one call | 5 s | 504, model_timeout |
| An access token's life | 5 minutes | 401, unauthenticated; a stream ends with expired |
| Programs one person keeps | 10 | 400, too_many_programs |
| A sign-in without a renewal | 30 minutes | Sign in again |
| A sign-in, in all | 10 hours | Sign in again |
An operator may set the first five otherwise on a platform. Where a
limit refuses a read or a stream, details.limit gives the
number in force.
Next: What stays the same says which of all this you may build on for good.