{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "markdown-1",
   "metadata": {},
   "source": [
    "# 2. A first model: next week is like this week\n",
    "\n",
    "The simplest forecast that works is to repeat the past. Tomorrow at 08:00 will\n",
    "cost about what today did at 08:00. Forecasters call such a rule **naive**, and\n",
    "every serious model has to beat it. If yours cannot, it is not worth its\n",
    "complexity.\n",
    "\n",
    "In this notebook you will write a model, run it on real data, score it and\n",
    "improve it."
   ]
  },
  {
   "cell_type": "code",
   "id": "code-2",
   "metadata": {},
   "source": [
    "from pathlib import Path\n",
    "\n",
    "import pandas as pd\n",
    "\n",
    "from bff import lab\n",
    "\n",
    "FOLDER = Path(\"my_first_model\")\n",
    "if not FOLDER.exists():          # only the first time: after that, the folder is yours\n",
    "    lab.new(FOLDER, kind=\"model\")\n",
    "print((FOLDER / \"main.py\").read_text())"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "markdown-3",
   "metadata": {},
   "source": [
    "## What a model is\n",
    "\n",
    "A model is one Python function in a file called `main.py`. Read the file above\n",
    "from the top.\n",
    "\n",
    "**The declaration**, `@bff.model(...)`, tells the platform about the model\n",
    "before it runs:\n",
    "\n",
    "- `title` and `description`: what people see of it, on a screen or in a list;\n",
    "- `parameters`: what a caller may change. Here, which series to repeat;\n",
    "- `outputs`: what it answers, with a unit. Here, `forecast`, in EUR/MWh;\n",
    "- `examples`: at least one call that the platform's checks run;\n",
    "- `time`: the most it may take, here 20 seconds.\n",
    "\n",
    "**The function** receives a `call` and returns `call.answer(...)`. The call\n",
    "carries three things:\n",
    "\n",
    "- `call.start` and `call.end`: the period to forecast, from `start` up to but\n",
    "  not including `end`;\n",
    "- `call.as_of`: the cut-off. Every read sees only what was known by then.\n",
    "\n",
    "This model reads the prices of one week before the period and shifts them a\n",
    "week forward. Monday's forecast is last Monday."
   ]
  },
  {
   "cell_type": "code",
   "id": "code-4",
   "metadata": {},
   "source": [
    "answer = lab.run(FOLDER)       # its first example, on the platform's data\n",
    "answer"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "code",
   "id": "code-5",
   "metadata": {},
   "source": [
    "answer.plot();"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "markdown-6",
   "metadata": {},
   "source": [
    "The solid line is the forecast. The dashed line is what really happened, as we\n",
    "know it today. The vertical line is the cut-off.\n",
    "\n",
    "## How good is it?\n",
    "\n",
    "Look at the gap between the lines. A single number describes it well: the\n",
    "**mean absolute error** (MAE), the average distance between forecast and truth,\n",
    "in EUR/MWh. Lower is better."
   ]
  },
  {
   "cell_type": "code",
   "id": "code-7",
   "metadata": {},
   "source": [
    "def mae(answer):\n",
    "    forecast = answer[\"forecast\"][\"value\"]\n",
    "    actual = lab.series(\"epf.price\", answer.start, answer.end)   # as known today\n",
    "    return (forecast - actual.reindex(forecast.index)).abs().mean()\n",
    "\n",
    "mae(answer)"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "markdown-8",
   "metadata": {},
   "source": [
    "One day proves little. Run the model for each day of a week, each time as known\n",
    "at 09:00 in Berlin the morning before, as a trader would. `lab.run` takes the\n",
    "period and the cut-off you give it:"
   ]
  },
  {
   "cell_type": "code",
   "id": "code-9",
   "metadata": {},
   "source": [
    "def score_week(folder):\n",
    "    scores = {}\n",
    "    for day in pd.date_range(\"2017-03-06\", \"2017-03-12\", tz=\"Europe/Berlin\"):\n",
    "        start = day.tz_convert(\"UTC\")\n",
    "        end = (day + pd.DateOffset(days=1)).tz_convert(\"UTC\")\n",
    "        as_of = (day - pd.Timedelta(hours=15)).tz_convert(\"UTC\")   # 09:00 the day before\n",
    "        scores[f\"{day:%a %d %b}\"] = mae(lab.run(folder, start=start, end=end, as_of=as_of))\n",
    "    return pd.Series(scores).round(1)\n",
    "\n",
    "week_before = score_week(FOLDER)\n",
    "week_before"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "markdown-10",
   "metadata": {},
   "source": [
    "## Make it better\n",
    "\n",
    "Last week is a week old. Yesterday is fresher. Would repeating yesterday do\n",
    "better?\n",
    "\n",
    "Open `my_first_model/main.py` in the file browser on the left. In the line\n",
    "`week = pd.Timedelta(days=7)`, change `days=7` to `days=1`. Save the file, then\n",
    "run the next cell. **Before you run it, predict which days improve\n",
    "and which get worse.**"
   ]
  },
  {
   "cell_type": "code",
   "id": "code-11",
   "metadata": {},
   "source": [
    "day_before = score_week(FOLDER)\n",
    "pd.DataFrame({\"week before\": week_before, \"day before\": day_before})"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "markdown-12",
   "metadata": {},
   "source": [
    "If you changed the file, you will see two things. Midweek days improve:\n",
    "Thursday looks like Wednesday. But Monday and Sunday get much worse. Monday\n",
    "after Sunday is the worst case: you forecast a working day from a quiet one.\n",
    "\n",
    "The fix is to use each rule where it works: the day before on Tuesday to\n",
    "Friday, the week before on Monday, Saturday and Sunday. That is the\n",
    "platform's `naive` example. Read it in `../examples/naive/main.py`, then score\n",
    "it:"
   ]
  },
  {
   "cell_type": "code",
   "id": "code-13",
   "metadata": {},
   "source": [
    "pd.DataFrame({\"week before\": week_before, \"day before\": day_before,\n",
    "              \"naive\": score_week(\"../examples/naive\")})"
   ],
   "execution_count": null,
   "outputs": []
  },
  {
   "cell_type": "markdown",
   "id": "markdown-14",
   "metadata": {},
   "source": [
    "Look along each row. The naive rule is not the best every day: this week, the\n",
    "week before happened to be closer on Wednesday and Friday. Now look down the\n",
    "columns. Naive is never far off, and its errors add up to the least. That is\n",
    "the lesson: judge a forecast over many days, never on one. A rule that wins a\n",
    "single day may simply have been lucky.\n",
    "\n",
    "Naive does well because it knows both patterns from the first notebook: the\n",
    "shape of the day and the shape of the week.\n",
    "\n",
    "## Try it yourself\n",
    "\n",
    "1. Compute the mean of each column with `.mean()`. How much does naive gain\n",
    "   over the other two?\n",
    "2. Score a week in summer, starting 2017-06-05. Does the same rule win?\n",
    "3. Put `days=7` back in your model, or keep your change: the folder is yours."
   ]
  }
 ],
 "nbformat": 4,
 "nbformat_minor": 5,
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python"
  }
 }
}
