Upload 203 files

.gitattributes

@@ -33,3 +33,7 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+docs/_static/logo.ai filter=lfs diff=lfs merge=lfs -text
+docs/documentation/valuing_actions/action_changes_gamestate.png filter=lfs diff=lfs merge=lfs -text
+docs/documentation/valuing_actions/action.gif filter=lfs diff=lfs merge=lfs -text
+docs/documentation/valuing_actions/default_xt_grid.png filter=lfs diff=lfs merge=lfs -text

CONTRIBUTING.rst

@@ -0,0 +1,205 @@
+Contributor guide
+=================
+
+This document lays out guidelines and advice for contributing to this project.
+If you're thinking of contributing, please start by reading this document and
+getting a feel for how contributing to this project works. If you have any
+questions, feel free to reach out to `Pieter Robberechts`_, the primary maintainer.
+
+.. _Pieter Robberechts: http://www.cs.kuleuven.be/cgi-bin/e-post.pl?epost=Pieter.Robberechts
+
+The guide is split into sections based on the type of contribution you're
+thinking of making.
+
+
+.. _bug-reports:
+
+Bug reports
+-----------
+
+Bug reports are hugely important! Before you raise one, though, please check
+through the `GitHub issues`_, **both open and closed**, to confirm that the bug
+hasn't been reported before.
+
+When filing an issue, make sure to answer these questions:
+
+- Which Python version are you using?
+- Which version of socceraction are you using?
+- What did you do?
+- What did you expect to see?
+- What did you see instead?
+
+The best way to get your bug fixed is to provide a test case,
+and/or steps to reproduce the issue.
+
+.. _GitHub issues: https://github.com/ML-KULeuven/socceraction/issues
+
+
+Feature requests
+----------------
+
+Socceraction is not actively developed. Its primary use is to enable
+reproducibility of our research. If you believe there is a feature missing,
+feel free to raise a feature request on the `Issue Tracker`_, but please do be
+aware that the overwhelming likelihood is that your feature request will not
+be accepted.
+
+.. _Issue tracker: https://github.com/ML-KULeuven/socceraction/issues
+
+
+Documentation contributions
+---------------------------
+
+Documentation improvements are always welcome! The documentation files live in
+the ``docs/`` directory of the codebase. They're written in
+`reStructuredText`_, and use `Sphinx`_ to generate the full suite of
+documentation.
+
+You do not have to set up a development environment to make small changes to
+the docs. Instead, you can `edit files directly on GitHub`_ and suggest changes.
+
+When contributing documentation, please do your best to follow the style of the
+documentation files. This means a soft-limit of 79 characters wide in your text
+files and a semiformal, yet friendly and approachable, prose style.
+
+When presenting Python code, use double-quoted strings (``"hello"`` instead of
+``'hello'``).
+
+.. _reStructuredText: http://docutils.sourceforge.net/rst.html
+.. _Sphinx: http://sphinx-doc.org/index.html
+.. _edit files directly on GitHub: https://docs.github.com/en/repositories/working-with-files/managing-files/editing-files
+
+
+Code contributions
+------------------
+
+If you intend to contribute code, do not feel the need to sit on your
+contribution until it is perfectly polished and complete. It helps everyone
+involved for you to seek feedback as early as you possibly can. Submitting an
+early, unfinished version of your contribution for feedback can save you from
+putting a lot of work into a contribution that is not suitable for the
+project.
+
+Setting up your development environment
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+You need Python 3.9+ and the following tools:
+
+- Poetry_
+- Nox_
+- nox-poetry_
+
+Install the package with development requirements:
+
+.. code:: console
+
+   $ poetry install
+   $ poetry self add poetry-plugin-export
+
+You can now run an interactive Python session.
+
+.. code:: console
+
+   $ poetry run python
+
+.. _Poetry: https://python-poetry.org/
+.. _Nox: https://nox.thea.codes/
+.. _nox-poetry: https://nox-poetry.readthedocs.io/
+
+Steps for submitting code
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+When contributing code, you'll want to follow this checklist:
+
+1. Fork the repository on GitHub.
+2. Run the tests to confirm they all pass on your system. If they don't, you'll
+   need to investigate why they fail. If you're unable to diagnose this
+   yourself, raise it as a bug report.
+3. Write tests that demonstrate your bug or feature. Ensure that they fail.
+4. Make your change.
+5. Run the entire test suite again, confirming that all tests pass *including
+   the ones you just added*.
+6. Make sure your code follows the code style discussed below.
+7. Send a GitHub Pull Request to the main repository's ``master`` branch.
+   GitHub Pull Requests are the expected method of code collaboration on this
+   project.
+
+Testing the project
+~~~~~~~~~~~~~~~~~~~
+
+Download the test data:
+
+.. code:: console
+
+   $ poetry run python tests/datasets/download.py
+
+Run the full test suite:
+
+.. code:: console
+
+   $ nox
+
+List the available Nox sessions:
+
+.. code:: console
+
+   $ nox --list-sessions
+
+You can also run a specific Nox session.
+For example, invoke the unit test suite like this:
+
+.. code:: console
+
+   $ nox --session=tests
+
+Unit tests are located in the ``tests`` directory,
+and are written using the pytest_ testing framework.
+
+.. _pytest: https://pytest.readthedocs.io/
+
+Code style
+~~~~~~~~~~~~
+
+The socceraction codebase uses the `PEP 8`_ code style. In addition, we have
+a few guidelines:
+
+- Line-length can exceed 79 characters, to 100, when convenient.
+- Line-length can exceed 100 characters, when doing otherwise would be *terribly* inconvenient.
+- Always use double-quoted strings (e.g. ``"soccer"``), unless a double-quote occurs within the string.
+
+To ensure all code conforms to this format. You can format the code using the
+pre-commit hooks.
+
+.. code:: console
+
+   $ nox --session=pre-commit
+
+Docstrings are to follow the `numpydoc guidelines`_.
+
+.. _PEP 8: https://pep8.org/
+.. _numpydoc guidelines: https://numpydoc.readthedocs.io/en/latest/format.html
+
+Submitting changes
+~~~~~~~~~~~~~~~~~~
+
+Open a `pull request`_ to submit changes to this project.
+
+Your pull request needs to meet the following guidelines for acceptance:
+
+- The Nox test suite must pass without errors and warnings.
+- Include unit tests.
+- If your changes add functionality, update the documentation accordingly.
+
+Feel free to submit early, though. We can always iterate on this.
+
+To run linting and code formatting checks before committing your change, you
+can install pre-commit as a Git hook by running the following command:
+
+.. code:: console
+
+   $ nox --session=pre-commit -- install
+
+It is recommended to open an issue before starting work on anything.
+
+.. _pull request: https://github.com/ML-KULeuven/socceraction/pulls
+.. github-only

LICENSE.rst

@@ -0,0 +1,22 @@
+MIT License
+===========
+
+Copyright (c) 2019 KU Leuven Machine Learning Research Group
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+**The software is provided "as is", without warranty of any kind, express or
+implied, including but not limited to the warranties of merchantability,
+fitness for a particular purpose and noninfringement. In no event shall the
+authors or copyright holders be liable for any claim, damages or other
+liability, whether in an action of contract, tort or otherwise, arising from,
+out of or in connection with the software or the use or other dealings in the
+software.**

Makefile

@@ -0,0 +1,50 @@
+.PHONY: init test lint pretty notebooks precommit_install bump_major bump_minor bump_patch clean
+
+BIN = .venv/bin/
+CODE = socceraction
+
+init:
+	python3 -m venv .venv
+	poetry install
+
+tests/datasets/statsbomb/:
+	$(BIN)python tests/datasets/download.py --download-statsbomb --convert-statsbomb
+
+tests/datasets/wyscout_public/:
+	$(BIN)python tests/datasets/download.py --download-wyscout --convert-wyscout
+
+tests/datasets/spadl/:
+	$(BIN)python tests/datasets/download.py --spadl
+
+test: tests/datasets/statsbomb/  tests/datasets/wyscout_public/ tests/datasets/spadl/
+	nox -rs tests -- $(args)
+
+mypy:
+	nox -rs mypy -- $(args)
+
+lint:
+	nox -rs lint -- $(args)
+
+pretty:
+	nox -rs pre-commit -- $(args)
+
+notebooks:
+	$(BIN)python -m nbconvert --execute --inplace --config=default.json public-notebooks/*.ipynb
+
+precommit_install:
+	nox -rs pre-commit -- install
+
+bump_major:
+	$(BIN)bumpversion major
+
+bump_minor:
+	$(BIN)bumpversion minor
+
+bump_patch:
+	$(BIN)bumpversion patch
+
+clean:
+	find . -type f -name "*.py[co]" -delete
+	find . -type d -name "__pycache__" -delete
+	rm -rf tests/datasets/wyscout_public
+	rm -rf tests/datasets/statsbomb

README.md

@@ -1,12 +1,66 @@
----
-title: Socr
-emoji: 🚀
-colorFrom: green
-colorTo: indigo
-sdk: streamlit
-sdk_version: 1.42.0
-app_file: app.py
-pinned: false
----
-
-Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
+<div align="center">
+	<img src="https://socceraction.readthedocs.io/en/latest/_static/logo_white.png" height="200">
+	<p>
+	<b>Convert soccer event stream data to the SPADL format<br/>and value on-the-ball player actions</b>
+	</p>
+	<br/>
+
+[![PyPi](https://img.shields.io/pypi/v/socceraction.svg)](https://pypi.org/project/socceraction)
+[![Python Version: 3.9+](https://img.shields.io/badge/Python-3.9+-blue.svg)](https://pypi.org/project/socceraction)
+[![Downloads](https://img.shields.io/pypi/dm/socceraction.svg)](https://pypistats.org/packages/socceraction)
+[![License: MIT](https://img.shields.io/badge/License-MIT-green.svg)](https://en.wikipedia.org/wiki/MIT_License)
+
+[![Build Status](https://github.com/ML-KULeuven/socceraction/workflows/CI/badge.svg)](https://github.com/ML-KULeuven/socceraction/actions?workflow=CI)
+[![Read the Docs](https://img.shields.io/readthedocs/socceraction/latest.svg?label=Read%20the%20Docs)](https://socceraction.readthedocs.io)
+[![Code coverage](https://codecov.io/gh/ML-KULeuven/socceraction/branch/master/graph/badge.svg)](https://codecov.io/gh/ML-KULeuven/socceraction)
+
+<br/>
+<br/>
+</div>
+
+Socceraction is a Python package for objectively quantifying the impact of the individual actions performed by soccer players using event stream data. The general idea is to assign a value to each on-the-ball action based on the action's impact on the game outcome, while accounting for the context in which the action happened. The video below gives a quick two-minute introduction to action values.
+
+<div align="center">
+
+https://user-images.githubusercontent.com/2175271/136857714-1d2c8706-7f2f-449d-818f-0e67fbb75400.mp4
+
+</div>
+
+## Features
+
+Socceraction contains the following components:
+
+- A set of API clients for **loading event stream data** from StatsBomb, Opta, Wyscout, Stats Perform and WhoScored as Pandas DataFrames using a unified data model. [Read more »](https://socceraction.readthedocs.io/en/latest/documentation/data/index.html)
+- Converters for each of these provider's proprietary data format to the **SPADL** and **atomic-SPADL** formats, which are unified and expressive languages for on-the-ball player actions. [Read more »](https://socceraction.readthedocs.io/en/latest/documentation/spadl/index.html)
+- An implementation of the **Expected Threat (xT)** possession value framework. [Read more »](https://socceraction.readthedocs.io/en/latest/documentation/valuing_actions/xT.html)
+- An implementation of the **VAEP** and **Atomic-VAEP** possession value frameworks. [Read more »](https://socceraction.readthedocs.io/en/latest/documentation/valuing_actions/vaep.html)
+
+## Installation / Getting started
+
+The recommended way to install `socceraction` is to simply use pip. The latest version officially supports Python 3.9 - 3.12.
+
+```sh
+$ pip install socceraction
+```
+
+The folder [`public-notebooks`](https://github.com/ML-KULeuven/socceraction/tree/master/public-notebooks) provides a demo of the full pipeline from raw StatsBomb event stream data to action values and player ratings. More detailed installation/usage instructions can be found in the [Documentation](https://socceraction.readthedocs.io/en/latest/).
+
+## Contributing
+
+All contributions, bug reports, bug fixes, documentation improvements, enhancements, and ideas are welcome. However, be aware that socceraction is not actively developed. It's primary use is to enable reproducibility of our research. If you believe there is a feature missing, feel free to raise a feature request, but please do be aware that the overwhelming likelihood is that your feature request will not be accepted.
+To learn more on how to contribute, see the [Contributor Guide](https://socceraction.readthedocs.io/en/latest/development/developer_guide.html).
+
+## Research
+
+If you make use of this package in your research, please consider citing the following papers:
+
+- Tom Decroos, Lotte Bransen, Jan Van Haaren, and Jesse Davis. **Actions speak louder than goals: Valuing player actions in soccer.** In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1851-1861. 2019. <br/>[ [pdf](http://doi.acm.org/10.1145/3292500.3330758) | [bibtex](https://github.com/ML-KULeuven/socceraction/blob/master/docs/_static/decroos19.bibtex) ]
+
+- Maaike Van Roy, Pieter Robberechts, Tom Decroos, and Jesse Davis. **Valuing on-the-ball actions in soccer: a critical comparison of XT and VAEP.** In Proceedings of the AAAI-20 Workshop on Artifical Intelligence in Team Sports. AI in Team Sports Organising Committee, 2020. <br/>[ [pdf](https://limo.libis.be/primo-explore/fulldisplay?docid=LIRIAS2913207&context=L&vid=KULeuven&search_scope=ALL_CONTENT&tab=all_content_tab&lang=en_US) | [bibtex](https://github.com/ML-KULeuven/socceraction/blob/master/docs/_static/vanroy20.bibtex) ]
+
+The Expected Threat (xT) framework was originally introduced by Karun Singh on his [blog](https://karun.in/blog/expected-threat.html) in 2019.
+
+## License
+
+Distributed under the terms of the [MIT license](https://opensource.org/licenses/MIT),
+socceraction is free and open source software. Although not strictly required, we appreciate it if you include a link to this repo or cite our research in your work if you make use of socceraction.

__init__.py

@@ -0,0 +1,13 @@
+"""
+SoccerAction
+~~~~~~~~~~~~
+
+SoccerAction is a Python package for objectively quantifying the impact of the
+individual actions performed by soccer players using event stream data.
+
+Full documentation is at <https://ml-kuleuven.github.io/socceraction/>.
+:copyright: (c) 2020 by DTAI KU Leuven.
+:license: MIT, see LICENSE for more details.
+"""
+
+__version__ = "1.5.3"

atomic/__init__.py

@@ -0,0 +1 @@
+"""Implements Atomic-SPADL and the Atomic-VAEP framework."""

atomic/spadl/__init__.py

@@ -0,0 +1,15 @@
+"""Implementation of the Atomic-SPADL language."""
+
+__all__ = [
+    "convert_to_atomic",
+    "AtomicSPADLSchema",
+    "bodyparts_df",
+    "actiontypes_df",
+    "add_names",
+    "play_left_to_right",
+]
+
+from .base import convert_to_atomic
+from .config import actiontypes_df, bodyparts_df
+from .schema import AtomicSPADLSchema
+from .utils import add_names, play_left_to_right

atomic/spadl/base.py

@@ -0,0 +1,236 @@
+"""Implements a converter for regular SPADL actions to atomic actions."""
+
+from typing import cast
+
+import pandas as pd
+from pandera.typing import DataFrame
+
+import socceraction.spadl.config as _spadl
+from socceraction.spadl.base import _add_dribbles
+from socceraction.spadl.schema import SPADLSchema
+
+from . import config as _atomicspadl
+from .schema import AtomicSPADLSchema
+
+
+def convert_to_atomic(actions: DataFrame[SPADLSchema]) -> DataFrame[AtomicSPADLSchema]:
+    """Convert regular SPADL actions to atomic actions.
+
+    Parameters
+    ----------
+    actions : pd.DataFrame
+        A SPADL dataframe.
+
+    Returns
+    -------
+    pd.DataFrame
+        The Atomic-SPADL dataframe.
+    """
+    atomic_actions = cast(pd.DataFrame, actions.copy())
+    atomic_actions = _extra_from_passes(atomic_actions)
+    atomic_actions = _add_dribbles(atomic_actions)  # for some reason this adds more dribbles
+    atomic_actions = _extra_from_shots(atomic_actions)
+    atomic_actions = _extra_from_fouls(atomic_actions)
+    atomic_actions = _convert_columns(atomic_actions)
+    atomic_actions = _simplify(atomic_actions)
+    return cast(DataFrame[AtomicSPADLSchema], atomic_actions)
+
+
+def _extra_from_passes(actions: pd.DataFrame) -> pd.DataFrame:
+    next_actions = actions.shift(-1)
+    same_team = actions.team_id == next_actions.team_id
+
+    passlike = [
+        "pass",
+        "cross",
+        "throw_in",
+        "freekick_short",
+        "freekick_crossed",
+        "corner_crossed",
+        "corner_short",
+        "clearance",
+        "goalkick",
+    ]
+    pass_ids = [_spadl.actiontypes.index(ty) for ty in passlike]
+
+    interceptionlike = [
+        "interception",
+        "tackle",
+        "keeper_punch",
+        "keeper_save",
+        "keeper_claim",
+        "keeper_pick_up",
+    ]
+    interception_ids = [_spadl.actiontypes.index(ty) for ty in interceptionlike]
+
+    samegame = actions.game_id == next_actions.game_id
+    sameperiod = actions.period_id == next_actions.period_id
+    # samephase = next_actions.time_seconds - actions.time_seconds < max_pass_duration
+    extra_idx = (
+        actions.type_id.isin(pass_ids)
+        & samegame
+        & sameperiod  # & samephase
+        & ~next_actions.type_id.isin(interception_ids)
+    )
+
+    prev = actions[extra_idx]
+    nex = next_actions[extra_idx]
+
+    extra = pd.DataFrame()
+    extra["game_id"] = prev.game_id
+    extra["original_event_id"] = prev.original_event_id
+    extra["period_id"] = prev.period_id
+    extra["action_id"] = prev.action_id + 0.1
+    extra["time_seconds"] = (prev.time_seconds + nex.time_seconds) / 2
+    extra["start_x"] = prev.end_x
+    extra["start_y"] = prev.end_y
+    extra["end_x"] = prev.end_x
+    extra["end_y"] = prev.end_y
+    extra["bodypart_id"] = _atomicspadl.bodyparts.index("foot")
+    extra["result_id"] = -1
+
+    offside = prev.result_id == _spadl.results.index("offside")
+    out = ((nex.type_id == _atomicspadl.actiontypes.index("goalkick")) & (~same_team)) | (
+        nex.type_id == _atomicspadl.actiontypes.index("throw_in")
+    )
+    ar = _atomicspadl.actiontypes
+    extra["type_id"] = -1
+    extra["type_id"] = (
+        extra.type_id.mask(same_team, ar.index("receival"))
+        .mask(~same_team, ar.index("interception"))
+        .mask(out, ar.index("out"))
+        .mask(offside, ar.index("offside"))
+    )
+    is_interception = extra["type_id"] == ar.index("interception")
+    extra["team_id"] = prev.team_id.mask(is_interception, nex.team_id)
+    extra["player_id"] = nex.player_id.mask(out | offside, prev.player_id).astype(
+        prev.player_id.dtype
+    )
+
+    actions = pd.concat([actions, extra], ignore_index=True, sort=False)
+    actions = actions.sort_values(["game_id", "period_id", "action_id"]).reset_index(drop=True)
+    actions["action_id"] = range(len(actions))
+    return actions
+
+
+def _extra_from_shots(actions: pd.DataFrame) -> pd.DataFrame:
+    next_actions = actions.shift(-1)
+
+    shotlike = ["shot", "shot_freekick", "shot_penalty"]
+    shot_ids = [_spadl.actiontypes.index(ty) for ty in shotlike]
+
+    samegame = actions.game_id == next_actions.game_id
+    sameperiod = actions.period_id == next_actions.period_id
+
+    shot = actions.type_id.isin(shot_ids)
+    goal = shot & (actions.result_id == _spadl.results.index("success"))
+    owngoal = actions.result_id == _spadl.results.index("owngoal")
+    next_corner_goalkick = next_actions.type_id.isin(
+        [
+            _atomicspadl.actiontypes.index("corner_crossed"),
+            _atomicspadl.actiontypes.index("corner_short"),
+            _atomicspadl.actiontypes.index("goalkick"),
+        ]
+    )
+    out = shot & next_corner_goalkick & samegame & sameperiod
+
+    extra_idx = goal | owngoal | out
+    prev = actions[extra_idx]
+    # nex = next_actions[extra_idx]
+
+    extra = pd.DataFrame()
+    extra["game_id"] = prev.game_id
+    extra["original_event_id"] = prev.original_event_id
+    extra["period_id"] = prev.period_id
+    extra["action_id"] = prev.action_id + 0.1
+    extra["time_seconds"] = prev.time_seconds  # + nex.time_seconds) / 2
+    extra["start_x"] = prev.end_x
+    extra["start_y"] = prev.end_y
+    extra["end_x"] = prev.end_x
+    extra["end_y"] = prev.end_y
+    extra["bodypart_id"] = prev.bodypart_id
+    extra["result_id"] = -1
+    extra["team_id"] = prev.team_id
+    extra["player_id"] = prev.player_id
+
+    ar = _atomicspadl.actiontypes
+    extra["type_id"] = -1
+    extra["type_id"] = (
+        extra.type_id.mask(out, ar.index("out"))
+        .mask(goal, ar.index("goal"))
+        .mask(owngoal, ar.index("owngoal"))
+    )
+    actions = pd.concat([actions, extra], ignore_index=True, sort=False)
+    actions = actions.sort_values(["game_id", "period_id", "action_id"]).reset_index(drop=True)
+    actions["action_id"] = range(len(actions))
+    return actions
+
+
+def _extra_from_fouls(actions: pd.DataFrame) -> pd.DataFrame:
+    yellow = actions.result_id == _spadl.results.index("yellow_card")
+    red = actions.result_id == _spadl.results.index("red_card")
+
+    prev = actions[yellow | red]
+    extra = pd.DataFrame()
+    extra["game_id"] = prev.game_id
+    extra["original_event_id"] = prev.original_event_id
+    extra["period_id"] = prev.period_id
+    extra["action_id"] = prev.action_id + 0.1
+    extra["time_seconds"] = prev.time_seconds  # + nex.time_seconds) / 2
+    extra["start_x"] = prev.end_x
+    extra["start_y"] = prev.end_y
+    extra["end_x"] = prev.end_x
+    extra["end_y"] = prev.end_y
+    extra["bodypart_id"] = prev.bodypart_id
+    extra["result_id"] = -1
+    extra["team_id"] = prev.team_id
+    extra["player_id"] = prev.player_id
+
+    ar = _atomicspadl.actiontypes
+    extra["type_id"] = -1
+    extra["type_id"] = extra.type_id.mask(yellow, ar.index("yellow_card")).mask(
+        red, ar.index("red_card")
+    )
+    actions = pd.concat([actions, extra], ignore_index=True, sort=False)
+    actions = actions.sort_values(["game_id", "period_id", "action_id"]).reset_index(drop=True)
+    actions["action_id"] = range(len(actions))
+    return actions
+
+
+def _convert_columns(actions: pd.DataFrame) -> pd.DataFrame:
+    actions["x"] = actions.start_x
+    actions["y"] = actions.start_y
+    actions["dx"] = actions.end_x - actions.start_x
+    actions["dy"] = actions.end_y - actions.start_y
+    return actions[
+        [
+            "game_id",
+            "original_event_id",
+            "action_id",
+            "period_id",
+            "time_seconds",
+            "team_id",
+            "player_id",
+            "x",
+            "y",
+            "dx",
+            "dy",
+            "type_id",
+            "bodypart_id",
+        ]
+    ]
+
+
+def _simplify(actions: pd.DataFrame) -> pd.DataFrame:
+    a = actions
+    ar = _atomicspadl.actiontypes
+
+    cornerlike = ["corner_crossed", "corner_short"]
+    corner_ids = [_spadl.actiontypes.index(ty) for ty in cornerlike]
+
+    freekicklike = ["freekick_crossed", "freekick_short", "shot_freekick"]
+    freekick_ids = [_spadl.actiontypes.index(ty) for ty in freekicklike]
+
+    a["type_id"] = a.type_id.mask(a.type_id.isin(corner_ids), ar.index("corner"))
+    a["type_id"] = a.type_id.mask(a.type_id.isin(freekick_ids), ar.index("freekick"))
+    return a

atomic/spadl/config.py

@@ -0,0 +1,48 @@
+"""Configuration of the Atomic-SPADL language.
+
+Attributes
+----------
+field_length : float
+    The length of a pitch (in meters).
+field_width : float
+    The width of a pitch (in meters).
+bodyparts : list(str)
+    The bodyparts used in the Atomic-SPADL language.
+actiontypes : list(str)
+    The action types used in the Atomic-SPADL language.
+
+"""
+
+import pandas as pd
+
+import socceraction.spadl.config as _spadl
+
+field_length = _spadl.field_length
+field_width = _spadl.field_width
+
+bodyparts = _spadl.bodyparts
+bodyparts_df = _spadl.bodyparts_df
+
+actiontypes = _spadl.actiontypes + [
+    "receival",
+    "interception",
+    "out",
+    "offside",
+    "goal",
+    "owngoal",
+    "yellow_card",
+    "red_card",
+    "corner",
+    "freekick",
+]
+
+
+def actiontypes_df() -> pd.DataFrame:
+    """Return a dataframe with the type id and type name of each Atomic-SPADL action type.
+
+    Returns
+    -------
+    pd.DataFrame
+        The 'type_id' and 'type_name' of each Atomic-SPADL action type.
+    """
+    return pd.DataFrame(list(enumerate(actiontypes)), columns=["type_id", "type_name"])

atomic/spadl/schema.py

@@ -0,0 +1,32 @@
+"""Schema for Atomic-SPADL actions."""
+
+from typing import Any, Optional
+
+import pandera as pa
+from pandera.typing import Series
+
+from . import config as spadlconfig
+
+
+class AtomicSPADLSchema(pa.SchemaModel):
+    """Definition of an Atomic-SPADL dataframe."""
+
+    game_id: Series[Any] = pa.Field()
+    original_event_id: Series[Any] = pa.Field(nullable=True)
+    action_id: Series[int] = pa.Field()
+    period_id: Series[int] = pa.Field(ge=1, le=5)
+    time_seconds: Series[float] = pa.Field(ge=0)
+    team_id: Series[Any] = pa.Field()
+    player_id: Series[Any] = pa.Field()
+    x: Series[float] = pa.Field(ge=0, le=spadlconfig.field_length)
+    y: Series[float] = pa.Field(ge=0, le=spadlconfig.field_width)
+    dx: Series[float] = pa.Field(ge=-spadlconfig.field_length, le=spadlconfig.field_length)
+    dy: Series[float] = pa.Field(ge=-spadlconfig.field_width, le=spadlconfig.field_width)
+    bodypart_id: Series[int] = pa.Field(isin=spadlconfig.bodyparts_df().bodypart_id)
+    bodypart_name: Optional[Series[str]] = pa.Field(isin=spadlconfig.bodyparts_df().bodypart_name)
+    type_id: Series[int] = pa.Field(isin=spadlconfig.actiontypes_df().type_id)
+    type_name: Optional[Series[str]] = pa.Field(isin=spadlconfig.actiontypes_df().type_name)
+
+    class Config:  # noqa: D106
+        strict = True
+        coerce = True

atomic/spadl/utils.py

@@ -0,0 +1,65 @@
+"""Utility functions for working with Atomic-SPADL dataframes."""
+
+from typing import cast
+
+from pandera.typing import DataFrame
+
+from . import config as spadlconfig
+from .schema import AtomicSPADLSchema
+
+
+def add_names(actions: DataFrame[AtomicSPADLSchema]) -> DataFrame[AtomicSPADLSchema]:
+    """Add the type name, result name and bodypart name to an Atomic-SPADL dataframe.
+
+    Parameters
+    ----------
+    actions : pd.DataFrame
+        An Atomic-SPADL dataframe.
+
+    Returns
+    -------
+    pd.DataFrame
+        The original dataframe with a 'type_name', 'result_name' and
+        'bodypart_name' appended.
+    """
+    return cast(
+        DataFrame[AtomicSPADLSchema],
+        actions.drop(columns=["type_name", "bodypart_name"], errors="ignore")
+        .merge(spadlconfig.actiontypes_df(), how="left")
+        .merge(spadlconfig.bodyparts_df(), how="left")
+        .set_index(actions.index),
+    )
+
+
+def play_left_to_right(
+    actions: DataFrame[AtomicSPADLSchema], home_team_id: int
+) -> DataFrame[AtomicSPADLSchema]:
+    """Perform all action in the same playing direction.
+
+    This changes the location of each action, such that all actions
+    are performed as if the team that executes the action plays from left to
+    right.
+
+    Parameters
+    ----------
+    actions : pd.DataFrame
+        The SPADL actins of a game.
+    home_team_id : int
+        The ID of the home team.
+
+    Returns
+    -------
+    list(pd.DataFrame)
+        All actions performed left to right.
+
+    See Also
+    --------
+    socceraction.atomic.vaep.features.play_left_to_right : For transforming gamestates.
+    """
+    ltr_actions = actions.copy()
+    away_idx = actions.team_id != home_team_id
+    ltr_actions.loc[away_idx, "x"] = spadlconfig.field_length - actions[away_idx]["x"].values
+    ltr_actions.loc[away_idx, "y"] = spadlconfig.field_width - actions[away_idx]["y"].values
+    ltr_actions.loc[away_idx, "dx"] = -actions[away_idx]["dx"].values
+    ltr_actions.loc[away_idx, "dy"] = -actions[away_idx]["dy"].values
+    return ltr_actions

atomic/vaep/__init__.py

@@ -0,0 +1,6 @@
+"""Implements the Atomic-VAEP framework."""
+
+from . import features, formula, labels
+from .base import AtomicVAEP
+
+__all__ = ["AtomicVAEP", "features", "labels", "formula"]

atomic/vaep/base.py

@@ -0,0 +1,80 @@
+"""Implements the Atomic-VAEP framework.
+
+Attributes
+----------
+xfns_default : list(callable)
+    The default VAEP features.
+
+"""
+
+from typing import Optional
+
+import socceraction.atomic.spadl as spadlcfg
+from socceraction.vaep.base import VAEP
+
+from . import features as fs
+from . import formula as vaep
+from . import labels as lab
+
+xfns_default = [
+    fs.actiontype,
+    fs.actiontype_onehot,
+    fs.bodypart,
+    fs.bodypart_onehot,
+    fs.time,
+    fs.team,
+    fs.time_delta,
+    fs.location,
+    fs.polar,
+    fs.movement_polar,
+    fs.direction,
+    fs.goalscore,
+]
+
+
+class AtomicVAEP(VAEP):
+    """
+    An implementation of the VAEP framework for atomic actions.
+
+    In contrast to the original VAEP framework [1]_ this extension
+    distinguishes the contribution of the player who initiates the action
+    (e.g., gives the pass) and the player who completes the action (e.g.,
+    receives the pass) [2]_.
+
+    Parameters
+    ----------
+    xfns : list
+        List of feature transformers (see :mod:`socceraction.atomic.vaep.features`)
+        used to describe the game states. Uses :attr:`~socceraction.vaep.base.xfns_default`
+        if None.
+    nb_prev_actions : int, default=3
+        Number of previous actions used to decscribe the game state.
+
+    See Also
+    --------
+    :class:`socceraction.vaep.VAEP` : Implementation of the original VAEP framework.
+
+    References
+    ----------
+    .. [1] Tom Decroos, Lotte Bransen, Jan Van Haaren, and Jesse Davis.
+        "Actions speak louder than goals: Valuing player actions in soccer." In
+        Proceedings of the 25th ACM SIGKDD International Conference on Knowledge
+        Discovery & Data Mining, pp. 1851-1861. 2019.
+    .. [2] Tom Decroos, Pieter Robberechts and Jesse Davis.
+        "Introducing Atomic-SPADL: A New Way to Represent Event Stream Data".
+        DTAI Sports Analytics Blog. https://dtai.cs.kuleuven.be/sports/blog/introducing-atomic-spadl:-a-new-way-to-represent-event-stream-data  # noqa
+        May 2020.
+    """
+
+    _spadlcfg = spadlcfg
+    _lab = lab
+    _fs = fs
+    _vaep = vaep
+
+    def __init__(
+        self,
+        xfns: Optional[list[fs.FeatureTransfomer]] = None,
+        nb_prev_actions: int = 3,
+    ) -> None:
+        xfns = xfns_default if xfns is None else xfns
+        super().__init__(xfns, nb_prev_actions)

atomic/vaep/features.py

@@ -0,0 +1,269 @@
+"""Implements the feature tranformers of the VAEP framework."""
+
+from typing import Any, Callable, Union
+
+import numpy as np
+import pandas as pd
+from pandera.typing import DataFrame
+
+import socceraction.atomic.spadl.config as atomicspadl
+from socceraction.atomic.spadl import AtomicSPADLSchema
+from socceraction.spadl import SPADLSchema
+from socceraction.vaep.features import (
+    actiontype,
+    bodypart,
+    bodypart_detailed,
+    bodypart_detailed_onehot,
+    bodypart_onehot,
+    gamestates,
+    player_possession_time,
+    simple,
+    speed,
+    team,
+    time,
+    time_delta,
+)
+
+__all__ = [
+    "feature_column_names",
+    "play_left_to_right",
+    "gamestates",
+    "actiontype",
+    "actiontype_onehot",
+    "bodypart",
+    "bodypart_detailed",
+    "bodypart_onehot",
+    "bodypart_detailed_onehot",
+    "team",
+    "time",
+    "time_delta",
+    "speed",
+    "location",
+    "polar",
+    "movement_polar",
+    "direction",
+    "goalscore",
+    "player_possession_time",
+]
+
+Actions = Union[DataFrame[SPADLSchema], DataFrame[AtomicSPADLSchema]]
+GameStates = list[Actions]
+Features = DataFrame[Any]
+FeatureTransfomer = Callable[[GameStates], Features]
+
+
+def feature_column_names(fs: list[FeatureTransfomer], nb_prev_actions: int = 3) -> list[str]:
+    """Return the names of the features generated by a list of transformers.
+
+    Parameters
+    ----------
+    fs : list(callable)
+        A list of feature transformers.
+    nb_prev_actions : int, default=3  # noqa: DAR103
+        The number of previous actions included in the game state.
+
+    Returns
+    -------
+    list(str)
+        The name of each generated feature.
+    """
+    spadlcolumns = [
+        "game_id",
+        "original_event_id",
+        "action_id",
+        "period_id",
+        "time_seconds",
+        "team_id",
+        "player_id",
+        "x",
+        "y",
+        "dx",
+        "dy",
+        "bodypart_id",
+        "bodypart_name",
+        "type_id",
+        "type_name",
+    ]
+    dummy_actions = pd.DataFrame(np.zeros((10, len(spadlcolumns))), columns=spadlcolumns)
+    for c in spadlcolumns:
+        if "name" in c:
+            dummy_actions[c] = dummy_actions[c].astype(str)
+    gs = gamestates(dummy_actions, nb_prev_actions)  # type: ignore
+    return list(pd.concat([f(gs) for f in fs], axis=1).columns)
+
+
+def play_left_to_right(gamestates: GameStates, home_team_id: int) -> GameStates:
+    """Perform all action in the same playing direction.
+
+    This changes the start and end location of each action, such that all actions
+    are performed as if the team plays from left to right.
+
+    Parameters
+    ----------
+    gamestates : GameStates
+        The game states of a game.
+    home_team_id : int
+        The ID of the home team.
+
+    Returns
+    -------
+    list(pd.DataFrame)
+        The game states with all actions performed left to right.
+    """
+    a0 = gamestates[0]
+    away_idx = a0.team_id != home_team_id
+    for actions in gamestates:
+        actions.loc[away_idx, "x"] = atomicspadl.field_length - actions[away_idx]["x"].values
+        actions.loc[away_idx, "y"] = atomicspadl.field_width - actions[away_idx]["y"].values
+        actions.loc[away_idx, "dx"] = -actions[away_idx]["dx"].values
+        actions.loc[away_idx, "dy"] = -actions[away_idx]["dy"].values
+    return gamestates
+
+
+@simple
+def actiontype_onehot(actions: Actions) -> Features:
+    """Get the one-hot-encoded type of each action.
+
+    Parameters
+    ----------
+    actions : Actions
+        The actions of a game.
+
+    Returns
+    -------
+    Features
+        A one-hot encoding of each action's type.
+    """
+    X = {}
+    for type_id, type_name in enumerate(atomicspadl.actiontypes):
+        col = "actiontype_" + type_name
+        X[col] = actions["type_id"] == type_id
+    return pd.DataFrame(X, index=actions.index)
+
+
+@simple
+def location(actions: Actions) -> Features:
+    """Get the location where each action started.
+
+    Parameters
+    ----------
+    actions : Actions
+        The actions of a game.
+
+    Returns
+    -------
+    Features
+        The 'x' and 'y' location of each action.
+    """
+    return actions[["x", "y"]]
+
+
+_goal_x = atomicspadl.field_length
+_goal_y = atomicspadl.field_width / 2
+
+
+@simple
+def polar(actions: Actions) -> Features:
+    """Get the polar coordinates of each action's start location.
+
+    The center of the opponent's goal is used as the origin.
+
+    Parameters
+    ----------
+    actions : Actions
+        The actions of a game.
+
+    Returns
+    -------
+    Features
+        The 'dist_to_goal' and 'angle_to_goal' of each action.
+    """
+    polardf = pd.DataFrame(index=actions.index)
+    dx = (_goal_x - actions["x"]).abs().values
+    dy = (_goal_y - actions["y"]).abs().values
+    polardf["dist_to_goal"] = np.sqrt(dx**2 + dy**2)
+    with np.errstate(divide="ignore", invalid="ignore"):
+        polardf["angle_to_goal"] = np.nan_to_num(np.arctan(dy / dx))
+    return polardf
+
+
+@simple
+def movement_polar(actions: Actions) -> Features:
+    """Get the distance covered and direction of each action.
+
+    Parameters
+    ----------
+    actions : Actions
+        The actions of a game.
+
+    Returns
+    -------
+    Features
+        The distance covered ('mov_d') and direction ('mov_angle') of each action.
+    """
+    mov = pd.DataFrame(index=actions.index)
+    mov["mov_d"] = np.sqrt(actions.dx**2 + actions.dy**2)
+    with np.errstate(divide="ignore", invalid="ignore"):
+        mov["mov_angle"] = np.arctan2(actions.dy, actions.dx)
+        mov.loc[actions.dy == 0, "mov_angle"] = 0  # fix float errors
+    return mov
+
+
+@simple
+def direction(actions: Actions) -> Features:
+    """Get the direction of the action as components of the unit vector.
+
+    Parameters
+    ----------
+    actions : Actions
+        The actions of a game.
+
+    Returns
+    -------
+    Features
+        The x-component ('dx') and y-compoment ('mov_angle') of the unit
+        vector of each action.
+    """
+    mov = pd.DataFrame(index=actions.index)
+    totald = np.sqrt(actions.dx**2 + actions.dy**2)
+    for d in ["dx", "dy"]:
+        # we don't want to give away the end location,
+        # just the direction of the ball
+        # We also don't want to divide by zero
+        mov[d] = actions[d].mask(totald > 0, actions[d] / totald)
+
+    return mov
+
+
+def goalscore(gamestates: GameStates) -> Features:
+    """Get the number of goals scored by each team after the action.
+
+    Parameters
+    ----------
+    gamestates : GameStates
+        The gamestates of a game.
+
+    Returns
+    -------
+    Features
+        The number of goals scored by the team performing the last action of the
+        game state ('goalscore_team'), by the opponent ('goalscore_opponent'),
+        and the goal difference between both teams ('goalscore_diff').
+    """
+    actions = gamestates[0]
+    teamA = actions["team_id"].values[0]
+    goals = actions.type_name == "goal"
+    owngoals = actions["type_name"].str.contains("owngoal")
+
+    teamisA = actions["team_id"] == teamA
+    teamisB = ~teamisA
+    goalsteamA = (goals & teamisA) | (owngoals & teamisB)
+    goalsteamB = (goals & teamisB) | (owngoals & teamisA)
+    goalscoreteamA = goalsteamA.cumsum() - goalsteamA
+    goalscoreteamB = goalsteamB.cumsum() - goalsteamB
+
+    scoredf = pd.DataFrame(index=actions.index)
+    scoredf["goalscore_team"] = (goalscoreteamA * teamisA) + (goalscoreteamB * teamisB)
+    scoredf["goalscore_opponent"] = (goalscoreteamB * teamisA) + (goalscoreteamA * teamisB)
+    scoredf["goalscore_diff"] = scoredf["goalscore_team"] - scoredf["goalscore_opponent"]
+    return scoredf

atomic/vaep/formula.py

@@ -0,0 +1,142 @@
+"""Implements the formula of the Atomic-VAEP framework."""
+
+import pandas as pd
+from pandera.typing import DataFrame, Series
+
+from socceraction.atomic.spadl import AtomicSPADLSchema
+
+
+def _prev(x: pd.Series) -> pd.Series:
+    prev_x = x.shift(1)
+    prev_x[:1] = x.values[0]
+    return prev_x
+
+
+def offensive_value(
+    actions: DataFrame[AtomicSPADLSchema], scores: Series[float], concedes: Series[float]
+) -> Series[float]:
+    r"""Compute the offensive value of each action.
+
+    VAEP defines the *offensive value* of an action as the change in scoring
+    probability before and after the action.
+
+    .. math::
+
+      \Delta P_{score}(a_{i}, t) = P^{k}_{score}(S_i, t) - P^{k}_{score}(S_{i-1}, t)
+
+    where :math:`P_{score}(S_i, t)` is the probability that team :math:`t`
+    which possesses the ball in state :math:`S_i` will score in the next 10
+    actions.
+
+    Parameters
+    ----------
+    actions : pd.DataFrame
+        SPADL action.
+    scores : pd.Series
+        The probability of scoring from each corresponding game state.
+    concedes : pd.Series
+        The probability of conceding from each corresponding game state.
+
+    Returns
+    -------
+    pd.Series
+        he ffensive value of each action.
+    """
+    sameteam = _prev(actions.team_id) == actions.team_id
+    prev_scores = _prev(scores) * sameteam + _prev(concedes) * (~sameteam)
+
+    # if the previous action was too long ago, the odds of scoring are now 0
+    # toolong_idx = (
+    #    abs(actions.time_seconds - _prev(actions.time_seconds)) > _samephase_nb
+    # )
+    # prev_scores[toolong_idx] = 0
+
+    # if the previous action was a goal, the odds of scoring are now 0
+    prevgoal_idx = _prev(actions.type_name).isin(["goal", "owngoal"])
+    prev_scores[prevgoal_idx] = 0
+
+    return scores - prev_scores
+
+
+def defensive_value(
+    actions: DataFrame[AtomicSPADLSchema], scores: Series[float], concedes: Series[float]
+) -> Series[float]:
+    r"""Compute the defensive value of each action.
+
+    VAEP defines the *defensive value* of an action as the change in conceding
+    probability.
+
+    .. math::
+
+      \Delta P_{concede}(a_{i}, t) = P^{k}_{concede}(S_i, t) - P^{k}_{concede}(S_{i-1}, t)
+
+    where :math:`P_{concede}(S_i, t)` is the probability that team :math:`t`
+    which possesses the ball in state :math:`S_i` will concede in the next 10
+    actions.
+
+    Parameters
+    ----------
+    actions : pd.DataFrame
+        SPADL action.
+    scores : pd.Series
+        The probability of scoring from each corresponding game state.
+    concedes : pd.Series
+        The probability of conceding from each corresponding game state.
+
+    Returns
+    -------
+    pd.Series
+        The defensive value of each action.
+    """
+    sameteam = _prev(actions.team_id) == actions.team_id
+    prev_concedes = _prev(concedes) * sameteam + _prev(scores) * (~sameteam)
+
+    # if the previous action was too long ago, the odds of scoring are now 0
+    # toolong_idx = (
+    #    abs(actions.time_seconds - _prev(actions.time_seconds)) > _samephase_nb
+    # )
+    # prev_concedes[toolong_idx] = 0
+
+    # if the previous action was a goal, the odds of conceding are now 0
+    prevgoal_idx = _prev(actions.type_name).isin(["goal", "owngoal"])
+    prev_concedes[prevgoal_idx] = 0
+
+    return -(concedes - prev_concedes)
+
+
+def value(
+    actions: DataFrame[AtomicSPADLSchema], Pscores: Series[float], Pconcedes: Series[float]
+) -> pd.DataFrame:
+    r"""Compute the offensive, defensive and VAEP value of each action.
+
+    The total VAEP value of an action is the difference between that action's
+    offensive value and defensive value.
+
+    .. math::
+
+      V_{VAEP}(a_i) = \Delta P_{score}(a_{i}, t) - \Delta P_{concede}(a_{i}, t)
+
+    Parameters
+    ----------
+    actions : pd.DataFrame
+        SPADL action.
+    Pscores : pd.Series
+        The probability of scoring from each corresponding game state.
+    Pconcedes : pd.Series
+        The probability of conceding from each corresponding game state.
+
+    Returns
+    -------
+    pd.DataFrame
+        The 'offensive_value', 'defensive_value' and 'vaep_value' of each action.
+
+    See Also
+    --------
+    :func:`~socceraction.vaep.formula.offensive_value`: The offensive value
+    :func:`~socceraction.vaep.formula.defensive_value`: The defensive value
+    """
+    v = pd.DataFrame()
+    v["offensive_value"] = offensive_value(actions, Pscores, Pconcedes)
+    v["defensive_value"] = defensive_value(actions, Pscores, Pconcedes)
+    v["vaep_value"] = v["offensive_value"] + v["defensive_value"]
+    return v

atomic/vaep/labels.py

@@ -0,0 +1,108 @@
+"""Implements the label tranformers of the Atomic-VAEP framework."""
+
+import pandas as pd
+from pandera.typing import DataFrame
+
+import socceraction.atomic.spadl.config as atomicspadl
+from socceraction.atomic.spadl import AtomicSPADLSchema
+
+
+def scores(actions: DataFrame[AtomicSPADLSchema], nr_actions: int = 10) -> pd.DataFrame:
+    """Determine whether the team possessing the ball scored a goal within the next x actions.
+
+    Parameters
+    ----------
+    actions : pd.DataFrame
+        The actions of a game.
+    nr_actions : int, default=10  # noqa: DAR103
+        Number of actions after the current action to consider.
+
+    Returns
+    -------
+    pd.DataFrame
+        A dataframe with a column 'scores' and a row for each action set to
+        True if a goal was scored by the team possessing the ball within the
+        next x actions; otherwise False.
+    """
+    # merging goals, owngoals and team_ids
+    goals = actions["type_id"] == atomicspadl.actiontypes.index("goal")
+    owngoals = actions["type_id"] == atomicspadl.actiontypes.index("owngoal")
+    y = pd.concat([goals, owngoals, actions["team_id"]], axis=1)
+    y.columns = ["goal", "owngoal", "team_id"]
+
+    # adding future results
+    for i in range(1, nr_actions):
+        for c in ["team_id", "goal", "owngoal"]:
+            shifted = y[c].shift(-i)
+            shifted[-i:] = y[c].iloc[len(y) - 1]
+            y["%s+%d" % (c, i)] = shifted
+
+    res = y["goal"]
+    for i in range(1, nr_actions):
+        gi = y["goal+%d" % i] & (y["team_id+%d" % i] == y["team_id"])
+        ogi = y["owngoal+%d" % i] & (y["team_id+%d" % i] != y["team_id"])
+        res = res | gi | ogi
+
+    return pd.DataFrame(res, columns=["scores"])
+
+
+def concedes(actions: DataFrame[AtomicSPADLSchema], nr_actions: int = 10) -> pd.DataFrame:
+    """Determine whether the team possessing the ball conceded a goal within the next x actions.
+
+    Parameters
+    ----------
+    actions : pd.DataFrame
+        The actions of a game.
+    nr_actions : int, default=10  # noqa: DAR103
+        Number of actions after the current action to consider.
+
+    Returns
+    -------
+    pd.DataFrame
+        A dataframe with a column 'concedes' and a row for each action set to
+        True if a goal was conceded by the team possessing the ball within the
+        next x actions; otherwise False.
+    """
+    # merging goals, owngoals and team_ids
+    goals = actions["type_id"] == atomicspadl.actiontypes.index("goal")
+    owngoals = actions["type_id"] == atomicspadl.actiontypes.index("owngoal")
+    y = pd.concat([goals, owngoals, actions["team_id"]], axis=1)
+    y.columns = ["goal", "owngoal", "team_id"]
+
+    # adding future results
+    for i in range(1, nr_actions):
+        for c in ["team_id", "goal", "owngoal"]:
+            shifted = y[c].shift(-i)
+            shifted[-i:] = y[c].iloc[len(y) - 1]
+            y["%s+%d" % (c, i)] = shifted
+
+    res = y["owngoal"]
+    for i in range(1, nr_actions):
+        gi = y["goal+%d" % i] & (y["team_id+%d" % i] != y["team_id"])
+        ogi = y["owngoal+%d" % i] & (y["team_id+%d" % i] == y["team_id"])
+        res = res | gi | ogi
+
+    return pd.DataFrame(res, columns=["concedes"])
+
+
+def goal_from_shot(actions: DataFrame[AtomicSPADLSchema]) -> pd.DataFrame:
+    """Determine whether a goal was scored from the current action.
+
+    This label can be use to train an xG model.
+
+    Parameters
+    ----------
+    actions : pd.DataFrame
+        The actions of a game.
+
+    Returns
+    -------
+    pd.DataFrame
+        A dataframe with a column 'goal' and a row for each action set to
+        True if a goal was scored from the current action; otherwise False.
+    """
+    goals = (actions["type_id"] == atomicspadl.actiontypes.index("shot")) & (
+        actions["type_id"].shift(-1) == atomicspadl.actiontypes.index("goal")
+    )
+
+    return pd.DataFrame(goals.rename("goal"))

data/__init__.py

@@ -0,0 +1,9 @@
+"""Implements serializers for the event data of various providers."""
+
+__all__ = [
+    "opta",
+    "statsbomb",
+    "wyscout",
+]
+
+from . import opta, statsbomb, wyscout

data/base.py

@@ -0,0 +1,211 @@
+"""Base class and utility functions for all event stream data serializers.
+
+A serializer should extend the 'EventDataLoader' class to (down)load event
+stream data.
+"""
+
+import base64
+import json
+import warnings
+from abc import ABC, abstractmethod
+from typing import Any, Union
+from urllib import request
+
+from pandera.typing import DataFrame
+
+JSONType = Union[str, int, float, bool, None, dict[str, Any], list[Any]]
+
+
+class ParseError(Exception):
+    """Exception raised when a file is not correctly formatted."""
+
+
+class MissingDataError(Exception):
+    """Exception raised when a field is missing in the input data."""
+
+
+class NoAuthWarning(UserWarning):
+    """Warning raised when no user credentials are provided."""
+
+
+def _remoteloadjson(path: str) -> JSONType:
+    """Load JSON data from a URL.
+
+    Parameters
+    ----------
+    path : str
+        URL of the data source.
+
+    Returns
+    -------
+    JSONType
+        A dictionary with the loaded JSON data.
+    """
+    return json.loads(request.urlopen(path).read())
+
+
+def _auth_remoteloadjson(user: str, passwd: str) -> None:
+    """Add a Authorization header to all requests.
+
+    Parameters
+    ----------
+    user : str
+        Username.
+    passwd : str
+        Password.
+    """
+    auth = base64.b64encode(f"{user}:{passwd}".encode())
+    opener = request.build_opener()
+    opener.addheaders = [("Authorization", f"Basic {auth.decode()}")]
+    request.install_opener(opener)
+
+
+def _localloadjson(path: str) -> JSONType:
+    """Load a dictionary from a JSON's filepath.
+
+    Parameters
+    ----------
+    path : str
+        JSON's filepath.
+
+    Returns
+    -------
+    JSONType
+        A dictionary with the data loaded.
+    """
+    with open(path, encoding="utf-8") as fh:
+        return json.load(fh)
+
+
+def _has_auth(creds: dict[str, str]) -> bool:
+    """Check if user credentials are provided.
+
+    Parameters
+    ----------
+    creds : dict
+        A dictionary with user credentials. It should contain "user" and
+        "passwd" keys.
+
+    Returns
+    -------
+    bool
+        True if user credentials are provided, False otherwise.
+    """
+    if creds.get("user") in [None, ""] or creds.get("passwd") in [None, ""]:
+        warnings.warn("Credentials were not supplied. Public data access only.", NoAuthWarning)
+        return False
+    return True
+
+
+def _expand_minute(minute: int, periods_duration: list[int]) -> int:
+    """Expand a timestamp with injury time of previous periods.
+
+    Parameters
+    ----------
+    minute : int
+        Timestamp in minutes.
+    periods_duration : List[int]
+        Total duration of each period in minutes.
+
+    Returns
+    -------
+    int
+        Timestamp expanded with injury time.
+    """
+    expanded_minute = minute
+    periods_regular = [45, 45, 15, 15, 0]
+    for period in range(len(periods_duration) - 1):
+        if minute > sum(periods_regular[: period + 1]):
+            expanded_minute += periods_duration[period] - periods_regular[period]
+        else:
+            break
+    return expanded_minute
+
+
+class EventDataLoader(ABC):
+    """Load event data either from a remote location or from a local folder.
+
+    Parameters
+    ----------
+    root : str
+        Root-path of the data.
+    getter : str
+        "remote" or "local"
+    """
+
+    @abstractmethod
+    def competitions(self) -> DataFrame[Any]:
+        """Return a dataframe with all available competitions and seasons.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available competitions and seasons. See
+            :class:`~socceraction.spadl.base.CompetitionSchema` for the schema.
+        """
+
+    @abstractmethod
+    def games(self, competition_id: int, season_id: int) -> DataFrame[Any]:
+        """Return a dataframe with all available games in a season.
+
+        Parameters
+        ----------
+        competition_id : int
+            The ID of the competition.
+        season_id : int
+            The ID of the season.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available games. See
+            :class:`~socceraction.spadl.base.GameSchema` for the schema.
+        """
+
+    @abstractmethod
+    def teams(self, game_id: int) -> DataFrame[Any]:
+        """Return a dataframe with both teams that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing both teams. See
+            :class:`~socceraction.spadl.base.TeamSchema` for the schema.
+        """
+
+    @abstractmethod
+    def players(self, game_id: int) -> DataFrame[Any]:
+        """Return a dataframe with all players that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all players. See
+            :class:`~socceraction.spadl.base.PlayerSchema` for the schema.
+        """
+
+    @abstractmethod
+    def events(self, game_id: int) -> DataFrame[Any]:
+        """Return a dataframe with the event stream of a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing the event stream. See
+            :class:`~socceraction.spadl.base.EventSchema` for the schema.
+        """

data/opta/__init__.py

@@ -0,0 +1,19 @@
+"""Module for loading Opta event data."""
+
+__all__ = [
+    "OptaLoader",
+    "OptaCompetitionSchema",
+    "OptaGameSchema",
+    "OptaPlayerSchema",
+    "OptaTeamSchema",
+    "OptaEventSchema",
+]
+
+from .loader import OptaLoader
+from .schema import (
+    OptaCompetitionSchema,
+    OptaEventSchema,
+    OptaGameSchema,
+    OptaPlayerSchema,
+    OptaTeamSchema,
+)

data/opta/loader.py

@@ -0,0 +1,478 @@
+"""Implements serializers for Opta data."""
+
+import copy
+import datetime
+import glob
+import os
+import re
+import warnings
+from collections.abc import Mapping
+from pathlib import Path
+from typing import Any, Optional, Union, cast
+
+import pandas as pd  # type: ignore
+from pandera.typing import DataFrame
+
+from socceraction.data.base import EventDataLoader
+
+from .parsers import (
+    F1JSONParser,
+    F7XMLParser,
+    F9JSONParser,
+    F24JSONParser,
+    F24XMLParser,
+    MA1JSONParser,
+    MA3JSONParser,
+    OptaParser,
+    WhoScoredParser,
+)
+from .schema import (
+    OptaCompetitionSchema,
+    OptaEventSchema,
+    OptaGameSchema,
+    OptaPlayerSchema,
+    OptaTeamSchema,
+)
+
+_jsonparsers = {
+    "f1": F1JSONParser,
+    "f9": F9JSONParser,
+    "f24": F24JSONParser,
+    "ma1": MA1JSONParser,
+    "ma3": MA3JSONParser,
+}
+
+_xmlparsers = {
+    "f7": F7XMLParser,
+    "f24": F24XMLParser,
+}
+
+_statsperformparsers = {
+    "ma1": MA1JSONParser,
+    "ma3": MA3JSONParser,
+}
+
+_whoscoredparsers = {
+    "whoscored": WhoScoredParser,
+}
+
+_eventtypesdf = pd.DataFrame(
+    [
+        (1, "pass"),
+        (2, "offside pass"),
+        (3, "take on"),
+        (4, "foul"),
+        (5, "out"),
+        (6, "corner awarded"),
+        (7, "tackle"),
+        (8, "interception"),
+        (9, "turnover"),
+        (10, "save"),
+        (11, "claim"),
+        (12, "clearance"),
+        (13, "miss"),
+        (14, "post"),
+        (15, "attempt saved"),
+        (16, "goal"),
+        (17, "card"),
+        (18, "player off"),
+        (19, "player on"),
+        (20, "player retired"),
+        (21, "player returns"),
+        (22, "player becomes goalkeeper"),
+        (23, "goalkeeper becomes player"),
+        (24, "condition change"),
+        (25, "official change"),
+        (26, "unknown26"),
+        (27, "start delay"),
+        (28, "end delay"),
+        (29, "unknown29"),
+        (30, "end"),
+        (31, "unknown31"),
+        (32, "start"),
+        (33, "unknown33"),
+        (34, "team set up"),
+        (35, "player changed position"),
+        (36, "player changed jersey number"),
+        (37, "collection end"),
+        (38, "temp_goal"),
+        (39, "temp_attempt"),
+        (40, "formation change"),
+        (41, "punch"),
+        (42, "good skill"),
+        (43, "deleted event"),
+        (44, "aerial"),
+        (45, "challenge"),
+        (46, "unknown46"),
+        (47, "rescinded card"),
+        (48, "unknown46"),
+        (49, "ball recovery"),
+        (50, "dispossessed"),
+        (51, "error"),
+        (52, "keeper pick-up"),
+        (53, "cross not claimed"),
+        (54, "smother"),
+        (55, "offside provoked"),
+        (56, "shield ball opp"),
+        (57, "foul throw in"),
+        (58, "penalty faced"),
+        (59, "keeper sweeper"),
+        (60, "chance missed"),
+        (61, "ball touch"),
+        (62, "unknown62"),
+        (63, "temp_save"),
+        (64, "resume"),
+        (65, "contentious referee decision"),
+        (66, "possession data"),
+        (67, "50/50"),
+        (68, "referee drop ball"),
+        (69, "failed to block"),
+        (70, "injury time announcement"),
+        (71, "coach setup"),
+        (72, "caught offside"),
+        (73, "other ball contact"),
+        (74, "blocked pass"),
+        (75, "delayed start"),
+        (76, "early end"),
+        (77, "player off pitch"),
+        (78, "temp card"),
+        (79, "coverage interruption"),
+        (80, "drop of ball"),
+        (81, "obstacle"),
+        (83, "attempted tackle"),
+        (84, "deleted after review"),
+        (10000, "offside given"),  # Seems specific to WhoScored
+    ],
+    columns=["type_id", "type_name"],
+)
+
+
+def _deepupdate(target: dict[Any, Any], src: dict[Any, Any]) -> None:
+    """Deep update target dict with src.
+
+    For each k,v in src: if k doesn't exist in target, it is deep copied from
+    src to target. Otherwise, if v is a list, target[k] is extended with
+    src[k]. If v is a set, target[k] is updated with v, If v is a dict,
+    recursively deep-update it.
+
+    Parameters
+    ----------
+    target: dict
+        The original dictionary which is updated.
+    src: dict
+        The dictionary with which `target` is updated.
+
+    Examples
+    --------
+    >>> t = {'name': 'ferry', 'hobbies': ['programming', 'sci-fi']}
+    >>> deepupdate(t, {'hobbies': ['gaming']})
+    >>> print(t)
+    {'name': 'ferry', 'hobbies': ['programming', 'sci-fi', 'gaming']}
+    """
+    for k, v in src.items():
+        if isinstance(v, list):
+            if k not in target:
+                target[k] = copy.deepcopy(v)
+            else:
+                target[k].extend(v)
+        elif isinstance(v, dict):
+            if k not in target:
+                target[k] = copy.deepcopy(v)
+            else:
+                _deepupdate(target[k], v)
+        elif isinstance(v, set):
+            if k not in target:
+                target[k] = v.copy()
+            else:
+                target[k].update(v.copy())
+        else:
+            target[k] = copy.copy(v)
+
+
+def _extract_ids_from_path(path: str, pattern: str) -> dict[str, Union[str, int]]:
+    regex = re.compile(
+        ".+?"
+        + re.escape(pattern)
+        .replace(r"\{competition_id\}", r"(?P<competition_id>[a-zA-Zà-üÀ-Ü0-9-_ ]+)")
+        .replace(r"\{season_id\}", r"(?P<season_id>[a-zA-Zà-üÀ-Ü0-9-_ ]+)")
+        .replace(r"\{game_id\}", r"(?P<game_id>[a-zA-Zà-üÀ-Ü0-9-_ ]+)")
+    )
+    m = re.match(regex, path)
+    if m is None:
+        raise ValueError(f"The filepath {path} does not match the format {pattern}.")
+    ids = m.groupdict()
+    return {k: int(v) if v.isdigit() else v for k, v in ids.items()}
+
+
+class OptaLoader(EventDataLoader):
+    """Load Opta data feeds from a local folder.
+
+    Parameters
+    ----------
+    root : str
+        Root-path of the data.
+    parser : str or dict
+        Either 'xml', 'json', 'statsperform', 'whoscored' or a dict with
+        a custom parser for each feed. The default xml parser supports F7 and
+        F24 feeds; the default json parser supports F1, F9 and F24 feeds, the
+        StatsPerform parser supports MA1 and MA3 feeds. Custom parsers can be
+        specified as::
+
+            {
+                'feed1_name': Feed1Parser
+                'feed2_name': Feed2Parser
+            }
+
+        where Feed1Parser and Feed2Parser are classes implementing
+        :class:`~socceraction.spadl.opta.OptaParser` and 'feed1_name' and
+        'feed2_name' are a unique ID for each feed that matches to the keys in
+        `feeds`.
+    feeds : dict
+        Glob pattern describing from which files the data from a specific game
+        can be retrieved. For example, if files are named::
+
+            f7-1-2021-17362.xml
+            f24-1-2021-17362.xml
+
+        use::
+
+            feeds = {
+                'f7': "f7-{competition_id}-{season_id}-{game_id}.xml",
+                'f24': "f24-{competition_id}-{season_id}-{game_id}.xml"
+            }
+
+    Raises
+    ------
+    ValueError
+        If an invalid parser is provided.
+    """
+
+    def __init__(  # noqa: C901
+        self,
+        root: str,
+        parser: Union[str, Mapping[str, type[OptaParser]]] = "xml",
+        feeds: Optional[dict[str, str]] = None,
+    ) -> None:
+        self.root = root
+        if parser == "json":
+            if feeds is None:
+                feeds = {
+                    "f1": "f1-{competition_id}-{season_id}.json",
+                    "f9": "f9-{competition_id}-{season_id}-{game_id}.json",
+                    "f24": "f24-{competition_id}-{season_id}-{game_id}.json",
+                }
+            self.parsers = self._get_parsers_for_feeds(_jsonparsers, feeds)
+        elif parser == "xml":
+            if feeds is None:
+                feeds = {
+                    "f7": "f7-{competition_id}-{season_id}-{game_id}.xml",
+                    "f24": "f24-{competition_id}-{season_id}-{game_id}.xml",
+                }
+            self.parsers = self._get_parsers_for_feeds(_xmlparsers, feeds)
+        elif parser == "statsperform":
+            if feeds is None:
+                feeds = {
+                    "ma1": "ma1-{competition_id}-{season_id}.json",
+                    "ma3": "ma3-{competition_id}-{season_id}-{game_id}.json",
+                }
+            self.parsers = self._get_parsers_for_feeds(_statsperformparsers, feeds)
+        elif parser == "whoscored":
+            if feeds is None:
+                feeds = {
+                    "whoscored": "{competition_id}-{season_id}-{game_id}.json",
+                }
+            self.parsers = self._get_parsers_for_feeds(_whoscoredparsers, feeds)
+        elif isinstance(parser, dict):
+            if feeds is None:
+                raise ValueError("You must specify a feed for each parser.")
+            self.parsers = self._get_parsers_for_feeds(parser, feeds)
+        else:
+            raise ValueError("Invalid parser provided.")
+        self.feeds = {k: str(Path(v)) for k, v in feeds.items()}
+
+    def _get_parsers_for_feeds(
+        self, available_parsers: Mapping[str, type[OptaParser]], feeds: dict[str, str]
+    ) -> Mapping[str, type[OptaParser]]:
+        """Select the appropriate parser for each feed.
+
+        Parameters
+        ----------
+        available_parsers : dict(str, OptaParser)
+            Dictionary with all available parsers.
+        feeds : dict(str, str)
+            All feeds that should be parsed.
+
+        Returns
+        -------
+        dict(str, OptaParser)
+            A mapping between all feeds that should be parsed and the
+            corresponding parser class.
+
+        Warns
+        -----
+        Raises a warning if there is no parser available for any of the
+        provided feeds.
+        """
+        parsers = {}
+        for feed in feeds:
+            if feed in available_parsers:
+                parsers[feed] = available_parsers[feed]
+            else:
+                warnings.warn(f"No parser available for {feed} feeds. This feed is ignored.")
+        return parsers
+
+    def competitions(self) -> DataFrame[OptaCompetitionSchema]:
+        """Return a dataframe with all available competitions and seasons.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available competitions and seasons. See
+            :class:`~socceraction.spadl.opta.OptaCompetitionSchema` for the schema.
+        """
+        data: dict[int, dict[str, Any]] = {}
+        loaded_seasons = set()
+        for feed, feed_pattern in self.feeds.items():
+            glob_pattern = feed_pattern.format(competition_id="*", season_id="*", game_id="*")
+            feed_files = glob.glob(os.path.join(self.root, glob_pattern))
+            for ffp in feed_files:
+                ids = _extract_ids_from_path(ffp, feed_pattern)
+                # For efficiency, we only parse one game for each season. This
+                # only works if both the competition and season are part of
+                # the file name.
+                competition_id = ids.get("competition_id")
+                season_id = ids.get("season_id")
+                if competition_id is not None and season_id is not None:
+                    if (competition_id, season_id) in loaded_seasons:
+                        continue
+                    else:
+                        loaded_seasons.add((competition_id, season_id))
+                parser = self.parsers[feed](ffp, **ids)
+                _deepupdate(data, parser.extract_competitions())
+        return cast(DataFrame[OptaCompetitionSchema], pd.DataFrame(list(data.values())))
+
+    def games(self, competition_id: int, season_id: int) -> DataFrame[OptaGameSchema]:
+        """Return a dataframe with all available games in a season.
+
+        Parameters
+        ----------
+        competition_id : int
+            The ID of the competition.
+        season_id : int
+            The ID of the season.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available games. See
+            :class:`~socceraction.spadl.opta.OptaGameSchema` for the schema.
+        """
+        data: dict[int, dict[str, Any]] = {}
+        for feed, feed_pattern in self.feeds.items():
+            glob_pattern = feed_pattern.format(
+                competition_id=competition_id, season_id=season_id, game_id="*"
+            )
+            feed_files = glob.glob(os.path.join(self.root, glob_pattern))
+            for ffp in feed_files:
+                ids = _extract_ids_from_path(ffp, feed_pattern)
+                parser = self.parsers[feed](ffp, **ids)
+                _deepupdate(data, parser.extract_games())
+        return cast(DataFrame[OptaGameSchema], pd.DataFrame(list(data.values())))
+
+    def teams(self, game_id: int) -> DataFrame[OptaTeamSchema]:
+        """Return a dataframe with both teams that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing both teams. See
+            :class:`~socceraction.spadl.opta.OptaTeamSchema` for the schema.
+        """
+        data: dict[int, dict[str, Any]] = {}
+        for feed, feed_pattern in self.feeds.items():
+            glob_pattern = feed_pattern.format(competition_id="*", season_id="*", game_id=game_id)
+            feed_files = glob.glob(os.path.join(self.root, glob_pattern))
+            for ffp in feed_files:
+                ids = _extract_ids_from_path(ffp, feed_pattern)
+                parser = self.parsers[feed](ffp, **ids)
+                _deepupdate(data, parser.extract_teams())
+        return cast(DataFrame[OptaTeamSchema], pd.DataFrame(list(data.values())))
+
+    def players(self, game_id: int) -> DataFrame[OptaPlayerSchema]:
+        """Return a dataframe with all players that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all players. See
+            :class:`~socceraction.spadl.opta.OptaPlayerSchema` for the schema.
+        """
+        data: dict[int, dict[str, Any]] = {}
+        for feed, feed_pattern in self.feeds.items():
+            glob_pattern = feed_pattern.format(competition_id="*", season_id="*", game_id=game_id)
+            feed_files = glob.glob(os.path.join(self.root, glob_pattern))
+            for ffp in feed_files:
+                ids = _extract_ids_from_path(ffp, feed_pattern)
+                parser = self.parsers[feed](ffp, **ids)
+                _deepupdate(data, parser.extract_players())
+        df_players = pd.DataFrame(list(data.values()))
+        df_players["game_id"] = game_id
+        return cast(DataFrame[OptaPlayerSchema], df_players)
+
+    def events(self, game_id: int) -> DataFrame[OptaEventSchema]:
+        """Return a dataframe with the event stream of a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing the event stream. See
+            :class:`~socceraction.spadl.opta.OptaEventSchema` for the schema.
+        """
+        data: dict[int, dict[str, Any]] = {}
+        for feed, feed_pattern in self.feeds.items():
+            glob_pattern = feed_pattern.format(competition_id="*", season_id="*", game_id=game_id)
+            feed_files = glob.glob(os.path.join(self.root, glob_pattern))
+            for ffp in feed_files:
+                ids = _extract_ids_from_path(ffp, feed_pattern)
+                parser = self.parsers[feed](ffp, **ids)
+                _deepupdate(data, parser.extract_events())
+        events = (
+            pd.DataFrame(list(data.values()))
+            .merge(_eventtypesdf, on="type_id", how="left")
+            .sort_values(
+                ["game_id", "period_id", "minute", "second", "timestamp"], kind="mergesort"
+            )
+            .reset_index(drop=True)
+        )
+
+        # sometimes pre-match events has -3, -2 and -1 seconds
+        events.loc[events.second < 0, "second"] = 0
+        events = events.sort_values(
+            ["game_id", "period_id", "minute", "second", "timestamp"], kind="mergesort"
+        )
+
+        # deleted events has wrong datetime which occurs OutOfBoundsDatetime error
+        events = events[events.type_id != 43]
+        events = events[
+            ~(
+                (events.timestamp < datetime.datetime(1900, 1, 1))
+                | (events.timestamp > datetime.datetime(2100, 1, 1))
+            )
+        ]
+
+        return cast(DataFrame[OptaEventSchema], events)

data/opta/parsers/__init__.py

@@ -0,0 +1,23 @@
+"""Parsers for Opta(-derived) event streams."""
+
+__all__ = [
+    "OptaParser",
+    "F1JSONParser",
+    "F9JSONParser",
+    "F24JSONParser",
+    "F7XMLParser",
+    "F24XMLParser",
+    "MA1JSONParser",
+    "MA3JSONParser",
+    "WhoScoredParser",
+]
+
+from .base import OptaParser
+from .f1_json import F1JSONParser
+from .f7_xml import F7XMLParser
+from .f9_json import F9JSONParser
+from .f24_json import F24JSONParser
+from .f24_xml import F24XMLParser
+from .ma1_json import MA1JSONParser
+from .ma3_json import MA3JSONParser
+from .whoscored import WhoScoredParser

data/opta/parsers/base.py

@@ -0,0 +1,179 @@
+"""Base class for all Opta(-derived) event stream parsers.
+
+A parser reads a single data file and should extend the 'OptaParser' class to
+extract data about competitions, games, players, teams and events that is
+encoded in the file.
+
+"""
+
+import json  # type: ignore
+from typing import Any, Optional
+
+from lxml import objectify
+
+
+class OptaParser:
+    """Extract data from an Opta data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    def __init__(self, path: str, **kwargs: Any) -> None:  # noqa: ANN401
+        raise NotImplementedError
+
+    def extract_competitions(self) -> dict[tuple[Any, Any], dict[str, Any]]:
+        """Return a dictionary with all available competitions.
+
+        Returns
+        -------
+        dict
+            A mapping between (competion ID, season ID) tuples and the
+            information available about each competition in the data stream.
+        """
+        return {}
+
+    def extract_games(self) -> dict[Any, dict[str, Any]]:
+        """Return a dictionary with all available games.
+
+        Returns
+        -------
+        dict
+            A mapping between game IDs and the information available about
+            each game in the data stream.
+        """
+        return {}
+
+    def extract_teams(self) -> dict[Any, dict[str, Any]]:
+        """Return a dictionary with all available teams.
+
+        Returns
+        -------
+        dict
+            A mapping between team IDs and the information available about
+            each team in the data stream.
+        """
+        return {}
+
+    def extract_players(self) -> dict[tuple[Any, Any], dict[str, Any]]:
+        """Return a dictionary with all available players.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, player ID) tuples and the information
+            available about each player in the data stream.
+        """
+        return {}
+
+    def extract_lineups(self) -> dict[Any, dict[str, Any]]:
+        """Return a dictionary with the lineup of each team.
+
+        Returns
+        -------
+        dict
+            A mapping between team IDs and the information available about
+            each team's lineup in the data stream.
+        """
+        return {}
+
+    def extract_events(self) -> dict[tuple[Any, Any], dict[str, Any]]:
+        """Return a dictionary with all available events.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, event ID) tuples and the information
+            available about each event in the data stream.
+        """
+        return {}
+
+
+class OptaJSONParser(OptaParser):
+    """Extract data from an Opta JSON data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    def __init__(self, path: str, **kwargs: Any) -> None:  # noqa: ANN401
+        with open(path, encoding="utf-8") as fh:
+            self.root = json.load(fh)
+
+
+class OptaXMLParser(OptaParser):
+    """Extract data from an Opta XML data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    def __init__(self, path: str, **kwargs: Any) -> None:  # noqa: ANN401
+        with open(path, "rb") as fh:
+            self.root = objectify.fromstring(fh.read())
+
+
+def assertget(dictionary: dict[str, Any], key: str) -> Any:  # noqa: ANN401
+    """Return the value of the item with the specified key.
+
+    In contrast to the default `get` method, this version will raise an
+    assertion error if the given key is not present in the dict.
+
+    Parameters
+    ----------
+    dictionary : dict
+        A Python dictionary.
+    key : str
+        A key in the dictionary.
+
+    Returns
+    -------
+    Any
+        Returns the value for the specified key if the key is in the dictionary.
+
+    Raises
+    ------
+    AssertionError
+        If the given key could not be found in the dictionary.
+    """
+    value = dictionary.get(key)
+    assert value is not None, "KeyError: " + key + " not found in " + str(dictionary)
+    return value
+
+
+def _get_end_x(qualifiers: dict[int, Any]) -> Optional[float]:
+    try:
+        # pass
+        if 140 in qualifiers:
+            return float(qualifiers[140])
+        # blocked shot
+        if 146 in qualifiers:
+            return float(qualifiers[146])
+        # passed the goal line
+        if 102 in qualifiers:
+            return float(100)
+        return None
+    except ValueError:
+        return None
+
+
+def _get_end_y(qualifiers: dict[int, Any]) -> Optional[float]:
+    try:
+        # pass
+        if 141 in qualifiers:
+            return float(qualifiers[141])
+        # blocked shot
+        if 147 in qualifiers:
+            return float(qualifiers[147])
+        # passed the goal line
+        if 102 in qualifiers:
+            return float(qualifiers[102])
+        return None
+    except ValueError:
+        return None

data/opta/parsers/f1_json.py

@@ -0,0 +1,103 @@
+"""JSON parser for Opta F1 feeds."""
+
+from datetime import datetime
+from typing import Any
+
+from ...base import MissingDataError
+from .base import OptaJSONParser, assertget
+
+
+class F1JSONParser(OptaJSONParser):
+    """Extract data from a Opta F1 data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    def _get_feed(self) -> dict[str, Any]:
+        for node in self.root:
+            if "OptaFeed" in node["data"].keys():
+                return node
+        raise MissingDataError
+
+    def _get_doc(self) -> dict[str, Any]:
+        f1 = self._get_feed()
+        data = assertget(f1, "data")
+        optafeed = assertget(data, "OptaFeed")
+        optadocument = assertget(optafeed, "OptaDocument")
+        return optadocument
+
+    def extract_competitions(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all available competitions.
+
+        Returns
+        -------
+        dict
+            A mapping between (competion ID, season ID) tuples and the
+            information available about each competition in the data stream.
+        """
+        optadocument = self._get_doc()
+        attr = assertget(optadocument, "@attributes")
+        competition_id = int(assertget(attr, "competition_id"))
+        season_id = int(assertget(attr, "season_id"))
+        competition = {
+            # Fields required by the base schema
+            "season_id": season_id,
+            "season_name": str(assertget(attr, "season_id")),
+            "competition_id": competition_id,
+            "competition_name": assertget(attr, "competition_name"),
+        }
+        return {(competition_id, season_id): competition}
+
+    def extract_games(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all available games.
+
+        Returns
+        -------
+        dict
+            A mapping between game IDs and the information available about
+            each game in the data stream.
+        """
+        optadocument = self._get_doc()
+        attr = assertget(optadocument, "@attributes")
+        matchdata = assertget(optadocument, "MatchData")
+        matches = {}
+        for match in matchdata:
+            matchattr = assertget(match, "@attributes")
+            matchinfo = assertget(match, "MatchInfo")
+            matchinfoattr = assertget(matchinfo, "@attributes")
+            game_id = int(assertget(matchattr, "uID")[1:])
+            matches[game_id] = {
+                # Fields required by the base schema
+                "game_id": game_id,
+                "competition_id": int(assertget(attr, "competition_id")),
+                "season_id": int(assertget(attr, "season_id")),
+                "game_day": int(assertget(matchinfoattr, "MatchDay")),
+                "game_date": datetime.strptime(assertget(matchinfo, "Date"), "%Y-%m-%d %H:%M:%S"),
+                # home_team_id=see below,
+                # away_team_id=see below,
+                # Optional fields
+                # home_score=see below,
+                # away_score=see below,
+                # duration=?
+                # referee=?
+                # venue=?,
+                # attendance=?
+                # home_manager=?
+                # away_manager=?
+            }
+            teamdata = assertget(match, "TeamData")
+            for team in teamdata:
+                teamattr = assertget(team, "@attributes")
+                side = assertget(teamattr, "Side")
+                teamid = assertget(teamattr, "TeamRef")
+                score = assertget(teamattr, "Score")
+                if side == "Home":
+                    matches[game_id]["home_team_id"] = int(teamid[1:])
+                    matches[game_id]["home_score"] = int(score)
+                else:
+                    matches[game_id]["away_team_id"] = int(teamid[1:])
+                    matches[game_id]["away_score"] = int(score)
+        return matches

data/opta/parsers/f24_json.py

@@ -0,0 +1,123 @@
+"""JSON parser for Opta F24 feeds."""
+
+from datetime import datetime
+from typing import Any
+
+from ...base import MissingDataError
+from .base import OptaJSONParser, _get_end_x, _get_end_y, assertget
+
+
+class F24JSONParser(OptaJSONParser):
+    """Extract data from a Opta F24 data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    def _get_doc(self) -> dict[str, Any]:
+        for node in self.root:
+            if "Games" in node["data"].keys():
+                return node
+        raise MissingDataError
+
+    def extract_games(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all available games.
+
+        Returns
+        -------
+        dict
+            A mapping between game IDs and the information available about
+            each game in the data stream.
+        """
+        f24 = self._get_doc()
+
+        data = assertget(f24, "data")
+        games = assertget(data, "Games")
+        game = assertget(games, "Game")
+        attr = assertget(game, "@attributes")
+
+        game_id = int(assertget(attr, "id"))
+        game_dict = {
+            game_id: {
+                # Fields required by the base schema
+                "game_id": game_id,
+                "season_id": int(assertget(attr, "season_id")),
+                "competition_id": int(assertget(attr, "competition_id")),
+                "game_day": int(assertget(attr, "matchday")),
+                "game_date": datetime.strptime(
+                    assertget(assertget(attr, "game_date"), "locale"), "%Y-%m-%dT%H:%M:%S.%fZ"
+                ).replace(tzinfo=None),
+                "home_team_id": int(assertget(attr, "home_team_id")),
+                "away_team_id": int(assertget(attr, "away_team_id")),
+                # Fields required by the opta schema
+                # home_score=?
+                # away_score=?
+                # duration=?
+                # referee=?
+                # venue=?,
+                # attendance=?
+                # Optional fields
+                # home_manager=?
+                # away_manager=?
+            }
+        }
+        return game_dict
+
+    def extract_events(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all available events.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, event ID) tuples and the information
+            available about each event in the data stream.
+        """
+        f24 = self._get_doc()
+
+        data = assertget(f24, "data")
+        games = assertget(data, "Games")
+        game = assertget(games, "Game")
+        game_attr = assertget(game, "@attributes")
+        game_id = int(assertget(game_attr, "id"))
+
+        events = {}
+        for element in assertget(game, "Event"):
+            attr = element["@attributes"]
+            timestamp = attr["TimeStamp"].get("locale") if attr.get("TimeStamp") else None
+            timestamp = datetime.strptime(timestamp, "%Y-%m-%dT%H:%M:%S.%fZ")
+            qualifiers = {
+                int(q["@attributes"]["qualifier_id"]): q["@attributes"]["value"]
+                for q in element.get("Q", [])
+            }
+            start_x = float(assertget(attr, "x"))
+            start_y = float(assertget(attr, "y"))
+            end_x = _get_end_x(qualifiers)
+            end_y = _get_end_y(qualifiers)
+
+            event_id = int(assertget(attr, "id"))
+            events[(game_id, event_id)] = {
+                # Fields required by the base schema
+                "game_id": game_id,
+                "event_id": event_id,
+                "period_id": int(assertget(attr, "period_id")),
+                "team_id": int(assertget(attr, "team_id")),
+                "player_id": int(assertget(attr, "player_id")),
+                "type_id": int(assertget(attr, "type_id")),
+                # type_name=?, # added in the opta loader
+                # Fields required by the opta schema
+                "timestamp": timestamp,
+                "minute": int(assertget(attr, "min")),
+                "second": int(assertget(attr, "sec")),
+                "outcome": bool(int(attr.get("outcome", 1))),
+                "start_x": start_x,
+                "start_y": start_y,
+                "end_x": end_x if end_x is not None else start_x,
+                "end_y": end_y if end_y is not None else start_y,
+                "qualifiers": qualifiers,
+                # Optional fields
+                "assist": bool(int(attr.get("assist", 0))),
+                "keypass": bool(int(attr.get("keypass", 0))),
+            }
+        return events

data/opta/parsers/f24_xml.py

@@ -0,0 +1,108 @@
+"""XML parser for Opta F24 feeds."""
+
+from datetime import datetime
+from typing import Any
+
+from lxml import objectify
+
+from .base import OptaXMLParser, _get_end_x, _get_end_y, assertget
+
+
+class F24XMLParser(OptaXMLParser):
+    """Extract data from a Opta F24 data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    def _get_doc(self) -> objectify.ObjectifiedElement:
+        return self.root
+
+    def extract_games(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all available games.
+
+        Returns
+        -------
+        dict
+            A mapping between game IDs and the information available about
+            each game in the data stream.
+        """
+        optadocument = self._get_doc()
+        game_elem = optadocument.find("Game")
+        attr = game_elem.attrib
+        game_id = int(assertget(attr, "id"))
+        game_dict = {
+            # Fields required by the base schema
+            "game_id": game_id,
+            "season_id": int(assertget(attr, "season_id")),
+            "competition_id": int(assertget(attr, "competition_id")),
+            "game_day": int(assertget(attr, "matchday")),
+            "game_date": datetime.strptime(assertget(attr, "game_date"), "%Y-%m-%dT%H:%M:%S"),
+            "home_team_id": int(assertget(attr, "home_team_id")),
+            "away_team_id": int(assertget(attr, "away_team_id")),
+            # Optional fields
+            "home_score": int(assertget(attr, "home_score")),
+            "away_score": int(assertget(attr, "away_score")),
+            # duration=?
+            # referee=?
+            # venue=?
+            # attendance=?
+            # home_manager=?
+            # away_manager=?
+        }
+        return {game_id: game_dict}
+
+    def extract_events(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all available events.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, event ID) tuples and the information
+            available about each event in the data stream.
+        """
+        optadocument = self._get_doc()
+        game_elm = optadocument.find("Game")
+        game_id = int(assertget(game_elm.attrib, "id"))
+        events = {}
+        for event_elm in game_elm.iterchildren("Event"):
+            attr = dict(event_elm.attrib)
+            event_id = int(assertget(attr, "id"))
+
+            qualifiers = {
+                int(qualifier_elm.attrib["qualifier_id"]): qualifier_elm.attrib.get("value")
+                for qualifier_elm in event_elm.iterchildren("Q")
+            }
+            start_x = float(assertget(attr, "x"))
+            start_y = float(assertget(attr, "y"))
+            end_x = _get_end_x(qualifiers)
+            end_y = _get_end_y(qualifiers)
+
+            events[(game_id, event_id)] = {
+                # Fields required by the base schema
+                "game_id": game_id,
+                "event_id": event_id,
+                "period_id": int(assertget(attr, "period_id")),
+                "team_id": int(assertget(attr, "team_id")),
+                "player_id": int(attr["player_id"]) if "player_id" in attr else None,
+                "type_id": int(assertget(attr, "type_id")),
+                # type_name=?, # added in the opta loader
+                # Fields required by the opta schema
+                "timestamp": datetime.strptime(
+                    assertget(attr, "timestamp"), "%Y-%m-%dT%H:%M:%S.%f"
+                ),
+                "minute": int(assertget(attr, "min")),
+                "second": int(assertget(attr, "sec")),
+                "outcome": bool(int(attr["outcome"])) if "outcome" in attr else None,
+                "start_x": start_x,
+                "start_y": start_y,
+                "end_x": end_x if end_x is not None else start_x,
+                "end_y": end_y if end_y is not None else start_y,
+                "qualifiers": qualifiers,
+                # Optional fields
+                "assist": bool(int(attr.get("assist", 0))),
+                "keypass": bool(int(attr.get("keypass", 0))),
+            }
+        return events

data/opta/parsers/f7_xml.py

@@ -0,0 +1,250 @@
+"""XML parser for Opta F7 feeds."""
+
+from datetime import datetime
+from typing import Any
+
+from lxml import objectify
+
+from .base import OptaXMLParser, assertget
+
+
+class F7XMLParser(OptaXMLParser):
+    """Extract data from a Opta F7 data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    def _get_doc(self) -> objectify.ObjectifiedElement:
+        optadocument = self.root.find("SoccerDocument")
+        return optadocument
+
+    def _get_stats(self, obj: objectify.ObjectifiedElement) -> dict[str, Any]:
+        stats = {}
+        for stat in obj.find("Stat"):
+            stats[stat.attrib["Type"]] = stat.text
+        return stats
+
+    def _get_name(self, obj: objectify.ObjectifiedElement) -> str:
+        if "Known" in obj:
+            return obj.Known
+        return obj.First + " " + obj.Last
+
+    def extract_competitions(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all available competitions.
+
+        Returns
+        -------
+        dict
+            A mapping between (competion ID, season ID) tuples and the
+            information available about each competition in the data stream.
+        """
+        optadocument = self._get_doc()
+        competition = optadocument.Competition
+        competition_id = int(competition.attrib["uID"][1:])
+        stats = self._get_stats(competition)
+        season_id = int(assertget(stats, "season_id"))
+        competition_dict = {
+            # Fields required by the base schema
+            "competition_id": competition_id,
+            "season_id": season_id,
+            "season_name": assertget(stats, "season_name"),
+            "competition_name": competition.Name.text,
+        }
+        return {(competition_id, season_id): competition_dict}
+
+    def extract_games(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all available games.
+
+        Returns
+        -------
+        dict
+            A mapping between game IDs and the information available about
+            each game in the data stream.
+        """
+        optadocument = self._get_doc()
+        competition = optadocument.Competition
+        competition_id = int(competition.attrib["uID"][1:])
+        competition_stats = self._get_stats(competition)
+        match_info = optadocument.MatchData.MatchInfo
+        game_id = int(optadocument.attrib["uID"][1:])
+        stats = self._get_stats(optadocument.MatchData)
+        team_data_elms = {
+            t.attrib["Side"]: t for t in optadocument.MatchData.iterchildren("TeamData")
+        }
+        team_officials = {}
+        for t in optadocument.iterchildren("Team"):
+            side = (
+                "Home"
+                if int(team_data_elms["Home"].attrib["TeamRef"][1:]) == int(t.attrib["uID"][1:])
+                else "Away"
+            )
+            for m in t.iterchildren("TeamOfficial"):
+                if m.attrib["Type"] == "Manager":
+                    team_officials[side] = m
+
+        game_dict = {
+            # Fields required by the base schema
+            "game_id": game_id,
+            "season_id": int(assertget(competition_stats, "season_id")),
+            "competition_id": competition_id,
+            "game_day": int(competition_stats["matchday"])
+            if "matchday" in competition_stats
+            else None,
+            "game_date": datetime.strptime(match_info.Date.text, "%Y%m%dT%H%M%S%z").replace(
+                tzinfo=None
+            ),
+            "home_team_id": int(
+                assertget(assertget(team_data_elms, "Home").attrib, "TeamRef")[1:]
+            ),
+            "away_team_id": int(
+                assertget(assertget(team_data_elms, "Away").attrib, "TeamRef")[1:]
+            ),
+            # Optional fields
+            "home_score": int(assertget(assertget(team_data_elms, "Home").attrib, "Score")),
+            "away_score": int(assertget(assertget(team_data_elms, "Away").attrib, "Score")),
+            "duration": int(stats["match_time"]),
+            "referee": self._get_name(optadocument.MatchData.MatchOfficial.OfficialName),
+            "venue": optadocument.Venue.Name.text,
+            "attendance": int(match_info.Attendance),
+            "home_manager": self._get_name(team_officials["Home"].PersonName)
+            if "Home" in team_officials
+            else None,
+            "away_manager": self._get_name(team_officials["Away"].PersonName)
+            if "Away" in team_officials
+            else None,
+        }
+        return {game_id: game_dict}
+
+    def extract_teams(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all available teams.
+
+        Returns
+        -------
+        dict
+            A mapping between team IDs and the information available about
+            each team in the data stream.
+        """
+        optadocument = self._get_doc()
+        team_elms = list(optadocument.iterchildren("Team"))
+        teams = {}
+        for team_elm in team_elms:
+            team_id = int(assertget(team_elm.attrib, "uID")[1:])
+            teams[team_id] = {
+                # Fields required by the base schema
+                "team_id": team_id,
+                "team_name": team_elm.Name.text,
+            }
+        return teams
+
+    def extract_lineups(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with the lineup of each team.
+
+        Returns
+        -------
+        dict
+            A mapping between team IDs and the information available about
+            each team's lineup in the data stream.
+        """
+        optadocument = self._get_doc()
+
+        stats = {}
+        for stat in optadocument.MatchData.find("Stat"):
+            stats[stat.attrib["Type"]] = stat.text
+
+        lineup_elms = optadocument.MatchData.iterchildren("TeamData")
+        lineups = {}
+        for team_elm in lineup_elms:
+            # lineup attributes
+            team_id = int(team_elm.attrib["TeamRef"][1:])
+            lineups[team_id] = {
+                "formation": team_elm.attrib["Formation"],
+                "score": int(team_elm.attrib["Score"]),
+                "side": team_elm.attrib["Side"],
+                "players": {},
+            }
+            # substitutes
+            subst_elms = team_elm.iterchildren("Substitution")
+            subst = [subst_elm.attrib for subst_elm in subst_elms]
+            # red_cards
+            booking_elms = team_elm.iterchildren("Booking")
+            red_cards = {
+                int(booking_elm.attrib["PlayerRef"][1:]): int(booking_elm.attrib["Min"])
+                for booking_elm in booking_elms
+                if "CardType" in booking_elm.attrib
+                and booking_elm.attrib["CardType"] in ["Red", "SecondYellow"]
+                and "PlayerRef" in booking_elm.attrib  # not defined if a coach receives a red card
+            }
+            # players
+            player_elms = team_elm.PlayerLineUp.iterchildren("MatchPlayer")
+            for player_elm in player_elms:
+                player_id = int(player_elm.attrib["PlayerRef"][1:])
+                sub_on = int(
+                    next(
+                        (
+                            item["Time"]
+                            for item in subst
+                            if "Retired" not in item and item["SubOn"] == f"p{player_id}"
+                        ),
+                        stats["match_time"] if player_elm.attrib["Status"] == "Sub" else 0,
+                    )
+                )
+                sub_off = int(
+                    next(
+                        (item["Time"] for item in subst if item["SubOff"] == f"p{player_id}"),
+                        stats["match_time"]
+                        if player_id not in red_cards
+                        else red_cards[player_id],
+                    )
+                )
+                minutes_played = sub_off - sub_on
+                lineups[team_id]["players"][player_id] = {
+                    "starting_position_id": int(player_elm.attrib["Formation_Place"]),
+                    "starting_position_name": player_elm.attrib["Position"],
+                    "jersey_number": int(player_elm.attrib["ShirtNumber"]),
+                    "is_starter": int(player_elm.attrib["Formation_Place"]) != 0,
+                    "minutes_played": minutes_played,
+                }
+        return lineups
+
+    def extract_players(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all available players.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, player ID) tuples and the information
+            available about each player in the data stream.
+        """
+        optadocument = self._get_doc()
+        game_id = int(optadocument.attrib["uID"][1:])
+        lineups = self.extract_lineups()
+        team_elms = list(optadocument.iterchildren("Team"))
+        players = {}
+        for team_elm in team_elms:
+            team_id = int(team_elm.attrib["uID"][1:])
+            for player_elm in team_elm.iterchildren("Player"):
+                player_id = int(player_elm.attrib["uID"][1:])
+                player = {
+                    # Fields required by the base schema
+                    "game_id": game_id,
+                    "team_id": team_id,
+                    "player_id": player_id,
+                    "player_name": self._get_name(player_elm.PersonName),
+                    "is_starter": lineups[team_id]["players"][player_id]["is_starter"],
+                    "minutes_played": lineups[team_id]["players"][player_id]["minutes_played"],
+                    "jersey_number": lineups[team_id]["players"][player_id]["jersey_number"],
+                    # Fields required by the opta schema
+                    "starting_position": lineups[team_id]["players"][player_id][
+                        "starting_position_name"
+                    ],
+                    # Optional fields
+                    # height="?",
+                    # weight="?",
+                    # age="?",
+                }
+                players[(game_id, player_id)] = player
+
+        return players

data/opta/parsers/f9_json.py

@@ -0,0 +1,302 @@
+"""JSON parser for Opta F9 feeds."""
+
+from datetime import datetime
+from typing import Any, Optional
+
+from ...base import MissingDataError
+from .base import OptaJSONParser, assertget
+
+
+class F9JSONParser(OptaJSONParser):
+    """Extract data from a Opta F9 data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    def _get_feed(self) -> dict[str, Any]:
+        for node in self.root:
+            if "OptaFeed" in node["data"].keys():
+                return node
+        raise MissingDataError
+
+    def _get_doc(self) -> dict[str, Any]:
+        f9 = self._get_feed()
+        data = assertget(f9, "data")
+        optafeed = assertget(data, "OptaFeed")
+        optadocument = assertget(optafeed, "OptaDocument")[0]
+        return optadocument
+
+    def _get_stats(self, obj: dict[str, Any]) -> dict[str, Any]:
+        if "Stat" not in obj:
+            return {}
+
+        stats = {}
+        statobj = obj["Stat"] if isinstance(obj["Stat"], list) else [obj["Stat"]]
+        for stat in statobj:
+            stats[stat["@attributes"]["Type"]] = stat["@value"]
+        return stats
+
+    def _get_name(self, obj: dict[str, Any]) -> Optional[str]:
+        if "Known" in obj and obj["Known"].strip():
+            return obj["Known"]
+        if "First" in obj and "Last" in obj and obj["Last"].strip() or obj["First"].strip():
+            return (obj["First"] + " " + obj["Last"]).strip()
+        return None
+
+    def extract_games(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all available games.
+
+        Returns
+        -------
+        dict
+            A mapping between game IDs and the information available about
+            each game in the data stream.
+        """
+        optadocument = self._get_doc()
+        attr = assertget(optadocument, "@attributes")
+        matchdata = assertget(optadocument, "MatchData")
+        competition = assertget(optadocument, "Competition")
+        competitionstat = self._get_stats(competition)
+        venue = assertget(optadocument, "Venue")
+        matchofficial = assertget(matchdata, "MatchOfficial")
+        matchinfo = assertget(matchdata, "MatchInfo")
+        matchstat = self._get_stats(matchdata)
+        teamdata = assertget(matchdata, "TeamData")
+        scores = {}
+        for t in teamdata:
+            scores[t["@attributes"]["Side"]] = t["@attributes"]["Score"]
+
+        game_id = int(assertget(attr, "uID")[1:])
+        game_dict = {
+            # Fields required by the base schema
+            "game_id": game_id,
+            "competition_id": int(assertget(assertget(competition, "@attributes"), "uID")[1:]),
+            "season_id": assertget(competitionstat, "season_id"),
+            "game_day": competitionstat["matchday"] if "matchday" in competitionstat else None,
+            "game_date": datetime.strptime(
+                assertget(matchinfo, "Date"), "%Y%m%dT%H%M%S%z"
+            ).replace(tzinfo=None),
+            # home_team_id=see below
+            # away_team_id=see below
+            # Optional fields
+            "home_score": int(scores["Home"]),
+            "away_score": int(scores["Away"]),
+            "duration": int(assertget(matchstat, "match_time")),
+            "referee": self._get_name(matchofficial["OfficialName"])
+            if "OfficialName" in matchofficial
+            else None,
+            "venue": venue["Name"] if "Name" in venue else None,
+            "attendance": int(matchinfo["Attendance"]) if "Attendance" in matchinfo else None,
+            # home_manager=see below
+            # away_manager=see below
+        }
+        for team in teamdata:
+            teamattr = assertget(team, "@attributes")
+            side = assertget(teamattr, "Side")
+            teamid = assertget(teamattr, "TeamRef")
+            score = assertget(teamattr, "Score")
+            manager = (
+                self._get_name(team["TeamOfficial"]["PersonName"])
+                if "TeamOfficial" in team
+                else None
+            )
+            if side == "Home":
+                game_dict["home_team_id"] = int(teamid[1:])
+                game_dict["home_score"] = int(score)
+                game_dict["home_manager"] = manager
+            else:
+                game_dict["away_team_id"] = int(teamid[1:])
+                game_dict["away_score"] = int(score)
+                game_dict["away_manager"] = manager
+        return {game_id: game_dict}
+
+    def extract_teams(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all available teams.
+
+        Returns
+        -------
+        dict
+            A mapping between team IDs and the information available about
+            each team in the data stream.
+        """
+        optadocument = self._get_doc()
+        root_teams = assertget(optadocument, "Team")
+
+        teams = {}
+        for team in root_teams:
+            if "id" in team.keys():
+                nameobj = team.get("nameObj")
+                team_id = int(team["id"])
+                teams[team_id] = {
+                    # Fields required by the base schema
+                    "team_id": team_id,
+                    "team_name": nameobj.get("name"),
+                }
+        return teams
+
+    def extract_players(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all available players.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, player ID) tuples and the information
+            available about each player in the data stream.
+        """
+        optadocument = self._get_doc()
+        attr = assertget(optadocument, "@attributes")
+        game_id = int(assertget(attr, "uID")[1:])
+        root_teams = assertget(optadocument, "Team")
+        lineups = self.extract_lineups()
+
+        players = {}
+        for team in root_teams:
+            team_id = int(team["@attributes"]["uID"].replace("t", ""))
+            for player in team["Player"]:
+                player_id = int(player["@attributes"]["uID"].replace("p", ""))
+
+                assert "nameObj" in player["PersonName"]
+                nameobj = player["PersonName"]["nameObj"]
+                if not nameobj.get("is_unknown"):
+                    player = {
+                        # Fields required by the base schema
+                        "game_id": game_id,
+                        "team_id": team_id,
+                        "player_id": player_id,
+                        "player_name": self._get_name(player["PersonName"]),
+                        # is_starter=
+                        # minutes_played=
+                        # jersey_number=
+                        # Fields required by the opta schema
+                        # starting_position=
+                        # Optional fields
+                        # height="?",
+                        # weight="?",
+                        # age="?",
+                    }
+                    if player_id in lineups[team_id]["players"]:
+                        player = dict(
+                            **player,
+                            jersey_number=lineups[team_id]["players"][player_id]["jersey_number"],
+                            starting_position=lineups[team_id]["players"][player_id][
+                                "starting_position_name"
+                            ],
+                            is_starter=lineups[team_id]["players"][player_id]["is_starter"],
+                            minutes_played=lineups[team_id]["players"][player_id][
+                                "minutes_played"
+                            ],
+                        )
+                    players[(game_id, player_id)] = player
+        return players
+
+    def extract_lineups(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with the lineup of each team.
+
+        Raises
+        ------
+        MissingDataError
+            If teams data is not available in the stream.
+
+        Returns
+        -------
+        dict
+            A mapping between team IDs and the information available about
+            each team's lineup in the data stream.
+        """
+        optadocument = self._get_doc()
+        attr = assertget(optadocument, "@attributes")
+
+        try:
+            rootf9 = optadocument["MatchData"]["TeamData"]
+        except KeyError as e:
+            raise MissingDataError from e
+        matchstats = optadocument["MatchData"]["Stat"]
+        matchstats = [matchstats] if isinstance(matchstats, dict) else matchstats
+        matchstatsdict = {stat["@attributes"]["Type"]: stat["@value"] for stat in matchstats}
+
+        lineups: dict[int, dict[str, Any]] = {}
+        for team in rootf9:
+            # lineup attributes
+            team_id = int(team["@attributes"]["TeamRef"].replace("t", ""))
+            lineups[team_id] = {"players": {}}
+            # substitutes
+            subst = [s["@attributes"] for s in team["Substitution"]]
+            # red cards
+            red_cards = {
+                int(e["@attributes"]["PlayerRef"].replace("p", "")): e["@attributes"]["Time"]
+                for e in team.get("Booking", [])
+                if "CardType" in e["@attributes"]
+                and e["@attributes"]["CardType"] in ["Red", "SecondYellow"]
+                and "PlayerRef" in e["@attributes"]  # not defined if a coach receives a red card
+            }
+            for player in team["PlayerLineUp"]["MatchPlayer"]:
+                attr = player["@attributes"]
+                player_id = int(attr["PlayerRef"].replace("p", ""))
+                playerstatsdict = {
+                    stat["@attributes"]["Type"]: stat["@value"] for stat in player["Stat"]
+                }
+                sub_on = next(
+                    (
+                        item["Time"]
+                        for item in subst
+                        if "Retired" not in item and item["SubOn"] == f"p{player_id}"
+                    ),
+                    matchstatsdict["match_time"] if attr["Status"] == "Sub" else 0,
+                )
+                sub_off = next(
+                    (item["Time"] for item in subst if item["SubOff"] == f"p{player_id}"),
+                    matchstatsdict["match_time"]
+                    if player_id not in red_cards
+                    else red_cards[player_id],
+                )
+                minutes_played = sub_off - sub_on
+                lineups[team_id]["players"][player_id] = dict(
+                    jersey_number=attr["ShirtNumber"],
+                    starting_position_name=attr["Position"],
+                    starting_position_id=attr["position_id"],
+                    is_starter=attr["Status"] == "Start",
+                    minutes_played=minutes_played,
+                    **playerstatsdict,
+                )
+        return lineups
+
+    def extract_teamgamestats(self) -> list[dict[str, Any]]:
+        """Return some aggregated statistics of each team.
+
+        Raises
+        ------
+        MissingDataError
+            If teams data is not available in the stream.
+
+        Returns
+        -------
+        list(dict)
+            A dictionary with aggregated team statistics for each team.
+        """
+        optadocument = self._get_doc()
+        attr = assertget(optadocument, "@attributes")
+        game_id = int(assertget(attr, "uID")[1:])
+
+        try:
+            rootf9 = optadocument["MatchData"]["TeamData"]
+        except KeyError as e:
+            raise MissingDataError from e
+        teams_gamestats = []
+        for team in rootf9:
+            attr = team["@attributes"]
+            statsdict = self._get_stats(team)
+
+            team_gamestats = dict(
+                game_id=game_id,
+                team_id=int(attr["TeamRef"].replace("t", "")),
+                side=attr["Side"],
+                score=attr["Score"],
+                shootout_score=attr["ShootOutScore"],
+                **statsdict,
+            )
+
+            teams_gamestats.append(team_gamestats)
+        return teams_gamestats

data/opta/parsers/ma1_json.py

@@ -0,0 +1,264 @@
+"""JSON parser for Stats Perform MA1 feeds."""
+
+from datetime import datetime
+from typing import Any, Optional
+
+from ...base import MissingDataError
+from .base import OptaJSONParser, assertget
+
+
+class MA1JSONParser(OptaJSONParser):
+    """Extract data from a Stats Perform MA1 data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    def _get_matches(self) -> list[dict[str, Any]]:
+        if "matchInfo" in self.root:
+            return [self.root]
+        if "match" in self.root:
+            return self.root["match"]
+        raise MissingDataError
+
+    def _get_match_info(self, match: dict[str, Any]) -> dict[str, Any]:
+        if "matchInfo" in match:
+            return match["matchInfo"]
+        raise MissingDataError
+
+    def _get_live_data(self, match: dict[str, Any]) -> dict[str, Any]:
+        if "liveData" in match:
+            return match["liveData"]
+        return {}
+
+    def _get_name(self, obj: dict[str, Any]) -> Optional[str]:
+        if "name" in obj:
+            return assertget(obj, "name")
+        if "firstName" in obj:
+            return f"{assertget(obj, 'firstName')} {assertget(obj, 'lastName')}"
+        return None
+
+    @staticmethod
+    def _extract_team_id(teams: list[dict[str, str]], side: str) -> Optional[str]:
+        for team in teams:
+            team_side = assertget(team, "position")
+            if team_side == side:
+                team_id = assertget(team, "id")
+                return team_id
+        raise MissingDataError
+
+    def extract_competitions(self) -> dict[tuple[str, str], dict[str, Any]]:
+        """Return a dictionary with all available competitions.
+
+        Returns
+        -------
+        dict
+            A mapping between (competion ID, season ID) tuples and the
+            information available about each competition in the data stream.
+        """
+        competitions = {}
+        for match in self._get_matches():
+            match_info = self._get_match_info(match)
+            season = assertget(match_info, "tournamentCalendar")
+            season_id = assertget(season, "id")
+            competition = assertget(match_info, "competition")
+            competition_id = assertget(competition, "id")
+            competitions[(competition_id, season_id)] = {
+                "season_id": season_id,
+                "season_name": assertget(season, "name"),
+                "competition_id": competition_id,
+                "competition_name": assertget(competition, "name"),
+            }
+        return competitions
+
+    def extract_games(self) -> dict[str, dict[str, Any]]:
+        """Return a dictionary with all available games.
+
+        Returns
+        -------
+        dict
+            A mapping between game IDs and the information available about
+            each game in the data stream.
+        """
+        games = {}
+        for match in self._get_matches():
+            match_info = self._get_match_info(match)
+            game_id = assertget(match_info, "id")
+            season = assertget(match_info, "tournamentCalendar")
+            competition = assertget(match_info, "competition")
+            contestant = assertget(match_info, "contestant")
+            game_date = assertget(match_info, "date")
+            game_time = assertget(match_info, "time")
+            game_datetime = f"{game_date} {game_time}"
+            venue = assertget(match_info, "venue")
+            games[game_id] = {
+                # Fields required by the base schema
+                "game_id": game_id,
+                "competition_id": assertget(competition, "id"),
+                "season_id": assertget(season, "id"),
+                "game_day": int(match_info["week"]) if "week" in match_info else None,
+                "game_date": datetime.strptime(game_datetime, "%Y-%m-%dZ %H:%M:%SZ"),
+                "home_team_id": self._extract_team_id(contestant, "home"),
+                "away_team_id": self._extract_team_id(contestant, "away"),
+                # Optional fields
+                # home_score=?,
+                # away_score=?,
+                # duration=?,
+                # referee=?,
+                "venue": venue["shortName"] if "shortName" in venue else None,
+                # attendance=?,
+                # home_manager=?,
+                # away_manager=?,
+            }
+            live_data = self._get_live_data(match)
+            if "matchDetails" in live_data:
+                match_details = assertget(live_data, "matchDetails")
+                if "matchLengthMin" in match_details:
+                    games[game_id]["duration"] = assertget(match_details, "matchLengthMin")
+                if "scores" in match_details:
+                    scores = assertget(match_details, "scores")
+                    games[game_id]["home_score"] = assertget(scores, "total")["home"]
+                    games[game_id]["away_score"] = assertget(scores, "total")["away"]
+                if "matchDetailsExtra" in live_data:
+                    extra_match_details = assertget(live_data, "matchDetailsExtra")
+                    if "attendance" in extra_match_details:
+                        games[game_id]["attendance"] = int(
+                            assertget(extra_match_details, "attendance")
+                        )
+                    if "matchOfficial" in extra_match_details:
+                        for official in assertget(extra_match_details, "matchOfficial"):
+                            if official["type"] == "Main":
+                                games[game_id]["referee"] = self._get_name(official)
+        return games
+
+    def extract_teams(self) -> dict[str, dict[str, Any]]:
+        """Return a dictionary with all available teams.
+
+        Returns
+        -------
+        dict
+            A mapping between team IDs and the information available about
+            each team in the data stream.
+        """
+        teams = {}
+        for match in self._get_matches():
+            match_info = self._get_match_info(match)
+            contestants = assertget(match_info, "contestant")
+            for contestant in contestants:
+                team_id = assertget(contestant, "id")
+                team = {
+                    "team_id": team_id,
+                    "team_name": assertget(contestant, "name"),
+                }
+                teams[team_id] = team
+        return teams
+
+    def extract_players(self) -> dict[tuple[str, str], dict[str, Any]]:  # noqa: C901
+        """Return a dictionary with all available players.
+
+        Returns
+        -------
+        dict
+            A mapping between player IDs and the information available about
+            each player in the data stream.
+        """
+        players = {}
+        subs = self.extract_substitutions()
+        for match in self._get_matches():
+            match_info = self._get_match_info(match)
+            game_id = assertget(match_info, "id")
+            live_data = self._get_live_data(match)
+            if "lineUp" not in live_data:
+                continue
+            red_cards = {
+                e["playerId"]: e["timeMin"]
+                for e in live_data.get("card", [])
+                if "type" in e
+                and e["type"] in ["Y2C", "RC"]
+                and "playerId" in e  # not defined if a coach receives a red card
+            }
+            lineups = assertget(live_data, "lineUp")
+            for lineup in lineups:
+                team_id = assertget(lineup, "contestantId")
+                players_in_lineup = assertget(lineup, "player")
+                for individual in players_in_lineup:
+                    player_id = assertget(individual, "playerId")
+                    players[(game_id, player_id)] = {
+                        # Fields required by the base schema
+                        "game_id": game_id,
+                        "team_id": team_id,
+                        "player_id": player_id,
+                        "player_name": self._get_name(individual),
+                        "is_starter": assertget(individual, "position") != "Substitute",
+                        # minutes_played="?",
+                        "jersey_number": assertget(individual, "shirtNumber"),
+                        # Fields required by the opta schema
+                        "starting_position": assertget(individual, "position"),
+                    }
+                    if "matchDetails" in live_data and "substitute" in live_data:
+                        match_details = assertget(live_data, "matchDetails")
+                        if "matchLengthMin" not in match_details:
+                            continue
+                        # Determine when player entered the pitch
+                        is_starter = assertget(individual, "position") != "Substitute"
+                        sub_in = [
+                            s
+                            for s in subs.values()
+                            if s["game_id"] == game_id and s["player_in_id"] == player_id
+                        ]
+                        if is_starter:
+                            minute_start = 0
+                        elif len(sub_in) == 1:
+                            minute_start = sub_in[0]["minute"]
+                        else:
+                            minute_start = None
+                        # Determine when player left the pitch
+                        sub_out = [
+                            s
+                            for s in subs.values()
+                            if s["game_id"] == game_id and s["player_out_id"] == player_id
+                        ]
+                        duration = assertget(match_details, "matchLengthMin")
+                        minute_end = duration
+                        if len(sub_out) == 1:
+                            minute_end = sub_out[0]["minute"]
+                        elif player_id in red_cards:
+                            minute_end = red_cards[player_id]
+                        # Determin time on the pitch
+                        if is_starter or minute_start is not None:
+                            players[(game_id, player_id)]["minutes_played"] = (
+                                minute_end - minute_start
+                            )
+                        else:
+                            players[(game_id, player_id)]["minutes_played"] = 0
+        return players
+
+    def extract_substitutions(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all substitution events.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, player ID) tuples and the information
+            available about each substitution in the data stream.
+        """
+        subs = {}
+        for match in self._get_matches():
+            match_info = self._get_match_info(match)
+            game_id = assertget(match_info, "id")
+            live_data = self._get_live_data(match)
+            if "substitute" not in live_data:
+                continue
+            for e in assertget(live_data, "substitute"):
+                sub_id = assertget(e, "playerOnId")
+                subs[(game_id, sub_id)] = {
+                    "game_id": game_id,
+                    "team_id": assertget(e, "contestantId"),
+                    "period_id": int(assertget(e, "periodId")),
+                    "minute": int(assertget(e, "timeMin")),
+                    "player_in_id": assertget(e, "playerOnId"),
+                    "player_out_id": assertget(e, "playerOffId"),
+                }
+        return subs

data/opta/parsers/ma3_json.py

@@ -0,0 +1,355 @@
+"""JSON parser for Stats Perform MA3 feeds."""
+
+from datetime import datetime
+from typing import Any, Optional
+
+import pandas as pd
+
+from ...base import MissingDataError
+from .base import OptaJSONParser, _get_end_x, _get_end_y, assertget
+
+
+class MA3JSONParser(OptaJSONParser):
+    """Extract data from a Stats Perform MA3 data stream.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    """
+
+    _position_map = {
+        1: "Goalkeeper",
+        2: "Defender",
+        3: "Midfielder",
+        4: "Forward",
+        5: "Substitute",
+    }
+
+    def _get_match_info(self) -> dict[str, Any]:
+        if "matchInfo" in self.root:
+            return self.root["matchInfo"]
+        raise MissingDataError
+
+    def _get_live_data(self) -> dict[str, Any]:
+        if "liveData" in self.root:
+            return self.root["liveData"]
+        raise MissingDataError
+
+    def extract_competitions(self) -> dict[tuple[str, str], dict[str, Any]]:
+        """Return a dictionary with all available competitions.
+
+        Returns
+        -------
+        dict
+            A mapping between competion IDs and the information available about
+            each competition in the data stream.
+        """
+        match_info = self._get_match_info()
+        season = assertget(match_info, "tournamentCalendar")
+        competition = assertget(match_info, "competition")
+        competition_id = assertget(competition, "id")
+        season_id = assertget(season, "id")
+        season = {
+            # Fields required by the base schema
+            "season_id": season_id,
+            "season_name": assertget(season, "name"),
+            "competition_id": competition_id,
+            "competition_name": assertget(competition, "name"),
+        }
+        return {(competition_id, season_id): season}
+
+    def extract_games(self) -> dict[str, dict[str, Any]]:
+        """Return a dictionary with all available games.
+
+        Returns
+        -------
+        dict
+            A mapping between game IDs and the information available about
+            each game in the data stream.
+        """
+        match_info = self._get_match_info()
+        game_id = assertget(match_info, "id")
+        season = assertget(match_info, "tournamentCalendar")
+        competition = assertget(match_info, "competition")
+        contestant = assertget(match_info, "contestant")
+        game_date = assertget(match_info, "date")[0:10]
+        game_time = assertget(match_info, "time")[0:8]
+        game_datetime = f"{game_date}T{game_time}"
+        venue = assertget(match_info, "venue")
+        game_obj = {
+            "game_id": game_id,
+            "competition_id": assertget(competition, "id"),
+            "season_id": assertget(season, "id"),
+            "game_day": int(match_info["week"]) if "week" in match_info else None,
+            "game_date": datetime.strptime(game_datetime, "%Y-%m-%dT%H:%M:%S"),
+            "home_team_id": self._extract_team_id(contestant, "home"),
+            "away_team_id": self._extract_team_id(contestant, "away"),
+            "venue": assertget(venue, "shortName"),
+        }
+        live_data = self._get_live_data()
+        if "matchDetails" in live_data:
+            match_details = assertget(live_data, "matchDetails")
+            if "matchLengthMin" in match_details:
+                game_obj["duration"] = assertget(match_details, "matchLengthMin")
+            if "scores" in match_details:
+                scores = assertget(match_details, "scores")
+                game_obj["home_score"] = assertget(scores, "total")["home"]
+                game_obj["away_score"] = assertget(scores, "total")["away"]
+
+        return {game_id: game_obj}
+
+    def extract_teams(self) -> dict[str, dict[str, Any]]:
+        """Return a dictionary with all available teams.
+
+        Returns
+        -------
+        dict
+            A mapping between team IDs and the information available about
+            each team in the data stream.
+        """
+        match_info = self._get_match_info()
+        contestants = assertget(match_info, "contestant")
+        teams = {}
+        for contestant in contestants:
+            team_id = assertget(contestant, "id")
+            team = {
+                # Fields required by the base schema
+                "team_id": team_id,
+                "team_name": assertget(contestant, "name"),
+            }
+            teams[team_id] = team
+        return teams
+
+    def extract_players(self) -> dict[tuple[str, str], dict[str, Any]]:  # noqa: C901
+        """Return a dictionary with all available players.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, player ID) tuples and the information
+            available about each player in the data stream.
+        """
+        match_info = self._get_match_info()
+        game_id = assertget(match_info, "id")
+        live_data = self._get_live_data()
+        events = assertget(live_data, "event")
+
+        game_duration = self._extract_duration()
+        playerid_to_name = {}
+
+        players_data: dict[str, list[Any]] = {
+            "starting_position_id": [],
+            "player_id": [],
+            "team_id": [],
+            "position_in_formation": [],
+            "jersey_number": [],
+        }
+        red_cards = {}
+
+        for event in events:
+            event_type = assertget(event, "typeId")
+            if event_type == 34:
+                team_id = assertget(event, "contestantId")
+                qualifiers = assertget(event, "qualifier")
+                for q in qualifiers:
+                    qualifier_id = assertget(q, "qualifierId")
+                    value = assertget(q, "value")
+                    value = value.split(", ")
+                    if qualifier_id == 30:
+                        players_data["player_id"] += value
+                        team = [team_id for _ in range(len(value))]
+                        players_data["team_id"] += team
+                    elif qualifier_id == 44:
+                        value = [int(v) for v in value]
+                        players_data["starting_position_id"] += value
+                    elif qualifier_id == 131:
+                        value = [int(v) for v in value]
+                        players_data["position_in_formation"] += value
+                    elif qualifier_id == 59:
+                        value = [int(v) for v in value]
+                        players_data["jersey_number"] += value
+            elif event_type == 17 and "playerId" in event:
+                qualifiers = assertget(event, "qualifier")
+                for q in qualifiers:
+                    qualifier_id = assertget(q, "qualifierId")
+                    if qualifier_id in [32, 33]:
+                        red_cards[event["playerId"]] = event["timeMin"]
+
+            player_id = event.get("playerId")
+            if player_id is None:
+                continue
+            player_name = assertget(event, "playerName")
+            if player_id not in playerid_to_name:
+                playerid_to_name[player_id] = player_name
+
+        df_players_data = pd.DataFrame.from_dict(players_data)  # type: ignore
+
+        substitutions = list(self.extract_substitutions().values())
+        substitutions_columns = ["player_id", "team_id", "minute_start", "minute_end"]
+        df_substitutions = pd.DataFrame(substitutions, columns=substitutions_columns)
+        df_substitutions = df_substitutions.groupby(["player_id", "team_id"]).max().reset_index()
+        df_substitutions["minute_start"] = df_substitutions["minute_start"].fillna(0)
+        df_substitutions["minute_end"] = df_substitutions["minute_end"].fillna(game_duration)
+
+        if df_substitutions.empty:
+            df_players_data["minute_start"] = 0
+            df_players_data["minute_end"] = game_duration
+        else:
+            df_players_data = df_players_data.merge(
+                df_substitutions, on=["team_id", "player_id"], how="left"
+            )
+        df_players_data["minute_end"] = df_players_data.apply(
+            lambda row: red_cards[row["player_id"]]
+            if row["player_id"] in red_cards
+            else row["minute_end"],
+            axis=1,
+        )
+
+        df_players_data["is_starter"] = df_players_data["position_in_formation"] > 0
+        df_players_data.loc[
+            df_players_data["is_starter"] & df_players_data["minute_start"].isnull(),
+            "minute_start",
+        ] = 0
+        df_players_data.loc[
+            df_players_data["is_starter"] & df_players_data["minute_end"].isnull(), "minute_end"
+        ] = game_duration
+
+        df_players_data["minutes_played"] = (
+            (df_players_data["minute_end"] - df_players_data["minute_start"]).fillna(0).astype(int)
+        )
+
+        players = {}
+        for _, player in df_players_data.iterrows():
+            if player.minutes_played > 0:
+                players[(game_id, player.player_id)] = {
+                    # Fields required by the base schema
+                    "game_id": game_id,
+                    "team_id": player.team_id,
+                    "player_id": player.player_id,
+                    "player_name": playerid_to_name[player.player_id],
+                    "is_starter": player.is_starter,
+                    "minutes_played": player.minutes_played,
+                    "jersey_number": player.jersey_number,
+                    # Fields required by the opta schema
+                    "starting_position": self._position_map.get(
+                        player.starting_position_id, "Unknown"
+                    ),
+                }
+        return players
+
+    def extract_events(self) -> dict[tuple[str, int], dict[str, Any]]:
+        """Return a dictionary with all available events.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, event ID) tuples and the information
+            available about each event in the data stream.
+        """
+        match_info = self._get_match_info()
+        live_data = self._get_live_data()
+        game_id = assertget(match_info, "id")
+
+        events = {}
+        for element in assertget(live_data, "event"):
+            timestamp_string = assertget(element, "timeStamp")
+            timestamp = self._convert_timestamp(timestamp_string)
+
+            qualifiers = {
+                int(q["qualifierId"]): q.get("value") for q in element.get("qualifier", [])
+            }
+            start_x = float(assertget(element, "x"))
+            start_y = float(assertget(element, "y"))
+            end_x = _get_end_x(qualifiers)
+            end_y = _get_end_y(qualifiers)
+
+            event_id = int(assertget(element, "id"))
+            event = {
+                # Fields required by the base schema
+                "game_id": game_id,
+                "event_id": event_id,
+                "period_id": int(assertget(element, "periodId")),
+                "team_id": assertget(element, "contestantId"),
+                "player_id": element.get("playerId"),
+                "type_id": int(assertget(element, "typeId")),
+                # Fields required by the opta schema
+                "timestamp": timestamp,
+                "minute": int(assertget(element, "timeMin")),
+                "second": int(assertget(element, "timeSec")),
+                "outcome": bool(int(element.get("outcome", 1))),
+                "start_x": start_x,
+                "start_y": start_y,
+                "end_x": end_x if end_x is not None else start_x,
+                "end_y": end_y if end_y is not None else start_y,
+                "qualifiers": qualifiers,
+                # Optional fields
+                "assist": bool(int(element.get("assist", 0))),
+                "keypass": bool(int(element.get("keyPass", 0))),
+            }
+            events[(game_id, event_id)] = event
+        return events
+
+    def extract_substitutions(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all substitution events.
+
+        Returns
+        -------
+        dict
+            A mapping between player IDs and the information available about
+            each substitution in the data stream.
+        """
+        live_data = self._get_live_data()
+        events = assertget(live_data, "event")
+
+        subs = {}
+        for e in events:
+            event_type = assertget(e, "typeId")
+            if event_type in (18, 19):
+                sub_id = assertget(e, "playerId")
+                substitution_data = {
+                    "player_id": assertget(e, "playerId"),
+                    "team_id": assertget(e, "contestantId"),
+                }
+                if event_type == 18:
+                    substitution_data["minute_end"] = assertget(e, "timeMin")
+                else:
+                    substitution_data["minute_start"] = assertget(e, "timeMin")
+                subs[sub_id] = substitution_data
+        return subs
+
+    def _extract_duration(self) -> int:
+        live_data = self._get_live_data()
+        events = assertget(live_data, "event")
+
+        game_duration = 90
+
+        for event in events:
+            event_type = assertget(event, "typeId")
+            if event_type == 30:
+                # todo: add 1st half time
+                qualifiers = assertget(event, "qualifier")
+                for q in qualifiers:
+                    qualifier = assertget(q, "qualifierId")
+                    if qualifier == 209:
+                        new_duration = assertget(event, "timeMin")
+                        if new_duration > game_duration:
+                            game_duration = new_duration
+
+        return game_duration
+
+    @staticmethod
+    def _extract_team_id(teams: list[dict[str, str]], side: str) -> Optional[str]:
+        for team in teams:
+            team_side = assertget(team, "position")
+            if team_side == side:
+                team_id = assertget(team, "id")
+                return team_id
+        raise MissingDataError
+
+    @staticmethod
+    def _convert_timestamp(timestamp_string: str) -> datetime:
+        try:
+            return datetime.strptime(timestamp_string, "%Y-%m-%dT%H:%M:%S.%fZ")
+        except ValueError:
+            return datetime.strptime(timestamp_string, "%Y-%m-%dT%H:%M:%SZ")

data/opta/parsers/whoscored.py

@@ -0,0 +1,421 @@
+"""JSON parser for WhoScored feeds."""
+
+import json  # type: ignore
+import re
+from datetime import datetime, timedelta
+from typing import Any, Optional
+
+from ...base import MissingDataError
+from .base import OptaParser, _get_end_x, _get_end_y, assertget
+
+
+def _position_mapping(formation: str, x: float, y: float) -> str:
+    if x == 0 and y == 5:
+        return "GK"
+    return "Unknown"
+
+
+class WhoScoredParser(OptaParser):
+    """Extract data from a JSON data stream scraped from WhoScored.
+
+    Parameters
+    ----------
+    path : str
+        Path of the data file.
+    competition_id : int
+        ID of the competition to which the provided data file belongs. If
+        None, this information is extracted from a field 'competition_id' in
+        the JSON.
+    season_id : int
+        ID of the season to which the provided data file belongs. If None,
+        this information is extracted from a field 'season_id' in the JSON.
+    game_id : int
+        ID of the game to which the provided data file belongs. If None, this
+        information is extracted from a field 'game_id' in the JSON.
+    """
+
+    def __init__(  # noqa: C901
+        self,
+        path: str,
+        competition_id: Optional[int] = None,
+        season_id: Optional[int] = None,
+        game_id: Optional[int] = None,
+    ) -> None:
+        with open(path, encoding="utf-8") as fh:
+            self.root = json.load(fh)
+
+        if competition_id is None:
+            try:
+                competition_id = int(assertget(self.root, "competition_id"))
+            except AssertionError as e:
+                raise MissingDataError(
+                    """Could not determine the competition id. Add it to the
+                    file path or include a field 'competition_id' in the
+                    JSON."""
+                ) from e
+        self.competition_id = competition_id
+
+        if season_id is None:
+            try:
+                season_id = int(assertget(self.root, "season_id"))
+            except AssertionError as e:
+                raise MissingDataError(
+                    """Could not determine the season id. Add it to the file
+                    path or include a field 'season_id' in the JSON."""
+                ) from e
+        self.season_id = season_id
+
+        if game_id is None:
+            try:
+                game_id = int(assertget(self.root, "game_id"))
+            except AssertionError as e:
+                raise MissingDataError(
+                    """Could not determine the game id. Add it to the file
+                    path or include a field 'game_id' in the JSON."""
+                ) from e
+        self.game_id = game_id
+
+    def _get_period_id(self, event: dict[str, Any]) -> int:
+        period = assertget(event, "period")
+        period_id = int(assertget(period, "value"))
+        return period_id
+
+    def _get_period_milliseconds(self, event: dict[str, Any]) -> int:
+        period_minute_limits = assertget(self.root, "periodMinuteLimits")
+        period_id = self._get_period_id(event)
+        if period_id == 16:  # Pre-match
+            return 0
+        if period_id == 14:  # Post-game
+            return 0
+        minute = int(assertget(event, "minute"))
+        period_minute = minute
+        if period_id > 1:
+            period_minute = minute - period_minute_limits[str(period_id - 1)]
+        period_second = period_minute * 60 + int(event.get("second", 0))
+        return period_second * 1000
+
+    def extract_games(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all available games.
+
+        Returns
+        -------
+        dict
+            A mapping between game IDs and the information available about
+            each game in the data stream.
+        """
+        team_home = assertget(self.root, "home")
+        team_away = assertget(self.root, "away")
+        game_dict = {
+            # Fields required by the base schema
+            "game_id": self.game_id,
+            "season_id": self.season_id,
+            "competition_id": self.competition_id,
+            "game_day": None,  # Cannot be determined from the data stream
+            "game_date": datetime.strptime(
+                assertget(self.root, "startTime"), "%Y-%m-%dT%H:%M:%S"
+            ),  # Dates are UTC
+            "home_team_id": int(assertget(team_home, "teamId")),
+            "away_team_id": int(assertget(team_away, "teamId")),
+            # Optional fields
+            "home_score": int(assertget(assertget(self.root["home"], "scores"), "running")),
+            "away_score": int(assertget(assertget(self.root["away"], "scores"), "running")),
+            "duration": int(self.root.get("expandedMaxMinute"))
+            if "expandedMaxMinute" in self.root
+            else None,
+            "referee": self.root.get("referee", {}).get("name"),
+            "venue": self.root.get("venueName"),
+            "attendance": int(self.root.get("attendance")) if "attendance" in self.root else None,
+            "home_manager": team_home.get("managerName"),
+            "away_manager": team_away.get("managerName"),
+        }
+        return {self.game_id: game_dict}
+
+    def extract_teams(self) -> dict[int, dict[str, Any]]:
+        """Return a dictionary with all available teams.
+
+        Returns
+        -------
+        dict
+            A mapping between team IDs and the information available about
+            each team in the data stream.
+        """
+        teams = {}
+        for side in [self.root["home"], self.root["away"]]:
+            team_id = int(assertget(side, "teamId"))
+            teams[team_id] = {
+                # Fields required by the base schema
+                "team_id": team_id,
+                "team_name": assertget(side, "name"),
+            }
+        return teams
+
+    def extract_players(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all available players.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, player ID) tuples and the information
+            available about each player in the data stream.
+        """
+        game_id = self.game_id
+        player_gamestats = self.extract_playergamestats()
+        players = {}
+        for team in [self.root["home"], self.root["away"]]:
+            team_id = int(assertget(team, "teamId"))
+            for p in team["players"]:
+                player_id = int(assertget(p, "playerId"))
+                players[(game_id, player_id)] = {
+                    # Fields required by the base schema
+                    "game_id": game_id,
+                    "team_id": team_id,
+                    "player_id": player_id,
+                    "player_name": assertget(p, "name"),
+                    "is_starter": bool(p.get("isFirstEleven", False)),
+                    "minutes_played": player_gamestats[(game_id, player_id)]["minutes_played"],
+                    "jersey_number": player_gamestats[(game_id, player_id)]["jersey_number"],
+                    # Fields required by the opta schema
+                    "starting_position": player_gamestats[(game_id, player_id)]["position_code"],
+                    # Optional fields
+                    # WhoScored retrieves player details for the current date,
+                    # not for the game date. Hence, we do not innclude this
+                    # info.
+                    # age=int(p["age"]),
+                    # height=float(p.get("height", float("NaN"))),
+                    # weight=float(p.get("weight", float("NaN"))),
+                }
+        return players
+
+    def extract_events(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all available events.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, event ID) tuples and the information
+            available about each event in the data stream.
+        """
+        events = {}
+
+        time_start_str = assertget(self.root, "startTime")
+        time_start = datetime.strptime(time_start_str, "%Y-%m-%dT%H:%M:%S")
+        for attr in self.root["events"]:
+            event_id = int(assertget(attr, "id" if "id" in attr else "eventId"))
+            eventtype = attr.get("type", {})
+            start_x = float(assertget(attr, "x"))
+            start_y = float(assertget(attr, "y"))
+            minute = int(assertget(attr, "expandedMinute"))
+            second = int(attr.get("second", 0))
+            qualifiers = {
+                int(q["type"]["value"]): q.get("value", True) for q in attr.get("qualifiers", [])
+            }
+            end_x = attr.get("endX", _get_end_x(qualifiers))
+            end_y = attr.get("endY", _get_end_y(qualifiers))
+            events[(self.game_id, event_id)] = {
+                # Fields required by the base schema
+                "game_id": self.game_id,
+                "event_id": event_id,
+                "period_id": self._get_period_id(attr),
+                "team_id": int(assertget(attr, "teamId")),
+                "player_id": int(attr.get("playerId")) if "playerId" in attr else None,
+                "type_id": int(assertget(eventtype, "value")),
+                # type_name=assertget(eventtype, "displayName"),  # added in the opta loader
+                # Fields required by the opta schema
+                # Timestamp is not availe in the data stream. The returned
+                # timestamp  is not accurate, but sufficient for camptability
+                # with the other Opta data streams.
+                "timestamp": (time_start + timedelta(seconds=(minute * 60 + second))),
+                "minute": minute,
+                "second": second,
+                "outcome": bool(attr["outcomeType"].get("value"))
+                if "outcomeType" in attr
+                else None,
+                "start_x": start_x,
+                "start_y": start_y,
+                "end_x": end_x if end_x is not None else start_x,
+                "end_y": end_y if end_y is not None else start_y,
+                "qualifiers": qualifiers,
+                # Optional fields
+                "related_player_id": int(attr.get("relatedPlayerId"))
+                if "relatedPlayerId" in attr
+                else None,
+                "touch": bool(attr.get("isTouch", False)),
+                "goal": bool(attr.get("isGoal", False)),
+                "shot": bool(attr.get("isShot", False)),
+                # assist=bool(attr.get('assist')) if "assist" in attr else None,
+                # keypass=bool(attr.get('keypass')) if "keypass" in attr else None,
+            }
+
+        return events
+
+    def extract_substitutions(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return a dictionary with all substitution events.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, player ID) tuples and the information
+            available about each substitution in the data stream.
+        """
+        subs = {}
+        subonevents = [e for e in self.root["events"] if e["type"].get("value") == 19]
+        for e in subonevents:
+            sub_id = int(assertget(e, "playerId"))
+            sub = {
+                "game_id": self.game_id,
+                "team_id": int(assertget(e, "teamId")),
+                "period_id": self._get_period_id(e),
+                "period_milliseconds": self._get_period_milliseconds(e),
+                "player_in_id": int(assertget(e, "playerId")),
+                "player_out_id": int(assertget(e, "relatedPlayerId")),
+            }
+            subs[(self.game_id, sub_id)] = sub
+        return subs
+
+    def extract_positions(self) -> dict[tuple[int, int, int], dict[str, Any]]:  # noqa: C901
+        """Return a dictionary with each player's position during a game.
+
+        Returns
+        -------
+        dict
+            A mapping between (game ID, player ID, epoch ID) tuples and the
+            information available about each player's position in the data stream.
+        """
+        positions = {}
+        for t in [self.root["home"], self.root["away"]]:
+            team_id = int(assertget(t, "teamId"))
+            for f in assertget(t, "formations"):
+                fpositions = assertget(f, "formationPositions")
+                playersIds = assertget(f, "playerIds")
+                formation = assertget(f, "formationName")
+
+                period_end_minutes = assertget(self.root, "periodEndMinutes")
+                period_minute_limits = assertget(self.root, "periodMinuteLimits")
+                start_minute = int(assertget(f, "startMinuteExpanded"))
+                end_minute = int(assertget(f, "endMinuteExpanded"))
+                for period_id in sorted(period_end_minutes.keys()):
+                    if period_end_minutes[period_id] > start_minute:
+                        break
+                period_id = int(period_id)
+                period_minute = start_minute
+                if period_id > 1:
+                    period_minute = start_minute - period_minute_limits[str(period_id - 1)]
+
+                for i, p in enumerate(fpositions):
+                    player_id = int(playersIds[i])
+                    x = float(assertget(p, "vertical"))
+                    y = float(assertget(p, "horizontal"))
+                    position_code = _position_mapping(formation, x, y)
+                    positions[(self.game_id, player_id, start_minute)] = {
+                        "game_id": self.game_id,
+                        "team_id": team_id,
+                        "player_id": player_id,
+                        "period_id": period_id,
+                        "period_milliseconds": (period_minute * 60 * 1000),
+                        "start_milliseconds": (start_minute * 60 * 1000),
+                        "end_milliseconds": (end_minute * 60 * 1000),
+                        "formation_scheme": formation,
+                        "player_position": position_code,
+                        "player_position_x": x,
+                        "player_position_y": y,
+                    }
+        return positions
+
+    def extract_teamgamestats(self) -> dict[tuple[int, int], dict[str, Any]]:
+        """Return some aggregated statistics of each team in a game.
+
+        Returns
+        -------
+        list(dict)
+            A dictionary with aggregated team statistics for each team.
+        """
+        teams_gamestats = {}
+        teams = [self.root["home"], self.root["away"]]
+        for team in teams:
+            team_id = int(assertget(team, "teamId"))
+            statsdict = {}
+            for name in team["stats"]:
+                if isinstance(team["stats"][name], dict):
+                    statsdict[_camel_to_snake(name)] = sum(team["stats"][name].values())
+
+            scores = assertget(team, "scores")
+            teams_gamestats[(self.game_id, team_id)] = dict(
+                game_id=self.game_id,
+                team_id=team_id,
+                side=assertget(team, "field"),
+                score=assertget(scores, "fulltime"),
+                shootout_score=scores.get("penalty"),
+                **{k: statsdict[k] for k in statsdict if not k.endswith("Success")},
+            )
+
+        return teams_gamestats
+
+    def extract_playergamestats(self) -> dict[tuple[int, int], dict[str, Any]]:  # noqa: C901
+        """Return some aggregated statistics of each player in a game.
+
+        Returns
+        -------
+        dict(dict)
+            A dictionary with aggregated team statistics for each player.
+        """
+        players_gamestats = {}
+        for team in [self.root["home"], self.root["away"]]:
+            team_id = int(assertget(team, "teamId"))
+            red_cards = {
+                e["playerId"]: e["expandedMinute"]
+                for e in team.get("incidentEvents", [])
+                if "cardType" in e
+                and e["cardType"]["displayName"] in ["Red", "SecondYellow"]
+                and "playerId" in e  # not defined if a coach receives a red card
+            }
+            for player in team["players"]:
+                statsdict = {
+                    _camel_to_snake(name): sum(stat.values())
+                    for name, stat in player["stats"].items()
+                }
+                stats = [k for k in statsdict if not k.endswith("success")]
+
+                player_id = int(assertget(player, "playerId"))
+                p = dict(
+                    game_id=self.game_id,
+                    team_id=team_id,
+                    player_id=player_id,
+                    is_starter=bool(player.get("isFirstEleven", False)),
+                    position_code=player.get("position", None),
+                    jersey_number=int(player.get("shirtNo", 0)),
+                    mvp=bool(player.get("isManOfTheMatch", False)),
+                    **{k: statsdict[k] for k in stats},
+                )
+                if "subbedInExpandedMinute" in player:
+                    p["minute_start"] = player["subbedInExpandedMinute"]
+                if "subbedOutExpandedMinute" in player:
+                    p["minute_end"] = player["subbedOutExpandedMinute"]
+                if player_id in red_cards:
+                    p["minute_end"] = red_cards[player_id]
+
+                # Did not play
+                p["minutes_played"] = 0
+                # Played the full game
+                if p["is_starter"] and "minute_end" not in p:
+                    p["minute_start"] = 0
+                    p["minute_end"] = self.root["expandedMaxMinute"]
+                    p["minutes_played"] = self.root["expandedMaxMinute"]
+                # Started but substituted out
+                elif p["is_starter"] and "minute_end" in p:
+                    p["minute_start"] = 0
+                    p["minutes_played"] = p["minute_end"]
+                # Substitud in and played the remainder of the game
+                elif "minute_start" in p and "minute_end" not in p:
+                    p["minute_end"] = self.root["expandedMaxMinute"]
+                    p["minutes_played"] = self.root["expandedMaxMinute"] - p["minute_start"]
+                # Substitud in and out
+                elif "minute_start" in p and "minute_end" in p:
+                    p["minutes_played"] = p["minute_end"] - p["minute_start"]
+
+                players_gamestats[(self.game_id, player_id)] = p
+        return players_gamestats
+
+
+def _camel_to_snake(name: str) -> str:
+    s1 = re.sub("(.)([A-Z][a-z]+)", r"\1_\2", name)
+    return re.sub("([a-z0-9])([A-Z])", r"\1_\2", s1).lower()

data/opta/schema.py

@@ -0,0 +1,86 @@
+"""SPADL schema for Opta data."""
+
+from typing import Optional
+
+import pandas as pd
+import pandera as pa
+from pandera.typing import DateTime, Object, Series
+
+from socceraction.data.schema import (
+    CompetitionSchema,
+    EventSchema,
+    GameSchema,
+    PlayerSchema,
+    TeamSchema,
+)
+
+
+class OptaCompetitionSchema(CompetitionSchema):
+    """Definition of a dataframe containing a list of competitions and seasons."""
+
+
+class OptaGameSchema(GameSchema):
+    """Definition of a dataframe containing a list of games."""
+
+    home_score: Optional[Series[int]] = pa.Field(nullable=True)
+    """The final score of the home team."""
+    away_score: Optional[Series[int]] = pa.Field(nullable=True)
+    """The final score of the away team."""
+    duration: Optional[Series[int]] = pa.Field(nullable=True)
+    """The total duration of the game in minutes."""
+    referee: Optional[Series[str]] = pa.Field(nullable=True)
+    """The name of the referee."""
+    venue: Optional[Series[str]] = pa.Field(nullable=True)
+    """The name of the stadium where the game was played."""
+    attendance: Optional[Series[int]] = pa.Field(nullable=True)
+    """The number of people who attended the game."""
+    home_manager: Optional[Series[str]] = pa.Field(nullable=True)
+    """The name of the manager of the home team."""
+    away_manager: Optional[Series[str]] = pa.Field(nullable=True)
+    """The name of the manager of the away team."""
+
+
+class OptaPlayerSchema(PlayerSchema):
+    """Definition of a dataframe containing the list of players of a game."""
+
+    starting_position: Series[str]
+    """The starting position of the player."""
+
+
+class OptaTeamSchema(TeamSchema):
+    """Definition of a dataframe containing the list of teams of a game."""
+
+
+class OptaEventSchema(EventSchema):
+    """Definition of a dataframe containing event stream data of a game."""
+
+    timestamp: Series[DateTime]
+    """Time in the match the event takes place, recorded to the millisecond."""
+    minute: Series[int]
+    """The minutes on the clock at the time of this event."""
+    second: Series[int] = pa.Field(ge=0, le=59)
+    """The second part of the timestamp."""
+    outcome: Series[bool]
+    """Whether the event had a successful outcome or not."""
+    start_x: Series[float] = pa.Field(nullable=True)
+    """The x coordinate of the location where the event started."""
+    start_y: Series[float] = pa.Field(nullable=True)
+    """The y coordinate of the location where the event started."""
+    end_x: Series[float] = pa.Field(nullable=True)
+    """The x coordinate of the location where the event ended."""
+    end_y: Series[float] = pa.Field(nullable=True)
+    """The y coordinate of the location where the event ended."""
+    qualifiers: Series[Object]
+    """A JSON object containing the Opta qualifiers of the event."""
+    assist: Optional[Series[bool]]
+    """Whether the event was an assist or not."""
+    keypass: Optional[Series[bool]]
+    """Whether the event was a keypass or not."""
+    goal: Optional[Series[bool]]
+    """Whether the event was a goal or not."""
+    shot: Optional[Series[bool]]
+    """Whether the event was a shot or not."""
+    touch: Optional[Series[bool]]
+    """Whether the event was a on-the-ball action or not."""
+    related_player_id: Optional[Series[pd.Int64Dtype]] = pa.Field(nullable=True)
+    """The ID of a second player that was involved in this event."""

data/schema.py

@@ -0,0 +1,110 @@
+"""Base schemas used by all event stream serializers.
+
+Each serializer should create dataframes that contain at least the fields
+included in these base schemas. Each serializer can add different additional
+fields on top.
+
+"""
+
+import pandas as pd
+import pandera as pa
+from pandera.typing import DateTime, Object, Series
+
+
+class CompetitionSchema(pa.SchemaModel):
+    """Definition of a dataframe containing a list of competitions and seasons."""
+
+    season_id: Series[Object] = pa.Field()
+    """The unique identifier for the season."""
+    season_name: Series[str] = pa.Field()
+    """The name of the season."""
+    competition_id: Series[Object] = pa.Field()
+    """The unique identifier for the competition."""
+    competition_name: Series[str] = pa.Field()
+    """The name of the competition."""
+
+    class Config:  # noqa: D106
+        strict = True
+        coerce = True
+
+
+class GameSchema(pa.SchemaModel):
+    """Definition of a dataframe containing a list of games."""
+
+    game_id: Series[Object] = pa.Field()
+    """The unique identifier for the game."""
+    season_id: Series[Object] = pa.Field()
+    """The unique identifier for the season."""
+    competition_id: Series[Object] = pa.Field()
+    """The unique identifier for the competition."""
+    game_day: Series[pd.Int64Dtype] = pa.Field(nullable=True)
+    """Number corresponding to the weeks or rounds into the competition this game is."""
+    game_date: Series[DateTime] = pa.Field()
+    """The date when the game was played."""
+    home_team_id: Series[Object] = pa.Field()
+    """The unique identifier for the home team in this game."""
+    away_team_id: Series[Object] = pa.Field()
+    """The unique identifier for the away team in this game."""
+
+    class Config:  # noqa: D106
+        strict = True
+        coerce = True
+
+
+class TeamSchema(pa.SchemaModel):
+    """Definition of a dataframe containing the list of teams of a game."""
+
+    team_id: Series[Object] = pa.Field()
+    """The unique identifier for the team."""
+    team_name: Series[str] = pa.Field()
+    """The name of the team."""
+
+    class Config:  # noqa: D106
+        strict = True
+        coerce = True
+
+
+class PlayerSchema(pa.SchemaModel):
+    """Definition of a dataframe containing the list of players on the teamsheet of a game."""
+
+    game_id: Series[Object] = pa.Field()
+    """The unique identifier for the game."""
+    team_id: Series[Object] = pa.Field()
+    """The unique identifier for the player's team."""
+    player_id: Series[Object] = pa.Field()
+    """The unique identifier for the player."""
+    player_name: Series[str] = pa.Field()
+    """The name of the player."""
+    is_starter: Series[bool] = pa.Field()
+    """Whether the player is in the starting lineup."""
+    minutes_played: Series[int] = pa.Field()
+    """The number of minutes the player played in the game."""
+    jersey_number: Series[int] = pa.Field()
+    """The player's jersey number."""
+
+    class Config:  # noqa: D106
+        strict = True
+        coerce = True
+
+
+class EventSchema(pa.SchemaModel):
+    """Definition of a dataframe containing event stream data of a game."""
+
+    game_id: Series[Object] = pa.Field()
+    """The unique identifier for the game."""
+    event_id: Series[Object] = pa.Field()
+    """The unique identifier for the event."""
+    period_id: Series[int] = pa.Field()
+    """The unique identifier for the part of the game in which the event took place."""
+    team_id: Series[Object] = pa.Field(nullable=True)
+    """The unique identifier for the team this event relates to."""
+    player_id: Series[Object] = pa.Field(nullable=True)
+    """The unique identifier for the player this event relates to."""
+    type_id: Series[int] = pa.Field()
+    """The unique identifier for the type of this event."""
+    type_name: Series[str] = pa.Field()
+    """The name of the type of this event."""
+
+    class Config:  # noqa: D106
+        strict = True
+        coerce = True

data/statsbomb/__init__.py

@@ -0,0 +1,20 @@
+"""Module for loading StatsBomb event data."""
+
+__all__ = [
+    "StatsBombLoader",
+    "extract_player_games",
+    "StatsBombCompetitionSchema",
+    "StatsBombGameSchema",
+    "StatsBombPlayerSchema",
+    "StatsBombTeamSchema",
+    "StatsBombEventSchema",
+]
+
+from .loader import StatsBombLoader, extract_player_games
+from .schema import (
+    StatsBombCompetitionSchema,
+    StatsBombEventSchema,
+    StatsBombGameSchema,
+    StatsBombPlayerSchema,
+    StatsBombTeamSchema,
+)

data/statsbomb/loader.py

@@ -0,0 +1,495 @@
+"""Implements serializers for StatsBomb data."""
+
+import os
+from typing import Any, Optional, cast
+
+import pandas as pd  # type: ignore
+from pandera.typing import DataFrame
+
+try:
+    from statsbombpy import sb
+except ImportError:
+    sb = None
+
+from socceraction.data.base import (
+    EventDataLoader,
+    ParseError,
+    _expand_minute,
+    _localloadjson,
+)
+
+from .schema import (
+    StatsBombCompetitionSchema,
+    StatsBombEventSchema,
+    StatsBombGameSchema,
+    StatsBombPlayerSchema,
+    StatsBombTeamSchema,
+)
+
+
+class StatsBombLoader(EventDataLoader):
+    """Load Statsbomb data either from a remote location or from a local folder.
+
+    To load remote data, this loader uses the `statsbombpy
+    <https://github.com/statsbomb/statsbombpy>`__ package. Data can be retrieved
+    from the StatsBomb API and from the `Open Data GitHub repo
+    <https://github.com/statsbomb/open-data/>`__.
+    API access is for paying customers only. Authentication can be done by
+    setting environment variables named ``SB_USERNAME`` and ``SB_PASSWORD`` to
+    your login credentials. Alternatively, pass your login credentials using
+    the ``creds`` parameter.
+    StatsBomb's open data can be accessed without the need of authentication
+    but its use is subject to a `user agreement
+    <https://github.com/statsbomb/open-data/blob/master/LICENSE.pdf>`__.
+
+    To load local data, point ``root`` to the root folder of the data. This folder
+    should use the same directory structure as used in the Open Data GitHub repo.
+
+    Parameters
+    ----------
+    getter : str
+        "remote" or "local"
+    root : str, optional
+        Root-path of the data. Only used when getter is "local".
+    creds: dict, optional
+        Login credentials in the format {"user": "", "passwd": ""}. Only used
+        when getter is "remote".
+    """
+
+    def __init__(
+        self,
+        getter: str = "remote",
+        root: Optional[str] = None,
+        creds: Optional[dict[str, str]] = None,
+    ) -> None:
+        if getter == "remote":
+            if sb is None:
+                raise ImportError(
+                    """The 'statsbombpy' package is required. Install with 'pip install statsbombpy'."""
+                )
+            self._creds = creds or sb.DEFAULT_CREDS
+            self._local = False
+        elif getter == "local":
+            if root is None:
+                raise ValueError("""The 'root' parameter is required when loading local data.""")
+            self._local = True
+            self._root = root
+        else:
+            raise ValueError("Invalid getter specified")
+
+    def competitions(self) -> DataFrame[StatsBombCompetitionSchema]:
+        """Return a dataframe with all available competitions and seasons.
+
+        Raises
+        ------
+        ParseError
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available competitions and seasons. See
+            :class:`~socceraction.spadl.statsbomb.StatsBombCompetitionSchema` for the schema.
+        """
+        cols = [
+            "season_id",
+            "competition_id",
+            "competition_name",
+            "country_name",
+            "competition_gender",
+            "season_name",
+        ]
+        if self._local:
+            obj = _localloadjson(str(os.path.join(self._root, "competitions.json")))
+        else:
+            obj = list(sb.competitions(fmt="dict", creds=self._creds).values())
+        if not isinstance(obj, list):
+            raise ParseError("The retrieved data should contain a list of competitions")
+        if len(obj) == 0:
+            return cast(DataFrame[StatsBombCompetitionSchema], pd.DataFrame(columns=cols))
+        return cast(DataFrame[StatsBombCompetitionSchema], pd.DataFrame(obj)[cols])
+
+    def games(self, competition_id: int, season_id: int) -> DataFrame[StatsBombGameSchema]:
+        """Return a dataframe with all available games in a season.
+
+        Parameters
+        ----------
+        competition_id : int
+            The ID of the competition.
+        season_id : int
+            The ID of the season.
+
+        Raises
+        ------
+        ParseError
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available games. See
+            :class:`~socceraction.spadl.statsbomb.StatsBombGameSchema` for the schema.
+        """
+        cols = [
+            "game_id",
+            "season_id",
+            "competition_id",
+            "competition_stage",
+            "game_day",
+            "game_date",
+            "home_team_id",
+            "away_team_id",
+            "home_score",
+            "away_score",
+            "venue",
+            "referee",
+        ]
+        if self._local:
+            obj = _localloadjson(
+                str(os.path.join(self._root, "matches", f"{competition_id}", f"{season_id}.json"))
+            )
+        else:
+            obj = list(
+                sb.matches(competition_id, season_id, fmt="dict", creds=self._creds).values()
+            )
+        if not isinstance(obj, list):
+            raise ParseError("The retrieved data should contain a list of games")
+        if len(obj) == 0:
+            return cast(DataFrame[StatsBombGameSchema], pd.DataFrame(columns=cols))
+        gamesdf = pd.DataFrame(_flatten(m) for m in obj)
+        gamesdf["kick_off"] = gamesdf["kick_off"].fillna("12:00:00.000")
+        gamesdf["match_date"] = pd.to_datetime(
+            gamesdf[["match_date", "kick_off"]].agg(" ".join, axis=1)
+        )
+        gamesdf.rename(
+            columns={
+                "match_id": "game_id",
+                "match_date": "game_date",
+                "match_week": "game_day",
+                "stadium_name": "venue",
+                "referee_name": "referee",
+                "competition_stage_name": "competition_stage",
+            },
+            inplace=True,
+        )
+        if "venue" not in gamesdf:
+            gamesdf["venue"] = None
+        if "referee" not in gamesdf:
+            gamesdf["referee"] = None
+        return cast(DataFrame[StatsBombGameSchema], gamesdf[cols])
+
+    def _lineups(self, game_id: int) -> list[dict[str, Any]]:
+        if self._local:
+            obj = _localloadjson(str(os.path.join(self._root, "lineups", f"{game_id}.json")))
+        else:
+            obj = list(sb.lineups(game_id, fmt="dict", creds=self._creds).values())
+        if not isinstance(obj, list):
+            raise ParseError("The retrieved data should contain a list of teams")
+        if len(obj) != 2:
+            raise ParseError("The retrieved data should contain two teams")
+        return obj
+
+    def teams(self, game_id: int) -> DataFrame[StatsBombTeamSchema]:
+        """Return a dataframe with both teams that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Raises
+        ------
+        ParseError  # noqa: DAR402
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing both teams. See
+            :class:`~socceraction.spadl.statsbomb.StatsBombTeamSchema` for the schema.
+        """
+        cols = ["team_id", "team_name"]
+        obj = self._lineups(game_id)
+        return cast(DataFrame[StatsBombTeamSchema], pd.DataFrame(obj)[cols])
+
+    def players(self, game_id: int) -> DataFrame[StatsBombPlayerSchema]:
+        """Return a dataframe with all players that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Raises
+        ------
+        ParseError  # noqa: DAR402
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all players. See
+            :class:`~socceraction.spadl.statsbomb.StatsBombPlayerSchema` for the schema.
+        """
+        cols = [
+            "game_id",
+            "team_id",
+            "player_id",
+            "player_name",
+            "nickname",
+            "jersey_number",
+            "is_starter",
+            "starting_position_id",
+            "starting_position_name",
+            "minutes_played",
+        ]
+
+        obj = self._lineups(game_id)
+        playersdf = pd.DataFrame(_flatten_id(p) for lineup in obj for p in lineup["lineup"])
+        playergamesdf = extract_player_games(self.events(game_id))
+        playersdf = pd.merge(
+            playersdf,
+            playergamesdf[
+                ["player_id", "team_id", "position_id", "position_name", "minutes_played"]
+            ],
+            on="player_id",
+        )
+        playersdf["game_id"] = game_id
+        playersdf["position_name"] = playersdf["position_name"].replace(0, "Substitute")
+        playersdf["position_id"] = playersdf["position_id"].fillna(0).astype(int)
+        playersdf["is_starter"] = playersdf["position_id"] != 0
+        playersdf.rename(
+            columns={
+                "player_nickname": "nickname",
+                "country_name": "country",
+                "position_id": "starting_position_id",
+                "position_name": "starting_position_name",
+            },
+            inplace=True,
+        )
+        return cast(DataFrame[StatsBombPlayerSchema], playersdf[cols])
+
+    def events(self, game_id: int, load_360: bool = False) -> DataFrame[StatsBombEventSchema]:
+        """Return a dataframe with the event stream of a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+        load_360 : bool
+            Whether to load the 360 data.
+
+        Raises
+        ------
+        ParseError
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing the event stream. See
+            :class:`~socceraction.spadl.statsbomb.StatsBombEventSchema` for the schema.
+        """
+        cols = [
+            "game_id",
+            "event_id",
+            "period_id",
+            "team_id",
+            "player_id",
+            "type_id",
+            "type_name",
+            "index",
+            "timestamp",
+            "minute",
+            "second",
+            "possession",
+            "possession_team_id",
+            "possession_team_name",
+            "play_pattern_id",
+            "play_pattern_name",
+            "team_name",
+            "duration",
+            "extra",
+            "related_events",
+            "player_name",
+            "position_id",
+            "position_name",
+            "location",
+            "under_pressure",
+            "counterpress",
+        ]
+        # Load the events
+        if self._local:
+            obj = _localloadjson(str(os.path.join(self._root, "events", f"{game_id}.json")))
+        else:
+            obj = list(sb.events(game_id, fmt="dict", creds=self._creds).values())
+        if not isinstance(obj, list):
+            raise ParseError("The retrieved data should contain a list of events")
+        if len(obj) == 0:
+            return cast(DataFrame[StatsBombEventSchema], pd.DataFrame(columns=cols))
+
+        eventsdf = pd.DataFrame(_flatten_id(e) for e in obj)
+        eventsdf["match_id"] = game_id
+        eventsdf["timestamp"] = pd.to_timedelta(eventsdf["timestamp"])
+        eventsdf["related_events"] = eventsdf["related_events"].apply(
+            lambda d: d if isinstance(d, list) else []
+        )
+        eventsdf["under_pressure"] = eventsdf["under_pressure"].fillna(False).astype(bool)
+        eventsdf["counterpress"] = eventsdf["counterpress"].fillna(False).astype(bool)
+        eventsdf.rename(
+            columns={"id": "event_id", "period": "period_id", "match_id": "game_id"},
+            inplace=True,
+        )
+        if not load_360:
+            return cast(DataFrame[StatsBombEventSchema], eventsdf[cols])
+
+        # Load the 360 data
+        cols_360 = ["visible_area_360", "freeze_frame_360"]
+        if self._local:
+            obj = _localloadjson(str(os.path.join(self._root, "three-sixty", f"{game_id}.json")))
+        else:
+            obj = sb.frames(game_id, fmt="dict", creds=self._creds)
+        if not isinstance(obj, list):
+            raise ParseError("The retrieved data should contain a list of frames")
+        if len(obj) == 0:
+            eventsdf["visible_area_360"] = None
+            eventsdf["freeze_frame_360"] = None
+            return cast(DataFrame[StatsBombEventSchema], eventsdf[cols + cols_360])
+        framesdf = pd.DataFrame(obj).rename(
+            columns={
+                "event_uuid": "event_id",
+                "visible_area": "visible_area_360",
+                "freeze_frame": "freeze_frame_360",
+            },
+        )[["event_id", "visible_area_360", "freeze_frame_360"]]
+        return cast(
+            DataFrame[StatsBombEventSchema],
+            pd.merge(eventsdf, framesdf, on="event_id", how="left")[cols + cols_360],
+        )
+
+
+def extract_player_games(events: pd.DataFrame) -> pd.DataFrame:
+    """Extract player games [player_id, game_id, minutes_played] from statsbomb match events.
+
+    Parameters
+    ----------
+    events : pd.DataFrame
+        DataFrame containing StatsBomb events of a single game.
+
+    Returns
+    -------
+    player_games : pd.DataFrame
+        A DataFrame with the number of minutes played by each player during the game.
+    """
+    # get duration of each period
+    periods = pd.DataFrame(
+        [
+            {"period_id": 1, "minute": 45},
+            {"period_id": 2, "minute": 45},
+            {"period_id": 3, "minute": 15},
+            {"period_id": 4, "minute": 15},
+            # Shoot-outs should not contritbute to minutes played
+            # {"period_id": 5, "minute": 0},
+        ]
+    ).set_index("period_id")
+    periods_minutes = (
+        events.loc[events.type_name == "Half End", ["period_id", "minute"]]
+        .drop_duplicates()
+        .set_index("period_id")
+        .sort_index()
+        .subtract(periods.cumsum().shift(1).fillna(0))
+        .minute.dropna()
+        .astype(int)
+        .tolist()
+    )
+    # get duration of entire match
+    game_minutes = sum(periods_minutes)
+
+    game_id = events.game_id.mode().values[0]
+    players = {}
+    # Red cards
+    red_cards = events[
+        events.apply(
+            lambda x: any(
+                e in x.extra
+                and "card" in x.extra[e]
+                and x.extra[e]["card"]["name"] in ["Second Yellow", "Red Card"]
+                for e in ["foul_committed", "bad_behaviour"]
+            ),
+            axis=1,
+        )
+    ]
+    # stats for starting XI
+    for startxi in events[events.type_name == "Starting XI"].itertuples():
+        team_id, team_name = startxi.team_id, startxi.team_name
+        for player in startxi.extra["tactics"]["lineup"]:
+            player = _flatten_id(player)
+            player = {
+                **player,
+                **{
+                    "game_id": game_id,
+                    "team_id": team_id,
+                    "team_name": team_name,
+                    "minutes_played": game_minutes,
+                },
+            }
+            player_red_card = red_cards[red_cards.player_id == player["player_id"]]
+            if len(player_red_card) > 0:
+                red_card_minute = player_red_card.iloc[0].minute
+                player["minutes_played"] = _expand_minute(red_card_minute, periods_minutes)
+            players[player["player_id"]] = player
+    # stats for substitutions
+    for substitution in events[events.type_name == "Substitution"].itertuples():
+        exp_sub_minute = _expand_minute(substitution.minute, periods_minutes)
+        replacement = {
+            "player_id": substitution.extra["substitution"]["replacement"]["id"],
+            "player_name": substitution.extra["substitution"]["replacement"]["name"],
+            "minutes_played": game_minutes - exp_sub_minute,
+            "team_id": substitution.team_id,
+            "game_id": game_id,
+            "team_name": substitution.team_name,
+        }
+        player_red_card = red_cards[red_cards.player_id == replacement["player_id"]]
+        if len(player_red_card) > 0:
+            red_card_minute = player_red_card.iloc[0].minute
+            replacement["minutes_played"] = (
+                _expand_minute(red_card_minute, periods_minutes) - exp_sub_minute
+            )
+        players[replacement["player_id"]] = replacement
+        players[substitution.player_id]["minutes_played"] = exp_sub_minute
+    pg = pd.DataFrame(players.values()).fillna(0)
+    for col in pg.columns:
+        if "_id" in col:
+            pg[col] = pg[col].astype(int)  # pylint: disable=E1136,E1137
+    return pg
+
+
+def _flatten_id(d: dict[str, dict[str, Any]]) -> dict[str, Any]:
+    newd = {}
+    extra = {}
+    for k, v in d.items():
+        if isinstance(v, dict):
+            if "id" in v and "name" in v:
+                newd[k + "_id"] = v["id"]
+                newd[k + "_name"] = v["name"]
+            else:
+                extra[k] = v
+        else:
+            newd[k] = v
+    newd["extra"] = extra
+    return newd
+
+
+def _flatten(d: dict[str, dict[str, Any]]) -> dict[str, Any]:
+    newd = {}
+    for k, v in d.items():
+        if isinstance(v, dict):
+            if "id" in v and "name" in v:
+                newd[k + "_id"] = v["id"]
+                newd[k + "_name"] = v["name"]
+                newd[k + "_extra"] = {l: w for (l, w) in v.items() if l in ("id", "name")}
+            else:
+                newd = {**newd, **_flatten(v)}
+        else:
+            newd[k] = v
+    return newd

data/statsbomb/schema.py

@@ -0,0 +1,100 @@
+"""SPADL schema for StatsBomb data."""
+
+from typing import Optional
+
+import pandera as pa
+from pandera.typing import Object, Series, Timedelta
+
+from socceraction.data.schema import (
+    CompetitionSchema,
+    EventSchema,
+    GameSchema,
+    PlayerSchema,
+    TeamSchema,
+)
+
+
+class StatsBombCompetitionSchema(CompetitionSchema):
+    """Definition of a dataframe containing a list of competitions and seasons."""
+
+    country_name: Series[str]
+    """The name of the country the competition relates to."""
+    competition_gender: Series[str]
+    """The gender of the players competing in the competition."""
+
+
+class StatsBombGameSchema(GameSchema):
+    """Definition of a dataframe containing a list of games."""
+
+    competition_stage: Series[str]
+    """The name of the phase of the competition this game is in."""
+    home_score: Series[int]
+    """The final score of the home team."""
+    away_score: Series[int]
+    """The final score of the away team."""
+    venue: Series[str] = pa.Field(nullable=True)
+    """The name of the stadium where the game was played."""
+    referee: Series[str] = pa.Field(nullable=True)
+    """The name of the referee."""
+
+
+class StatsBombPlayerSchema(PlayerSchema):
+    """Definition of a dataframe containing the list of players of a game."""
+
+    nickname: Series[str] = pa.Field(nullable=True)
+    """The nickname of the player on the team."""
+    starting_position_id: Series[int]
+    """The unique identifier for the starting position of the player on the team."""
+    starting_position_name: Series[str]
+    """The name of the starting position of the player on the team."""
+
+
+class StatsBombTeamSchema(TeamSchema):
+    """Definition of a dataframe containing the list of teams of a game."""
+
+
+class StatsBombEventSchema(EventSchema):
+    """Definition of a dataframe containing event stream data of a game."""
+
+    index: Series[int]
+    """Sequence notation for the ordering of events within each match."""
+    timestamp: Series[Timedelta]
+    """Time in the match the event takes place, recorded to the millisecond."""
+    minute: Series[int]
+    """The minutes on the clock at the time of this event."""
+    second: Series[int] = pa.Field(ge=0, le=59)
+    """The second part of the timestamp."""
+    possession: Series[int]
+    """Indicates the current unique possession in the game."""
+    possession_team_id: Series[int]
+    """The ID of the team that started this possession in control of the ball."""
+    possession_team_name: Series[str]
+    """The name of the team that started this possession in control of the ball."""
+    play_pattern_id: Series[int]
+    """The ID of the play pattern relevant to this event."""
+    play_pattern_name: Series[str]
+    """The name of the play pattern relevant to this event."""
+    team_name: Series[str]
+    """The name of the team this event relates to."""
+    duration: Series[float] = pa.Field(nullable=True)
+    """If relevant, the length in seconds the event lasted."""
+    extra: Series[Object]
+    """A JSON string containing type-specific information."""
+    related_events: Series[Object]
+    """A comma separated list of the IDs of related events."""
+    player_name: Series[str] = pa.Field(nullable=True)
+    """The name of the player this event relates to."""
+    position_id: Series[float] = pa.Field(nullable=True)
+    """The ID of the position the player was in at the time of this event."""
+    position_name: Series[str] = pa.Field(nullable=True)
+    """The name of the position the player was in at the time of this event."""
+    location: Series[Object] = pa.Field(nullable=True)
+    """Array containing the x and y coordinates of the event."""
+    under_pressure: Series[bool] = pa.Field(nullable=True)
+    """Whether the action was performed while being pressured by an opponent."""
+    counterpress: Series[bool] = pa.Field(nullable=True)
+    """Pressing actions within 5 seconds of an open play turnover."""
+    visible_area_360: Optional[Series[Object]] = pa.Field(nullable=True)
+    """An array of coordinates describing the polygon visible to the camera / in the 360 frame."""
+    freeze_frame_360: Optional[Series[Object]] = pa.Field(nullable=True)
+    """An array of freeze frame objects."""

data/wyscout/__init__.py

@@ -0,0 +1,20 @@
+"""Module for loading Wyscout event data."""
+
+__all__ = [
+    "PublicWyscoutLoader",
+    "WyscoutLoader",
+    "WyscoutCompetitionSchema",
+    "WyscoutGameSchema",
+    "WyscoutPlayerSchema",
+    "WyscoutTeamSchema",
+    "WyscoutEventSchema",
+]
+
+from .loader import PublicWyscoutLoader, WyscoutLoader
+from .schema import (
+    WyscoutCompetitionSchema,
+    WyscoutEventSchema,
+    WyscoutGameSchema,
+    WyscoutPlayerSchema,
+    WyscoutTeamSchema,
+)

data/wyscout/loader.py

@@ -0,0 +1,849 @@
+"""Implements serializers for Wyscout data."""
+
+import glob
+import os
+import re
+import warnings
+from pathlib import Path
+from typing import Any, Callable, Optional, Union, cast
+from urllib.error import HTTPError
+from urllib.parse import urlparse
+from urllib.request import urlopen, urlretrieve
+from zipfile import ZipFile, is_zipfile
+
+import pandas as pd  # type: ignore
+from pandera.typing import DataFrame
+
+from ..base import (
+    EventDataLoader,
+    JSONType,
+    MissingDataError,
+    ParseError,
+    _auth_remoteloadjson,
+    _expand_minute,
+    _has_auth,
+    _localloadjson,
+    _remoteloadjson,
+)
+from .schema import (
+    WyscoutCompetitionSchema,
+    WyscoutEventSchema,
+    WyscoutGameSchema,
+    WyscoutPlayerSchema,
+    WyscoutTeamSchema,
+)
+
+
+class PublicWyscoutLoader(EventDataLoader):
+    """
+    Load the public Wyscout dataset.
+
+    This dataset is a public release of event stream data, collected by Wyscout
+    (https://wyscout.com/) containing all matches of the 2017/18 season of the
+    top-5 European leagues (La Liga, Serie A, Bundesliga, Premier League, Ligue
+    1), the FIFA World Cup 2018, and UEFA Euro Cup 2016. For a detailed
+    description, see Pappalardo et al. [1]_.
+
+    Parameters
+    ----------
+    root : str
+        Path where a local copy of the dataset is stored or where the
+        downloaded dataset should be stored.
+    download : bool
+        Whether to force a redownload of the data.
+
+    References
+    ----------
+    .. [1] Pappalardo, L., Cintia, P., Rossi, A. et al. A public data set of
+        spatio-temporal match events in soccer competitions. Sci Data 6, 236
+        (2019). https://doi.org/10.1038/s41597-019-0247-7
+    """
+
+    def __init__(self, root: Optional[str] = None, download: bool = False) -> None:
+        if root is None:
+            self.root = os.path.join(os.getcwd(), "wyscout_data")
+            os.makedirs(self.root, exist_ok=True)
+        else:
+            self.root = root
+
+        self.get = _localloadjson
+
+        if download or len(os.listdir(self.root)) == 0:
+            self._download_repo()
+
+        self._index = pd.DataFrame(
+            [
+                {
+                    "competition_id": 524,
+                    "season_id": 181248,
+                    "season_name": "2017/2018",
+                    "db_matches": "matches_Italy.json",
+                    "db_events": "events_Italy.json",
+                },
+                {
+                    "competition_id": 364,
+                    "season_id": 181150,
+                    "season_name": "2017/2018",
+                    "db_matches": "matches_England.json",
+                    "db_events": "events_England.json",
+                },
+                {
+                    "competition_id": 795,
+                    "season_id": 181144,
+                    "season_name": "2017/2018",
+                    "db_matches": "matches_Spain.json",
+                    "db_events": "events_Spain.json",
+                },
+                {
+                    "competition_id": 412,
+                    "season_id": 181189,
+                    "season_name": "2017/2018",
+                    "db_matches": "matches_France.json",
+                    "db_events": "events_France.json",
+                },
+                {
+                    "competition_id": 426,
+                    "season_id": 181137,
+                    "season_name": "2017/2018",
+                    "db_matches": "matches_Germany.json",
+                    "db_events": "events_Germany.json",
+                },
+                {
+                    "competition_id": 102,
+                    "season_id": 9291,
+                    "season_name": "2016",
+                    "db_matches": "matches_European_Championship.json",
+                    "db_events": "events_European_Championship.json",
+                },
+                {
+                    "competition_id": 28,
+                    "season_id": 10078,
+                    "season_name": "2018",
+                    "db_matches": "matches_World_Cup.json",
+                    "db_events": "events_World_Cup.json",
+                },
+            ]
+        ).set_index(["competition_id", "season_id"])
+        self._match_index = self._create_match_index().set_index("match_id")
+        self._cache: Optional[dict[str, Any]] = None
+
+    def _download_repo(self) -> None:
+        dataset_urls = {
+            "competitions": "https://ndownloader.figshare.com/files/15073685",
+            "teams": "https://ndownloader.figshare.com/files/15073697",
+            "players": "https://ndownloader.figshare.com/files/15073721",
+            "matches": "https://ndownloader.figshare.com/files/14464622",
+            "events": "https://ndownloader.figshare.com/files/14464685",
+        }
+        # download and unzip Wyscout open data
+        for url in dataset_urls.values():
+            url_obj = urlopen(url).geturl()
+            path = Path(urlparse(url_obj).path)
+            file_name = os.path.join(self.root, path.name)
+            file_local, _ = urlretrieve(url_obj, file_name)
+            if is_zipfile(file_local):
+                with ZipFile(file_local) as zip_file:
+                    zip_file.extractall(self.root)
+
+    def _create_match_index(self) -> pd.DataFrame:
+        df_matches = pd.concat(
+            [pd.DataFrame(self.get(path)) for path in glob.iglob(f"{self.root}/matches_*.json")]
+        )
+        df_matches.rename(
+            columns={
+                "wyId": "match_id",
+                "competitionId": "competition_id",
+                "seasonId": "season_id",
+            },
+            inplace=True,
+        )
+        return pd.merge(
+            df_matches[["match_id", "competition_id", "season_id"]],
+            self._index,
+            on=["competition_id", "season_id"],
+            how="left",
+        )
+
+    def competitions(self) -> DataFrame[WyscoutCompetitionSchema]:
+        """Return a dataframe with all available competitions and seasons.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available competitions and seasons. See
+            :class:`~socceraction.spadl.wyscout.WyscoutCompetitionSchema` for the schema.
+        """
+        path = os.path.join(self.root, "competitions.json")
+        df_competitions = pd.DataFrame(self.get(path))
+        df_competitions.rename(
+            columns={"wyId": "competition_id", "name": "competition_name"}, inplace=True
+        )
+        df_competitions["country_name"] = df_competitions.apply(
+            lambda x: x.area["name"] if x.area["name"] != "" else "International", axis=1
+        )
+        df_competitions["competition_gender"] = "male"
+        df_competitions = pd.merge(
+            df_competitions,
+            self._index.reset_index()[["competition_id", "season_id", "season_name"]],
+            on="competition_id",
+            how="left",
+        )
+        return cast(
+            DataFrame[WyscoutCompetitionSchema],
+            df_competitions.reset_index()[
+                [
+                    "competition_id",
+                    "season_id",
+                    "country_name",
+                    "competition_name",
+                    "competition_gender",
+                    "season_name",
+                ]
+            ],
+        )
+
+    def games(self, competition_id: int, season_id: int) -> DataFrame[WyscoutGameSchema]:
+        """Return a dataframe with all available games in a season.
+
+        Parameters
+        ----------
+        competition_id : int
+            The ID of the competition.
+        season_id : int
+            The ID of the season.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available games. See
+            :class:`~socceraction.spadl.wyscout.WyscoutGameSchema` for the schema.
+        """
+        path = os.path.join(self.root, self._index.at[(competition_id, season_id), "db_matches"])
+        df_matches = pd.DataFrame(self.get(path))
+        return cast(DataFrame[WyscoutGameSchema], _convert_games(df_matches))
+
+    def _lineups(self, game_id: int) -> list[dict[str, Any]]:
+        competition_id, season_id = self._match_index.loc[game_id, ["competition_id", "season_id"]]
+        path = os.path.join(self.root, self._index.at[(competition_id, season_id), "db_matches"])
+        df_matches = pd.DataFrame(self.get(path)).set_index("wyId")
+        return list(df_matches.at[game_id, "teamsData"].values())
+
+    def teams(self, game_id: int) -> DataFrame[WyscoutTeamSchema]:
+        """Return a dataframe with both teams that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing both teams. See
+            :class:`~socceraction.spadl.wyscout.WyscoutTeamSchema` for the schema.
+        """
+        path = os.path.join(self.root, "teams.json")
+        df_teams = pd.DataFrame(self.get(path)).set_index("wyId")
+        df_teams_match_id = pd.DataFrame(self._lineups(game_id))["teamId"]
+        df_teams_match = df_teams.loc[df_teams_match_id].reset_index()
+        return cast(DataFrame[WyscoutTeamSchema], _convert_teams(df_teams_match))
+
+    def players(self, game_id: int) -> DataFrame[WyscoutPlayerSchema]:
+        """Return a dataframe with all players that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all players. See
+            :class:`~socceraction.spadl.wyscout.WyscoutPlayerSchema` for the schema.
+        """
+        path = os.path.join(self.root, "players.json")
+        df_players = pd.DataFrame(self.get(path)).set_index("wyId")
+        lineups = self._lineups(game_id)
+        players_match = []
+        for team in lineups:
+            playerlist = team["formation"]["lineup"]
+            if team["formation"]["substitutions"] != "null":
+                for p in team["formation"]["substitutions"]:
+                    try:
+                        playerlist.append(
+                            next(
+                                item
+                                for item in team["formation"]["bench"]
+                                if item["playerId"] == p["playerIn"]
+                            )
+                        )
+                    except StopIteration:
+                        warnings.warn(
+                            f'A player with ID={p["playerIn"]} was substituted '
+                            f'in the {p["minute"]}th minute of game {game_id}, but '
+                            "could not be found on the bench."
+                        )
+            df = pd.DataFrame(playerlist)
+            df["side"] = team["side"]
+            df["team_id"] = team["teamId"]
+            players_match.append(df)
+        df_players_match = (
+            pd.concat(players_match)
+            .rename(columns={"playerId": "wyId"})
+            .set_index("wyId")
+            .join(df_players, how="left")
+        )
+        df_players_match.reset_index(inplace=True)
+        for c in ["shortName", "lastName", "firstName"]:
+            df_players_match[c] = df_players_match[c].apply(
+                lambda x: x.encode().decode("unicode-escape")
+            )
+        df_players_match = _convert_players(df_players_match)
+
+        # get minutes played
+        competition_id, season_id = self._match_index.loc[game_id, ["competition_id", "season_id"]]
+        path = os.path.join(self.root, self._index.at[(competition_id, season_id), "db_events"])
+        if self._cache is not None and self._cache["path"] == path:
+            df_events = self._cache["events"]
+        else:
+            df_events = pd.DataFrame(self.get(path)).set_index("matchId")
+            # avoid that this large json file has to be parsed again for
+            # each game when loading a batch of games from the same season
+            self._cache = {"path": path, "events": df_events}
+        match_events = df_events.loc[game_id].reset_index().to_dict("records")
+        mp = _get_minutes_played(lineups, match_events)
+        df_players_match = pd.merge(df_players_match, mp, on="player_id", how="right")
+        df_players_match["minutes_played"] = df_players_match.minutes_played.fillna(0)
+        df_players_match["game_id"] = game_id
+        return cast(DataFrame[WyscoutPlayerSchema], df_players_match)
+
+    def events(self, game_id: int) -> DataFrame[WyscoutEventSchema]:
+        """Return a dataframe with the event stream of a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing the event stream. See
+            :class:`~socceraction.spadl.wyscout.WyscoutEventSchema` for the schema.
+        """
+        competition_id, season_id = self._match_index.loc[game_id, ["competition_id", "season_id"]]
+        path = os.path.join(self.root, self._index.at[(competition_id, season_id), "db_events"])
+        if self._cache is not None and self._cache["path"] == path:
+            df_events = self._cache["events"]
+        else:
+            df_events = pd.DataFrame(self.get(path)).set_index("matchId")
+            # avoid that this large json file has to be parsed again for
+            # each game when loading a batch of games from the same season
+            self._cache = {"path": path, "events": df_events}
+        return cast(
+            DataFrame[WyscoutEventSchema], _convert_events(df_events.loc[game_id].reset_index())
+        )
+
+
+class WyscoutLoader(EventDataLoader):
+    """Load event data either from a remote location or from a local folder.
+
+    Parameters
+    ----------
+    root : str
+        Root-path of the data.
+    getter : str or callable, default: "remote"
+        "remote", "local" or a function that returns loads JSON data from a path.
+    feeds : dict(str, str)
+        Glob pattern for each feed that should be parsed. The default feeds for
+        a "remote" getter are::
+
+            {
+                'competitions': 'competitions',
+                'seasons': 'competitions/{season_id}/seasons',
+                'games': 'seasons/{season_id}/matches',
+                'events': 'matches/{game_id}/events?fetch=teams,players,match,substitutions'
+            }
+
+        The default feeds for a "local" getter are::
+
+            {
+                'competitions': 'competitions.json',
+                'seasons': 'seasons_{competition_id}.json',
+                'games': 'matches_{season_id}.json',
+                'events': 'matches/events_{game_id}.json',
+            }
+
+    creds: dict, optional
+        Login credentials in the format {"user": "", "passwd": ""}. Only used
+        when getter is "remote".
+    """
+
+    _wyscout_api: str = "https://apirest.wyscout.com/v2/"
+
+    def __init__(
+        self,
+        root: str = _wyscout_api,
+        getter: Union[str, Callable[[str], JSONType]] = "remote",
+        feeds: Optional[dict[str, str]] = None,
+        creds: Optional[dict[str, str]] = None,
+    ) -> None:
+        self.root = root
+
+        # Init credentials
+        if creds is None:
+            creds = {
+                "user": os.environ.get("WY_USERNAME", ""),
+                "passwd": os.environ.get("WY_PASSWORD", ""),
+            }
+
+        # Init getter
+        if getter == "remote":
+            self.get = _remoteloadjson
+            if _has_auth(creds):
+                _auth_remoteloadjson(creds["user"], creds["passwd"])
+        elif getter == "local":
+            self.get = _localloadjson
+        else:
+            self.get = getter  # type: ignore
+
+        # Set up feeds
+        if feeds is not None:
+            self.feeds = feeds
+        elif getter == "remote":
+            self.feeds = {
+                "seasons": "competitions/{competition_id}/seasons?fetch=competition",
+                "games": "seasons/{season_id}/matches",
+                "events": "matches/{game_id}/events?fetch=teams,players,match,coaches,referees,formations,substitutions",  # noqa: B950
+            }
+        elif getter == "local":
+            self.feeds = {
+                "competitions": "competitions.json",
+                "seasons": "seasons_{competition_id}.json",
+                "games": "matches_{season_id}.json",
+                "events": "matches/events_{game_id}.json",
+            }
+        else:
+            raise ValueError("No feeds specified.")
+
+    def _get_file_or_url(
+        self,
+        feed: str,
+        competition_id: Optional[int] = None,
+        season_id: Optional[int] = None,
+        game_id: Optional[int] = None,
+    ) -> list[str]:
+        competition_id_glob = "*" if competition_id is None else competition_id
+        season_id_glob = "*" if season_id is None else season_id
+        game_id_glob = "*" if game_id is None else game_id
+        glob_pattern = self.feeds[feed].format(
+            competition_id=competition_id_glob, season_id=season_id_glob, game_id=game_id_glob
+        )
+        if "*" in glob_pattern:
+            files = glob.glob(os.path.join(self.root, glob_pattern))
+            if len(files) == 0:
+                raise MissingDataError
+            return files
+        return [glob_pattern]
+
+    def competitions(
+        self, competition_id: Optional[int] = None
+    ) -> DataFrame[WyscoutCompetitionSchema]:
+        """Return a dataframe with all available competitions and seasons.
+
+        Parameters
+        ----------
+        competition_id : int, optional
+            The ID of the competition.
+
+        Raises
+        ------
+        ParseError
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available competitions and seasons. See
+            :class:`~socceraction.spadl.wyscout.WyscoutCompetitionSchema` for the schema.
+        """
+        # Get all competitions
+        if "competitions" in self.feeds:
+            competitions_url = self._get_file_or_url("competitions")[0]
+            path = os.path.join(self.root, competitions_url)
+            obj = self.get(path)
+            if not isinstance(obj, dict) or "competitions" not in obj:
+                raise ParseError(f"{path} should contain a list of competitions")
+            seasons_urls = [
+                self._get_file_or_url("seasons", competition_id=c["wyId"])[0]
+                for c in obj["competitions"]
+            ]
+        else:
+            seasons_urls = self._get_file_or_url("seasons", competition_id=competition_id)
+        # Get seasons in each competition
+        competitions = []
+        seasons = []
+        for seasons_url in seasons_urls:
+            try:
+                path = os.path.join(self.root, seasons_url)
+                obj = self.get(path)
+                if not isinstance(obj, dict) or "competition" not in obj or "seasons" not in obj:
+                    raise ParseError(
+                        f"{path} should contain a list of competition and list of seasons"
+                    )
+                competitions.append(obj["competition"])
+                seasons.extend([s["season"] for s in obj["seasons"]])
+            except FileNotFoundError:
+                warnings.warn(f"File not found: {seasons_url}")
+        df_competitions = _convert_competitions(pd.DataFrame(competitions))
+        df_seasons = _convert_seasons(pd.DataFrame(seasons))
+        # Merge into a single dataframe
+        return cast(
+            DataFrame[WyscoutCompetitionSchema],
+            pd.merge(df_competitions, df_seasons, on="competition_id"),
+        )
+
+    def games(self, competition_id: int, season_id: int) -> DataFrame[WyscoutGameSchema]:
+        """Return a dataframe with all available games in a season.
+
+        Parameters
+        ----------
+        competition_id : int
+            The ID of the competition.
+        season_id : int
+            The ID of the season.
+
+        Raises
+        ------
+        ParseError
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all available games. See
+            :class:`~socceraction.spadl.wyscout.WyscoutGameSchema` for the schema.
+        """
+        # Get all games
+        if "games" in self.feeds:
+            games_url = self._get_file_or_url(
+                "games", competition_id=competition_id, season_id=season_id
+            )[0]
+            path = os.path.join(self.root, games_url)
+            obj = self.get(path)
+            if not isinstance(obj, dict) or "matches" not in obj:
+                raise ParseError(f"{path} should contain a list of matches")
+            gamedetails_urls = [
+                self._get_file_or_url(
+                    "events",
+                    competition_id=competition_id,
+                    season_id=season_id,
+                    game_id=g["matchId"],
+                )[0]
+                for g in obj["matches"]
+            ]
+        else:
+            gamedetails_urls = self._get_file_or_url(
+                "events", competition_id=competition_id, season_id=season_id
+            )
+        games = []
+        for gamedetails_url in gamedetails_urls:
+            try:
+                path = os.path.join(self.root, gamedetails_url)
+                obj = self.get(path)
+                if not isinstance(obj, dict) or "match" not in obj:
+                    raise ParseError(f"{path} should contain a match")
+                games.append(obj["match"])
+            except FileNotFoundError:
+                warnings.warn(f"File not found: {gamedetails_url}")
+            except HTTPError:
+                warnings.warn(f"Resource not found: {gamedetails_url}")
+        df_games = _convert_games(pd.DataFrame(games))
+        return cast(DataFrame[WyscoutGameSchema], df_games)
+
+    def teams(self, game_id: int) -> DataFrame[WyscoutTeamSchema]:
+        """Return a dataframe with both teams that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Raises
+        ------
+        ParseError
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing both teams. See
+            :class:`~socceraction.spadl.wyscout.WyscoutTeamSchema` for the schema.
+        """
+        events_url = self._get_file_or_url("events", game_id=game_id)[0]
+        path = os.path.join(self.root, events_url)
+        obj = self.get(path)
+        if not isinstance(obj, dict) or "teams" not in obj:
+            raise ParseError(f"{path} should contain a list of matches")
+        teams = [t["team"] for t in obj["teams"].values() if t.get("team")]
+        df_teams = _convert_teams(pd.DataFrame(teams))
+        return cast(DataFrame[WyscoutTeamSchema], df_teams)
+
+    def players(self, game_id: int) -> DataFrame[WyscoutPlayerSchema]:
+        """Return a dataframe with all players that participated in a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Raises
+        ------
+        ParseError
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing all players. See
+            :class:`~socceraction.spadl.wyscout.WyscoutPlayerSchema` for the schema.
+        """
+        events_url = self._get_file_or_url("events", game_id=game_id)[0]
+        path = os.path.join(self.root, events_url)
+        obj = self.get(path)
+        if not isinstance(obj, dict) or "players" not in obj:
+            raise ParseError(f"{path} should contain a list of players")
+        players = [
+            player["player"]
+            for team in obj["players"].values()
+            for player in team
+            if player.get("player")
+        ]
+        df_players = _convert_players(pd.DataFrame(players).drop_duplicates("wyId"))
+        df_players = pd.merge(
+            df_players,
+            _get_minutes_played(obj["match"]["teamsData"], obj["events"]),
+            on="player_id",
+            how="right",
+        )
+        df_players["minutes_played"] = df_players.minutes_played.fillna(0)
+        df_players["game_id"] = game_id
+        return cast(DataFrame[WyscoutPlayerSchema], df_players)
+
+    def events(self, game_id: int) -> DataFrame[WyscoutEventSchema]:
+        """Return a dataframe with the event stream of a game.
+
+        Parameters
+        ----------
+        game_id : int
+            The ID of the game.
+
+        Raises
+        ------
+        ParseError
+            When the raw data does not adhere to the expected format.
+
+        Returns
+        -------
+        pd.DataFrame
+            A dataframe containing the event stream. See
+            :class:`~socceraction.spadl.wyscout.WyscoutEventSchema` for the schema.
+        """
+        events_url = self._get_file_or_url("events", game_id=game_id)[0]
+        path = os.path.join(self.root, events_url)
+        obj = self.get(path)
+        if not isinstance(obj, dict) or "events" not in obj:
+            raise ParseError(f"{path} should contain a list of events")
+        df_events = _convert_events(pd.DataFrame(obj["events"]))
+        return cast(DataFrame[WyscoutEventSchema], df_events)
+
+
+def _convert_competitions(competitions: pd.DataFrame) -> pd.DataFrame:
+    competitionsmapping = {
+        "wyId": "competition_id",
+        "name": "competition_name",
+        "gender": "competition_gender",
+    }
+    cols = ["competition_id", "competition_name", "country_name", "competition_gender"]
+    competitions["country_name"] = competitions.apply(
+        lambda x: x.area["name"] if x.area["name"] != "" else "International", axis=1
+    )
+    competitions = competitions.rename(columns=competitionsmapping)[cols]
+    return competitions
+
+
+def _convert_seasons(seasons: pd.DataFrame) -> pd.DataFrame:
+    seasonsmapping = {
+        "wyId": "season_id",
+        "name": "season_name",
+        "competitionId": "competition_id",
+    }
+    cols = ["season_id", "season_name", "competition_id"]
+    seasons = seasons.rename(columns=seasonsmapping)[cols]
+    return seasons
+
+
+def _convert_games(matches: pd.DataFrame) -> pd.DataFrame:
+    gamesmapping = {
+        "wyId": "game_id",
+        "dateutc": "game_date",
+        "competitionId": "competition_id",
+        "seasonId": "season_id",
+        "gameweek": "game_day",
+    }
+    cols = ["game_id", "competition_id", "season_id", "game_date", "game_day"]
+    games = matches.rename(columns=gamesmapping)[cols]
+    games["game_date"] = pd.to_datetime(games["game_date"])
+    games["home_team_id"] = matches.teamsData.apply(lambda x: _get_team_id(x, "home"))
+    games["away_team_id"] = matches.teamsData.apply(lambda x: _get_team_id(x, "away"))
+    return games
+
+
+def _get_team_id(teamsData: dict[int, Any], side: str) -> int:
+    for team_id, data in teamsData.items():
+        if data["side"] == side:
+            return int(team_id)
+    raise ValueError()
+
+
+def _convert_players(players: pd.DataFrame) -> pd.DataFrame:
+    playermapping = {
+        "wyId": "player_id",
+        "shortName": "nickname",
+        "firstName": "firstname",
+        "lastName": "lastname",
+        "birthDate": "birth_date",
+    }
+    cols = ["player_id", "nickname", "firstname", "lastname", "birth_date"]
+    df_players = players.rename(columns=playermapping)[cols]
+    df_players["player_name"] = df_players[["firstname", "lastname"]].agg(" ".join, axis=1)
+    df_players["birth_date"] = pd.to_datetime(df_players["birth_date"])
+    return df_players
+
+
+def _convert_teams(teams: pd.DataFrame) -> pd.DataFrame:
+    teammapping = {
+        "wyId": "team_id",
+        "name": "team_name_short",
+        "officialName": "team_name",
+    }
+    cols = ["team_id", "team_name_short", "team_name"]
+    return teams.rename(columns=teammapping)[cols]
+
+
+def _convert_events(raw_events: pd.DataFrame) -> pd.DataFrame:
+    eventmapping = {
+        "id": "event_id",
+        "match_id": "game_id",
+        "event_name": "type_name",
+        "sub_event_name": "subtype_name",
+    }
+    cols = [
+        "event_id",
+        "game_id",
+        "period_id",
+        "milliseconds",
+        "team_id",
+        "player_id",
+        "type_id",
+        "type_name",
+        "subtype_id",
+        "subtype_name",
+        "positions",
+        "tags",
+    ]
+    events = raw_events.copy()
+    # Camel case to snake case column names
+    pattern = re.compile(r"(?<!^)(?=[A-Z])")
+    events.columns = [pattern.sub("_", c).lower() for c in events.columns]
+    #
+    events["type_id"] = (
+        pd.to_numeric(
+            events["event_id"] if "event_id" in events.columns else None, errors="coerce"
+        )
+        .fillna(0)
+        .astype(int)
+    )
+    del events["event_id"]
+    events["subtype_id"] = (
+        pd.to_numeric(
+            events["sub_event_id"] if "sub_event_id" in events.columns else None, errors="coerce"
+        )
+        .fillna(0)
+        .astype(int)
+    )
+    del events["sub_event_id"]
+    events["period_id"] = events.match_period.apply(lambda x: wyscout_periods[x])
+    events["milliseconds"] = events.event_sec * 1000
+    return events.rename(columns=eventmapping)[cols]
+
+
+def _get_minutes_played(
+    teamsData: list[dict[str, Any]], events: list[dict[str, Any]]
+) -> pd.DataFrame:
+    # get duration of each period
+    periods_ts = {i: [0] for i in range(6)}
+    for e in events:
+        period_id = wyscout_periods[e["matchPeriod"]]
+        periods_ts[period_id].append(e["eventSec"])
+    periods_duration = [
+        round(max(periods_ts[i]) / 60) for i in range(5) if max(periods_ts[i]) != 0
+    ]
+    # get duration of entire match
+    duration = sum(periods_duration)
+
+    # get stats for each player
+    playergames: dict[int, dict[str, Any]] = {}
+    if isinstance(teamsData, dict):
+        teamsData = list(teamsData.values())
+    for teamData in teamsData:
+        formation = teamData.get("formation", {})
+        substitutions = formation.get("substitutions", [])
+        red_cards = {
+            player["playerId"]: _expand_minute(int(player["redCards"]), periods_duration)
+            for key in ["bench", "lineup"]
+            for player in formation.get(key, [])
+            if player["redCards"] != "0"
+        }
+        pg = {
+            player["playerId"]: {
+                "team_id": teamData["teamId"],
+                "player_id": player["playerId"],
+                "jersey_number": player.get("shirtNumber", 0),
+                "minutes_played": red_cards.get(player["playerId"], duration),
+                "is_starter": True,
+            }
+            for player in formation.get("lineup", [])
+        }
+
+        # correct minutes played for substituted players
+        if substitutions != "null":
+            for substitution in substitutions:
+                expanded_minute_sub = _expand_minute(substitution["minute"], periods_duration)
+                substitute = {
+                    "team_id": teamData["teamId"],
+                    "player_id": substitution["playerIn"],
+                    "jersey_number": next(
+                        (
+                            p.get("shirtNumber", 0)
+                            for p in formation.get("bench", [])
+                            if p["playerId"] == substitution["playerIn"]
+                        ),
+                        0,
+                    ),
+                    "minutes_played": duration - expanded_minute_sub,
+                    "is_starter": False,
+                }
+                if substitution["playerIn"] in red_cards:
+                    substitute["minutes_played"] = (
+                        red_cards[substitution["playerIn"]] - expanded_minute_sub
+                    )
+                pg[substitution["playerIn"]] = substitute
+                pg[substitution["playerOut"]]["minutes_played"] = expanded_minute_sub
+
+        playergames = {**playergames, **pg}
+    return pd.DataFrame(playergames.values())
+
+
+wyscout_periods = {"1H": 1, "2H": 2, "E1": 3, "E2": 4, "P": 5}

data/wyscout/schema.py

@@ -0,0 +1,48 @@
+"""SPADL schema for Wyscout data."""
+
+import pandera as pa
+from pandera.typing import DateTime, Object, Series
+
+from socceraction.data.schema import (
+    CompetitionSchema,
+    EventSchema,
+    GameSchema,
+    PlayerSchema,
+    TeamSchema,
+)
+
+
+class WyscoutCompetitionSchema(CompetitionSchema):
+    """Definition of a dataframe containing a list of competitions and seasons."""
+
+    country_name: Series[str]
+    competition_gender: Series[str]
+
+
+class WyscoutGameSchema(GameSchema):
+    """Definition of a dataframe containing a list of games."""
+
+
+class WyscoutPlayerSchema(PlayerSchema):
+    """Definition of a dataframe containing the list of teams of a game."""
+
+    firstname: Series[str]
+    lastname: Series[str]
+    nickname: Series[str] = pa.Field(nullable=True)
+    birth_date: Series[DateTime] = pa.Field(nullable=True)
+
+
+class WyscoutTeamSchema(TeamSchema):
+    """Definition of a dataframe containing the list of players of a game."""
+
+    team_name_short: Series[str]
+
+
+class WyscoutEventSchema(EventSchema):
+    """Definition of a dataframe containing event stream data of a game."""
+
+    milliseconds: Series[float]
+    subtype_id: Series[int]
+    subtype_name: Series[str]
+    positions: Series[Object]
+    tags: Series[Object]

docs/_static/custom.css

@@ -0,0 +1,50 @@
+a {
+  color: #26b079;
+}
+
+.toctree-l1 a:active,
+.toctree-l1 a:hover {
+  background-color: #676767;
+}
+
+.sidebar-logo {
+  max-width: 100%;
+}
+
+.sidebar-drawer {
+  width: calc(50% - 25em);
+  min-width: 22em;
+}
+
+.sidebar-drawer .sidebar-container {
+  width: 23em;
+}
+
+li.toctree-l2 {
+  font-size: 80%;
+}
+
+@media (max-width: 67em) {
+  .sidebar-drawer {
+    width: 22em;
+    left: -22em;
+  }
+  .sidebar-drawer .sidebar-container {
+    width: 22em;
+  }
+  li.toctree-l2 {
+    font-size: 75%;
+  }
+}
+
+/* autosummary table text */
+article .align-center,
+article .align-default {
+  text-align: left;
+}
+
+table.dataframe {
+  font-size: 80%;
+  margin-left: 0;
+  margin-right: 0;
+}

docs/_static/decroos19.bibtex

@@ -0,0 +1,17 @@
+@inproceedings{Decroos2019actions,
+ author = {Decroos, Tom and Bransen, Lotte and Van Haaren, Jan and Davis, Jesse},
+ title = {Actions Speak Louder Than Goals: Valuing Player Actions in Soccer},
+ booktitle = {Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining},
+ series = {KDD '19},
+ year = {2019},
+ isbn = {978-1-4503-6201-6},
+ location = {Anchorage, AK, USA},
+ pages = {1851--1861},
+ numpages = {11},
+ url = {http://doi.acm.org/10.1145/3292500.3330758},
+ doi = {10.1145/3292500.3330758},
+ acmid = {3330758},
+ publisher = {ACM},
+ address = {New York, NY, USA},
+ keywords = {event stream data, probabilistic classification, soccer match data, sports analytics, valuing actions},
+}

docs/_static/logo.ai

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b8f4add7db11daf2a6f1c77c8d26f84c0a227bea40e9b607f2930d15b75ae99e
+size 153178

docs/_static/vanroy20.bibtex

@@ -0,0 +1,11 @@
+@conference{vanroy2020,
+  author = {Van Roy, Maaike and Robberechts, Pieter and Decroos, Tom and Davis, Jesse},
+  title = {Valuing On-the-Ball Actions in Soccer: A Critical Comparison of xT and VAEP},
+  series = {AITS},
+  booktitle = {Proceedings of the {AAAI}-20 Workshop on Artifical Intelligence in Team Sports},
+  publisher = {AI in Team Sports Organising Committee},
+  month = {dec},
+  year = {2020},
+  abstract = {Objectively quantifying a soccer player's contributions within a match is a challenging and crucial task in soccer analytics. Many of the currently available metrics focus on measuring the quality of shots and assists only, although these represent less than 1% of all on-the-ball actions. Most recently, several approaches were proposed to bridge this gap. By valuing how actions increase or decrease the likelihood of yielding a goal, these models are effective tools for quantifying the performances of players for all sorts of actions. However, we lack an understanding of their differences, both conceptually and in practice. Therefore, this paper critically compares two such models: expected threat (xT) and valuing actions by estimating probabilities (VAEP). Both approaches exhibit variety in their design choices, that leads to different top player rankings and major differences in how they value specific actions.},
+  keywords = {soccer},
+}

docs/_templates/class.rst

@@ -0,0 +1,40 @@
+{{ fullname | escape | underline}}
+
+.. currentmodule:: {{ module }}
+
+.. autoclass:: {{ objname }}
+
+   {% block attributes %}
+   {% if attributes %}
+   .. rubric:: Attributes
+
+   .. autosummary::
+      :nosignatures:
+
+   {% for item in attributes %}
+     ~{{ name }}.{{ item }}
+   {%- endfor %}
+
+   {% endif %}
+   {% endblock %}
+
+   {% block methods %}
+   {% if methods %}
+   .. rubric:: Methods
+
+   .. autosummary::
+      :nosignatures:
+      :toctree: methods
+
+   {% for item in methods %}
+   {%- if item not in inherited_members %}
+      ~{{ name }}.{{ item }}
+   {%- endif %}
+   {%- endfor %}
+   {% endif %}
+
+   {%- if members and '__call__' in members %}
+      ~{{ name }}.__call__
+   {%- endif %}
+
+   {% endblock %}

docs/_templates/module.rst

@@ -0,0 +1,27 @@
+.. empty
+
+{{ fullname | escape | underline }}
+
+.. currentmodule:: {{ fullname }}
+
+.. automodule:: {{ fullname }}
+
+   {% block classes %}
+
+     {% for item in classes %}
+        .. autoclass:: {{ item }}
+           :members:
+           :member-order: bysource
+           :show-inheritance:
+           :exclude-members:
+     {%- endfor %}
+
+   {% endblock %}
+
+   {% block functions %}
+
+     {% for item in functions %}
+        .. autofunction:: {{ item }}
+     {%- endfor %}
+
+   {% endblock %}

docs/_templates/schema.rst

@@ -0,0 +1,19 @@
+{{ fullname | escape | underline}}
+
+.. currentmodule:: {{ module }}
+
+.. autoclass:: {{ objname }}
+
+   {% block attributes %}
+   {% if attributes %}
+   .. rubric:: Attributes
+
+   .. autosummary::
+      :nosignatures:
+
+   {% for item in attributes %}
+     ~{{ name }}.{{ item }}
+   {%- endfor %}
+
+   {% endif %}
+   {% endblock %}