The tech we use

Data pipeline

The data aggregation pipeline is built in python, using a variety of open source tools:

• dagster is the glue that orchestrates everything.
• the data is stored in a postgresql database.
• the pipeline is deployed in a docker swarm, using ansible.
• data is served via our API built using the rocket rust web framework.
• the code is hosted on Codeberg
• Everything is deployed from our CI, which uses woodpecker

This documentation site is built with Material for MkDocs.