Metadata
- Author: Adyen
- Full Title:: Data Engineering at Adyen
- Category:: 🗞️Articles
- Document tags:: Data team topologies
- URL:: https://medium.com/adyen/data-engineering-at-adyen-ccded12a6eb
- Finished date:: 2023-04-23
Highlights
data engineers at Adyen are responsible for creating high-quality, scalable, reusable and insightful datasets out of large volumes of raw data (View Highlight)
New highlights added 2023-04-23
At Adyen, we strive to give autonomy to our engineers. This has two advantages for our engineers: they can quickly make decisions within their area of expertise and can enjoy a safe environment for creativity and experimentation. Therefore, our data engineers are encouraged to choose projects based on skill, experience and interest within their team and even expand their work across teams (View Highlight)
We analyze and explore data using JupyterHub notebooks, tied to our Hadoop filesystem and Hive metastore (View Highlight)
we are also not really responsible for the functioning of the platform: this means we do not have to ensure there are enough airflow workers, or that we need to ingest raw data from event streams. This is done by our data platform engineers, and we trust them to do a good job 👍 (View Highlight)
(A team might consist of data analysts, data engineers and data scientists, headed by a technical team lead and product manager, who all work together to develop a product (View Highlight)
We extend our collaboration beyond our team in the following cases: (View Highlight)
Working on smaller tooling features (View Highlight)
Abstracting complex but commonly used logic away (View Highlight)
They suggest a fully embedded model with collaborations to build common tooling
you have a much clearer understanding of our work after reading this post. (View Highlight)