As a data engineer, you are the gatekeeper of all things infrastructure. You are responsible for ensuring the trading platforms run smoothly and we don’t acquire long term technical debt. You are the guardian of our machine learning models, datasets, feature stores and automation initiatives.
You are someone with;
4+ years of experience in Python, SQL, Terraform, Ansible, CI/CD pipelines
2-3+ years of experience in error, logging and observability platforms like Sentry, Datadog, Honeycomb
Experience with AWS Athena and Redshift is a plus
Knowledge of Pyspark or other distributed computing frameworks is a plus
3+ years of experience in data processing libraries like pandas, numpy, scipy and pyAthena
3+ years of experience in REST APIs, Flask, FastAPI
3+ years of experience in AWS or any other cloud computing platform
Prior experience in deploying data processing or ETL pipelines with large scale time series/financial/quantitative datasets
Prior experience in assessing code quality and building a codebase with proper engineering fundamentals
Exposure to interdisciplinary collaboration, especially involving domain expertise
Strong oral and written communication skills
Capacity to operate independently and make sense of limited directions, especially capable to identify the underlying unknowns
