Job Description
The Senior Data Engineer will be responsible for creating and optimizing our data systems, data pipelines and data sets. The ideal candidate is an experienced data pipeline builder and data wrangler who enjoys optimizing data systems and building them from the ground up. The Data Engineer will support our software developers and data analysts on data initiatives and will ensure optimal data delivery for all the projects. The hire must be self-directed and comfortable supporting the data needs of multiple teams, systems and products.
Job Responsibilities
- Confidently oversee work with data analysts, product, engineering, program managers and data scientists to understand and deliver on company-wide data needs.
- Architect and build efficient and reliable data pipelines to transport data across a number of platforms including Amazon Redshift, S3, Sagemaker, online caches and real-time systems.
- Oversee productionization of machine learning models.
- Craft code and technology design review best practices.
- Communicate, at scale, through presentations, dashboards, company-wide datasets, bots and more.
- Inculcate a collaborative environment which fosters data expertise and owns data quality.
Requirements
- 3+ years experience with programming languages such as Python, Scala or Java.
- 5+ years SQL experience.
- 3+ years experience with Cloud Platforms such as AWS and GCP.
- 3+ years experience with workflow management engines such as Apache Airflow or Argo Workflows.
- 3+ years experience in custom ETL design, implementation and maintenance.
- 3+ years experience with systems integration.
- 3+ years experience working with Amazon Redshift or other cloud or on-prem Big Data/MPP analytics platform (i.e. Netezza, Teradata, Snowflake, Google BigQuery, Azure Data Warehouse, or similar).
- 3+ years experience with Data Modeling.
- 3+ years experience analyzing data to discover opportunities and address gaps.
- 3+ years experience with Docker.
- 3+ years experience with database management and experience with MySQL, Aurora MySQL and Amazon Redshift.
- Great communication and collaborative abilities.
Preferred Qualifications
- 2+ years experience with Kubernetes and Argo Workflows.
- Experience with DBT.
- Experience with more than one coding language.
- 2+ years experience with designing and implementing real-time pipelines.
- Experience with data quality and validation.
- Experience with geolocation data.
- Experience with SQL performance tuning and end to end process optimization.
- Experience with anomaly/outlier detection.
- Experience with notebook-based model development workflows.
- Experience with Amazon Quicksight.
- Great team player and able to work efficiently with minimal supervision
Benefits
- A purposeful role with mentorship from an experienced founding team
- Fast paced career prospects in one of the hottest and well-funded tech startups in the region
- Competitive salary
- Excellent variable incentive plans
- Store credits to save on your monthly grocery bill
- Medical insurance (including family coverage)