Data Engineer

About Secretlab

Secretlab is a globally recognized gaming chair company serving over two million users, with major markets across the United States, Europe, and headquartered in Singapore.

Role Overview

We are searching for a Data Engineer (or Senior Data Engineer) to enhance and manage our data infrastructure and analytics capabilities. The exponential growth in demand for accessible, clean data streams and batch data requires innovative, startup-style data culture development.

Key Responsibilities

• Establish AWS Cloud infrastructure tailored to data engineering and data science needs.
• Implement robust security measures including VPC and SAML based on architectural best practices.
• Design comprehensive data models and structure for data warehouses and related systems.
• Create star-schema models and layers for analytics and machine learning using tools such as Airflow and Data Build Tool (DBT).
• Develop reusable template packages, including logging and AWS integration templates for team use.
• Maintain reliable data pipelines by employing best practices like unit testing and extensive logging to prevent technical debt.
• Balance MVP delivery with user interface, functionality, and system reliability to avoid overengineering or underengineering.
• Deliver custom data pipelines fulfilling feature requests and user stories.
• Work comfortably with SaaS platforms such as Fivetran, DBT, Amazon S3, and Snowflake.
• Effectively communicate requirements and provide guidance to junior team members.
• Ensure code review submissions are concise and easy to evaluate, aiming for most pull requests to pass within one or two reviews.

Weekly Workflow

• Participate in Agile sprints collaborating with the Business Intelligence team to manage backlog priorities.
• Develop various data models including star schemas and event-driven data marts.
• Engage in peer code reviews as part of the team's production process.
• Manage microbatch and batch data processing from sources such as Shopify.
• Automate DBT pipelines using orchestration tools like Airflow or Luigi.
• Establish connections to downstream Business Intelligence and Data Warehouse tools.
• Troubleshoot and resolve data processing errors and operational failures.
• Contribute to process enhancements and tooling decisions during weekly retrospectives.

Technical Requirements

• Proficiency in SQL and Python programming.
• Experience or familiarity with DevOps technologies including Git, Docker, and Terraform is advantageous.
• Knowledge of error logging, handling corrupt or bad records, and building fault-tolerant data pipelines.
• Understanding of scaling pipelines, continuous integration, database administration, data cleansing, and ensuring pipeline determinism.
• Experience working with cloud platforms such as AWS or Google Cloud Platform (GCP).

Personal Attributes

• Genuine enthusiasm for data, emerging data technologies, and creative problem solving for organizational data challenges.
• Transparent and upfront communication style, actively engaging in team and process improvements, open to candid feedback.
• Honest and pragmatic approach to self-assessment and project contributions.

Preferred Qualifications

• Prior experience contributing to scaling startup environments.
• Familiarity with Apache Spark.