DATA ENGINEERING
What We Do
Our mission is to provide a world-class platform that empowers the business to leverage data that will enhance, monitor, and support our products. We are responsible for data ingestion systems, processing pipelines, and various data stores all operating in the cloud. We operate at a petabyte scale, and support near real-time use cases as well as more traditional batch approaches.
What You'll Do
You will be responsible for designing, building, and maintaining our data infrastructure to ensure the reliability and efficiency of our data and systems used by our Machine Learning team. Your role will include creating and maintaining data pipelines that transform and load data from various products and managing the AWS infrastructure for our machine learning platform. Additionally, you will work with engineers, product managers, and data scientists to design and implement robust and scalable data services that support Epic's mission while ensuring our user's privacy.
In this role, you will
• Interact with product teams to understand how our safety systems interact with their data systems.
• Design and implement an automated end-to-end ETL process, including data anonymization, to prepare data for machine learning and ad hoc analysis.
• Manage and scale the tools and technologies we use to label data running on AWS.
• Devise database structure and technology for storing and efficiently accessing large data sets (millions of records) of different types (text, images, videos, etc.).
• Use and implement data extraction APIs.
• Write and invoke custom SQL procedures.
• Support data versioning strategies using automated tools.
What we're looking for
• Strong analytical background: BSc or MSc in Computer Science/Software Engineering or related subject - candidates without a degree are welcome as long as they have extensive hands-on experience.
• Experience in ETL technical design, automated data quality testing, QA and documentation, data warehousing, and data modeling.
• Experience with Python for interaction with Web Services (e.g., Rest and Postman).
• Experience with using and developing data APIs.
• Experience using AWS, Snowflake, or other comparable large-scale analytics platforms.
• Experience monitoring and managing databases (we use Elasticsearch / MongoDB / PostgreSQL).
• Experience with SQL.
• Experience with data versioning tools.
• Experience developing and maintaining data infrastructure for ETL pipelines, such as Apache Airflow.
Note to Recruitment Agencies: Epic does not accept any unsolicited resumes or approaches from any unauthorized third party (including recruitment or placement agencies) (i.e., a third party with whom we do not have a negotiated and validly executed agreement). We will not pay any fees to any unauthorized third party. Further details on these matters can be found here.
Jobcode: Reference SBJ-gx4j64-64-137-103-244-42 in your application.
