We’re a cognitive science company on a mission to optimize the measurement of brain health to advance the development of new medicines and to enable earlier clinical insights in healthcare.

That’s why we’re seeking a Senior Data Engineer responsible for building new data pipelines and converting legacy data pipelines from SAS to Azure Databricks to help the analytics and the data science team at Cogstate. The candidate will have a central critical role for establishing and maintaining the Azure infrastructure for building data pipelines, provide best practices and recommendations for using these tools, including but not limited to Azure Data Factory and Azure Databricks, and work closely with data scientists and be a point of contact for all things related to the data platform.

*Salary Range: $110,000 – $120,000 per year. Benefits listed below.

Core Responsibilities

• Design, develop, implement, and tune large-scale distributed systems and pipelines that process large volume of data; focusing on scalability, low-latency, and fault-tolerance in every system
• Developing scalable and re-usable frameworks for ingesting data into Azure Databricks, incorporating standards and best practices into engineering solutions
• Databricks engineering – query tuning, performance tuning, troubleshooting, and debugging pipelines.
• Deep understanding of ETL/ELT design methodologies, architecture, strategy, and tactics for complex ETL solution, including CI/CD skills.
• Develop high performance scripts in SQL/Python/etc. to achieve objectives of enterprise data, BI, data visualisation and analytics needs.
• Data processing/transformation using various technologies such as Apache Spark, SQL, Python/Scala and Azure cloud services.
• Understand current data engineering pipelines using legacy SAS tools and convert to modern pipelines using Databricks.
• Manage code versions in source control and coordinate changes across teams by leveraging BitBucket and Octopus Deploy
• Participate in architecture design and discussions, provide logical and physical data design, and database modelling
• Be part of the Agile team to collaborate and to help shape requirements, including working closely with business users to discover business rules and turn these into actionable work items
• Solve complex data issues around data integration, data quality, and other data processing incidents
• Work with business system owners to resolve source data issues and refine transformation rules
• Partner with engineering teams, data scientists, and data analysts to define requirements and design and develop the end solutions

Qualifications

• 5+ years’ experience as a data engineer.
• Must have 2+ years in implementing data engineering solutions with Spark, preferably Databricks.
• This is a hands-on role building data pipelines using Azure Databricks. Hands-on technical experience with Apache Spark is required.
• Must have deep expertise in one of the programming languages for data processes (Python, Scala). Experience with Python, PySpark, Hadoop, Hive and/or Spark to write data pipelines and data processing layers
• Must have worked with relational databases like SQL Server. Good SQL experience for writing complex SQL transformations.
• Strong knowledge on Databricks configuration, troubleshooting and performance tuning.
• Experience with development tools for CI/CD, unit and integration testing, automation and orchestration, including BitBucket, Azure Data Factory and Azure functions.
• Experience in a top tier consulting environment desirable

What’s In It For You

• Remote Work Practices: Cogstate is a virtual first company. Cogstate employees can work from anywhere where Cogstate is registered to business within the United States, Australia, or the United Kingdom!
• Generous Paid Time-off: Cogstate employees receive 20 days of vacation leave, 10 days of personal leave and 10 paid public holidays, which can be tailored to dates of personal, cultural, or religious significance to you.
• 401(k) Matching: As you invest in yourself and your future, Cogstate invests in you too: we match up to 3% of your yearly salary in Cogstate’s 401k program
• Competitive Salary: We offer competitive base salaries plus additional earning opportunities based on the position.
• Health, Dental & Vision Coverage: We’ve invested in comprehensive health & dental insurance options with competitive company contributions to help when you need it most. We also offer free vision insurance for all full-time employees.
• Short-Term & Long Term Disability Life Insurance: 100% employer sponsored
• Pre-Tax Benefits: Healthcare and Dependent Care Flexible Spending Accounts
• Learning & Development Opportunities: Cogstate offers a robust learning program from mentorships to scholarships to Coursera to improve knowledge or obtain certifications in applicable areas of interest.

Our Culture

We bring our whole selves to work every day. We’re courageous and we deliver together. We’re passionate individuals who enjoy working together. We’re brave enough and care enough to have the right conversations to get the best outcome and are famous for our can-do attitude. We see challenges as opportunities and move with pace to achieve our goals.

If you’re ready to help us in our journey to optimize the measurement of brain health around the world, please apply now!

• Applicants with disabilities may be entitled to reasonable accommodation under the terms of the Americans with Disabilities Act and certain state or local laws. A reasonable accommodation is a change in the way things are normally done which will ensure an equal employment opportunity without imposing undue hardship on the company. If you need assistance in applying please email PeopleandCulture@cogstate.com.