Principal Data Engineer

iO Associates is currently partnered with a custom software and technology consultancy that is seeking a skilled Principal Data Engineer.

Their in-house product teams’ partner with Fortune 500s to navigate the unknown – that space between an identified problem and a future-focused solution. They are currently seeking a Senior Data Engineer, as they continue to build out their development team.

Responsibilities

• Lead the design, development, and optimisation of ETL data pipelines using Golang or Python.
• Collaborate with cross-functional teams to architect scalable data solutions on platforms such as Databricks, Snowflake, or similar.
• Ensure data quality, security, and compliance standards are upheld across all pipelines.
• Mentor and guide junior team members, fostering a culture of continuous improvement.
• Troubleshoot and resolve complex technical issues to ensure seamless data flow.
• Contribute to strategic decisions regarding data architecture and technology stack.

Essential Skills & Experience

• Extensive experience developing ETL pipelines using Golang and/or Python.
• Proven expertise working with data platforms such as Databricks, Snowflake, or comparable solutions.
• Strong knowledge of distributed data processing and scalability best practices.
• Demonstrable leadership experience, with the ability to guide technical teams and influence project outcomes.
• Excellent problem-solving skills and attention to detail.
• Experience working in fast-paced, collaborative environments.
• Senior/ Principal level with Consulting experience

Desirable Skills & Experience

• Familiarity with cloud infrastructure services such as AWS, Azure, or GCP.
• Knowledge of containerisation and orchestration tools like Docker and Kubernetes.
• Prior experience in data engineering within large-scale enterprise environments.
• Exposure to DevOps practices and CI/CD pipelines.
• Advanced understanding of data security and compliance standards.

If you are seeking an opportunity to learn new technologies, make decisions that influence the path of a product roadmap, and impact hundreds and thousands of users every day with the products you build, apply now to dive deeper into this opportunity!