Databricks Application Engineer

Company Description
Showtime Consulting is a leading provider of Shielded Cloud and Digital Solutions across Australia and New Zealand. We specialise in delivering secure, enterprise-scale technology solutions across cloud, infrastructure, cybersecurity, DevSecOps, software engineering, data, and digital transformation programs.

We partner with government and enterprise organisations to build high-performing technology teams that deliver secure, resilient, and scalable solutions within complex and highly regulated environments.

The Role
We are seeking an experienced Databricks Application Engineer to join a Sydney-based team supporting the development of modern data and application platform capabilities.

This role will suit a hands-on engineer with experience across both data engineering and software development. You will support the transition from traditional data archival initiatives toward scalable Databricks-based application and platform solutions. Working closely with business and technology stakeholders, you will design, develop, and maintain data pipelines, application integrations, and modern platform services that support innovative business outcomes.

Key Responsibilities

• Design, build, and maintain scalable data pipelines using the Databricks platform.
• Develop and support data ingestion, transformation, and orchestration frameworks.
• Build and manage solutions using Lakeflow Connect and Lakeflow Pipelines.
• Develop scalable data processing solutions using Python and SQL.
• Ensure data quality, reliability, and performance across batch and near real-time workloads.
• Collaborate with architecture and platform teams to implement data governance and best practices.
• Design and develop modern web-based applications integrated with Databricks platforms.
• Build and consume REST APIs to support application and platform integrations.
• Implement secure API authentication and authorisation mechanisms.
• Contribute to application design, development, testing, deployment, and support activities.
• Support CI/CD pipelines and promote DevOps best practices across projects.
• Develop scalable and maintainable solutions aligned to enterprise architecture standards.
• Work closely with business stakeholders to translate requirements into technical solutions.
• Troubleshoot and resolve complex technical issues across data, platform, and application layers.

What You'll Need

• Experience across data engineering and software engineering.
• Hands-on experience with the Databricks native platform.
• Strong experience with Lakeflow Connect and Lakeflow Pipelines.
• Experience building data ingestion, transformation, and orchestration frameworks.
• Strong Python and SQL development skills.
• Solid understanding of modern web application architecture.
• Experience designing, developing, and integrating REST APIs.
• Strong understanding of API security, authentication, and authorisation concepts.
• Experience working across the full software development lifecycle.
• Familiarity with CI/CD practices and DevOps methodologies.
• Strong analytical, troubleshooting, and problem-solving skills.
• Ability to work collaboratively with business and technology stakeholders.
• Experience developing Databricks Apps is highly regarded.
• Cloud-native application development experience is advantageous.
• Knowledge of enterprise integration patterns and distributed systems is highly desirable.
• Experience working within Agile delivery environments will be highly regarded.
• Strong stakeholder engagement, communication, and collaboration skills.

Why Join Us?

• Work on modern Databricks-based data and application platforms.
• Build scalable solutions across data engineering and software development.
• Gain exposure to Lakeflow, cloud-native applications, APIs, and enterprise integrations.
• Collaborate with experienced architecture, platform, and delivery teams.
• Contribute to technology roadmap initiatives and future-ready platform capability.
• Join a consulting culture focused on innovation, technical excellence, and continuous improvement.