Role: Sr Data Engineer - Disney

Location: Glendale, CA (Hybrid - 2- 4 Days Onsite)

Full Time

In Person is interview is must

Mandatory skills: Databricks experience (primary requirement), Apache Airflow, Advanced SQL skills, Python, Spark / PySpark, Scala, Experience building and maintaining data pipelines and workflows

Job Description:

Key Responsibilities:

·       Design, write, test, and deploy data pipelines using PySpark, Scala, SQL, Python

·       Meet with stakeholders to gather requirements and translate them into scalable data platform solutions

·       Understanding of Databricks platform and developer tooling to diagnose errors, audit platform activity, and automate updates across pipelines, objects, and integrations

·       Ability to explain Spark architecture and pipeline behavior to stakeholders to diagnose root causes and recommend solutions

·       Provide solution architecture across AWS, Databricks, Kubernetes, and Airflow (MWAA), including cross-platform integrations

·       Manage Databricks platform governance, including Unity Catalog, ACLs, lineage, and data discovery and privacy tooling

·       Build and maintain Kubernetes containers and containerized utilities supporting deployed data platform services

·       Apply networking knowledge to troubleshoot connectivity and integration errors across platform components

·       Perform platform administration: provision and remove access, assess resource utilization, monitor platform health and cost, and evaluate stakeholder requests

·       Collaborate with engineers, architects, and product managers to drive Core Data platform success; participate in agile/scrum ceremonies

·       Maintain documentation of platform changes, standards, and pipeline configurations to support data quality and governance

Qualifications:

·       5+ years of data engineering experience developing and operating large-scale data pipelines

·       Deep hands-on experience with Databricks and Apache Spark (batch and streaming), including pipeline development in PySpark and/or Scala

·       Strong understanding of Spark architecture-executors, stages, partitioning, shuffle, and performance tuning-with ability to explain tradeoffs to technical and non-technical stakeholders

·       Proficiency with Databricks platform tooling (API, SDK, CLI) for automation, auditing, governance, and operational troubleshooting

·       Proficient in SQL with advanced performance tuning capabilities

·       Hands-on production experience with Airflow (MWAA) for orchestrating data pipelines

·       Experience managing Databricks platform governance: ACLs, Unity Catalog, lineage, and access provisioning

·       Proficiency in Python and at least one additional language (Scala, Kotlin, or SQL-driven pipeline tooling)