Databricks Engineer

Databricks Engineer - Alexandria, VA (Remote)

Must be a US Citizen and hold active IRS clearance - Public Trust or higher.

Job Description:

• Currently seeking a skilled Databricks Engineer to join our dynamic team.
• Full-Time, Remote Position.
• Responsible for designing, implementing, and optimizing data pipelines into Databricks.
• Ensure seamless data flow and efficient data processing, storage, and analysis.

Responsibilities:

• Develop and maintain ETL processes for data extraction, transformation, and loading.
• Design and implement data pipelines and workflows within Databricks.
• Collaborate with stakeholders to understand data requirements and provide solutions.
• Develop and maintain Python Notebooks for data analysis and processing.
• Optimize data processing jobs for performance and cost-efficiency.
• Ensure data quality through robust validation and cleansing techniques.
• Monitor and troubleshoot data pipeline issues for timely resolution.
• Utilize Terraform for infrastructure automation and scaling.
• Stay updated with the latest trends in data engineering and Databricks technologies.

Qualifications:

• U.S. Citizenship.
• Active IRS clearance - Public Trust or higher.
• Over 5 years of hands-on experience with Databricks.
• Proficiency in ETL tools and techniques.
• Strong skills in SQL, Python, and Spark.
• Familiarity with cloud platforms like AWS, Azure, or Google Cloud.
• Experience with big data technologies such as Apache Airflow.
• Knowledge of data warehousing concepts and best practices.
• Familiarity with data visualization tools like Tableau, Power BI, or Looker.
• Experience deploying Databricks infrastructure on cloud platforms.
• Proficiency in Terraform for infrastructure as code (IaC) solutions.
• Strong problem-solving skills and ability to troubleshoot complex data issues.
• Excellent communication and collaboration skills.