Data Engineer (Databricks)

Title

Data Engineer (Databricks)

Overview

This is a full-time, employee position. Please do not apply if you are seeking a C2C or 1099/W2 contract.

Must be located in one of the following: Atlanta, Chicago, Columbus, Dallas, Minneapolis, New York Area, St. Louis. This position is mostly Remote.

Daugherty Business Solutions brings a fresh approach to data engineering by delivering results through unmatched innovation and world-class technology and talent. This is why many of the most well-known companies in the world trust us with their mission-critical projects. As a team member at Daugherty, you will play an integral role in our company’s success and are recognized and valued for your contributions. We have an entrepreneurial culture with the maturity and security of a 35+ year company helping you to create your best work and take your career to the next level.

We are seeking a skilled and experienced Data Engineer with expertise in Databricks, Azure, and Spark to join our dynamic team. The ideal candidate will play a key role in designing, developing, and maintaining scalable data pipelines and analytics solutions to support our growing business needs. This position offers an exciting opportunity to work with cutting-edge technologies and collaborate with cross-functional teams to drive actionable insights from data.

Responsibilities

• Design, develop, and deploy end-to-end data pipelines using Databricks, Azure Data Factory, and Spark to ingest, process, and analyze large volumes of structured and unstructured data.
• Collaborate with data scientists, analysts, and other stakeholders to understand business requirements and translate them into technical solutions.
• Optimize and tune data pipelines for performance, reliability, and scalability to ensure efficient processing of data.
• Implement data governance and security best practices to ensure the integrity and confidentiality of data.
• Develop and maintain documentation, including data flow diagrams, technical specifications, and user guides.
• Stay up-to-date with emerging technologies and best practices in data engineering, cloud computing, and big data analytics.
  
  
  

Qualifications

• Proven experience as a Data Engineer, with at least 5 years of hands-on experience in designing and building data pipelines.
• Proficiency in Databricks, Azure services (e.g., Azure Data Lake Storage, Azure Synapse Analytics), and Spark for big data processing and analytics.
• Strong programming skills in Python, Scala, or Java, with experience in writing complex SQL queries.
• Experience with data modeling, ETL/ELT processes, and data warehousing concepts.
• Excellent problem-solving skills and attention to detail, with the ability to troubleshoot and debug complex data engineering issues.
• Strong communication and interpersonal skills, with the ability to collaborate effectively with cross-functional teams.
• Able to articulate ideas clearly, present findings persuasively, and build rapport with clients and team members.
• Ability to effectively communicate technical concepts to non-technical stakeholders .
• Relevant certifications in Databricks, Azure, or Spark (e.g., Azure Data Engineer, Databricks Certified Associate) are a plus.
• Bachelor's degree in Computer Science, Engineering, or related field; Master's degree preferred.
  
  
  

What We Commit to YOU:

• We provide many training opportunities like certifications, hackathons, lunch and learns and free access to Pluralsight, Udemy and other digital learning platforms.
• You will get to work with some of the most innovative teams in the IT marketplace and solve real strategic problems.
• We will invest in things that are important to you professionally and personally.
• We will build a relationship with you to accelerate your career.
• We will provide you with a team environment like no other. We are consistently ranked as a Top Workplace in many of our regions as voted by our employees.
• We provide opportunities to build community, be social and have fun with your colleagues.
• We provide a comprehensive compensation and benefits package.
  
  
  

Daugherty Business Solutions is an inclusive Equal Employment Opportunity employer that considers applicants without regard to gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

If you require accommodations or assistance to complete the online application process, please inform any recruiter you are working with (or send an email to careers@daugherty.com) and identify the type of accommodation or assistance you are requesting. Do not include any medical or health information in this email. The recruiting team will respond to your email promptly.