Senior Data Engineer

  • Location:

    Miami, Florida 33101 - United States

  • Sector:

    Analytics

  • Job ref:

    1432

  • Contact:

    Zack Cheatham

  • Expiry date:

    2023-08-25

  • Published:

    3 months ago

  • Location: Miami, Florida
  • Type: Direct Hire
  • Job #1432

Senior Data Engineer

South Florida or Remote 

The ideal candidate is a hands-on leader with a real passion for solving problems, has ability to wear multiple hats, and can work with our onshore and offshore teams to deliver on client challenges through their data engineering talents.  You’ll collaborate across all lines of business with a high-caliber team of experienced professionals and will have the opportunity to gain a broad base of experience working with mid-market companies across a wide spectrum of industries.

Your primary focus will be to lead the team through your data engineering expertise by designing and optimizing the data and data pipeline architecture across our client-base; designing and implementing data models that align with data warehouse standards; providing documentation, training and consulting with internal and external clients.

Essential Duties & Responsibilities

  • Provide end-to-end functional software landscape and solution design to support the business processes.

  • Evaluate requirements, map them to business processes, create functional and solution approaches, and ensure a proper implementation.

  • Provide direction to the implementation teams and work as intermediary across the local and offshore teams.

  • Must have demonstrably strong interpersonal and communication skills (both written and verbal), to include speaking clearly and persuasively in positive or negative situations

  • Experience with creating and supporting a complete ELT/ETL pipeline.

  • Knowledge of data movement tools, data stores, and cloud-based environments.

  • Experience in building and supporting a modern next generation data warehouse platform.

  • Deep understanding of data modeling and architecture as it relates to business goals and objectives

  • Exposure to and support of downstream data usages for visualization and data science.

  • Ability to explain complex technical challenges to technical and business audiences.

  • Standardize and implement code branching strategy

  • Enable developers with automation of deployments and testing frameworks

  • Document, implement and audit security and access permissions for the data platform

  • Follow and implement CI/CD and data pipelines for cloud data platform

  • Strong knowledge on AWS IAM and security policy management and service configurations and compliance management.

  • Must have extensive knowledge of release/change management process.

  • Experienced or Awareness with Scrum and Agile methodologies with geographically distributed teams.

  • Experience as DevOps skill is an added advantage

Required Skills / Experience

  • Strong hands-on coding experience creating data pipelines in Python (5+ years)

  • Experience in creating data pipelines using PySpark in Databricks (2+ years)

  • Strong hands-on coding experience in SQL (2+ years)

  • Experience with managing and scaling cloud data warehouse (Snowflake) 

  • Experience with AWS ecosystem – S3, Amazon RDS, Amazon ML ecosystem

  • Experience in GitHub integration, CI/CD pipeline development

  • Experience designing, building and maintaining ETL/ELT data pipelines (3+ years)

  • Experience using orchestration tools like Apache Airflow

  • Leveraging, designing and building dimensional modeling (3+ years)

  • Experience working with large datasets and big data technologies, preferably cloud-based, such as Snowflake, Databricks, or similar

  • Participate in driving best practices around data engineering software development processes

  • Bachelor’s degree in Computer Science or Software Engineering with master’s degree a plus

  • Excellent communication skills with the ability to effectively communicate complex issues and solutions at all levels

Attach a resume file. Accepted file types are DOC, DOCX, PDF, HTML, and TXT.

We are uploading your application. It may take a few moments to read your resume. Please wait!