Data Engineer

Data Engineer

Focus Areas:

Strong Data Modelling, Data Mining, Data Science skills preferred

Design and develop data solution features and data integration pipelines for batch and real-time data systems

- Develop pipeline/ETL code using traditional ETL tools, cloud-based platforms and technologies, and big data tools

- Troubleshoot and resolve data and processing issues

- Participate in architecture discussions, and propose solutions related to area of focus


Has significant experience with:

- building custom data pipelines

- data structure optimization and design for scalability


- distributed systems

- one or more RDBMS platforms and advanced SQL

- data processing on one or more cloud platforms (AWS, Azure, etc)

- multiple languages: Python, Java, Scala, Shell, etc

- working in an agile environment and with cross-functional teams


Nice to have:

- Kafka

- Cassandra

- Solr

- Elasticsearch

- Kafka 

- Microservices

- Mesos