Data Science Engineer Job in Dunnhumby
What we expect from you
Develop and maintain engineering tools/products needed for simple and efficient data science development
Analyse complex data pipelines to identify performance bottlenecks, and suggest robust ways to optimize the workload in reasonable costs
Proficiency in at least one programming language python/java/scala
Good knowledge of data engineering techniques, e.g. data ingestion, data processing , data validation, data publishing, data quality etc
Proficiency in Hadoop, PySpark, Pandas, NumPy and python version > 3.5
Good Working experience of Partitions, Joins, cache, HDFS, handling data skewness and code optimisation in general
Good knowledge of Airflow or any other data process orchestration tools like Nifi , Lungi
Good knowledge of SQL
Good knowledge of shell scripting and GIT workflows
Experience with container orchestration platforms like Kubernetes and containerization technologies like Docker
Experience in re-engineering, automating and productionising code
Great communication skills, both written and oral
Should have experience in handling/optimising various file formats like parquet, Avro etc
Should have good experience in working with Hadoop Ecosystem components (e.g. YARN, Spark UI, Hive, HDFS etc.) and cloud equivalents possibly in GCP
Working knowledge of web application development and application integration (REST APIs, Web services)
Your 4 Step Guide to Career Success
