Data Engineer Job in Dyson

The Role The Data Engineer will be responsible for designing and building the data pipelines, implement automation where necessary, as well as developing and implementing data processing pipelines and data storage solutions. They will also be involved in developing and maintaining data integration tools and will be responsible for ensuring that all data is securely stored and backed up. Solid experience in data engineering, cloud infrastructure, and modern data-driven systems are essential for this role. Strong analytical and problem-solving skills are key, as project requirements may evolve over time and the Data Engineer will need to be able to adapt and improve the data infrastructure accordingly. Responsibilities Developing scalable and reliable ETL pipelines and processes to ingest data from a large number and variety of data sources Developing a deep understanding of real-time data productions availability to inform on the real time metric definitions Maintaining and optimizing the performance of our data analytics infrastructure to ensure accurate, reliable and timely delivery of key insights for decision making Design and deliver the next-gen data lifecycle management suite of tools/frameworks, including ingestion and consumption on the top of the data lake to support real-time, API-based and serverless use-cases, along with batch as relevant Ensure that all data is securely stored and backed up. Identify and implement new technologies and tools to improve the data infrastructure. Monitor, troubleshoot, and resolve data infrastructure issues in a timely manner. Required Skills: Strong programming skills in at least one of the following languages: Python, Java, Scala, SQL, and a bonus in graph database (Node.js) Hands-on experience with big data technologies such as Hadoop, Spark, or Kafka Experience with cloud platforms such as AWS, GCP, or Azure, and their respective data services Experience with Docker, Kubernetes or other containerization tools Solid understanding of data modelling, data pipelines, database design principles Proficiency in SQL and experience with relational and non-relational databases Familiarity with data visualization tools such as Tableau, PowerBI or Looker Knowledge with monitoring tool such as Datadog or Grafana Experience with version control systems such as Git Excellent problem-solving skills and ability to work in a fast-paced environment Requirements Bachelor s degree in Computer Science, Information Systems, or other relevant field. 5+ years of experience in data engineering, cloud infrastructure, and modern data-driven systems from manufacturing industry preferably. Hands-on experience with large datasets, distributed computing systems, databases, data lakes, cloud-based solutions, and data integration. Solid experience with Python and Shell scripting. Strong understanding of API contracts, data integration patterns, and data security regulations. Demonstrable understanding of relevant data formats and protocols such as JSON, XML, etc. Ability to work independently and as part of a team. Excellent communication and interpersonal skills. Flexible in thinking and result oriented!