Data Engineer Job in Magna International

Role Summary As a Data Engineer, you will be responsible for designing, developing, and maintaining data pipelines and infrastructure that enable the efficient processing, storage, and retrieval of large-scale data sets. You will work with data scientists, analysts, and other stakeholders to understand their requirements, and collaborate with them to build solutions that meet their needs. You will have extensive experience in using Spark and Python for data engineering, and a good understanding of Cloud technologies such as AWS, Azure, or Google Cloud Platform. Key Responsibilities Design, develop, and maintain data pipelines and infrastructure that enable efficient processing, storage, and retrieval of large-scale data sets. Collaborate with data scientists, analysts, and other stakeholders to understand their requirements, and build solutions that meet their needs. Write Spark code and Python scripts to develop data processing workflows, and automate ETL processes. Work with Cloud technologies, such as AWS, Azure, or Google Cloud Platform, to design and deploy data infrastructure and services. Implement security, monitoring, and logging solutions for data pipelines and infrastructure. Develop data quality checks and data validation processes to ensure accuracy and consistency of data. Optimize data pipelines and infrastructure for performance, scalability, and reliability. Document data pipelines and infrastructure, and ensure that they are well-maintained. Key Qualifications/Requirements Extensive experience in data engineering using Spark and Python. Strong understanding of Cloud technologies such as AWS, Azure, or Google Cloud Platform. Experience with data processing frameworks and tools such as Hadoop, Kafka, or Flink. Proficiency in SQL and database technologies such as MySQL, PostgreSQL, or Oracle. Knowledge of data modeling, data warehousing, and data architecture. Experience with Agile methodologies and project management tools such as JIRA. Strong analytical and problem-solving skills. Excellent communication and collaboration skills, with the ability to work effectively in a team. Self-motivated and self-managed with a high degree of analytical skills (quick comprehension, abstract thinking, recognize relationships). Ability to be a strong team member and communicate effectively. Capability to prioritize and multi-task, and reasonably estimate work effort for tasks. Excellent English in spoken and written.