Senior Data Engineer Job in Kloud9

Roles & Responsibilities

• Design and Build Data pipelines in collaboration with Product Owners / BSA, Data Analysts and Business stake holders
• Work with Lead Data Engineers and Junior Data Engineers to ensure quality solutions are implemented with adherence to engineering best practices
• Build Reusable utilities, frameworks and user defined functions
• Evaluate and utilize new technologies/tools/frameworks centered around Orchestration Pipelines
• Provide technical leadership on Orchestration Pipelines to the project teams

Experience & Skills Fitment

• Overall 5+ Years of Experience in Data Modelling and ETL processing projects
• 3+ Years of Experience in designing and implementing large data pipelines using Spark, Hive on AWS with Python and pySpark
• 1+ Years of Experience in building end to end pipelines using Airflow DAG
• Hands on Experience in Data Ingestion, Data Cleansing, Data Transformation tools and techniques
• Hands on Experience with Data Quality and Data Lineage tools and frameworks
• Hands on Experience in Spark Batch and Streaming Data pipelines with Kafka
• Experience of working in Agile model
• Expert level knowledge on Coding and Design Best Practices
• Expert level experience in SQL
• Expert level Experience in Trouble shooting
• Experience supporting and working with cross-functional teams in a dynamic environment
• Excellent verbal and written communication skills in English
• Ability to communicate effectively with team members and business stakeholders

Nice to have:

• General knowledge/experience of AWS snowflake is an added advantage