Senior Big Data Engineer Job in Infocepts

Work and Technical Experience: Strong understanding of cloud and data engineering concepts in Big Data and AWS Strong SQL & DWH concepts Designing and Implementing Big data ETL pipeline for Data Ingestion, Storage, Processing & Consumption with Spark (Scala/Python) framework Optimizing existing Big Data Spark ETL pipelines Building and Deploying ETL jobs in AWS using Jenkins and other tools Role At least 7+ years of overall hands-on experience in data and analytics with good understanding on SQL Must be a minimum of 5+ years of hands-on association with Big Data and AWS Analytical services Big Data, HDFS, Hive, Spark, YARN AWS, EMR, EC2, S3, Athena, Airflow, Lambda Function as integrator between business needs and technology solutions, helping to create technology solutions to meet client business need Lead project efforts in defining scope, planning, executing, and reporting to stakeholders on strategic initiatives Understanding of EDW/ Big Data Lake system of business and creating High level design document and low level implementation document Designing Big data pipeline for Data Ingestion, Storage, Processing & Consumption