Big Data Engineers - Azure Job in Gspann Technologies, Inc

Skills and Experience

• Expertise in developing relevant Big Data/ETL data warehouse and building cloud-native data pipelines.
• Hands-on experience in Java, Spark, and SQL.
• Communicate effectively and confidently with business stakeholders and architects.
• Good understanding of ADLS (Azure Data Lake storage) and implementing Azure Data Factory pipeline.
• Expertise in Azure Data Factory, Azure Data Lake Azure SQL Data Warehouse, Azure Functions, Databricks.
• Strong experience in Object and Functional programming.
• Thorough knowledge of REST and SOAP-based APIs to extract data for data pipelines.
• Extensive experience in Hadoop and related processing frameworks such as Spark, Hive, Sqoop, etc.
• Good understanding of working with real-time data streams and Kafka Platform.
• Thorough knowledge of workflow orchestration tools like Apache Airflow design and deploy DAGs.
• Hands-on experience in performance and scalability tuning.
• Prior experience in Agile/Scrum application development using Jira.

Role and Responsibilities

• Actively participate in all phases of the software development life cycle, including requirement gathering, functional and technical design, development, testing, roll-out, and support.
• Solve complex business problems by utilizing a disciplined development methodology.
• Produce scalable, flexible, efficient, and supportable solutions using appropriate technologies.
• Analyze the source and target system data. Map the transformation that meets the requirement.
• Interact with the client and onsite coordinators during different phases of a project.