Senior Azure Data Engineering Consultant Job in Procogia

Job Description The Senior Data Engineer has the following responsibilities and duties: Create and maintain optimal data architecture pipeline. Perform root cause analysis on internal and external data, processes and share insights with different stakeholders using reporting and visualizations tools. Design infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources into Snowflake and other platforms Understand existing implementation and should enhance it as needed. Work with other data engineers, data ingestion specialists, and experts across cross-functional teams. Work on large datasets to meet functional and business requirements Requirements The Senior Data Engineering Consultant should have the following skills, education, and experience: Skills Required: Strong experience with Snowflake Data Warehouse Excellent SQL and Python, PySpark skills using Spark framework. Strong knowledge across Microsoft Azure cloud services to build scalable data pipelines and workflow management using ADF, Dataflow , usage of Databricks with PySpark Experience with ETL/ELT and familiarity of Data Warehouse concepts Expertise in designing and working with efficient Data Models in big data ecosystem. Experience with Docker/containerization Knowledge of API-based integration and security (Restful and SOAP) Experience with Data Visualization tools such as Power BI etc. Ability to learn fast and translate data into actionable results Excellent communication skills with onsite team and client stakeholders Preferred: Experience with JSON Programming Solid Linux programming skills Cosmos DB, Teradata, SQL Server (all nice to have) Stream-processing systems: Streaming-Analytics, IoT Hub, Event Hub, Kusto Queries (all nice to have)