Lead Data Engineer AWS Job in Infraveo Technologies

We are currently seeking a Lead AWS Platform Data Engineer to drive the evolution of our data infrastructure and unlock the full potential of our meter data. Currently, we leverage a suite of proprietary tools along with Git and Snowflake for our data warehousing needs.

Responsibilities:

• Leading the transition from proprietary tools to AWS-native technologies for meter data ingestion, integrating seamlessly with Snowflake. Develop solutions to facilitate easier data handling and identify data issues.
• Implementing processes and technologies to enable real-time or half-hour meter data ingestion, improving data timeliness and accuracy.
• Oversee the creation and management of a self-service data catalogues. Enable analysts to access and utilise data sources effectively for real-time reporting and analysis.
• Utilise AWS services (e.g. Lambda, Glue, Kinesis) to build and optimise data pipelines and infrastructure.
• Continuously monitor and enhance data quality and streamline data processes to ensure accuracy and efficiency.
• Work closely with stakeholders to understand their data needs, deliver tailored solutions, and support business objectives.
• Stay updated on the latest AWS technologies and data engineering best practices. Drive innovation in data processing and reporting solutions.
  
  

Requirements:

• Extensive experience with AWS services relevant to data ingestion and processing (e.g., Lambda, Glue, Kinesis) and data warehousing with Snowflake.
• Strong background in developing and optimising data pipelines and data management systems.
• Proven experience in developing and managing PowerBI reports and self-service data catalogues.
• Proficiency in programming languages such as Python and SQL.
• Experience with real-time data ingestion and processing solutions.
• Proven ability to lead and mentor a team of data engineers, with strong project management skills.
• Expertise in improving data quality and optimising data workflows.
  
  

Desirable:

• Experience with meter data ingestion systems and familiarity with industry-specific requirements.
• Knowledge of advanced analytics techniques and tools.
• Experience with integrating cloud data solutions with various data sources and systems.
• AWS Professional Certification or Data Analytics/Machine Learning.
  
  

Benefits:

• Work Location: Remote
• 5 days working​