Software Engineer 3 (lead Data Engineer)

Summary of Essential Job Functions: Design, develop, and maintain scalable and efficient data pipelines using Python, Pandas, Spark, and Dataflow. Strong expertise in relational databases, including SQL Server and MySQL, coupled with a deep understanding of cloud platforms. Construct and integrate large, complex data sets to fulfill both functional and non-functional business requirements. Analyze and integrate diverse systems and data sources through ETL processes and logic, ensuring timely and accurate data delivery to visualization teams and business stakeholders. Optimize and troubleshoot data pipelines to enhance performance, reliability, and scalability. Ensure data integrity and quality across the pipeline by implementing effective validation and testing strategies. Identify, design, and implement internal process improvements such as redesigning infrastructure for scalability, improving data delivery methods, and automating manual tasks. Collaborate with cross-functional teams to understand business data requirements and deliver solutions that meet organizational needs. Develop and maintain comprehensive documentation for data pipelines, including design specifications, code comments, and operational guides. Provide technical support for production issues, including troubleshooting, query optimization, and process enhancements. Offer technical guidance and mentorship to junior team members, promoting best practices and fostering continuous learning. Knowledge of data analytics tools such as PowerBI, Tableau, Data Studio, and Apache Superset is highly desirable. Work Experience & Qualifications: 8-10 years of experience in data design, development, and pipeline management. Excellent communication skills with the ability to engage effectively with both customers and internal teams. Bachelor s degree in Computer Science, Information Science, Data Science, or a related field. Qualification : Bachelors degree in Computer Science, Information Science, Data Science, or a related field.