Airflow Jobs in Chennai

Job Title: Data Engineer Location: Chennai (Work from Office) Job Type: Full-time Experience: 2 to 4 years Job Description Blackstraw Technologies Pvt Ltd is seeking a skilled Data Engineer to join our dynamic team. You will play a crucial role in transforming raw data into actionable insights by designing and maintaining robust data pipelines and systems. Your work will ensure alignment between data infrastructure and business goals, helping to drive efficiency and scalability. You will leverage your analytical skills and proficiency in various programming languages to integrate, process, and optimize data from diverse sources. You will also collaborate closely with data scientists and end-users to support data-driven decision-making. Key Responsibilities Design, build, and integrate data pipelines from multiple sources for analytics and operational purposes. Perform data integration and troubleshoot production and development issues. Develop, automate, and optimize ETL pipelines and workflows. Work with big data technologies including Hive, Spark, and Hadoop; optimize performance for large-scale data processing. Write and maintain efficient Spark/PySpark or Spark/Scala programs. Utilize strong SQL skills for querying and database design. Develop shell scripts (Unix/Linux) to automate data processing tasks. Collaborate with data scientists and various business teams to understand and fulfill data needs. Maintain and deploy ETL code following Agile methodologies. Handle service delivery tickets and provide support related to ETL tools. Participate in root cause analysis and problem resolution for data pipeline issues. Required Skills & Qualifications Bachelor s degree in Computer Science, Information Technology, or related field; Master s degree is a plus. 2 to 4 years of hands-on experience in data engineering or related roles. Strong expertise in ETL development and data integration tools. Proficiency in Spark (PySpark or Scala), Hive, and Hadoop ecosystem. Advanced SQL skills and experience with relational database management systems (RDBMS). Experience with shell scripting in Unix/Linux environments. Familiarity with Java, Python, or Scala programming languages is desirable. Knowledge of Cloud platforms (Azure preferred) is a plus. In-depth understanding of Retail domain data and business processes is advantageous. Excellent analytical, organizational, and communication skills. Experience with Agile project management and teamwork. About Blackstraw Technologies Pvt Ltd Founded in 2015 and fully operational since 2018, Blackstraw Technologies is a global software products and services company specializing in Artificial Intelligence (AI) and Machine Learning (ML) solutions. Our AI platform accelerates data labeling, model training, and deployment across cloud and on-premise environments. Our 250+ member team boasts over 900 years of combined experience, focusing on areas such as computer vision, natural language processing, and deep learning annotation tools. We enable enterprises worldwide to rapidly adopt AI in a cost-effective, plug-and-play manner. Headquartered in Florida, USA, Blackstraw has operations across Canada, the USA, and India. Qualification : Bachelors degree in Computer Science, Information Technology, or related field. Masters degree is a plus.

Business Intelligence Lead Experience: 6+ Years Location: Chennai Company: Kaleidofin About Kaleidofin Kaleidofin is a pioneering fintech platform creating a digital ecosystem to provide financial access for everyone, everywhere. Through our ki credit platform, we deliver credit health assessments, middleware integration, advanced risk management dashboards, and debt capital market structuring enabling financial institutions to serve underserved segments like women entrepreneurs, farmers, and MSMEs across India, Bangladesh, and Kenya. With over 6 million customers impacted and more than $3 billion facilitated in debt capital, Kaleidofin is backed by global investors including the Gates Foundation, Omidyar Network, and Blume Ventures. Recognized internationally by Forbes Asia and the United Nations, and winner of RBI s Swanari TechSprint and G20 TechSprint 2024, we are committed to innovation and impact. Role Overview We are seeking an experienced Business Intelligence Lead to drive data analytics and reporting efforts that inform strategic decisions across risk, credit, compliance, fraud detection, and product performance. The ideal candidate will bring strong leadership, a deep understanding of financial services data, and hands-on expertise in BI tools and data governance. Key Responsibilities Team Leadership & Development Lead and mentor the Data Analytics team; identify skill gaps and implement training plans. Data Analysis & Insights Leverage BI tools (Power BI, Tableau, Looker) to deliver actionable insights that drive business and partner strategies. Develop and optimize dashboards and reports aligned with business goals. Data Visualization & Reporting Create clear, interactive visualizations for diverse stakeholders, ensuring accessibility and timeliness. Manage robust data pipelines using ETL/ELT tools like Informatica, Talend, or Airflow. Collaboration & Stakeholder Management Partner with cross-functional teams to gather data requirements and support executive, regulatory, and product reporting needs. Present insights to senior leadership to promote data-driven decision-making. Process Improvement & Automation Automate routine reporting and continuously enhance BI tools and workflows for efficiency and accuracy. Data Governance & Compliance Maintain data quality standards and governance practices ensuring compliance with data privacy regulations. Interface with auditors and compliance teams as required. KPI Management Achieve >90% data accuracy through audits and corrections. Ensure >80% user adoption of BI products by monitoring engagement metrics. Deliver cost savings by evaluating BI-driven efficiency improvements. Who You Are Bachelor s or Master s degree in Computer Science, Data Science, Information Systems, or related fields. 6+ years in BI, analytics, or data architecture, with 3+ years in leadership or architect roles. Hands-on experience with BI platforms (Power BI, Tableau, Looker) and proficient in SQL querying. Strong knowledge of financial data domains: credit scoring, transactions, KYC, fraud, lending, payments. Experience with credit bureau data (Experian, Equifax, CRIF) preferred. Experience with cloud data platforms (AWS, GCP, Azure) and data warehousing concepts. Familiarity with machine learning applications in BI or data science is a plus. Relevant certifications such as Microsoft Data Analyst Associate or AWS Certified Data Analytics are advantageous. Join a high-impact team revolutionizing financial inclusion. Work with cutting-edge BI technologies in a fast-growing fintech startup. Thrive in a culture that values innovation, collaboration, and continuous learning. Qualification : Bachelors or Masters degree in Computer Science, Data Science, Information Systems, or related fields

About Arcadia: Arcadia is a leading technology company focused on solving the climate crisis by empowering energy innovators and consumers. We re revolutionizing the energy industry with our software and APIs, creating access to the data and clean energy necessary for building a decarbonized energy grid. Since our founding in 2014, Arcadia has been working to break the fossil fuel monopoly and offer high-quality clean energy options to consumers and small businesses. Today, we ve launched Arcadia Platform, a SaaS product that is transforming the energy industry into a data-driven network. With a $1.5B valuation and $500M raised, Arcadia is scaling its operations to solve one of the world s most significant challenges. We re building a diverse team of individuals who are passionate about making a difference. Join us in shaping a clean energy future! What We re Looking For: We are seeking a thoughtful, empathetic, and driven Engineering Manager to lead and grow our team of four engineers responsible for building data products for Arcadia s customers and internal stakeholders. You will guide the team to solve complex data engineering and analytics problems, while ensuring their professional growth and development. You will also work closely with the Director of Data and Analytics Engineering to develop roadmaps and strategies that align with the business needs. The role is based in Chennai, India, with a hybrid remote-first approach, and offers the opportunity to shape a world-class data ecosystem in a fast-paced, high-growth environment. What You'll Do: Empower and guide engineers: Lead a team of engineers in solving complex data problems within the energy sector. Work with cross-functional teams: Collaborate with Product, Design, and Marketing teams to define roadmaps and strategies for our Platform business. Drive technical excellence: Contribute to technical discussions and ensure that the team is building scalable data pipelines and assets. Mentor and develop team members: Provide ongoing mentorship, aligning team members skills, interests, and career aspirations with business needs. Facilitate regular feedback and performance discussions. Lead by example: Foster a culture of empathy and humanity while delivering results in a fast-paced environment. Align stakeholder expectations: Represent engineering concerns to ensure alignment with business priorities and goals. What Will Help You Succeed: Must-Haves: 6+ years of experience, with 2+ years in a leadership/management role on a data engineering or analytics engineering team. Deep understanding of modern data and analytics engineering practices, with the ability to contribute hands-on when necessary. Experience with Python (in a data orchestration context) and SQL (in a data modeling context). A proven track record of mentoring and growing direct reports through feedback, goal-setting, and career development. Ability to navigate the relationship between Engineering, Product, and Design teams. Nice-to-Haves: Experience with tools and technologies in Arcadia s data stack: Snowflake dbt (or similar Analytics Engineering tools) Prefect (Airflow experience is a plus) AWS, Docker, Kubernetes (basic familiarity) GitHub and GitHub Actions Experience with scalability challenges at fast-growing startups. Familiarity with automating internal processes to improve operational efficiency. Experience with roadmap development and execution. Background in the energy sector or related fields. Benefits: Competitive compensation based on market standards. Hybrid remote-first model, promoting flexibility in work arrangements. Flexible leave policy to support work-life balance. Medical insurance for you and your family (up to 6 members), including accident and life insurance. Annual performance cycle with rewards and recognition. Quarterly team engagement activities to enhance collaboration and team spirit. Learning & Development programs to support continuous growth and skill-building. A supportive engineering culture that emphasizes diversity, empathy, teamwork, trust, and efficiency. Why Arcadia? At Arcadia, we celebrate diversity and believe that unique perspectives are essential to our collective success. We are committed to equal employment opportunities and welcome applicants from all qualified candidates eligible to work in India, regardless of race, color, religion, gender, sexual orientation, gender identity or expression, national origin, age, disability, genetic information, or protected veteran status. We are committed to providing reasonable accommodations for individuals with disabilities during the application process, interviews, or employment. Please contact us if you require accommodation. How to Apply: If you are excited about building the next-generation data ecosystem and contributing to Arcadia s mission of a clean energy future, we encourage you to apply. We look forward to learning what you will uniquely bring to our team! Qualification : 6+ years of experience, with 2+ years in a leadership/management role on a data engineering or analytics engineering team.