Data Engineering Pipeline Jobs in Bengaluru

1194 Jobs Found

CO

Distinguished Engineer - Machine Learning Engineering

Capital One

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Distinguished Engineer Machine Learning Engineering Location: Bangalore Company: Capital One India About Us At Capital One India, we re redefining how technology powers financial services. Our teams work in a fast-paced, intellectually rigorous environment to tackle complex business challenges at scale. By harnessing the power of advanced analytics, data science, and machine learning, we create innovative, patentable solutions that transform customer experiences and drive the business forward. Team Overview: Machine Learning Experience (MLX) The MLX team leads Capital One s mission to build scalable, well-managed ML systems and platforms. We empower teams across the enterprise to develop, govern, and deploy machine learning models efficiently, securely, and at scale. From automated model governance to observability platforms, MLX enables end-to-end ML lifecycle management laying the foundation for AI-driven innovation across the organization. Role Overview We re looking for a Distinguished Engineer Machine Learning Engineering to join our MLX team. In this high-impact role, you'll architect and implement the platforms and tools that support model observability, automated governance, and ML model deployment at scale. This is an opportunity to drive enterprise-wide innovation and shape how ML is integrated into Capital One s core business systems. What You ll Do Design and build systems that capture and analyze large-scale model and feature metadata, including training metrics and runtime performance, to power model observability and governance automation. Partner with cross-functional teams including product managers, designers, and platform engineers to create scalable solutions that accelerate ML model lifecycle management. Lead efforts to enable automated governance decisions for ML models, ensuring compliance, auditability, and operational integrity. Architect and implement high-performance data pipelines that feed ML models with real-time and batch data. Contribute to the design and implementation of cloud-native ML systems using tools such as AWS, Kubernetes, and Terraform. Write clean, scalable, production-grade code in languages like Python, Go, or Java. Implement CI/CD pipelines, testing frameworks, and monitoring systems for ML applications. Drive the adoption of best practices in ML Ops, observability, and platform resilience. Basic Qualifications Master s Degree in Computer Science or related field. 15+ years of experience in software engineering or solution architecture. 10+ years building data-intensive, distributed computing systems. 10+ years programming in Python, Go, or Java. 8+ years of hands-on experience with industry-leading ML frameworks (e.g., Scikit-learn, TensorFlow, PyTorch, Dask, Spark). Preferred Qualifications PhD or Master's in Computer Science, Electrical Engineering, Mathematics, or related field. 5+ years of experience building, scaling, and optimizing production ML systems. Deep expertise in data preparation, feature engineering, and ML pipeline optimization. 10+ years writing performant, maintainable, and resilient production code. Strong experience deploying ML solutions on public cloud platforms (AWS, Azure, GCP). Expertise in distributed systems, file systems, or multi-node databases. Open-source contributor to ML tools or libraries. Published work in ML (papers, patents, blogs, etc.). 5+ years of experience in ML Ops (using MLflow, TFX, Kubeflow, etc.). Experience with LLMs and Generative AI applications (open-source or commercial models). Proven experience designing production-ready observability platforms for ML applications. Be at the forefront of building scalable, secure, and enterprise-grade ML platforms. Shape the future of AI and ML adoption in a top-tier financial institution. Collaborate with world-class engineers and data scientists. Solve real-world problems with high business impact. Thrive in a diverse, inclusive, and innovation-focused culture. Qualification : PhD or Master's in Computer Science, Electrical Engineering, Mathematics, or related field

Engineer Distinguished engineer Machine Machine engineer Learning
CO

Data Engineer

Capital One

1+ Year | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Data Engineer Location: Bangalore Company: Capital One India About Capital One At Capital One, we're redefining how technology solves real-world financial challenges. As a technology-driven company, we bring together talented engineers, data scientists, and designers to innovate at scale and deliver meaningful impact to millions of customers. If you're passionate about building powerful data solutions, exploring cutting-edge technologies, and working in a collaborative, fast-paced environment this is the place for you. About the Role As a Data Engineer at Capital One, you ll join a team of innovators who design and build next-generation data platforms and pipelines that power real-time decision-making. You ll collaborate across disciplines engineering, product, machine learning, and cloud infrastructure to transform how we leverage data at scale. What You ll Do Collaborate across Agile teams to design, develop, test, and deploy data-driven solutions. Build and support scalable data pipelines using modern data engineering tools and cloud services. Work on real-time and batch data processing systems that integrate with distributed microservices and ML platforms. Use programming languages such as Python, Java, or Scala with SQL, NoSQL, and cloud data warehouses like Redshift or Snowflake. Contribute to code reviews, unit testing, and performance optimization to ensure high-quality data systems. Partner with product managers and platform teams to deliver robust, cloud-native data solutions that power business decisions. Stay ahead of tech trends, share knowledge, and mentor junior engineers. Basic Qualifications Bachelor s degree in Computer Science, Engineering, or a related field. 1.5+ years of hands-on experience in application or data engineering (excluding internships). At least 1 year of experience working with big data technologies. Preferred Qualifications 3+ years of application/data engineering experience using Python, Scala, Java, or SQL. 1+ year of experience with cloud platforms (AWS, Azure, or GCP). 2+ years of experience with distributed computing tools (Spark, Hadoop, Hive, EMR, Kafka, etc.). 1+ year working on real-time streaming applications. 1+ year of experience with NoSQL databases (MongoDB, Cassandra). 1+ year of experience with data warehousing (Redshift, Snowflake). 2+ years working with Linux/Unix systems and shell scripting. Familiarity with Agile methodologies and modern DevOps practices. Why Join Capital One Work on high-impact data solutions at one of the world s most innovative financial institutions. Be part of a collaborative tech culture that values experimentation and learning. Access to top-tier tools, mentorship, and career development opportunities. Competitive compensation and benefits in a mission-driven environment. Qualification : Bachelors degree in Computer Science, Engineering, or a related field

Data Engineer Data Engineer Full-Time Senior data engineer
5N

Data Science Specialist

5c Network Pvt. Ltd.

1-6 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Position: Data Science Specialist Employment Type: Full-time Location: Bangalore, On-site Experience Required: 1 to 6 years (Mindset > Experience) About the Role: We re seeking a mission-driven, hands-on Data Analytics Specialist passionate about impact. This is not a typical analytics role you will drive key business metrics across clinical operations, stakeholder reporting, and automation. Collaborate closely with leadership, radiologists, operations, product teams, and AI engineers to ensure every decision is data-driven, actionable, and scalable. If you thrive on building dashboards, automating processes, streamlining pipelines, and driving growth through insights, this is your playground. Key Responsibilities: Own Business KPIs: Take full ownership of company-wide metrics, ensuring their accuracy, relevance, and actionability through stakeholder alignment. Data Engineering: Build and optimize ETL/ELT pipelines integrating data from Postgres, ClickHouse, and other sources. Dashboards & Reporting: Design and maintain intuitive dashboards (Metabase, Power BI, Tableau, Google Data Studio) that stakeholders actively use. Backend Data Transformation: Write clean, reusable code to convert raw data into actionable insights. API Development: Build and maintain internal APIs to serve analytics data to frontend and production systems. Email Automation: Develop real-time and scheduled email reports delivering dynamic insights to stakeholders. Spreadsheet Expertise: Automate and manipulate complex data in Excel/Google Sheets for detailed analysis and reporting. AI & Automation: Collaborate with AI teams to integrate predictive algorithms and automate analytics workflows. Business Acumen: Develop deep understanding of the teleradiology ecosystem clinical workflows, operations, financials, and tech platforms. Proactive Collaboration: Identify data gaps, flag inconsistencies, recommend improvements, and work closely with leadership. Required Skills: Strong Python skills for data analytics (Pandas, NumPy, FastAPI, Jupyter, etc.) Proficient in SQL (ClickHouse, PostgreSQL) with experience optimizing queries End-to-end dashboard ownership from data modeling to UI presentation Experience building or integrating APIs for analytics Advanced spreadsheet skills and formula-driven reporting Experience working with AI/ML models in applied settings (preferred) Hands-on backend data transformations, version control, and automation Familiarity with full-stack development frameworks (bonus) Bonus / Nice to Have: Exposure to dbt, Airflow, ChromaDB, Streamlit, Plotly Knowledge of data privacy, compliance, and healthcare analytics Experience building analytics platforms for SaaS or health-tech companies Mindset We re Looking For: 10x Hustler: Ready to learn fast and go the extra mile to solve problems. Obsessed with Accuracy: Data must be flawless before release. Extreme Ownership: Proactive in driving results without waiting for instructions. Fast Learner: Able to pick up new tools and concepts quickly. Business First, Code Later: Focused on impact, not just coding. Be part of India s fastest-growing teleradiology platform Direct access to leadership and real-world business impact Build game-changing analytics systems and influence strategy Gain exposure to AI, automation, product, and operations in one role Clear ramp-up roadmap and growth plan Expectations in First 3 Months: Build an end-to-end dashboard from scratch Clean up at least one messy data pipeline Automate a stakeholder email update Validate accuracy across critical business metrics Understand full-stack platform structure and propose an improvement

Data Science Data Science Specialist Data specialist
PS

Senior Associate Data Engineering L2

Publicis Sapient

6+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Senior Associate Data Engineering L2 Location: Bengaluru, India Department: Engineering Data Employment Type: Full-Time About the Role As a Senior Associate Data Engineering (L2) at Publicis Sapient, you will lead technical solutions that drive digital transformation by building scalable, high-performance data platforms. You ll be responsible for translating business and technical requirements into modern, data-centric solutions using Big Data technologies, cloud services (Azure), and advanced data engineering practices. Key Responsibilities Design and implement data ingestion, integration, and transformation processes from multiple heterogeneous sources in both batch and real-time. Build scalable data platforms using Hadoop stack components such as HDFS, Kafka, Spark, Hive, NiFi, Oozie, Airflow, Flink, and Storm. Develop real-time analytics, aggregation, and search features to support various data-driven applications. Collaborate closely with cross-functional teams on data infrastructure, computation frameworks, and data visualization. Apply cloud-native principles and Azure services to build and deploy data pipelines. Ensure performance optimization and data pipeline tuning. Work with NoSQL and MPP platforms like MongoDB, Cassandra, Redshift, Azure SQL DW, HBase, BigQuery. Contribute to infrastructure, automation, and DevOps for data pipelines using CI/CD practices. Ensure data governance, lineage, and cataloging using tools like Collibra or Alation. Required Qualifications 6 8 years of professional experience in software/data engineering. Minimum of 3 years hands-on experience with Big Data technologies. Strong programming expertise in Java (preferred), Scala, or Python. Expertise in the Hadoop ecosystem and real-time stream processing tools (Kafka, Pulsar, Spark Streaming, etc.). Hands-on experience with Azure data services (e.g., Data Factory, Synapse, ADLS, Databricks). Experience working with modern ETL tools (Informatica, Talend, etc.) and traditional RDBMS platforms (Oracle, PostgreSQL, SQL Server, MySQL). Bachelor's degree in Computer Science, Engineering, or related field. Nice to Have Certifications in Azure Data Engineer, GCP Big Data, or related cloud specializations. Experience with distributed messaging frameworks (ActiveMQ, RabbitMQ, Solace). Familiarity with microservices architecture and search technologies (Elasticsearch). Performance tuning of distributed data processing systems. Exposure to data governance, security, and metadata management. Benefits and Culture at Publicis Sapient Gender-neutral workplace policies 18 paid holidays annually Generous parental leave + new parent transition support Flexible work arrangements Access to Employee Assistance Programs (wellness & well-being) A dynamic culture focused on learning, creativity, and collaboration Qualification : Bachelor's degree in Computer Science, Engineering, or related field.

Senior Associate Senior associate Data Data Associate
SI

Business Technology Data Engineer

Samsara Inc

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Position: Business Technology Data Engineer Location: Bengaluru, India (Hybrid 3 days onsite) Company: Samsara Technologies India Pvt. Ltd. About Samsara Samsara (NYSE: IOT) is a leader in the Connected Operations Cloud, enabling businesses across industries like transportation, logistics, manufacturing, and field services to harness IoT data for safety, efficiency, and sustainability improvements. Samsara helps organizations digitize physical operations at scale, improving outcomes that impact global infrastructure. Role Overview Samsara is seeking a Business Technology Data Engineer to join its Data & Analytics team within the Business Technology division. In this role, you will design, build, and optimize end-to-end data pipelines and infrastructure for various business-critical systems across CRM, marketing, support, and product platforms. You'll collaborate with teams across the company to build reliable and scalable data solutions that power reporting, automation, and analytics. This hybrid role requires working 3 days per week from the Bengaluru office and 2 days remotely, with working hours aligned to India Standard Time (IST). Key Responsibilities Data Engineering & Platform Development Design and maintain ETL/ELT pipelines that integrate and transform data across business systems. Build scalable data infrastructure to support advanced analytics and real-time reporting needs. Write Python and SQL scripts for data ingestion, transformation, and validation. Data Integration & Enablement Work with diverse data sources: CRM, product telemetry, marketing automation, support ticketing, and order flow systems. Develop and support data lake and data warehouse solutions using Snowflake, Redshift, Databricks, or BigQuery. Ensure interoperability between applications and data layers. Performance & Quality Monitor and optimize pipeline performance, implement observability and alerting. Improve data quality, lineage, and governance across systems. Partner with internal stakeholders (e.g., Sales Ops, Marketing Ops, Analytics) to deliver reliable data products. Minimum Qualifications Bachelor s degree in Computer Science, Data Engineering, or related field. 5+ years of professional experience in data engineering. 3+ years experience building and maintaining end-to-end pipelines in a modern data stack. Strong in SQL and Python. Hands-on experience with: ETL tools: Fivetran, dbt Cloud: AWS (preferred), GCP, or Azure Databases: MySQL, PostgreSQL, Oracle, or similar Data Warehouses: Snowflake, Redshift, BigQuery, Databricks Preferred Qualifications Familiarity with API-based ingestion, serverless architecture (Lambda, API Gateway, SQS, etc.). Experience with monitoring tools (DataDog, CloudWatch, Splunk). Comfortable engaging stakeholders to translate business needs into data solutions. Proficiency in Docker, Kubernetes, or AWS Fargate is a plus. Qualification : Bachelors degree in Computer Science, Data Engineering, or related field

Business Technology Data Business Data Data technology
SY

Senior Data Engineer

Synechron

8+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Position Title: Senior Data Engineer Databricks, PySpark, Cloud Platforms Location: Bengaluru Bellandur (GTP) Employment Type: Full-time Job Summary Synechron is looking for a Senior Data Engineer to join our advanced analytics team in Bengaluru. In this role, you will architect and build scalable, high-performance data pipelines that power data science, analytics, and business intelligence initiatives. You ll work with modern tools including Databricks, PySpark, and cloud data platforms, while collaborating across teams to ensure high-quality, secure, and efficient data solutions. Key Responsibilities Design, develop, and maintain large-scale, secure, and efficient data pipelines using Databricks, PySpark, and cloud-native tools. Partner with data scientists, analysts, and business stakeholders to translate requirements into robust data solutions. Integrate data from various structured, semi-structured, and streaming sources. Ensure high standards for data quality, performance optimization, security, and cost efficiency. Drive data pipeline automation, orchestration, and monitoring using tools like Airflow. Lead troubleshooting efforts, performance tuning, and enhancements of existing pipelines. Stay informed about emerging data technologies and recommend adoption where relevant. Technical Skills Core Expertise Programming: Python (expert), SQL (advanced), PySpark. Platforms: Databricks (clusters, notebooks, workflows), AWS/Azure/GCP. Data Orchestration: Apache Airflow (or similar). Data Warehousing: Snowflake (preferred), data modeling, ETL/ELT pipelines. Streaming: Kafka or other stream processing tools. DevOps: CI/CD (GitLab CI, Jenkins), version control (Git), containerization (Docker/Kubernetes preferred). Security: Familiarity with encryption, access controls, and compliance best practices. Experience 8+ years of experience in data engineering or related roles. Proven expertise in developing and deploying scalable data pipelines using Databricks, PySpark, and SQL. Hands-on experience with cloud platforms (AWS, Azure, or GCP). Strong background in data warehousing, especially with Snowflake. Exposure to real-time data processing and orchestration tools. Experience implementing CI/CD pipelines for data workflows is a plus. Daily Responsibilities Build and optimize data ingestion, transformation, and storage workflows. Collaborate with cross-functional teams to align data solutions with business objectives. Monitor, troubleshoot, and continuously improve pipeline performance. Conduct data quality checks, ensure governance and compliance standards. Contribute to technical documentation, code reviews, and team knowledge sharing. Qualifications Bachelor s or Master s degree in Computer Science, IT, or related field. Relevant certifications (e.g., Databricks Certified Data Engineer, AWS Certified Data Analytics) are preferred. Professional Competencies Strong problem-solving and analytical mindset. Effective communicator with ability to collaborate across technical and non-technical teams. Time management and prioritization skills under tight deadlines. Proactive leadership and a passion for innovation. Commitment to ethical data use and data security. Diversity & Inclusion at Synechron Synechron is committed to building an inclusive, diverse, and equitable workplace. Through our global Same Difference DEI initiative, we celebrate and support people from all backgrounds, including race, gender, sexual orientation, religion, age, disability, and more. We offer flexible work arrangements, continuous learning, internal mobility, and mentoring programs to support every employee s growth. Qualification : Bachelors or Masters degree in Computer Science, IT, or related field

Senior Data Engineer Senior engineer Data Engineer
CT

Data Analyst

Camsdata Technologies India Pvt. Ltd.

2+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Data Analyst Bangalore, India Location: Bangalore (Bengaluru) Experience: 2 to 8 Years Industry: IT / Data Analytics Job Summary: We are looking for a detail-oriented Data Analyst with strong skills in SQL, Python, and Excel to extract actionable insights and optimize data pipelines on popular cloud platforms. The ideal candidate will be passionate about clean, reproducible code and thrive in a collaborative team environment. Key Responsibilities: Write efficient SQL queries and develop data extraction, transformation, and loading (ETL/ELT) pipelines on cloud platforms Automate data extraction and data insertion into Management Information Systems (MIS) using Python Analyze data sets and extract actionable business insights to support decision-making Maintain high standards of data quality and ensure reproducible, clean code Collaborate effectively with cross-functional teams and communicate findings clearly Continuously learn and adapt to new analytics tools and techniques Preferred Skills & Qualifications: Strong proficiency in SQL, Python, and Excel Experience building and optimizing data pipelines on cloud platforms such as Google Cloud Platform (GCP) Familiarity with Google BigQuery, Metabase, CleverTap, Google Data Studio, Firebase, and Google Analytics Hands-on experience with data visualization tools like Tableau and Google Data Studio Knowledge of ETL/ELT pipeline development and data flow architecture Excellent attention to detail with a passion for clean and efficient coding Strong interpersonal and communication skills to work collaboratively with diverse teams Good to Have: Experience with automated reporting and dashboard creation Prior exposure to marketing analytics tools and user behavior tracking Work with cutting-edge cloud data technologies and analytics tools Opportunity to grow your skills in data engineering and visualization Collaborative work culture focused on continuous learning and innovation

Data Analyst Data analyst Full-Time Data Analysis
CO

Senior Data Engineer

Cognite

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Senior Data Engineer Location: Bengaluru Department: Global Strategic Services Data Engineering EMEA Type: Full-Time | Hybrid About Cognite Cognite is a global SaaS leader driving industrial digital transformation. Our platforms Cognite Data Fusion and Cognite Atlas AI enable companies across Energy, Utilities, Manufacturing, and Chemicals to solve complex challenges using AI, contextual data, and automation. Cognite is backed by top-tier investors and recognized with global innovation awards. Our Values Impact: We deliver results that matter. Ownership: We take initiative, act inclusively, and embrace accountability. Relentless: We pursue excellence and innovation with resilience. About the Role As a Senior Data Engineer, you ll lead the development of scalable data solutions that empower critical industries to make informed decisions. You ll work on high-impact projects across regions, collaborating with solution architects, data scientists, and product teams. This is a growth role with room to influence product direction and mentor junior engineers. Key Responsibilities Architect and implement robust data pipelines using Cognite Data Fusion, Python, SQL, and REST APIs. Lead integrations, data modeling, and transformation tasks using cloud-native technologies. Design custom data models for discovery, mapping, and cleansing industrial data. Collaborate closely with cross-functional teams to deliver digital solutions. Conduct code reviews and champion engineering best practices. Contribute to Cognite s SDKs and internal tools. Translate customer use cases into scalable, reusable data engineering frameworks. Mentor team members and support customer onboarding when needed. What You ll Bring Bachelor s or Master s in Computer Science or related field (or equivalent experience). 5+ years in a data-intensive, customer-facing engineering role. Expertise in Python, SQL, REST APIs, and pipeline orchestration. Experience with distributed computing, Kubernetes, and cloud platforms (Azure, GCP). Familiarity with data from industrial domains like oil & gas or manufacturing is a plus. Strong DevOps mindset with hands-on experience in Git, CI/CD, and deployments. Proactive and collaborative; able to work independently and solve complex challenges. A growth mindset with willingness to ask for help and share knowledge openly. Be part of a global team spanning 70+ nationalities with strong DEI focus. Work at our Bengaluru office (Rathi Legacy, Hoodi) in a modern hybrid environment. Enjoy flat hierarchy, fast decision-making, and high ownership culture. Collaborate with world-class professionals on industry-transforming projects. Shape the future of industrial data and drive real-world impact at scale. If you re passionate about solving meaningful problems with cutting-edge data technologies, apply today. We welcome applicants from all backgrounds and experiences you might be the perfect fit, even if you don t meet every single requirement. Qualification : Bachelors or Masters in Computer Science or related field (or equivalent experience).

Senior Data Engineer Senior engineer Data Engineer
NE

Lead Data Scientist

Neuron7.ai

10+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Lead Data Scientist Location: Bengaluru, India Employment Type: Full-time, Hybrid About Neuron7.ai Neuron7.ai is a rapidly growing AI-first SaaS company creating service intelligence products that are transforming industries such as high-tech devices, manufacturing, and medical devices. Backed by top Silicon Valley venture capitalists and an esteemed group of angel investors, we are recognized as a startup to watch. Our AI-driven platform helps enterprises make accurate service decisions at scale by delivering service predictions in seconds. By analyzing structured and unstructured data, alongside insights from leading experts, we empower service leaders to improve key metrics like first-call resolution, turnaround time, and service margins. At Neuron7.ai, you'll be part of a dynamic and innovative team that is redefining service intelligence. We value creativity, collaboration, and a commitment to pushing boundaries. Our goal is to lead the AI revolution in service decision-making, and we want you to be a part of that journey. About the Team As a Lead Data Scientist, you ll play a pivotal role in designing, developing, and deploying NLP-driven solutions for processing and analyzing large volumes of unstructured data. Your deep technical expertise will help launch AI products that directly impact users. You ll also mentor junior team members, ensuring technical excellence and fostering growth within the team. You'll collaborate closely with cross-functional teams and customers to solve complex challenges and create impactful AI solutions. What You ll Do: Lead NLP Development: Oversee the development and deployment of NLP-based solutions for processing and analyzing unstructured data at scale. Machine Learning Model Development: Design, train, and optimize machine learning models using popular libraries such as PyTorch, NLTK, and Scikit-learn. Cloud Deployment: Architect and deploy AI/ML products on cloud platforms such as Azure, GCP, or AWS. Collaboration: Work with data engineering teams to ensure seamless integration of AI models into production systems. Data Analytics: Perform advanced SQL analytics to extract actionable insights from structured datasets. Mentorship: Guide and mentor junior data scientists, fostering a culture of technical excellence within the team. Customer Engagement: Work closely with customers to understand their needs and translate those into technical solutions. Communicate complex technical concepts to non-technical stakeholders. Stay Current: Keep up-to-date with the latest advancements in NLP and machine learning to ensure best practices and cutting-edge solutions are applied. What We re Looking For: Experience: Minimum 10 years of experience in data science, specifically focused on NLP and unstructured data processing. Track Record: Proven experience in launching NLP-driven products to live users. Technical Expertise: Expertise in Python and libraries such as PyTorch, NLTK, and Scikit-learn. Hands-on experience with Transformer-based models (e.g., BERT, GPT). Cloud Proficiency: Strong experience with cloud platforms like Azure, GCP, or AWS for hosting and deploying AI/ML products. Data Engineering Knowledge: Familiarity with data engineering pipelines and best practices for scalable AI model deployment. SQL Proficiency: Proficient in SQL for data manipulation and advanced analytics. Problem-Solving: Excellent problem-solving skills with a strong ability to work with large-scale datasets. Leadership & Communication: Strong interpersonal and communication skills, with the ability to mentor team members and interact effectively with customers. Preferred Skills: MLOps Knowledge: Understanding of MLOps practices for model deployment, monitoring, and lifecycle management. Data Processing Tools: Familiarity with tools like Airflow, Spark, or similar frameworks for data processing. Customer-Facing Experience: Background in working with customer-facing applications and APIs to deliver AI-powered solutions. What We Do and Value: At Neuron7.ai, we prioritize integrity, innovation, and a customer-centric approach. Our mission is to enhance service decision-making using advanced AI technology, and we are committed to delivering excellence in all aspects of our work. Company Perks & Benefits: Competitive salary, equity, and spot bonuses Paid sick leave Latest MacBook Pro for your work Comprehensive health insurance Paid parental leave Hybrid work model with flexibility to work from our vibrant Bengaluru office Our Commitment to Diversity and Inclusion: Neuron7.ai is committed to fostering a diverse and inclusive workplace. We ensure equal employment opportunities without discrimination based on race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, marital status, or any other characteristic protected by law. If you re passionate about building innovative AI-driven solutions and want to be part of a forward-thinking team, we d love to hear from you!

Lead Data Data lead Scientist Data scientist
NE

Senior Data Engineer

Neuron7.ai

4+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Senior Data Engineer Location: Bengaluru, India Employment Type: Full-time, On-site About Neuron7.ai Neuron7.ai is a rapidly growing AI-first SaaS company that is revolutionizing the world of service intelligence. Backed by leading Silicon Valley venture capitalists and a distinguished group of angel investors, we are recognized as a startup to watch. Our AI-driven platform delivers service predictions in seconds by analyzing structured and unstructured data, alongside insights from top experts. We specialize in complex service environments like high-tech devices, manufacturing, and medical devices, empowering service leaders to optimize key metrics like first-call resolution, turnaround time, and service margins. At Neuron7.ai, you will be part of a dynamic, innovative team that is pushing the boundaries of service intelligence. We value creativity, collaboration, and a relentless commitment to innovation. This is your chance to join a fast-growing startup and help redefine how AI impacts the future of service optimization. About the Role We are building a real-time analytics platform to improve uptime of connected devices and minimize maintenance costs. This involves collecting, analyzing, and leveraging large volumes of sensor data for observability and predictive analytics. As a Senior Data Engineer, you will have the opportunity to build and maintain scalable, high-performance data systems using modern data engineering tools. The ideal candidate will have hands-on experience with Python, Scala, ELT processes, Lakehouse architecture, Apache Spark, Databricks, real-time streaming systems like Kafka and Flink, and observability tools such as the ELK stack, Prometheus, and Grafana. What You ll Do: Data Model Development: Design and implement advanced data models to extract insights from complex datasets. Collaborative Problem Solving: Work with cross-functional teams to devise data-driven solutions for service optimization. Experimental Analysis: Conduct experiments and analyses to enhance service predictions and outcomes. Actionable Insights: Present findings and actionable recommendations to stakeholders, translating complex data insights into clear, understandable concepts. Mentorship: Guide and mentor junior team members, fostering a collaborative and knowledge-sharing environment. What We re Looking For: Experience: At least 4 years of hands-on experience in data engineering, with a focus on building large-scale data systems. Data Pipeline Expertise: Strong proficiency in Python and Scala for developing data pipelines. Big Data Tools: Proven experience with Apache Spark and Databricks for big data processing and analytics. Observability Tools: Hands-on experience with observability tools like Prometheus, Grafana, and the ELK stack or OpenSearch. Real-Time Data Streaming: Experience working with real-time streaming systems such as Kafka and Flink. Cloud Platforms: Familiarity with cloud platforms like Azure, GCP, or AWS. Problem-Solving Skills: Strong analytical and problem-solving abilities to work with complex data systems in fast-paced environments. Collaboration: Excellent communication and teamwork skills, with the ability to work effectively with cross-functional teams. What We Do and Value: At Neuron7.ai, we prioritize integrity, innovation, and a customer-centric approach. Our mission is to leverage advanced AI technology to enhance service decision-making and we are dedicated to delivering excellence in all aspects of our work. Company Perks & Benefits: Competitive salary, equity, and spot bonuses Paid sick leave Latest MacBook Pro for your work Comprehensive health insurance Paid parental leave Flexible work arrangements work from home or from our vibrant Bengaluru office Our Commitment to Diversity and Inclusion: Neuron7.ai is committed to fostering a diverse and inclusive workplace. We ensure equal employment opportunities without discrimination based on race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, marital status, or any other characteristic protected by law. If you re passionate about using data to drive service intelligence and want to be part of a forward-thinking team, we d love to hear from you!

Senior Data Engineer Senior engineer Data Engineer
ME

Engineering Manager- Data

Meesho

8+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Engineering Manager Data Location: Bangalore, Karnataka | Department: Tech About the Team At Meesho, we power millions of orders daily with zero downtime thanks to our cutting-edge data and backend systems. Our engineering culture is built on speed, ownership, and a growth mindset. We embrace failures as learning opportunities and foster a 'Founder s Mindset' across all engineering teams. We're passionate technologists, team players, and storytellers just as comfortable building robust data platforms as we are bonding over books, games, or chai. If you're ready to lead a team building high-impact, real-time data systems that touch millions of users, come join us. About the Role We are seeking an experienced Engineering Manager Data to lead our data engineering initiatives. You ll be responsible for building scalable data platforms, managing distributed systems, and ensuring data accuracy, accessibility, and real-time analytics capabilities. This role combines technical depth, strategic thinking, and team leadership to build next-gen data systems powering Meesho s mission. What You Will Do Design and allocate technical tasks based on Meesho s data engineering standards and architecture. Lead and mentor a team of data engineers; conduct regular performance reviews and provide growth feedback. Oversee development of scalable, distributed data systems across streaming and batch pipelines. Collaborate with Product, Design, and Sales teams to create innovative data-driven products and features. Monitor project progress, drive sprint planning, and report on key metrics using OKRs. Promote engineering best practices, data quality standards, and cloud-native development. What You Will Need Bachelor s or Master s in Computer Science or a related field. 8+ years of overall experience, with 4+ years managing software/data engineering teams. Hands-on expertise in Java, Python, or Go and multithreading. Experience with distributed data systems, including Apache Spark, Kafka, and streaming architectures. Deep understanding of relational and NoSQL databases. Proficient in cloud platforms (preferably AWS) and containerized deployments. Experience in data modeling, pipeline architecture, and data quality frameworks. Familiarity with BI tools like Tableau, Metabase, or Superset. Good to have: Knowledge of Trino, Flink, Presto, Druid, Pinot, and real-time analytics platforms. Strong project and stakeholder management; ability to lead cross-functional teams. About Us Meesho is India s fastest-growing e-commerce platform on a mission to democratize internet commerce for every Indian. With 1.75M+ sellers and millions of active customers, we offer zero commission, the lowest shipping costs, and unparalleled reach across every serviceable pincode in India. We're building Meesho to be Meri Shop for the next billion users empowering small businesses and first-time entrepreneurs to sell and thrive online. Our Culture & Total Rewards Our culture is rooted in impact, innovation, and inclusivity. Guided by 11 core principles or "Mantras", we empower our teams to think big, move fast, and grow constantly. Daily rituals like Reflections and Listen or Die ensure everyone has a voice and a growth path. What We Offer: Competitive Compensation including salary, performance bonuses, and equity. Wellness & Insurance via our MeeCare program, with extensive coverage for employees and families, plus mental health and wellness initiatives. Work-Life Balance with flexible work hours, generous leave policies, and parental support. Learning & Development with courses, mentorship, and career mobility through our Internal Mobility Program. Perks & Benefits including salary advances, relocation support, recreational discounts, and employee gifts. Join us to lead the data transformation at Meesho and help shape the future of commerce for India s next billion users. Apply now to make your mark! Qualification : Bachelors or Masters in Computer Science or a related field.

Engineering Manager Engineering manager Manager engineering Data
WL

Python/apache Airflow Developer

Wipro Limited

Fresher | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Python/Apache Airflow Developer Location: Bengaluru, India Company: Wipro Limited (NYSE: WIT, BSE: 507685, NSE: WIPRO) Company Overview Wipro Limited is a leading technology services and consulting company focused on building innovative solutions that address clients' most complex digital transformation needs. Leveraging our holistic portfolio of capabilities in consulting, design, engineering, and operations, we help clients realize their boldest ambitions and build future-ready, sustainable businesses. With over 230,000 employees and business partners across 65 countries, we deliver on the promise of helping our customers, colleagues, and communities thrive in an ever-changing world. For more information, visit www.wipro.com. Role Purpose The purpose of this role is to design, test, and maintain software programs for operating systems or applications to be deployed at the client end and ensure they meet 100% quality assurance parameters. Position Overview We are seeking a skilled Python and Apache Airflow Developer with a strong background in data engineering to help develop and maintain robust data pipelines and support business intelligence initiatives. Key Responsibilities Develop and maintain data pipelines using Apache Airflow and Python. Collaborate with business managers to understand data requirements. Optimize existing data workflows for performance and reliability. Manage SQL and NoSQL databases for business intelligence. Ensure data quality and security. Troubleshoot data pipeline issues and ensure smooth data flow. Required Qualifications Proven experience in Python programming and data engineering. Hands-on experience with Apache Airflow. Strong understanding of SQL and experience with SQL databases. Familiarity with NoSQL databases. Experience in data modeling and ETL processes. Must have: Python and Apache Airflow, with preferably 5+ years of experience. About Wipro Wipro is constantly evolving, and we re looking for passionate individuals who want to build a modern digital transformation partner with bold ambitions. Join Wipro, where we empower you to design your own reinvention. Realize your ambitions and take part in our global, purpose-driven business. Applications from people with disabilities are explicitly welcome.

Python Apache Airflow Apache Airflow Developer
IN

Data Integration Developer- Talend

Infocepts

4-7 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Position: Data Integration Developer Talend Location: Bangalore Employment Type: Full-time Experience Required: 4 to 7 years Overview: We are seeking a Data Integration Developer with strong expertise in Talend to join our growing data engineering team. This role is ideal for professionals who are passionate about designing and building robust, scalable ETL/ELT pipelines and integrating enterprise data across cloud and on-premise platforms. You will work closely with data architects, business analysts, and reporting teams to support data transformation, integration, and analytics efforts. Key Responsibilities: ETL/ELT Solutions Development: Design, develop, and deploy scalable ETL/ELT processes using Talend Data Integration (on-prem and cloud). Data Transformation & Integration: Implement data transformation using stored procedures and other data processing techniques. Report Development Support: Collaborate with reporting teams to support dashboard/report creation using tools like MicroStrategy and Power BI. Documentation & Data Governance: Create detailed pipeline documentation, configurations, and data workflows. Performance Optimization: Monitor pipeline performance, identify bottlenecks, and troubleshoot data integration issues. Essential Skills: Hands-on experience with Talend Suite (Talend Data Integration, Talend Cloud) Experience with Snowflake and writing Stored Procedures Knowledge of ETL/ELT design, data warehousing, and data modeling Familiarity with JSON, XML, CSV data formats Experience with data quality frameworks and job monitoring/scheduling tools Proficiency in Agile methodology and tools like JIRA Desirable Skills: Experience with reporting tools such as MicroStrategy and Power BI Skills in scripting languages: Python, Java, or Shell Understanding of cloud platforms (AWS, GCP, Azure) and related data services Qualifications: Bachelor s or Master s degree in Computer Science, Engineering, or a related field Minimum of 3+ years of Talend development experience Qualities We Value: Strong verbal and written communication skills Experience in Agile and cross-functional team environments Problem-solving mindset and ability to work under minimal supervision Quick learner with a passion for data and process improvement Work with modern cloud data integration and enterprise tools Collaborate with global clients and expert teams Be part of cutting-edge data platform transformation initiatives Apply today and be part of our mission to deliver next-gen, data-driven insights. Qualification : Bachelors or Masters degree in Computer Science, Engineering, or a related field

Data Integration Data Integration Developer Data Developer
IN

Senior Data Scientist

Infocepts

5-7 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Position: Senior Data Scientist Location: Bangalore Employment Type: Full-time Experience Required: 5 to 7 years Purpose of the Position: The Senior Data Scientist will play a key role in designing, developing, and implementing machine learning models and algorithms to solve complex business problems. The role involves collaborating with data scientists, software engineers, and business stakeholders to deliver scalable, efficient machine learning solutions that drive innovation and improve business outcomes. Key Responsibilities: Model Development and Optimization: Develop, train, and optimize machine learning models to meet business objectives. Ensure models are accurate, efficient, and scalable. Data Pipeline and Infrastructure: Design and maintain robust data pipelines to support machine learning workflows. Ensure data quality and integrity throughout the data lifecycle. Deployment and Monitoring: Deploy machine learning models into production. Monitor model performance and implement improvements as needed. Collaboration and Communication: Work with cross-functional teams to understand business requirements and translate them into technical solutions. Communicate complex technical concepts to non-technical stakeholders. Research and Innovation: Stay up to date with the latest advancements in machine learning and AI. Experiment with new techniques and technologies to enhance the organization's capabilities. Essential Skills: Machine Learning Frameworks: Experience with ML frameworks such as TensorFlow, PyTorch, and Scikit-learn. Programming Languages: Strong skills in Python and R. Data Processing: Experience with tools like Pandas, NumPy, and Spark. Model Deployment: Experience with Docker, Kubernetes, MLflow, and FastAPI. Databases: Strong experience in databases like Snowflake, MongoDB, and Cosmo DB. Statistical Analysis: Strong foundation in statistics and probability. Model Management: Experience establishing model management, monitoring, support, and maintenance frameworks. ML Lifecycle: Experience setting up ML lifecycle processes and governance policies, as well as providing L0, L1, and L2 support. Desired Skills: Deep Learning: Familiarity with deep learning frameworks like TensorFlow or PyTorch. Natural Language Processing (NLP): Understanding of NLP techniques and applications. Cloud Computing: Experience with cloud platforms like AWS, Azure, or Google Cloud. Big Data Technologies: Knowledge of big data technologies such as Hive, Pig, or Cassandra. Software Engineering: Proficiency in software development and version control systems like Git. Qualifications: Education: Master's degree or Ph.D. in Computer Science, Data Science, Statistics, Mathematics, or a related field. Experience: Over 5 years of experience in machine learning, with a proven track record of developing and deploying models. Certifications: Relevant certifications in machine learning or data science are a plus. Key Qualities: Analytical Thinking: Strong analytical and problem-solving skills. Communication: Excellent verbal and written communication skills. Team Player: Ability to work effectively in a collaborative team environment. Adaptability: Flexibility to adapt to changing business needs and technologies. Innovation: A proactive approach to exploring new ideas and technologies. Apply now to join our dynamic team and lead innovation through machine learning and AI! Qualification : Master's degree or Ph.D. in Computer Science, Data Science, Statistics, Mathematics, or a related field.

Senior Data Scientist Senior Scientist Data scientist
IN

Data Engineer - Snowflake And Dbt

Infocepts

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Position: Data Integration Developer (Snowflake & DBT) Location: Bangalore Employment Type: Full-time Experience Required: 5 to 7 years Purpose of the Position: This role seeks candidates who are passionate about data and motivated to address the organization's analytics challenges. As a Data Integration Developer (Snowflake & DBT), you will play a crucial role in assisting our clients with their data and analytics journey. Your responsibilities will include building robust data pipelines, enhancing data consistency, and developing the necessary infrastructure to support analytics platforms. Key Responsibilities: Technical Expertise: Develop and optimize ETL processes using Snowflake capabilities. Build and maintain advanced data models and ETL processes using DBT (Data Build Tool). Ensure efficient data transformation and integration within the data warehouse. Collaboration: Work closely with data analysts, engineers, and stakeholders to understand business requirements and translate them into technical solutions. Facilitate effective communication and collaboration across teams. Quality Assurance: Implement and oversee data quality checks and testing procedures. Ensure data accuracy, consistency, and compliance with governance policies. Documentation and Reporting: Create and maintain comprehensive documentation for ETL processes and project workflows. Provide regular updates and reports on project progress and performance metrics. Compliance and Security: Ensure all data engineering activities comply with data governance and security policies. Implement measures to protect data integrity and confidentiality. Essential Skills: Proficient in SQL: Skilled in writing efficient queries on large datasets, including CTE and database table design. Snowflake Expertise: Experience with Snowflake or other database technologies, including SQL tuning. ETL Development: Skilled in developing and optimizing ETL processes using Snowflake and other tools. DBT Experience: Familiar with DBT Core/DBT Cloud and fundamental data modeling concepts (3NF, Star, Snowflake). Data Analysis and Profiling: Experienced in data analysis and profiling tasks. Qualifications: Overall Work Experience: More than 5 years of experience, with at least 2-3 years of experience in Snowflake and DBT. Education: Bachelor's degree in Computer Science, Engineering, or a related field (Master's degree is a plus). Key Qualities: Decision Making: Ability to influence and implement change with confidence and sound decisions. Problem-Solving: Tackles problems head-on with a logical, systematic, and practical approach; follows up with developers. Communication: Ability to consult, write, and present persuasively. Team Collaboration: Works effectively in self-organized, cross-functional teams. Apply now and become an integral part of our innovative team working on cutting-edge data integration solutions! Qualification : Bachelor's degree in Computer Science, Engineering, or a related field (Master's degree is a plus).

Data Engineer Data Engineer Snowflake Snowflake engineer
IB

Data Engineer

International Business Machines

7+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Azure Data Engineer IBM Consulting Client Innovation Center Location: Bengaluru, India Experience: 7+ Years (Minimum 4+ Years in Azure Technologies) Job Type: Full-Time Education Required: Bachelor s Degree (Master s preferred) Introduction: Join IBM Consulting at our Client Innovation Center in Bengaluru, where we deliver deep technical and industry expertise to a wide range of public and private sector clients. Our delivery centers focus on innovation, agility, and adoption of next-gen technologies to transform businesses locally and globally. Your Role & Responsibilities: Design and develop scalable data engineering solutions on Microsoft Azure. Build, optimize, and manage data pipelines and ETL workflows using tools like Azure Data Factory, ADLS, and Databricks. Write clean, efficient code using PySpark, PL/SQL, and Spark SQL. Integrate and manage data from various sources, both structured and unstructured. Work with SQL, Postgres, Cassandra, and Cosmos DB. Utilize Azure services such as Stream Analytics, SQL DW, Azure Functions, ARM Templates, and Analysis Services. Implement and maintain serverless architectures for modern data solutions. Ensure high-quality data delivery, security, and compliance across systems. Collaborate across teams and contribute to solution design, code reviews, and documentation. Required Skills & Experience: Bachelor s degree in Computer Science, Information Systems, or related technical field. 7+ years total experience in Data Engineering, with 4+ years in Azure-based data projects. Strong hands-on experience with: Azure Data Factory, Data Lake (ADLS), Databricks Programming: Python, PySpark, PL/SQL, Spark SQL Databases: SQL, Postgres, Cassandra, Cosmos DB Solid grasp of data warehousing, relational databases, and cloud-based architectures. Familiarity with version control tools like Git and CI/CD pipelines. Preferred Skills & Qualifications: Master s Degree in a relevant field. Experience with: ARM Templates, Azure Functions, Serverless Architectures Object-oriented scripting languages (Python, Scala, etc.) Excellent problem-solving and communication skills. Ability to work in a fast-paced environment and collaborate effectively with teams and stakeholders. What You ll Get: Work on high-impact, global projects with cutting-edge Azure technologies. Be part of a collaborative and forward-thinking IBM team. Opportunities for professional growth, upskilling, and certifications. A dynamic work culture focused on innovation, agility, and client success. Join us in Bengaluru and be part of IBM s journey in shaping the future of data engineering. Qualification : Bachelors degree in Computer Science, Information Systems, or related technical field.

Data Engineer Data Engineer Platforms Data Platforms
TE

Sr. Data Engineer

Trellissoft Engineering Services Pvt Ltd

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Job Title: Data Engineer Location: Bengaluru, Karnataka Experience: 5 to 8 Years Work Modality: Full-time (Work from office) Job Description: We are looking for an experienced Data Engineer to join our team and take responsibility for designing, developing, and maintaining scalable ETL/ELT pipelines. This is a full-time position based in Bengaluru, Karnataka, and you will be collaborating with cross-functional teams to define data requirements and ensure data accuracy, consistency, and integrity. Your role will also involve optimizing data workflows, automating processes, and ensuring high availability and reliability of data pipelines. Key Responsibilities: ETL/ELT Pipeline Development: Design, develop, and maintain scalable ETL/ELT pipelines to support data transformation and integration processes. Data Warehouse & Data Lake Optimization: Build and optimize data warehouses, data lakes, and real-time streaming solutions to support large-scale data operations. Collaboration & Data Requirements: Collaborate with cross-functional teams, such as product, data science, and analytics teams, to define data requirements and ensure data accuracy and consistency. Database Structure & Schema Management: Develop and maintain database structures and schemas to ensure efficient data storage and retrieval. Data Workflow Optimization: Optimize data workflows for performance, reliability, and scalability, ensuring the highest level of efficiency. Data Security & Compliance: Implement data security, governance, and compliance best practices to ensure that data is handled securely and meets industry standards. Pipeline Monitoring & Troubleshooting: Monitor, troubleshoot, and improve data pipelines to ensure uptime, reliability, and smooth data processing. Process Automation: Automate data-related processes to improve efficiency and reduce manual intervention, increasing the overall speed of data flow. Required Qualifications: Experience: 5+ years of experience in data engineering or 3-4 years of experience as a Data Engineer. Technical Skills: Strong proficiency in SQL and database management systems such as PostgreSQL, MySQL, SQL Server, etc. Experience with ETL tools such as Pentaho, Talend, Cdata, and SSIS. Exposure to Python, Java, or Scala for data processing is a plus. Experience with big data technologies such as Apache Spark, Hadoop, or Kafka. Familiarity with cloud services (AWS, Azure) and data storage solutions such as S3, Redshift, Snowflake, or BigQuery. Strong knowledge of data modeling, warehousing concepts, and data architecture best practices. Soft Skills: Excellent communication skills with the ability to collaborate effectively across teams. Strong problem-solving skills and the ability to work with large, complex datasets. What We Offer: Competitive Salary: Attractive salary based on experience and expertise. Collaborative Work Environment: Work in a dynamic and fast-paced environment with a team that fosters innovation and collaboration. Growth Opportunities: Opportunities to enhance your skills and career growth in the data engineering field. Comprehensive Benefits: Benefits package designed to support work-life balance and overall employee well-being.

Sr. Data Engineer Sr. engineer Data Engineer
KT

Data Engineer

Kpit Technologies

5-8 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Job/Position Summary: Data Engineer Responsibilities: Implement data pipelines that meet design and are efficient, scalable, and maintainable. Implement best practices including proper use of source control, participation in code reviews, data validation and testing. Timely deliveries while working on projects. Act as advisor/mentor and helps junior data engineers in their deliverables. Must Have Skills: Should have experience of at least 4+ years with Data Engineering. Strong experience of design, implementation and fine-tuning big data processing pipelines in production environment. Experience with big tools like Hadoop, Spark, Kafka, Hive, Databricks. Experience in programming at least one of with Python, Java, Scala, Shell Script. Experience with relational SQL and NO SQL databases like PostgresSQL, MYSQL, Cassandra etc. Experience with any data visualization tool (Plotly, Tableau, Power BI, Google Data Studio, Quick sight etc.). Good To Have Skills: Should have Basic Knowledge of CI/CD Pipeline. Experience in working on at least one Cloud (AWS or Azure or GCP). For AWS: - Experience with AWS Cloud services like EC2, S3, EMR, RDS, Athena, Glue, Lambda, EMR. For Azure: -Experience with Azure Cloud services like Azure Blob/Data Lake GEN2, Delta Lake, Databricks, Azure SQL, Azure DevOps, Azure Data Factory, Power BI. For GCP: - Experience with GCP Cloud services Big Query, Cloud Storage bucket, DataProc, Dataflow, Pub Sub, Cloud Function, Data Studio. Sound familiarity in Versioning tools (Git, SVN etc.). Experience Mentoring students is desirable. Knowledge of latest developments in Machine Learning, Deep Learning, Optimization in Automotive domain. Open minded approach to explore multiple algorithms to design optimal solution. History of contribution to articles/blogs/whitepapers etc. in Analytics. History of contribution to Open Source. Requirement: ESSENTIAL SKILLS /COMPETENCIES Data Engineering Hadoop Kafka CI/CD Cloud

Data Engineer Data Engineer Full-Time Data Engineering
M&

Data Engineer - Platform Generative Ai

Mckinsey & Company

3+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Your Impact We are seeking a passionate Data Engineer with expertise in Python development who is excited about cloud-based data engineering using AWS services. You will be an integral part of a dynamic, multi-disciplinary team, working closely with digital product professionals, data scientists, cloud engineers, and other stakeholders. As a key member of a global team working on our generative AI initiative, you will be based in one of our European offices. McKinsey s Tech Ecosystem function is responsible for developing and delivering all technology solutions for the firm s internal use, and your role will be crucial in driving the development of data solutions to support generative AI applications. You will work with a team of data engineers to develop robust data ingestion pipelines and enhance data processing capabilities that integrate data into systems used by AI applications. Your responsibilities will include writing Python code, creating tests, developing and maintaining GitHub Action CICD pipelines, and managing AWS-based infrastructure and Docker containers. Your Growth As a member of the global team working on our generative AI initiative, you will play a key role in shaping and accelerating the delivery of McKinsey's target state data platform, which will enable AI use-cases. You will be part of our cloud-first approach, transforming data platforms and analytical applications across the firm. Working closely with multidisciplinary teams, you will contribute to building cutting-edge data solutions in a fast-paced, innovative environment. McKinsey s Tech Ecosystem function is responsible for developing all technology solutions for the firm s internal needs, and you ll have the opportunity to shape how these solutions evolve. Your Qualifications and Skills 3+ years of professional experience as a Data Engineer, with a focus on cloud-based data engineering using AWS services Expertise in Python development and a strong understanding of clean code, modularity, error handling, and test automation Extensive experience with relational databases and data pipeline performance Hands-on experience with Docker and CI/CD pipelines (e.g., GitHub Actions) Strong execution focus, with the ability to work independently in complex, fast-paced environments and deliver results Demonstrable experience in solving data pipeline performance issues and diagnostics Interest in generative AI and machine learning topics Experience with Kedro framework is a plus Opinionated and confident in sharing ideas, willing to speak up at all levels Familiarity with Agile principles and product development methodologies Excellent problem-solving skills and the ability to analyze and resolve complex data engineering challenges Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams

Data Engineer Data Engineer Platform Data platform
M&

Data Engineer Ii

Mckinsey & Company

2-5 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Your Impact As a Data Engineer at QuantumBlack, you will collaborate with stakeholders, data scientists, and internal teams to develop and implement impactful data products and solutions. Your key responsibilities will include building and maintaining technical platforms for advanced analytics, designing scalable and reproducible data pipelines for machine learning, and ensuring information security within data environments. You will assess clients' data quality, map data fields to hypotheses, and prepare data for use in analytics models. Additionally, you will contribute to R&D projects, internal asset development, and participate in cross-functional problem-solving sessions with a variety of stakeholders, including C-level executives, to create innovative analytics solutions. You will be based in Gurugram, joining a global data engineering community, and work within cross-functional and Agile project teams alongside project managers, data scientists, machine learning engineers, other data engineers, and industry experts. You will collaborate directly with our clients, ranging from data owners and users to C-level executives. You will be aligned with one of our industry-focused practices: Pharma & Medical Products (PMP) or Global Energy & Materials (GEM). In these practices, you ll work on solving the most critical challenges for our clients in these sectors. PMP focuses on advancing the development and delivery of life-saving medicines and medical treatments, while GEM supports industries like chemicals, steel, mining, and energy to achieve operational excellence. GEMx and PMPx, the assetization arm of these practices, focus on creating reusable digital and analytics assets to support client work. As part of this team, you will help shape impactful solutions for large organizations, developing capabilities for sustained impact. Your Growth In this role, you will contribute to the frameworks and libraries that our teams of Data Scientists and Engineers use to progress from data to meaningful impact. You will have the opportunity to guide global companies through data science solutions, helping them transform and enhance performance across industries including healthcare, automotive, energy, and elite sports. Real-World Impact: You ll gain unique learning and development opportunities globally. Fusing Tech & Leadership: Work with the latest technologies and methodologies, with access to top-tier learning programs. Multidisciplinary Teamwork: Collaborate with data scientists, engineers, project managers, UX designers, and more to enhance performance. Innovative Work Culture: Creativity, passion, and wellness are central to our modern work environment, which includes insightful talks, training sessions, and a focus on work-life balance. Striving for Diversity: We celebrate diversity, with colleagues from over 40 nationalities, appreciating the value that diverse perspectives bring to the workplace. You are a highly collaborative individual who prioritizes impact over agenda. You enjoy learning from colleagues, challenging ideas thoughtfully, and working together to improve processes and solve problems. You believe in iterative change, experimenting with new approaches, and advancing quickly through constant learning and improvement. While we value using the right tech for the right task, our team often leverages technologies such as Python, PySpark, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions like AWS, GCP, and Azure. Your Role as a Data Engineer Collaboration: Work with business stakeholders, data scientists, and internal teams to create extraordinary, domain-focused data products (reusable assets) and deliver them to clients. Domain Expertise: Develop deep understanding of client industries and use creative techniques to deliver meaningful impact. Technical Platforms: Build and maintain technical platforms for advanced analytics engagements, spanning both data science and data engineering work. Data Pipelines: Design and implement robust, modular, scalable, deployable, and reproducible data pipelines for machine learning. Data Management: Ensure the security of data environments and compliance with information security standards. Data Wrangling: Assess data quality, map data fields to hypotheses, and prepare data for analytics models. Contribute to R&D: Participate in internal asset development and contribute to R&D projects to drive innovation. Cross-functional Problem Solving: Collaborate with internal teams and clients, including data owners and C-level executives, to create impactful analytics solutions. Your Qualifications and Skills Bachelor s degree in computer science or related field; Master's degree is a plus. 2-5 years of relevant work experience. Proficiency in at least one programming language such as Python, Scala, or Java. Strong experience with distributed processing frameworks (e.g., Spark, Hadoop, EMR) and SQL. Experience in commercial client-facing projects, particularly in close-knit teams. Ability to work with structured, semi-structured, and unstructured data, and identify linkages across disparate data sets. Clear communication skills to explain complex solutions effectively. Understanding of information security principles to ensure compliant handling of client data. Experience with cloud platforms (AWS, Azure, Google Cloud, Databricks) is highly desirable. Experience with CI/CD processes using GitHub Actions, CircleCI, or similar, and end-to-end pipeline development including application deployment is a plus. Qualification : Bachelors degree in computer science or related field; Master's degree is a plus.

Data Engineer Data Engineer Ii Engineer ii

1 - 20 of 0 jobs

* No exact matches found. Showing closest results instead
Sort by:

No results found

Modify search criteria or create an alert to get relevant jobs as soon as they’re posted

Create an alert

Continue to Save

Please login to your jobseeker account, or create a new one to save this job.

Feedback

Share Feedback