BIG Data Processing Jobs in Bengaluru

Lead Machine Learning Engineer - NLP Location: Bengaluru About Us: Observe.AI Observe.AI is the leading AI agent platform for customer experience. It enables enterprises to deploy AI agents that automate customer interactions, delivering natural conversations for customers with predictable outcomes for the business. Observe.AI combines advanced speech understanding, workflow automation, and enterprise-grade governance to execute end-to-end workflows with AI agents. It also enables teams to guide and augment human agents with AI copilots, and analyze 100% of human and AI interactions for insights, coaching, and quality management. Companies like DoorDash, Affordable Care, Signify Health, and Verida use Observe.AI to transform customer experiences every day by accelerating service speed, increasing operational efficiency, and strengthening customer loyalty across every channel. You will be shaping how AI transforms real-world challenges in the contact center space. As part of our world-class ML team, you ll work on developing cutting-edge LLM-powered solutions & Agentic AI, building end-to-end processing pipelines, and handling production challenges at scale millions of interactions daily. If you are truly an engineer at heart, excited about turning breakthroughs in multi-agent systems, LLMs, NLP, and ML into practical outcomes through applied research, and building scalable production systems, you will feel right at home. You ll also have the opportunity to publish in top conferences, and influence Observe.AI s product and platform strategy. What you ll be doing Design & develop state-of-the-art LLM-powered AI capabilities and Agentic AI/ Multi-agent systems end-to-end, from ideation to production for Observe.AI s product offerings, in a fast-paced startup environment. Work with cutting-edge tools and technologies in Machine Learning, Deep Learning & Natural Language Processing, including LLMs and LLM-powered technologies/ paradigms, including Agentic AI. Build/ maintain highly scalable production systems that power AI capabilities on Observe.AI product/ platform. Optimize ML models and processing pipelines for performance, cost-effectiveness, and scale. Work with a world-class ML team in building exciting stuff, mentor juniors, and influence peers/ stakeholders. Collaborate cross-team with engineers, product managers, customer-facing teams, and customers to understand pain points and business opportunities. Keep up-to-date with the latest ML/ DL/ NLP literature and influence the technological evolution of Observe.AI platform. Contribute to the community through tech blogs and publishing papers in ML/ NLP conferences like EMNLP, ACL, etc. What you ll bring to the role Education: Bachelor s or Master s degree in Computer Science or related disciplines from a top-tier institution with exposure to ML/ DL/ NLP/ NLU. An engineering mindset with the competencies of an applied scientist. 5+ years of industry experience in building large-scale NLP/ NLU systems, with recent experience in building LLM-powered applications and Agentic systems. Strong understanding of the fundamentals of ML and NLP/ NLU, and practical aspects of building ML systems in production. Good understanding of recent advances in building LLM-powered applications, and multi-agent systems at scale. Excellent implementation skills in Python and Machine Learning Frameworks such as Pytorch, Tensorflow, HuggingFace, etc., and deploying/ maintaining scalable machine learning systems in production. Ability to provide thought leadership in one or more technical areas of interest to Observe.AI, and influence product development. Excellent communication, collaboration skills, and presentation skills. Experience with Spoken Language Understanding is a plus. Published papers in top NLP/ NLU conferences or workshops are a plus. Relevant open-source contributions are a plus. Perks & Benefits Medical Insurance: Excellent options and free online doctor consultations. Leave Policies: Yearly privilege and sick leaves as per Karnataka S&E Act, generous holidays (National and Festive) recognition and parental leave policies. Learning & Development fund to support your continuous learning journey and professional development. Fun events to build culture across the organization. Flexible benefit plans for tax exemptions (i.e. Meal card, PF, etc.). Qualification : Bachelors or Masters degree in Computer Science or related disciplines from a top-tier institution with exposure to ML/ DL/ NLP/ NLU

Position: Mechatronics & Big Data Scientist Developer Department: Research & Development Reporting To: General Manager Location: Bengaluru Key Responsibilities Machine Learning: Select features, build, and optimize classifiers using advanced machine learning techniques. Data Mining: Perform data mining using state-of-the-art methods to extract valuable insights from large datasets. Data Enhancement: Extend the company s datasets with third-party data sources when necessary to improve model accuracy and relevance. Data Collection & Processing: Improve data collection procedures to include all necessary information for building analytic systems. Data Cleansing & Integrity: Process, cleanse, and verify the integrity of data used for analysis to ensure reliable results. Ad-hoc Analysis: Perform ad-hoc analysis as needed, presenting the results in a clear, actionable manner. Anomaly Detection: Design and implement automated anomaly detection systems, tracking their performance over time to ensure accuracy. Behavioral Competencies Data-Driven: Strong inclination toward working with data and applying analytical thinking to solve complex problems. Detail-Oriented: Meticulous in data analysis and system development to ensure quality and precision in results. Skills and Expertise Machine Learning Algorithms Strong understanding of machine learning techniques and algorithms such as k-NN, Naive Bayes, SVM, Decision Forests, etc. Data Science Tools Experience with common data science toolkits like R, Weka, NumPy, and MatLab. Proficiency in at least one (preferably NumPy or R) is highly desirable. Data Visualization Skilled in data visualization tools such as D3.js, GGplot, or similar. Database Management Experience with query languages such as SQL, Hive, Pig, NiFi, or others depending on the company s stack. Familiarity with NoSQL databases like InfluxDB, MongoDB, Cassandra, HBase. Statistical Analysis Strong applied statistics skills, including distributions, statistical testing, and regression analysis. Programming Skills Good scripting and programming skills in languages like PHP, Slim, SQL, and Laravel. Big Data Technologies Knowledge of Hadoop, HDFS, NiFi, and other big data platforms and technologies. Qualifications Essential: MTech, MS, or equivalent in Mechatronics, Computer Science, or a related field. Experience: Minimum 2 years of hands-on experience in developing SDKs and working with Big Data platforms. Proven track record in machine learning, data mining, and data science projects. Qualification : MTech, MS, or equivalent in Mechatronics, Computer Science, or a related field

Data Engineering Lead Bengaluru | Engineering | Full-Time About Fam (formerly FamPay) Fam is India s first payments app designed for everyone aged 11 and above. FamApp enables seamless online and offline payments through UPI and FamCard. Our mission is to empower over **250 million young Indians** to start their financial journey early, becoming financially aware and confident. Founded in 2019 by IIT Roorkee alumni, Fam is backed by top-tier investors including Elevation Capital, Y-Combinator, Peak XV (Sequoia Capital India), Venture Highway, and angels like Kunal Shah and Amrish Rao. About the Role We re looking for a visionary **Data Engineering Lead** to take **end-to-end ownership** of Fam s data ecosystem from data ingestion and storage to processing and delivering actionable insights. You ll **define the data strategy and architecture** that supports both batch and **real-time** use cases, ensuring scalability, reliability, and governance across the organization. You will be instrumental in enabling accurate, complete, and trusted data flow that powers business intelligence, analytics, and product decision-making. This role involves **leadership, strategic thinking**, and hands-on problem solving. What You ll Do Own the full data lifecycle: ingestion, organization, storage, processing, and presentation. Define and execute **data architecture and strategy** aligned with operational and analytical goals. Build **scalable, reliable, and observable data systems** supporting batch and near real-time processing. Ensure **data quality, governance, and compliance**, proactively resolving discrepancies. Collaborate with product, engineering, and business teams to define, track, and optimize key metrics. Anticipate data-related challenges and implement preventive solutions. Lead, mentor, and grow the data engineering team, fostering innovation and accountability. Must-Haves 10+ years experience in data engineering, including proven leadership of teams or projects. Expertise designing, building, and scaling end-to-end data pipelines and systems. Deep understanding of the data lifecycle, from ingestion through business reporting. Strong communication skills and ability to collaborate across technical and business teams. Solid knowledge of **data governance, quality assurance, and compliance standards**. Experience with observability and proactive monitoring for data systems. Proficiency in Python and SQL; familiarity with Scala or Java. Hands-on experience with streaming and batch data frameworks. Experience designing large-scale data lakes and warehouses with best practices for schema evolution and partitioning. Strong background with **cloud platforms (AWS, GCP, or Azure)**. Fintech or regulated industry experience is a plus. Good to Have Fintech-specific data experience, including regulatory compliance and reporting. Deployment experience with **real-time analytics** and event-driven architectures. Familiarity with containerization and infrastructure tools like Docker, Kubernetes, Terraform. Knowledge of data observability tools (Monte Carlo, Databand, etc.). Exposure to **ML pipelines** and model deployment. Solve challenging problems at the intersection of big data, real-time processing, and fintech. Lead impactful data initiatives at a rapidly growing startup. Collaborate with a world-class team of engineers, data scientists, and product leaders. Competitive compensation, equity, and benefits. Clear career growth opportunities in leadership and innovation. Perks That Go Beyond the Paycheck Relocation assistance for a smooth move. Free office meals (lunch & dinner). Generous leave policies (birthday, period, parental support, and more). Salary advances and loan policies for financial support. Quarterly rewards, recognition, and referral incentives. Access to the latest gadgets and tools. Comprehensive health insurance with mental health support. Tax benefits like food coupons, phone allowances, and leasing options. Retirement benefits including PF contribution, leave encashment, and gratuity. About FamApp FamApp focuses on financial inclusion for the next generation by offering UPI and card payments to users aged 11+. Our flagship product, FamX, integrates UPI and card payments seamlessly, helping users manage, save, and learn about their finances effortlessly. With over **10 million users**, FamApp is revolutionizing how young Indians transact eliminating the need to carry cash and offering customizable FamX cards with personal doodles for a fun, unique payment experience. Join Our Dynamic Team At Fam, we foster a people-first culture with flexible work schedules, generous leave, comprehensive health benefits, and mental health support. You ll be part of a passionate, talented, and fun team shaping the future of fintech for India s youth.

Job Title: Sr. Data Science Engineer Location: Bangalore Experience: 5 8+ Years Job Description As a Senior Data Science Engineer, you will develop and maintain scalable data pipelines and manage API integrations to support growing data volume and complexity. You will play a key role in ensuring data quality and enabling accurate AI model development through effective data handling and automation. Responsibilities Monitor data quality and implement processes for data cleansing and validation. Analyze data to troubleshoot and resolve data-related issues promptly. Develop automation workflows for efficient data labeling, preparation, and augmentation to improve AI model accuracy and utility. Requirements Proven experience as a Data Scientist, Data Analyst, or related role, with experience in data mining. Strong proficiency in data manipulation using Python or R; familiarity with Scala, Java, or C++ for raw data processing is a plus. Experience with business intelligence tools (e.g., Tableau) and big data frameworks like Hadoop and Spark. Strong mathematical foundation, especially in statistics and algebra. Advanced SQL skills with experience in database development, data migration, and integration. Expertise in exploratory data analysis and familiarity with common data science toolkits. Ability to communicate complex data insights clearly and effectively to non-technical stakeholders. Familiarity with data management tools and experience applying machine learning and AI techniques.

Data Engineer Location: Bangalore Company: Capital One India About Capital One At Capital One, we're redefining how technology solves real-world financial challenges. As a technology-driven company, we bring together talented engineers, data scientists, and designers to innovate at scale and deliver meaningful impact to millions of customers. If you're passionate about building powerful data solutions, exploring cutting-edge technologies, and working in a collaborative, fast-paced environment this is the place for you. About the Role As a Data Engineer at Capital One, you ll join a team of innovators who design and build next-generation data platforms and pipelines that power real-time decision-making. You ll collaborate across disciplines engineering, product, machine learning, and cloud infrastructure to transform how we leverage data at scale. What You ll Do Collaborate across Agile teams to design, develop, test, and deploy data-driven solutions. Build and support scalable data pipelines using modern data engineering tools and cloud services. Work on real-time and batch data processing systems that integrate with distributed microservices and ML platforms. Use programming languages such as Python, Java, or Scala with SQL, NoSQL, and cloud data warehouses like Redshift or Snowflake. Contribute to code reviews, unit testing, and performance optimization to ensure high-quality data systems. Partner with product managers and platform teams to deliver robust, cloud-native data solutions that power business decisions. Stay ahead of tech trends, share knowledge, and mentor junior engineers. Basic Qualifications Bachelor s degree in Computer Science, Engineering, or a related field. 1.5+ years of hands-on experience in application or data engineering (excluding internships). At least 1 year of experience working with big data technologies. Preferred Qualifications 3+ years of application/data engineering experience using Python, Scala, Java, or SQL. 1+ year of experience with cloud platforms (AWS, Azure, or GCP). 2+ years of experience with distributed computing tools (Spark, Hadoop, Hive, EMR, Kafka, etc.). 1+ year working on real-time streaming applications. 1+ year of experience with NoSQL databases (MongoDB, Cassandra). 1+ year of experience with data warehousing (Redshift, Snowflake). 2+ years working with Linux/Unix systems and shell scripting. Familiarity with Agile methodologies and modern DevOps practices. Why Join Capital One Work on high-impact data solutions at one of the world s most innovative financial institutions. Be part of a collaborative tech culture that values experimentation and learning. Access to top-tier tools, mentorship, and career development opportunities. Competitive compensation and benefits in a mission-driven environment. Qualification : Bachelors degree in Computer Science, Engineering, or a related field

Position Title: Data Architect Department: R&D Engineering Location: Bangalore Experience: 15+ Years Industry: SaaS / Conversational Engagement / Customer Experience Technology About Acqueon: Acqueon is a leading provider of conversational engagement software that enables customer-centric enterprises to proactively engage with their customers across voice, messaging, and email channels. By leveraging a powerful data platform, predictive models, and intelligent workflows, we help brands enhance customer experience, improve collections, and drive revenue growth. With over 200 global clients, Acqueon is at the forefront of AI-powered customer engagement. Role Overview: We are seeking a visionary and technically hands-on Data Architect to lead the development of enterprise-scale data platforms and engineering solutions. You will work closely with Product Owners, Engineering Leadership, and cross-functional teams to define and execute a strategic technology roadmap aligned with Acqueon s business goals. As a key member of our R&D team, you ll lead the design and development of highly scalable, low-latency, fault-tolerant data systems, while mentoring top-tier engineering talent and driving high-impact product features. Key Responsibilities: Architect & Lead: Design and lead development of scalable data architectures and solutions supporting real-time and batch processing, analytics, and enterprise applications. Strategic Ownership: Define and implement the data strategy, technology roadmap, and long-term architecture vision for Acqueon s platforms. Leadership: Manage and mentor a team of senior developers and engineers, fostering innovation, ownership, and delivery excellence. Cross-functional Collaboration: Work with Product, Sales, Engineering, and Customer teams to align on feature development and delivery strategy. Project Management: Oversee the end-to-end delivery of complex features, ensuring adherence to timelines, scalability, and quality standards. System Design: Review architecture and design for robustness, performance, and fault tolerance, including multi-region, high-availability setups. R&D Enablement: Collaborate with international R&D teams and align development efforts across global product initiatives. Innovation & Optimization: Drive architectural decisions, recommend performance improvements, and ensure best practices for enterprise-scale data solutions. Required Skills & Experience: Education: Bachelor s or Master s in Computer Science, IT, or related field. Experience: 15+ years in software development and data architecture, with leadership experience in managing engineering teams. Architecture Expertise: Proven experience in designing scalable, concurrent, distributed, and highly available data systems. Database Proficiency: Strong in SQL/NoSQL databases Experience with MS SQL, Aerospike, DynamoDB, Snowflake In-depth knowledge of micro-partitions, cluster keys, warehouse cloning, time travel in Snowflake Strong in writing and tuning complex stored procedures ETL & Pipelines: Experience in building ETL pipelines and integrating data from S3, Kinesis Streams, APIs Cloud & DevOps: Strong understanding of Docker, AWS, and cloud-native deployment architectures Setting up multi-region resilience, disaster recovery strategies Technologies: Elasticsearch, AWS data services, container orchestration Big Data & Analytics: Exposure to analytical processing and statistical modeling is a plus Leadership: Strong project management skills, stakeholder engagement, and team mentoring experience Preferred Qualifications: Background in customer engagement, VDI, Cybersecurity, or Secure Access technologies Previous experience working with distributed R&D and product teams Knowledge of Acqueon, Citrix, VMware, Omnissa platforms is a plus Certifications in AWS, Snowflake, or similar technologies are an advantage Soft Skills & Behavioral Traits: Strong verbal and written communication skills Strategic thinking with hands-on execution ability High accountability and ownership mindset Ability to work in a fast-paced, dynamic, startup-like environment Comfortable with ambiguity and context-switching Team player with the ability to lead by influence and collaboration Be a part of a fast-growing, AI-driven SaaS company disrupting the customer engagement space Work on cutting-edge technologies with global product teams Ownership of end-to-end solutions and ability to shape the data platform of the future A culture that promotes innovation, agility, and career growth

Position: Manager - Analytics Location: SEZ4, Bangalore, Karnataka, India Department: Advanced Analytics Employment Type: Subexian Experience Required: 6 to 9 years Job Overview: We are seeking an experienced Manager - Analytics to lead a team of talented data scientists in delivering impactful data-driven solutions. The ideal candidate will have a solid background in analytical methodologies, including machine learning, statistical modeling, and business analytics. You ll be responsible for collaborating with business stakeholders, driving the design and development of advanced analytics models, and ensuring the delivery of high-quality insights that support strategic decision-making. Key Responsibilities: Analytical Methodology: Research, evaluate, and implement new analytical methodologies and approaches to build innovative data-driven solutions. Utilize a blend of mathematics, statistics, machine learning, and business knowledge to solve complex problems. Business Collaboration: Work closely with business leaders and data owners to define key problems and challenges, ensuring the solutions developed have a significant impact on the business. Team Leadership: Lead and mentor a team of junior data scientists, guiding them through the development, deployment, and refinement of machine learning models in a production setting. Advanced Modeling: Utilize a wide range of analytical models, including reliability models, Markov models, stochastic models, Bayesian modeling, classification models, neural networks, and more to address business challenges effectively. Hands-On Expertise: Apply strong hands-on skills in R and Python, using libraries such as NLTK and Sklearn for developing machine learning models and data analysis. Large Data Management: Work with large datasets and distributed computing frameworks like MapReduce, Hadoop, and Hive, ensuring scalable and efficient data processing. Continuous Improvement: Stay up to date with the latest advancements in analytics and machine learning, continually enhancing models and methodologies to deliver cutting-edge insights. Required Technical Skills: Hands-on Experience in Python & R: Strong proficiency in R and Python programming, with experience in libraries such as NLTK, Sklearn, and other machine learning frameworks. Advanced Statistical & Machine Learning Techniques: Proven expertise in at least one of the following: Reliability models Markov Models Stochastic models Bayesian Modelling Classification Models Cluster Analysis Neural Networks NLP (Natural Language Processing) Deep Learning Non-parametric Methods Multivariate Statistics Big Data Tools: Experience working with large datasets and distributed computing platforms such as MapReduce, Hadoop, Hive, etc. Soft Skills: Leadership & Collaboration: Strong leadership skills with the ability to mentor a team of junior data scientists. Excellent collaboration and communication skills to work with business and technical teams. Problem-Solving & Critical Thinking: Ability to approach complex business problems analytically and come up with data-driven solutions. Continuous Learning: A passion for staying up to date with the latest trends and advancements in analytics, machine learning, and data science. At Subex, you ll have the opportunity to work on cutting-edge projects that make a tangible impact on the business. If you're passionate about leading analytics teams and leveraging data science to drive business success, we would love to have you on board.

Position: Data Architect Location: Remote/Hybrid | Bengaluru, IND We're searching for a highly skilled and experienced Data Architect to join our team. If you have a deep understanding of big data technologies and extensive experience with Hadoop, Python, Snowflake, and Databricks, you're the ideal candidate. You'll be responsible for designing, implementing, and managing complex data architectures that support our critical business needs and objectives. What You'll Do: Design & Architecture Leadership: Design scalable and efficient data architecture solutions that meet current and future business data needs. Lead the development of data models, schemas, and databases, ensuring alignment with business requirements. Architect and implement robust data solutions on leading cloud platforms (AWS, Azure, or GCP). Data Management & Governance: Develop and maintain robust data pipelines and ETL processes using Hadoop, Databricks, and other essential tools. Oversee data integration and quality efforts to ensure consistency and reliability across the organization. Implement data governance best practices, focusing on data security, privacy, and compliance. Collaboration & Mentorship: Work closely with data engineers, data scientists, and business stakeholders to translate data requirements into effective technical solutions. Provide technical leadership and mentorship to junior data engineers and architects. Collaborate with cross-functional teams to ensure data solutions align perfectly with overall business goals. Optimization & Innovation: Optimize existing data architectures for peak performance, scalability, and cost-efficiency. Monitor and troubleshoot data systems to ensure high availability and reliability. Continuously evaluate and recommend new tools and technologies to improve our data architecture. What You'll Bring: Experience: 10+ years in data architecture, data engineering, or a related field. Big Data Expertise: Proven experience with Hadoop ecosystems (HDFS, MapReduce, Hive, HBase). Programming Prowess: Strong programming skills in Python for data processing and automation. Data Platform Mastery: Hands-on experience with Snowflake for data warehousing and Databricks for analytics. Cloud Fluency: Extensive experience with cloud platforms (AWS, Azure, GCP) and their data services. Data Modeling: Familiarity with data modeling tools and methodologies. Core Skills: Deep understanding of big data technologies and distributed computing. Strong problem-solving skills to design solutions for complex data challenges. Excellent communication skills, able to explain complex technical concepts clearly to diverse audiences. Proficient in SQL and database performance tuning. Experience with CI/CD pipelines and automation in data environments. Education: Bachelor's degree in Computer Science, Information Technology, or a related field. Preferred Qualifications: Advanced Degree: A Master's degree in a related field. Cloud Certifications: Certifications like AWS Certified Data Analytics, Google Professional Data Engineer, or Microsoft Certified: Azure Data Engineer Associate. Additional Languages: Experience with other programming languages like Java or Scala. Machine Learning Integration: Knowledge of machine learning frameworks and their integration with data pipelines.

Senior Associate Data Engineering L2 Location: Bengaluru, India Department: Engineering Data Employment Type: Full-Time About the Role As a Senior Associate Data Engineering (L2) at Publicis Sapient, you will lead technical solutions that drive digital transformation by building scalable, high-performance data platforms. You ll be responsible for translating business and technical requirements into modern, data-centric solutions using Big Data technologies, cloud services (Azure), and advanced data engineering practices. Key Responsibilities Design and implement data ingestion, integration, and transformation processes from multiple heterogeneous sources in both batch and real-time. Build scalable data platforms using Hadoop stack components such as HDFS, Kafka, Spark, Hive, NiFi, Oozie, Airflow, Flink, and Storm. Develop real-time analytics, aggregation, and search features to support various data-driven applications. Collaborate closely with cross-functional teams on data infrastructure, computation frameworks, and data visualization. Apply cloud-native principles and Azure services to build and deploy data pipelines. Ensure performance optimization and data pipeline tuning. Work with NoSQL and MPP platforms like MongoDB, Cassandra, Redshift, Azure SQL DW, HBase, BigQuery. Contribute to infrastructure, automation, and DevOps for data pipelines using CI/CD practices. Ensure data governance, lineage, and cataloging using tools like Collibra or Alation. Required Qualifications 6 8 years of professional experience in software/data engineering. Minimum of 3 years hands-on experience with Big Data technologies. Strong programming expertise in Java (preferred), Scala, or Python. Expertise in the Hadoop ecosystem and real-time stream processing tools (Kafka, Pulsar, Spark Streaming, etc.). Hands-on experience with Azure data services (e.g., Data Factory, Synapse, ADLS, Databricks). Experience working with modern ETL tools (Informatica, Talend, etc.) and traditional RDBMS platforms (Oracle, PostgreSQL, SQL Server, MySQL). Bachelor's degree in Computer Science, Engineering, or related field. Nice to Have Certifications in Azure Data Engineer, GCP Big Data, or related cloud specializations. Experience with distributed messaging frameworks (ActiveMQ, RabbitMQ, Solace). Familiarity with microservices architecture and search technologies (Elasticsearch). Performance tuning of distributed data processing systems. Exposure to data governance, security, and metadata management. Benefits and Culture at Publicis Sapient Gender-neutral workplace policies 18 paid holidays annually Generous parental leave + new parent transition support Flexible work arrangements Access to Employee Assistance Programs (wellness & well-being) A dynamic culture focused on learning, creativity, and collaboration Qualification : Bachelor's degree in Computer Science, Engineering, or related field.

Data Architect Bangalore, India Location: Bangalore (Bengaluru) Experience: 10 to 15 Years Industry: IT & Data Systems Job Summary: We are seeking an experienced Data Architect with a strong background in designing and implementing enterprise-scale data solutions. The ideal candidate will have expertise in building data lakes, warehouses, and pipelines, with deep knowledge of cloud platforms, data management, and industry best practices. Key Responsibilities: Design, develop, and maintain complex data architectures including data lakes, data warehouses, data marts, and efficient schema design Build and optimize scalable data pipelines for extraction, transformation, and loading (ETL/ELT) processes Apply Agile methodologies in project delivery and collaborate within cross-functional teams Perform data profiling, cleansing, conversion, and ensure high-quality data management for both structured and unstructured data Implement CI/CD and Infrastructure as Code (IaC) practices using tools like GitHub, Jenkins, CloudFormation, and Azure Resource Manager Manage database systems and tools such as PostgreSQL, Oracle, Snowflake, Teradata, MongoDB, Hadoop, and others Utilize data modeling tools like Erwin, Power Designer, and Toad for effective data architecture design Leverage cloud platforms including AWS and Microsoft Azure, with hands-on experience in services like AWS Glue, DMS, Lambda, Azure Data Factory, Synapse, and Data Lake Storage Work with programming and scripting languages including SQL, PL/SQL, Python, Spark, YAML, and JSON Use containerization and automation tools such as Docker, Ansible, and NodeJS for efficient deployment Ensure compliance with cybersecurity principles and frameworks such as NIST Lead data governance initiatives and enforce best practices in data quality and security Preferred Qualifications: ITIL certification and experience with Agile methodology Knowledge of code review and version control best practices, especially in GitHub Familiarity with data science tools and AI/ML frameworks like R, Keras, or TensorFlow Experience with natural language processing (NLP) and machine learning concepts Background in regulated industries, with pharma manufacturing experience highly preferred Exposure to multi-site, global IT projects and manufacturing operations Lead innovative data architecture projects within a dynamic and fast-paced environment Work with cutting-edge cloud technologies and big data ecosystems Collaborate with global teams on impactful enterprise solutions Access to professional growth opportunities in data governance, AI, and cloud technologies

Job Title: Staff Machine Learning Engineer Location: Bengaluru, Karnataka, India Job Type: Full-Time (Hybrid Work Model) Experience Level: 6-10+ Years About Eightfold.ai: At Eightfold.ai, we re revolutionizing the way organizations find, manage, and empower talent. As a leader in AI-driven HR tech, our groundbreaking platform is transforming industries by using artificial intelligence to solve talent management challenges at scale. We are looking for exceptional engineers to join our team and be at the forefront of innovation in the field of AI. About the AI/ML Team: The AI/ML team at Eightfold is passionate about building cutting-edge solutions and pushing the boundaries of applied machine learning. We work with massive datasets and tackle complex, real-world challenges to develop state-of-the-art AI models. Our models are transforming how companies approach talent management, and we are looking for a Staff Machine Learning Engineer to help lead this journey. What You Will Do: As a Staff Machine Learning Engineer, you will play a critical role in leading and inspiring our ML team while architecting and implementing scalable, robust ML pipelines for our core products. Lead and Inspire: Mentor and guide a team of talented ML engineers, fostering a collaborative environment that encourages innovation. Architect and Implement: Design and build scalable ML pipelines that power Eightfold s core products. Innovate and Optimize: Develop novel algorithms and continuously enhance the performance, accuracy, and scalability of our models. Solve Complex Problems: Work on challenging real-world problems, including NLP, recommendation systems, and predictive analytics. Stay Ahead of the Curve: Research and implement cutting-edge techniques in deep learning, reinforcement learning, and other emerging fields. LLM & NLP Expertise: Design and implement scalable solutions using Large Language Models (LLMs), fine-tuning them for specific NLP tasks, and developing LLM-powered applications. What You Bring: Required Skills & Experience: Deep Expertise in ML & AI: Strong foundation in Machine Learning, Deep Learning, and Natural Language Processing (NLP) with a proven track record of developing and deploying ML models at scale. Technical Leadership: Experience leading and mentoring teams, with excellent communication and collaboration skills. Problem-Solving Mindset: Ability to analyze complex problems and develop innovative solutions. Passion for AI: Genuine enthusiasm for pushing the boundaries of AI and its real-world applications. Advanced Skills in Python & ML Frameworks: Proficiency in Python and relevant ML frameworks (TensorFlow, PyTorch), along with experience in big data technologies like Hadoop and Spark. CS Fundamentals & Coding: Strong foundation in computer science fundamentals and coding languages (Python, C/C++, Java, Scala, R). GenAI & LLM Expertise: Experience in developing and deploying LLM-powered applications, with knowledge of LLM fine-tuning techniques, optimization strategies, and ethical considerations. Nice to Haves: 6-10+ years of experience in Machine Learning and AI. MS or PhD in Computer Science or related field. Publications in top-tier AI conferences. Contributions to open-source ML projects. Familiarity with Generative AI (GenAI), Transformers, and LLMs. Impactful Work: Play a pivotal role in shaping the future of AI and its applications in talent management. Innovative Environment: Work on cutting-edge projects with world-class experts in the field of AI/ML. Career Growth: Accelerate your career in a dynamic, fast-paced environment with abundant opportunities for professional growth. Hybrid Work Model: Embrace a flexible work environment with the ability to collaborate remotely and in-person at our Bengaluru or Noida offices twice a week starting February 1, 2024. Competitive Benefits: Comprehensive family medical, vision, and dental coverage, a competitive base salary, equity awards, and discretionary bonuses or commissions. How to Apply: If you're a passionate AI/ML engineer ready to take on challenging problems and make a meaningful impact, we d love to hear from you. Join Eightfold.ai and help shape the future of AI-powered talent intelligence! Qualification : MS or PhD in Computer Science or a related field

Senior Data Engineer Location: Bengaluru, India Employment Type: Full-time, On-site About Neuron7.ai Neuron7.ai is a rapidly growing AI-first SaaS company that is revolutionizing the world of service intelligence. Backed by leading Silicon Valley venture capitalists and a distinguished group of angel investors, we are recognized as a startup to watch. Our AI-driven platform delivers service predictions in seconds by analyzing structured and unstructured data, alongside insights from top experts. We specialize in complex service environments like high-tech devices, manufacturing, and medical devices, empowering service leaders to optimize key metrics like first-call resolution, turnaround time, and service margins. At Neuron7.ai, you will be part of a dynamic, innovative team that is pushing the boundaries of service intelligence. We value creativity, collaboration, and a relentless commitment to innovation. This is your chance to join a fast-growing startup and help redefine how AI impacts the future of service optimization. About the Role We are building a real-time analytics platform to improve uptime of connected devices and minimize maintenance costs. This involves collecting, analyzing, and leveraging large volumes of sensor data for observability and predictive analytics. As a Senior Data Engineer, you will have the opportunity to build and maintain scalable, high-performance data systems using modern data engineering tools. The ideal candidate will have hands-on experience with Python, Scala, ELT processes, Lakehouse architecture, Apache Spark, Databricks, real-time streaming systems like Kafka and Flink, and observability tools such as the ELK stack, Prometheus, and Grafana. What You ll Do: Data Model Development: Design and implement advanced data models to extract insights from complex datasets. Collaborative Problem Solving: Work with cross-functional teams to devise data-driven solutions for service optimization. Experimental Analysis: Conduct experiments and analyses to enhance service predictions and outcomes. Actionable Insights: Present findings and actionable recommendations to stakeholders, translating complex data insights into clear, understandable concepts. Mentorship: Guide and mentor junior team members, fostering a collaborative and knowledge-sharing environment. What We re Looking For: Experience: At least 4 years of hands-on experience in data engineering, with a focus on building large-scale data systems. Data Pipeline Expertise: Strong proficiency in Python and Scala for developing data pipelines. Big Data Tools: Proven experience with Apache Spark and Databricks for big data processing and analytics. Observability Tools: Hands-on experience with observability tools like Prometheus, Grafana, and the ELK stack or OpenSearch. Real-Time Data Streaming: Experience working with real-time streaming systems such as Kafka and Flink. Cloud Platforms: Familiarity with cloud platforms like Azure, GCP, or AWS. Problem-Solving Skills: Strong analytical and problem-solving abilities to work with complex data systems in fast-paced environments. Collaboration: Excellent communication and teamwork skills, with the ability to work effectively with cross-functional teams. What We Do and Value: At Neuron7.ai, we prioritize integrity, innovation, and a customer-centric approach. Our mission is to leverage advanced AI technology to enhance service decision-making and we are dedicated to delivering excellence in all aspects of our work. Company Perks & Benefits: Competitive salary, equity, and spot bonuses Paid sick leave Latest MacBook Pro for your work Comprehensive health insurance Paid parental leave Flexible work arrangements work from home or from our vibrant Bengaluru office Our Commitment to Diversity and Inclusion: Neuron7.ai is committed to fostering a diverse and inclusive workplace. We ensure equal employment opportunities without discrimination based on race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, marital status, or any other characteristic protected by law. If you re passionate about using data to drive service intelligence and want to be part of a forward-thinking team, we d love to hear from you!

Data Scientist with GenAI-2 Location: Bengaluru, India Company: Wipro Limited (NYSE: WIT, BSE: 507685, NSE: WIPRO) Company Overview Wipro Limited is a leading technology services and consulting company focused on building innovative solutions that address clients most complex digital transformation needs. Leveraging our holistic portfolio of capabilities in consulting, design, engineering, and operations, we help clients realize their boldest ambitions and build future-ready, sustainable businesses. With over 230,000 employees and business partners across 65 countries, we deliver on the promise of helping our customers, colleagues, and communities thrive in an ever-changing world. Job Description Key Responsibilities: Research, design, develop, and modify computer vision and machine learning algorithms and models, leveraging technologies such as Caffe, Torch, or TensorFlow. Shape product strategy for highly contextualized applied ML/AI solutions by engaging with customers, solution teams, discovery workshops, and prototyping initiatives. Help build a high-impact ML/AI team by supporting recruitment, training, and development of team members. Serve as an evangelist by engaging in the broader ML/AI community through research, speaking/teaching, formal collaborations, and/or other channels. Design integrations of and tune machine learning and computer vision algorithms. Research and prototype techniques and algorithms for object detection and recognition. Use Convolutional Neural Networks (CNN) for performing image classification and object detection. Familiarity with Embedded Vision Processing systems. Work with open-source tools & platforms for statistical modeling, data extraction, and analysis. Construct, train, evaluate, and tune neural networks. Mandatory Skills: Proficiency in Java, C++, Python. Experience with Deep Learning frameworks such as Caffe, Torch, TensorFlow. Experience with image/video vision libraries like OpenCV, Clarifai, Google Cloud Vision. Expertise in Supervised & Unsupervised Learning. Development of feature learning, text mining, and prediction models (e.g., deep learning, collaborative filtering, SVM, random forest) on big data computation platforms (Hadoop, Spark, HIVE, Tableau). Familiarity with technologies such as Tableau, Hadoop, Spark, HBase, Kafka. Experience: 2-5 years of work or educational experience in Machine Learning or Artificial Intelligence. Experience in creating and applying Machine Learning algorithms to a variety of real-world problems with large datasets. Building scalable machine learning systems and data-driven products working with cross-functional teams. Experience with cloud services like AWS, Microsoft, IBM, and Google Cloud. Experience with Natural Language Processing, text understanding, classification, pattern recognition, recommendation systems, targeting systems, ranking systems, or similar fields. Nice to Have: Contribution to research communities and/or efforts, including publishing papers at conferences such as NIPS, ICML, ACL, CVPR, etc. Education: BA/BS (advanced degree preferable) in Computer Science, Engineering, or related technical field or equivalent practical experience. About Wipro Wipro is building a modern digital transformation business with bold ambitions. Join a team that values reinvention of yourself, your career, and your skills. Wipro is a place that empowers you to design your own career reinvention, evolve, and grow. Applications from people with disabilities are explicitly welcome. Qualification : BA/BS (advanced degree preferable) in Computer Science, Engineering, or related technical field or equivalent practical experience.

Machine Learning Engineer Full-Time - Bengaluru, India - Data Science / Artificial Intelligence / Engineering Join our dynamic Data Science / Artificial Intelligence / Engineering team in Bengaluru, India as a Full-Time Machine Learning Engineer and play a key role in driving data-driven innovation! We are seeking a skilled and results-oriented Machine Learning Engineer to design, build, and deploy scalable machine learning models that address real-world business challenges. You will collaborate closely with data scientists, engineers, and product managers to transform raw data into actionable insights and integrate intelligent features into our products. As a Machine Learning Engineer, you will be responsible for the complete lifecycle of machine learning models and pipelines, from design and development to seamless deployment for a variety of applications. This includes classification, regression, clustering, recommendation systems, and time-series forecasting. You will leverage your expertise to preprocess and analyze large and complex datasets, extracting meaningful features and valuable insights. Collaboration with cross-functional teams will be crucial as you identify strategic ML opportunities and define clear success metrics. A key aspect of this role involves optimizing machine learning models for peak performance, scalability, and accuracy within production environments. You will build robust APIs or efficient microservices to integrate these models seamlessly into our applications, utilizing tools such as Flask or FastAPI. Continuous improvement is paramount, and you will be responsible for the ongoing monitoring and retraining of models based on their performance and any signs of data drift. Staying at the forefront of the field is essential, and you will be expected to stay updated with the latest ML research and emerging technologies, applying them to continuously enhance our product capabilities. Key Responsibilities: Design, develop, and deploy machine learning models and pipelines for diverse applications including classification, regression, clustering, recommendation, and time-series forecasting. Preprocess and analyze large datasets to extract meaningful features and actionable insights. Collaborate effectively with cross-functional teams to identify strategic ML opportunities and define clear success metrics. Optimize models for maximum performance, scalability, and accuracy in production environments. Build robust APIs or efficient microservices to integrate ML models into applications using tools like Flask or FastAPI. Continuously monitor and retrain models based on performance metrics and potential data drift. Stay updated with the latest ML research and technologies and apply them to enhance product capabilities. Minimum Qualifications: Bachelor s or Master s degree in Computer Science, Data Science, Statistics, or a related field. 2+ years of proven experience as a Machine Learning Engineer or in a similar role. Strong proficiency in Python and key ML libraries such as Scikit-learn, XGBoost, TensorFlow, or PyTorch. Practical experience working with both SQL and NoSQL databases. Solid knowledge of essential data preprocessing, effective feature engineering, and robust model evaluation techniques. Familiarity with standard software engineering practices, including version control (Git), thorough code reviews, and efficient CI/CD pipelines. Preferred Qualifications: Prior experience with deep learning, natural language processing (NLP), or computer vision. Familiarity with major cloud services like AWS, GCP, or Azure (especially SageMaker, Vertex AI, etc.). Understanding of modern MLOps tools and practices (e.g., MLflow, Kubeflow, DVC). Practical experience with containerization and orchestration tools (Docker, Kubernetes). Knowledge of big data tools (e.g., Spark, Hadoop) is considered a significant plus. What We Offer: Competitive salary and performance-based incentives to reward your contributions. Comprehensive health insurance and valuable wellness benefits to support your well-being. Dedicated learning and development programs for continuous professional growth. Exciting opportunities to work on impactful, real-world AI/ML projects with significant scale. A collaborative, inclusive, and innovative work culture that fosters teamwork and creativity. Flexible working hours and a hybrid work model to promote a healthy work-life balance.

Lead Data Engineer | ThoughtWorks | Bangalore, India Location: Bangalore, India Employment Type: Full-time, Regular Industry: Information Technology About ThoughtWorks At ThoughtWorks, we're a global technology consultancy that integrates strategy, design, and engineering to drive digital innovation. For over 30 years, we've worked alongside our clients to deliver solutions that challenge the status status quo. With a diverse and inclusive team, we empower each other to grow through shared learning, fostering an environment where innovation thrives. Our commitment to a cultivation culture is key to our success, and we re looking for a Lead Data Engineer to join our Bangalore team and lead transformative projects. Job Overview As a Lead Data Engineer at ThoughtWorks, you will be responsible for designing, developing, and operating modern data architectures that meet client business objectives. You will lead and manage data engineering projects end-to-end, from strategic planning to hands-on coding, ensuring the delivery of scalable and efficient data solutions. Working with cutting-edge technologies, you ll collaborate with stakeholders, clients, and cross-functional teams to implement data-driven strategies that address complex business challenges. Key Responsibilities Project Leadership: Lead and manage data engineering projects from inception to completion, including goal-setting, scope definition, and ensuring on-time delivery in collaboration with cross-functional teams. Data Architecture & Solution Design: Collaborate with clients to design modern data architecture and implement end-to-end solutions that meet key business objectives. Create intricate data processing pipelines to address complex business problems. Stakeholder Collaboration: Work closely with stakeholders to understand business objectives and identify opportunities to leverage data and data quality improvements. Data Modeling & Governance: Develop data models using modern modeling techniques and implement them using appropriate technologies. Ensure compliance with data governance, security, and privacy requirements. Scalable Implementations: Partner with data scientists to design scalable implementations of their models, ensuring the solutions are robust and efficient. Clean, Iterative Code: Write clean, modular code based on TDD (Test-Driven Development) and implement continuous delivery practices to support and operate data pipelines. Technology Guidance: Advise clients on distributed storage and computing technologies, selecting the best options to fit their business needs. Data Quality Strategy: Define and incorporate data quality strategies into daily work processes to ensure high standards and compliance. Job Qualifications Technical Skills: Proven experience in data engineering and system design, with a focus on building Big Data architecture and data pipelines within distributed systems. Deep knowledge of data modeling and hands-on experience with modern data engineering tools and platforms. Strong programming skills, with expertise in building scalable, high-quality data pipelines using languages like Python, Java, Scala, or others. Experience with distributed storage platforms (e.g., Hadoop, Amazon S3, etc.) and distributed processing platforms (e.g., Spark, Flink). Experience working with SQL, NoSQL, data lakes, and other data storage technologies. Familiarity with data visualization techniques and ability to communicate insights effectively across varying audiences. Professional Skills: Stakeholder Management: Strong ability to liaise between clients and other key stakeholders, ensuring trust and buy-in throughout projects. Adaptability & Resilience: Comfortable handling ambiguity and finding innovative solutions to complex challenges. Leadership & Mentorship: Experienced in coaching and mentoring team members, fostering a culture of professional growth and accountability. Risk & Conflict Management: Skilled in managing risks and resolving conflicts, driving projects forward despite challenges. Relationship Building: Natural at cultivating strong relationships with clients, stakeholders, and internal teams to create new opportunities. What You Bring to the Team Leadership: A proven track record in leading high-performance teams and supporting colleagues in their professional development. Curiosity & Innovation: A passion for data and technology and a willingness to continually learn and push the boundaries of what's possible. Collaboration: Ability to work closely with cross-functional teams and stakeholders to design and implement innovative data solutions. At ThoughtWorks, we believe in giving you the autonomy to carve out your unique career path, while providing support through development programs and a vibrant culture of learning. You ll work on exciting projects with a diverse team, solve complex challenges, and make an impact at a global scale. Join ThoughtWorks and be part of a global community of innovators. Together, we turn curiosity into action and creativity into impactful solutions.

Sr. Data Engineer - AWS - Big Data Location:Bangalore Type of Employment: Full-Time Experience Required: 7 to 10 years Job Overview: We are seeking a highly skilled Sr. Data Engineer with expertise in AWS cloud technologies and Big Data to join our Cloud Data Architect Team at Infocepts. In this critical role, you will design and implement robust data solutions using technologies like EMR, Athena, PySpark, AWS Lambda, S3, and other AWS services. The ideal candidate will have a strong foundation in database concepts and SQL and will be responsible for building scalable data pipelines to support high-performance data processing. Key Responsibilities: Technology Assessment and Design: Study the existing technology landscape and evaluate current data integration frameworks. Assist in designing complex Big Data use cases leveraging AWS services. Documentation and Stakeholder Communication: Prepare and maintain comprehensive project documentation, adhering to quality guidelines and schedules. Work closely with Architects and Project Managers to provide accurate estimations, scoping, and scheduling assistance. Clearly communicate design decisions and conduct Proof-of-Concepts to validate new solutions before implementation. Process Improvement and Automation: Identify areas for process automation to improve efficiency and team productivity. Provide expert guidance and troubleshooting support to junior Data Engineers. Training and Knowledge Sharing: Develop and deliver technology-focused training sessions for the team, ensuring continuous knowledge sharing. Share expertise through Expert Knowledge Sharing sessions with Client Stakeholders. Essential Skills: AWS Services Expertise: In-depth knowledge of S3, EC2, EMR, Athena, AWS Glue, and Lambda. Big Data Technologies: Proficiency with Apache Spark, Databricks, and Big Data table formats such as Delta Lake (open-source). Data Warehousing: Strong understanding of data warehousing concepts and architectures. Programming Skills: Advanced programming skills in Python for building data pipelines. SQL Expertise: Strong SQL skills for data transformation, aggregation, and querying large datasets. ETL Workflow Development: Expertise in creating ETL workflows with complex transformations (e.g., SCD, deduplication, aggregation). Orchestration Tools: Familiarity with orchestration tools like Apache Airflow. MPP Databases: Experience with at least one MPP database (e.g., AWS Redshift, Snowflake, SingleStore). Cloud Databases: Exposure to cloud databases like Snowflake or AWS Aurora. Desirable Skills: Cloud Databases: Familiarity with Snowflake, AWS Aurora. Big Data Technologies: Experience with Hadoop and Hive. AWS Certification: Associate or Professional Level AWS Certification. Advanced Knowledge of Big Data Solutions: Exposure to big data tools and frameworks on cloud platforms. Qualifications: Experience: 7+ years of overall IT experience, with 5+ years specifically focused on AWS-related projects. Educational Background: Bachelor's degree in Computer Science, Engineering, or a related field (Master's degree is a plus). Technical Certifications: Demonstrated commitment to continuous learning through certifications or relevant training. Qualities: Strong analytical and problem-solving skills to deep dive into complex technical challenges.

Position: Senior Data Scientist Location: Bangalore Employment Type: Full-time Experience Required: 5 to 7 years Purpose of the Position: The Senior Data Scientist will play a key role in designing, developing, and implementing machine learning models and algorithms to solve complex business problems. The role involves collaborating with data scientists, software engineers, and business stakeholders to deliver scalable, efficient machine learning solutions that drive innovation and improve business outcomes. Key Responsibilities: Model Development and Optimization: Develop, train, and optimize machine learning models to meet business objectives. Ensure models are accurate, efficient, and scalable. Data Pipeline and Infrastructure: Design and maintain robust data pipelines to support machine learning workflows. Ensure data quality and integrity throughout the data lifecycle. Deployment and Monitoring: Deploy machine learning models into production. Monitor model performance and implement improvements as needed. Collaboration and Communication: Work with cross-functional teams to understand business requirements and translate them into technical solutions. Communicate complex technical concepts to non-technical stakeholders. Research and Innovation: Stay up to date with the latest advancements in machine learning and AI. Experiment with new techniques and technologies to enhance the organization's capabilities. Essential Skills: Machine Learning Frameworks: Experience with ML frameworks such as TensorFlow, PyTorch, and Scikit-learn. Programming Languages: Strong skills in Python and R. Data Processing: Experience with tools like Pandas, NumPy, and Spark. Model Deployment: Experience with Docker, Kubernetes, MLflow, and FastAPI. Databases: Strong experience in databases like Snowflake, MongoDB, and Cosmo DB. Statistical Analysis: Strong foundation in statistics and probability. Model Management: Experience establishing model management, monitoring, support, and maintenance frameworks. ML Lifecycle: Experience setting up ML lifecycle processes and governance policies, as well as providing L0, L1, and L2 support. Desired Skills: Deep Learning: Familiarity with deep learning frameworks like TensorFlow or PyTorch. Natural Language Processing (NLP): Understanding of NLP techniques and applications. Cloud Computing: Experience with cloud platforms like AWS, Azure, or Google Cloud. Big Data Technologies: Knowledge of big data technologies such as Hive, Pig, or Cassandra. Software Engineering: Proficiency in software development and version control systems like Git. Qualifications: Education: Master's degree or Ph.D. in Computer Science, Data Science, Statistics, Mathematics, or a related field. Experience: Over 5 years of experience in machine learning, with a proven track record of developing and deploying models. Certifications: Relevant certifications in machine learning or data science are a plus. Key Qualities: Analytical Thinking: Strong analytical and problem-solving skills. Communication: Excellent verbal and written communication skills. Team Player: Ability to work effectively in a collaborative team environment. Adaptability: Flexibility to adapt to changing business needs and technologies. Innovation: A proactive approach to exploring new ideas and technologies. Apply now to join our dynamic team and lead innovation through machine learning and AI! Qualification : Master's degree or Ph.D. in Computer Science, Data Science, Statistics, Mathematics, or a related field.

Research Engineer Location: Bangalore, Karnataka, India Job Type: Full-Time Experience Level: 0-8 years Company: IBM Research India (IRL) Introduction: IBM Research is the innovation engine of IBM and is the largest industrial research organization in the world. With 12 labs across 6 continents and over 3200 researchers globally, we produce more patents daily than any other organization. At IBM Research India (IRL), we are shaping the future of computing in areas like AI, Hybrid Cloud, and Quantum Computing. Our work is at the forefront of breakthrough innovations in Foundation Models, AI systems, large-scale data engineering, and more. We are looking for top talent to join us in our exciting and dynamic projects, pushing the boundaries of innovation. As a Research Engineer, you will work on pioneering research and development in the most cutting-edge fields of AI and computing. Role Overview: The Research Engineer role at IBM India Research Lab (IRL) involves working on challenging, dynamic, and highly innovative projects in the fields of AI, machine learning, and data systems. Your responsibilities will span multiple areas including optimizing AI models for large-scale distributed systems, pre-training foundation models, and developing real-world use cases that leverage IBM s infrastructure and models. Key Responsibilities: Optimized Runtime Stacks for Foundation Models: Work on fine-tuning, inference serving, and large-scale data engineering for AI models. Focus on multi-stage tuning, reinforcement learning, inference-time compute, and preparing data for complex AI systems. Model Optimization Across Accelerators: Develop solutions to optimize models for multi-accelerator environments, particularly focusing on IBM s AIU accelerator. Work on compiler optimizations, specialized kernels, libraries, and tools to enhance model performance. Pre-training and Deployment of Foundation Models: Participate in pre-training language models and multi-modal foundation models. Work on distributed training procedures, model alignment, and creating pipelines for various tasks, including LLM-generated data pipelines. Research and Use Case Development: Develop and implement use cases that effectively leverage infrastructure and models to drive real-world value. Contribute to creating frameworks for human-data collection and deploying models on user-centric platforms. Required Education and Experience: Education: A Master s degree in Computer Science, AI, or related fields from a top institution. Experience: 0-8 years of experience working with modern ML techniques, including but not limited to model architectures, data processing, fine-tuning techniques, reinforcement learning, distributed training, and inference optimizations. Technical Skills: Experience with big data platforms such as Ray and Spark. Experience with Pytorch FSDP and HuggingFace libraries. Proficiency in programming with Python or web development technologies. Mindset and Attitude: A growth mindset and pragmatic approach to problem-solving. Preferred Experience: Research Experience: Peer-reviewed research at top machine learning or systems conferences. Advanced Technical Skills: Experience working with pytorch.compile, CUDA, Triton kernels, GPU scheduling, and memory management. Open Source Contributions: Experience working within open-source communities, contributing to or developing open-source projects. Innovative Environment: Be at the forefront of technological innovation, working on cutting-edge projects in AI, quantum computing, and more. Global Impact: Work on projects that influence both academic research and commercial product development, making a global impact. Career Development: IBM offers abundant opportunities for learning and growth, with access to the latest technologies and research. Collaborative Culture: Work with a diverse team of world-class researchers and engineers in a collaborative, open-source-driven environment. Apply today and become a part of the team that s redefining innovation. Qualification : A Masters degree in Computer Science, AI, or related fields from a top institution.

Job Title: Data Engineer Location: Bengaluru, Karnataka Experience: 5 to 8 Years Work Modality: Full-time (Work from office) Job Description: We are looking for an experienced Data Engineer to join our team and take responsibility for designing, developing, and maintaining scalable ETL/ELT pipelines. This is a full-time position based in Bengaluru, Karnataka, and you will be collaborating with cross-functional teams to define data requirements and ensure data accuracy, consistency, and integrity. Your role will also involve optimizing data workflows, automating processes, and ensuring high availability and reliability of data pipelines. Key Responsibilities: ETL/ELT Pipeline Development: Design, develop, and maintain scalable ETL/ELT pipelines to support data transformation and integration processes. Data Warehouse & Data Lake Optimization: Build and optimize data warehouses, data lakes, and real-time streaming solutions to support large-scale data operations. Collaboration & Data Requirements: Collaborate with cross-functional teams, such as product, data science, and analytics teams, to define data requirements and ensure data accuracy and consistency. Database Structure & Schema Management: Develop and maintain database structures and schemas to ensure efficient data storage and retrieval. Data Workflow Optimization: Optimize data workflows for performance, reliability, and scalability, ensuring the highest level of efficiency. Data Security & Compliance: Implement data security, governance, and compliance best practices to ensure that data is handled securely and meets industry standards. Pipeline Monitoring & Troubleshooting: Monitor, troubleshoot, and improve data pipelines to ensure uptime, reliability, and smooth data processing. Process Automation: Automate data-related processes to improve efficiency and reduce manual intervention, increasing the overall speed of data flow. Required Qualifications: Experience: 5+ years of experience in data engineering or 3-4 years of experience as a Data Engineer. Technical Skills: Strong proficiency in SQL and database management systems such as PostgreSQL, MySQL, SQL Server, etc. Experience with ETL tools such as Pentaho, Talend, Cdata, and SSIS. Exposure to Python, Java, or Scala for data processing is a plus. Experience with big data technologies such as Apache Spark, Hadoop, or Kafka. Familiarity with cloud services (AWS, Azure) and data storage solutions such as S3, Redshift, Snowflake, or BigQuery. Strong knowledge of data modeling, warehousing concepts, and data architecture best practices. Soft Skills: Excellent communication skills with the ability to collaborate effectively across teams. Strong problem-solving skills and the ability to work with large, complex datasets. What We Offer: Competitive Salary: Attractive salary based on experience and expertise. Collaborative Work Environment: Work in a dynamic and fast-paced environment with a team that fosters innovation and collaboration. Growth Opportunities: Opportunities to enhance your skills and career growth in the data engineering field. Comprehensive Benefits: Benefits package designed to support work-life balance and overall employee well-being.

Job/Position Summary: Data Engineer Responsibilities: Implement data pipelines that meet design and are efficient, scalable, and maintainable. Implement best practices including proper use of source control, participation in code reviews, data validation and testing. Timely deliveries while working on projects. Act as advisor/mentor and helps junior data engineers in their deliverables. Must Have Skills: Should have experience of at least 4+ years with Data Engineering. Strong experience of design, implementation and fine-tuning big data processing pipelines in production environment. Experience with big tools like Hadoop, Spark, Kafka, Hive, Databricks. Experience in programming at least one of with Python, Java, Scala, Shell Script. Experience with relational SQL and NO SQL databases like PostgresSQL, MYSQL, Cassandra etc. Experience with any data visualization tool (Plotly, Tableau, Power BI, Google Data Studio, Quick sight etc.). Good To Have Skills: Should have Basic Knowledge of CI/CD Pipeline. Experience in working on at least one Cloud (AWS or Azure or GCP). For AWS: - Experience with AWS Cloud services like EC2, S3, EMR, RDS, Athena, Glue, Lambda, EMR. For Azure: -Experience with Azure Cloud services like Azure Blob/Data Lake GEN2, Delta Lake, Databricks, Azure SQL, Azure DevOps, Azure Data Factory, Power BI. For GCP: - Experience with GCP Cloud services Big Query, Cloud Storage bucket, DataProc, Dataflow, Pub Sub, Cloud Function, Data Studio. Sound familiarity in Versioning tools (Git, SVN etc.). Experience Mentoring students is desirable. Knowledge of latest developments in Machine Learning, Deep Learning, Optimization in Automotive domain. Open minded approach to explore multiple algorithms to design optimal solution. History of contribution to articles/blogs/whitepapers etc. in Analytics. History of contribution to Open Source. Requirement: ESSENTIAL SKILLS /COMPETENCIES Data Engineering Hadoop Kafka CI/CD Cloud