Data Processing Jobs in Bengaluru

Position: Mechatronics & Big Data Scientist Developer Department: Research & Development Reporting To: General Manager Location: Bengaluru Key Responsibilities Machine Learning: Select features, build, and optimize classifiers using advanced machine learning techniques. Data Mining: Perform data mining using state-of-the-art methods to extract valuable insights from large datasets. Data Enhancement: Extend the company s datasets with third-party data sources when necessary to improve model accuracy and relevance. Data Collection & Processing: Improve data collection procedures to include all necessary information for building analytic systems. Data Cleansing & Integrity: Process, cleanse, and verify the integrity of data used for analysis to ensure reliable results. Ad-hoc Analysis: Perform ad-hoc analysis as needed, presenting the results in a clear, actionable manner. Anomaly Detection: Design and implement automated anomaly detection systems, tracking their performance over time to ensure accuracy. Behavioral Competencies Data-Driven: Strong inclination toward working with data and applying analytical thinking to solve complex problems. Detail-Oriented: Meticulous in data analysis and system development to ensure quality and precision in results. Skills and Expertise Machine Learning Algorithms Strong understanding of machine learning techniques and algorithms such as k-NN, Naive Bayes, SVM, Decision Forests, etc. Data Science Tools Experience with common data science toolkits like R, Weka, NumPy, and MatLab. Proficiency in at least one (preferably NumPy or R) is highly desirable. Data Visualization Skilled in data visualization tools such as D3.js, GGplot, or similar. Database Management Experience with query languages such as SQL, Hive, Pig, NiFi, or others depending on the company s stack. Familiarity with NoSQL databases like InfluxDB, MongoDB, Cassandra, HBase. Statistical Analysis Strong applied statistics skills, including distributions, statistical testing, and regression analysis. Programming Skills Good scripting and programming skills in languages like PHP, Slim, SQL, and Laravel. Big Data Technologies Knowledge of Hadoop, HDFS, NiFi, and other big data platforms and technologies. Qualifications Essential: MTech, MS, or equivalent in Mechatronics, Computer Science, or a related field. Experience: Minimum 2 years of hands-on experience in developing SDKs and working with Big Data platforms. Proven track record in machine learning, data mining, and data science projects. Qualification : MTech, MS, or equivalent in Mechatronics, Computer Science, or a related field

Data Engineering Lead Bengaluru | Engineering | Full-Time About Fam (formerly FamPay) Fam is India s first payments app designed for everyone aged 11 and above. FamApp enables seamless online and offline payments through UPI and FamCard. Our mission is to empower over **250 million young Indians** to start their financial journey early, becoming financially aware and confident. Founded in 2019 by IIT Roorkee alumni, Fam is backed by top-tier investors including Elevation Capital, Y-Combinator, Peak XV (Sequoia Capital India), Venture Highway, and angels like Kunal Shah and Amrish Rao. About the Role We re looking for a visionary **Data Engineering Lead** to take **end-to-end ownership** of Fam s data ecosystem from data ingestion and storage to processing and delivering actionable insights. You ll **define the data strategy and architecture** that supports both batch and **real-time** use cases, ensuring scalability, reliability, and governance across the organization. You will be instrumental in enabling accurate, complete, and trusted data flow that powers business intelligence, analytics, and product decision-making. This role involves **leadership, strategic thinking**, and hands-on problem solving. What You ll Do Own the full data lifecycle: ingestion, organization, storage, processing, and presentation. Define and execute **data architecture and strategy** aligned with operational and analytical goals. Build **scalable, reliable, and observable data systems** supporting batch and near real-time processing. Ensure **data quality, governance, and compliance**, proactively resolving discrepancies. Collaborate with product, engineering, and business teams to define, track, and optimize key metrics. Anticipate data-related challenges and implement preventive solutions. Lead, mentor, and grow the data engineering team, fostering innovation and accountability. Must-Haves 10+ years experience in data engineering, including proven leadership of teams or projects. Expertise designing, building, and scaling end-to-end data pipelines and systems. Deep understanding of the data lifecycle, from ingestion through business reporting. Strong communication skills and ability to collaborate across technical and business teams. Solid knowledge of **data governance, quality assurance, and compliance standards**. Experience with observability and proactive monitoring for data systems. Proficiency in Python and SQL; familiarity with Scala or Java. Hands-on experience with streaming and batch data frameworks. Experience designing large-scale data lakes and warehouses with best practices for schema evolution and partitioning. Strong background with **cloud platforms (AWS, GCP, or Azure)**. Fintech or regulated industry experience is a plus. Good to Have Fintech-specific data experience, including regulatory compliance and reporting. Deployment experience with **real-time analytics** and event-driven architectures. Familiarity with containerization and infrastructure tools like Docker, Kubernetes, Terraform. Knowledge of data observability tools (Monte Carlo, Databand, etc.). Exposure to **ML pipelines** and model deployment. Solve challenging problems at the intersection of big data, real-time processing, and fintech. Lead impactful data initiatives at a rapidly growing startup. Collaborate with a world-class team of engineers, data scientists, and product leaders. Competitive compensation, equity, and benefits. Clear career growth opportunities in leadership and innovation. Perks That Go Beyond the Paycheck Relocation assistance for a smooth move. Free office meals (lunch & dinner). Generous leave policies (birthday, period, parental support, and more). Salary advances and loan policies for financial support. Quarterly rewards, recognition, and referral incentives. Access to the latest gadgets and tools. Comprehensive health insurance with mental health support. Tax benefits like food coupons, phone allowances, and leasing options. Retirement benefits including PF contribution, leave encashment, and gratuity. About FamApp FamApp focuses on financial inclusion for the next generation by offering UPI and card payments to users aged 11+. Our flagship product, FamX, integrates UPI and card payments seamlessly, helping users manage, save, and learn about their finances effortlessly. With over **10 million users**, FamApp is revolutionizing how young Indians transact eliminating the need to carry cash and offering customizable FamX cards with personal doodles for a fun, unique payment experience. Join Our Dynamic Team At Fam, we foster a people-first culture with flexible work schedules, generous leave, comprehensive health benefits, and mental health support. You ll be part of a passionate, talented, and fun team shaping the future of fintech for India s youth.

Senior Data Engineer Enterprise Data Platform Location: Bengaluru Department: Business Technology Data Engineering Experience: 5+ Years Employment Type: Full-Time About Okta Okta is The World s Identity Company. We empower people to securely use any technology, anywhere, on any device. Through our Okta and Auth0 platforms, we provide secure access, authentication, and automation placing identity at the center of security and growth for thousands of organizations. We value diverse perspectives and lifelong learners. We re not looking for someone who checks every box we re looking for someone who will make us better with their unique experiences. Team: Business Technology Data Engineering The Data Engineering team at Okta supports cross-functional partners by building scalable, secure, and high-performing platforms. These platforms power decision-making and business processes across sales, marketing, engineering, finance, product, and operations. As part of this team, you ll contribute to data solutions that fuel Okta s hyper-growth. You will have the opportunity to work with cutting-edge technologies in cloud infrastructure, data lakes, automation, and CI/CD pipelines. The Role: Senior Data Engineer As a Senior Data Engineer, you will design, build, and manage modern data pipelines, infrastructure, and automation frameworks. You ll help scale our enterprise data platform using tools such as Snowflake, dbt, Airflow, Databricks, and AWS, while ensuring security, observability, and performance. You ll also contribute to CI/CD pipelines, infrastructure as code (IaC), and secure development lifecycle practices, enabling consistent, efficient, and secure delivery of data solutions. Key Responsibilities Platform Development & Infrastructure Design and maintain scalable data pipelines and platforms using Snowflake, AWS, Databricks, dbt, and Airflow. Manage infrastructure with Terraform, enabling repeatable and consistent deployments. Develop and maintain robust CI/CD pipelines using GitHub Actions, GitLab, or Jenkins. Containerize data services using Docker for better scalability and portability. Security & Compliance Implement and enforce secure development lifecycle practices, integrating tools like DAST, SAST, SCA, and Secret Scanning into pipelines. Conduct vulnerability scanning and apply patches to ensure system integrity. Ensure data security and compliance with industry standards and regulations. Collaboration & Innovation Collaborate with data engineers, data scientists, and analysts across business units to ensure data availability and integrity. Identify opportunities for automation and optimization within the data platform. Stay updated on emerging technologies and drive adoption of best practices. Must-Have Skills Bachelor s degree in Computer Science, Engineering, or a related technical field. 5+ years of experience in data engineering, including: Advanced SQL and ETL development with Airflow and dbt. Experience with data warehouses such as Snowflake, Redshift, or BigQuery. Strong hands-on experience with AWS (S3, Lambda, EC2, EMR, EKS). 2+ years of experience managing CI/CD pipelines using tools like GitHub Actions, GitLab, Jenkins, or ArgoCD. Experience with Terraform and Docker. Proficiency in backend languages such as Python, Java, or Go. Preferred Skills Experience with lakehouse architectures like Databricks, including knowledge of Delta Lake and Apache Iceberg. Background in infrastructure security, vulnerability management, and observability tooling. High Impact: Help build and scale the data platform that powers Okta s global business. Cutting-Edge Stack: Work with best-in-class technologies like AWS, Snowflake, dbt, Terraform, and Databricks. Collaborative Culture: Join a diverse, inclusive, and globally distributed team that values knowledge sharing and continuous learning. Career Growth: Shape the future of Okta s data engineering practice while expanding your technical and leadership skills. Bring your passion for data, cloud, and automation and let s shape the future of secure, scalable enterprise data platforms together. Qualification : Bachelors degree in Computer Science, Engineering, or a related technical field

Senior Data Engineer Location: Bengaluru Department: Global Strategic Services Data Engineering EMEA Type: Full-Time | Hybrid About Cognite Cognite is a global SaaS leader driving industrial digital transformation. Our platforms Cognite Data Fusion and Cognite Atlas AI enable companies across Energy, Utilities, Manufacturing, and Chemicals to solve complex challenges using AI, contextual data, and automation. Cognite is backed by top-tier investors and recognized with global innovation awards. Our Values Impact: We deliver results that matter. Ownership: We take initiative, act inclusively, and embrace accountability. Relentless: We pursue excellence and innovation with resilience. About the Role As a Senior Data Engineer, you ll lead the development of scalable data solutions that empower critical industries to make informed decisions. You ll work on high-impact projects across regions, collaborating with solution architects, data scientists, and product teams. This is a growth role with room to influence product direction and mentor junior engineers. Key Responsibilities Architect and implement robust data pipelines using Cognite Data Fusion, Python, SQL, and REST APIs. Lead integrations, data modeling, and transformation tasks using cloud-native technologies. Design custom data models for discovery, mapping, and cleansing industrial data. Collaborate closely with cross-functional teams to deliver digital solutions. Conduct code reviews and champion engineering best practices. Contribute to Cognite s SDKs and internal tools. Translate customer use cases into scalable, reusable data engineering frameworks. Mentor team members and support customer onboarding when needed. What You ll Bring Bachelor s or Master s in Computer Science or related field (or equivalent experience). 5+ years in a data-intensive, customer-facing engineering role. Expertise in Python, SQL, REST APIs, and pipeline orchestration. Experience with distributed computing, Kubernetes, and cloud platforms (Azure, GCP). Familiarity with data from industrial domains like oil & gas or manufacturing is a plus. Strong DevOps mindset with hands-on experience in Git, CI/CD, and deployments. Proactive and collaborative; able to work independently and solve complex challenges. A growth mindset with willingness to ask for help and share knowledge openly. Be part of a global team spanning 70+ nationalities with strong DEI focus. Work at our Bengaluru office (Rathi Legacy, Hoodi) in a modern hybrid environment. Enjoy flat hierarchy, fast decision-making, and high ownership culture. Collaborate with world-class professionals on industry-transforming projects. Shape the future of industrial data and drive real-world impact at scale. If you re passionate about solving meaningful problems with cutting-edge data technologies, apply today. We welcome applicants from all backgrounds and experiences you might be the perfect fit, even if you don t meet every single requirement. Qualification : Bachelors or Masters in Computer Science or related field (or equivalent experience).

Data Engineer Experience: 5+ Years Location: Bangalore, Karnataka, India Job Type: Full Time Job Summary We are looking for a talented Data Engineer with over 5 years of experience and a strong foundation in machine learning development to join our team in Bangalore. The ideal candidate will have hands-on expertise in Python programming, machine learning basics, and computer vision techniques like custom object detection and OCR. Key Responsibilities Develop and maintain data pipelines supporting machine learning models and data analytics. Implement custom object detection algorithms and OCR solutions using computer vision techniques. Utilize Python ML libraries such as OpenCV, SciPy, NumPy, Matplotlib, Pandas, Scikit-learn, Keras, PyTorch, and TensorFlow. Collaborate with data scientists and software engineers to optimize data workflows and ML model deployment. Ensure data quality, integrity, and scalability within data infrastructure. Troubleshoot and improve existing machine learning systems and pipelines. Required Skills Minimum of 5 years of experience in data engineering or related roles. At least 2 years of hands-on experience as a Machine Learning developer. Strong programming skills in Python. Solid understanding of machine learning fundamentals. Practical experience with custom object detection and OCR applications. Proficiency in key Python libraries for machine learning and data processing. Ability to work collaboratively in cross-functional teams. Qualifications Any graduate degree from a recognized university. Work with cutting-edge machine learning technologies in a supportive, innovative environment in Bangalore. Grow your career by solving complex problems and building impactful data solutions with a passionate team. Qualification : Any graduate degree from a recognized university.

Data Annotation Specialist Location: Bengaluru, India About Neuron7.ai Neuron7.ai is a rapidly growing AI-first SaaS company that s redefining service intelligence. Backed by leading venture capitalists in Silicon Valley and a distinguished group of angel investors, Neuron7.ai is a startup recognized for its innovation and market traction. Our platform delivers service predictions by analyzing both structured and unstructured data, enabling enterprises in industries like high-tech devices, manufacturing, and medical devices to make accurate service decisions at scale. We re looking for a Data Annotation Specialist to join our team for a 6-month contract. This role involves reviewing and annotating technical documents, converting them into visual formats like graphs and flowcharts, and ensuring precision in technical specifications. The ideal candidate will possess strong communication skills, a deep understanding of technical documentation, and hands-on experience with data visualization tools. At Neuron7.ai, you'll be part of an innovative, dynamic team that's redefining service intelligence. We value creativity, collaboration, and a commitment to pushing boundaries. If you're passionate about transforming complex data into easily digestible visuals, this is the perfect role for you. About the Team Join a team of passionate professionals dedicated to using AI to drive impactful service solutions. Our culture promotes learning, growth, and encourages everyone to contribute ideas and drive projects forward. What You ll Do: Document Review & Annotation: Thoroughly analyze technical documents and annotate key data points accurately, ensuring they are aligned with the original content. Graph & Flowchart Creation: Transform complex technical specifications into clear, concise, and accurate visual formats like graphs, charts, and flowcharts. Data Visualization: Develop easy-to-understand visual data representations to enhance the comprehension of technical documentation. Collaboration: Work closely with internal technical teams to ensure that data and processes are correctly represented and visualized. Quality Control: Ensure that all annotations and visuals meet the highest accuracy standards and align with the original documentation. Clear Communication: Effectively communicate technical details, both in writing and verbally, with internal teams and stakeholders. Time Management: Manage multiple tasks efficiently while meeting deadlines, and be flexible with working hours, primarily in the PST time zone. What You ll Need: Education: A Bachelor s degree (or higher). Curiosity and Analytical Thinking: Strong curiosity about how people think and express ideas, and an ability to distill and clarify complex information for AI and machine learning applications. Communication Skills: Exceptional written and verbal communication skills to convey complex technical concepts clearly. Technical Document Comprehension: Ability to read and understand technical documents, accurately annotating them with precision. Graph & Flowchart Creation: Hands-on experience creating graphs, charts, and flowcharts to visually represent technical data. Attention to Detail: A keen eye for detail to ensure accuracy in annotation and data visualization. Technical Aptitude: Comfort with working with technical information and transforming it into easy-to-understand visuals. Flexible Working Hours: Ability to work flexible hours, primarily within the PST time zone. Immediate Availability: Must be able to start immediately and commit to the full 6-month contract. What We d Like to See: Experience with Diagramming Tools: Familiarity with tools like Microsoft Visio or similar for diagramming and process mapping. Data Annotation Experience: Previous experience in data annotation would be a plus. Multilingual Skills: Fluency in an international language other than English is a plus. Our Commitment to Diversity and Inclusion Neuron7.ai is committed to fostering a diverse and inclusive workplace. We ensure equal employment opportunities without discrimination or harassment based on race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, marital status, or any other characteristic protected by law. If you're excited about using data to drive service intelligence and want to be part of a forward-thinking team, we d love to hear from you! Qualification : A Bachelors degree (or higher).

Sr. Data Engineer - AWS - Big Data Location:Bangalore Type of Employment: Full-Time Experience Required: 7 to 10 years Job Overview: We are seeking a highly skilled Sr. Data Engineer with expertise in AWS cloud technologies and Big Data to join our Cloud Data Architect Team at Infocepts. In this critical role, you will design and implement robust data solutions using technologies like EMR, Athena, PySpark, AWS Lambda, S3, and other AWS services. The ideal candidate will have a strong foundation in database concepts and SQL and will be responsible for building scalable data pipelines to support high-performance data processing. Key Responsibilities: Technology Assessment and Design: Study the existing technology landscape and evaluate current data integration frameworks. Assist in designing complex Big Data use cases leveraging AWS services. Documentation and Stakeholder Communication: Prepare and maintain comprehensive project documentation, adhering to quality guidelines and schedules. Work closely with Architects and Project Managers to provide accurate estimations, scoping, and scheduling assistance. Clearly communicate design decisions and conduct Proof-of-Concepts to validate new solutions before implementation. Process Improvement and Automation: Identify areas for process automation to improve efficiency and team productivity. Provide expert guidance and troubleshooting support to junior Data Engineers. Training and Knowledge Sharing: Develop and deliver technology-focused training sessions for the team, ensuring continuous knowledge sharing. Share expertise through Expert Knowledge Sharing sessions with Client Stakeholders. Essential Skills: AWS Services Expertise: In-depth knowledge of S3, EC2, EMR, Athena, AWS Glue, and Lambda. Big Data Technologies: Proficiency with Apache Spark, Databricks, and Big Data table formats such as Delta Lake (open-source). Data Warehousing: Strong understanding of data warehousing concepts and architectures. Programming Skills: Advanced programming skills in Python for building data pipelines. SQL Expertise: Strong SQL skills for data transformation, aggregation, and querying large datasets. ETL Workflow Development: Expertise in creating ETL workflows with complex transformations (e.g., SCD, deduplication, aggregation). Orchestration Tools: Familiarity with orchestration tools like Apache Airflow. MPP Databases: Experience with at least one MPP database (e.g., AWS Redshift, Snowflake, SingleStore). Cloud Databases: Exposure to cloud databases like Snowflake or AWS Aurora. Desirable Skills: Cloud Databases: Familiarity with Snowflake, AWS Aurora. Big Data Technologies: Experience with Hadoop and Hive. AWS Certification: Associate or Professional Level AWS Certification. Advanced Knowledge of Big Data Solutions: Exposure to big data tools and frameworks on cloud platforms. Qualifications: Experience: 7+ years of overall IT experience, with 5+ years specifically focused on AWS-related projects. Educational Background: Bachelor's degree in Computer Science, Engineering, or a related field (Master's degree is a plus). Technical Certifications: Demonstrated commitment to continuous learning through certifications or relevant training. Qualities: Strong analytical and problem-solving skills to deep dive into complex technical challenges.

Position: Cloud Data Engineer AWS Big Data Location: Bangalore, India Employment Type: Full-time Experience Required: 5 to 8 years Purpose of the Position: Join the Infocepts Cloud Data Architect Team as a Cloud Data Engineer and help design and implement cutting-edge big data solutions on AWS. You will leverage your expertise in EMR, Athena, PySpark, S3, AWS Lambda, and SQL to develop robust and scalable data platforms. Key Responsibilities: Technology Assessment and Design: Assess existing technology landscape and data integration frameworks. Design complex Big Data use cases using AWS services under guidance of the Architect. Support architectural decision-making by evaluating trade-offs in cost, performance, and durability. Recommend optimizations to existing data infrastructure. Documentation and Stakeholder Communication: Create project documentation adhering to quality and delivery standards. Collaborate closely with Architects and Project Managers for scoping, estimation, and planning. Present design decisions to technical and business stakeholders clearly. Conduct PoCs and design review sessions. Process Improvement and Automation: Identify and suggest opportunities for automation and process enhancements. Mentor junior engineers and support technical problem solving. Training and Knowledge Sharing: Prepare and deliver internal training on AWS and Big Data topics. Lead client knowledge sharing sessions and contribute to case studies. Essential Skills: In-depth experience with AWS services: S3, EC2, EMR, Athena, Glue, Lambda Familiarity with MPP databases like Redshift, Snowflake, or SingleStore Proficiency in Apache Spark and Databricks Strong programming skills in Python Experience building data pipelines using AWS and Databricks Knowledge of Big Data file formats such as Delta Lake Advanced SQL skills for large-scale data manipulation Hands-on experience with Apache Airflow or similar orchestration tools Strong understanding of ETL workflows and data warehousing concepts Desirable Skills: Cloud databases: AWS Aurora, Snowflake Experience with Hadoop and Hive AWS Certifications (Associate or Professional level) are a plus Qualifications: Bachelor s degree in Computer Science, Engineering, or related field (Master s preferred) Overall 5+ years of IT experience with at least 3 years in AWS Big Data projects Ongoing learning and technical certifications are strongly encouraged Key Qualities: Strong problem-solving and analytical thinking Self-driven with a passion for emerging data technologies Excellent communication and client presentation skills Ability to work in cross-functional, agile teams Apply now to be part of a high-impact data transformation team working on large-scale cloud data projects! Qualification : Bachelors degree in Computer Science, Engineering, or related field (Masters preferred)

Job Title: Software Engineer Location: Bengaluru, India Company: Google Minimum Qualifications: Bachelor's degree in Computer Science or a related technical field, or equivalent practical experience. 5 years of experience with one or more general-purpose programming languages, including but not limited to: Java, C/C++, Python, Objective C, JavaScript, or Go. 4 years of experience with Distributed Processing, System Design, and High-Performance Computing. Preferred Qualifications: Experience in solving ambiguous problems in complex environments. Experience in building distributed systems that are scalable and efficient. Ability to design and complete projects independently, taking ownership of the end-to-end process. About the Job: Google's software engineers develop next-generation technologies that change how billions of users connect, explore, and interact with information and one another. Our products need to handle information at massive scale and extend well beyond web search. We're looking for engineers who bring fresh ideas from all areas, including information retrieval, distributed computing, large-scale system design, networking, data storage, security, artificial intelligence, natural language processing, UI design, and mobile. The list of technologies we engage with is always growing. As a software engineer, you will work on a specific project critical to Google s needs, with opportunities to switch teams and projects as you and our fast-paced business grow and evolve. We need our engineers to be versatile, to display leadership qualities, and to be enthusiastic to take on new problems across the full-stack as we continue to push technology forward. Core Team Overview: The Core team builds the technical foundation behind Google s flagship products. We are the owners and advocates for the underlying design elements, developer platforms, product components, and infrastructure that make up the backbone of Google s services. These are essential building blocks for creating excellent, safe, and coherent user experiences that also drive the pace of innovation for every developer at Google. We look across Google s products to build central solutions, break down technical barriers, and strengthen existing systems. As the Core team, we have the unique opportunity to impact critical technical decisions across the company, influencing the direction of the most foundational and influential aspects of Google s technology. Responsibilities: Work on a core indexing platform that is pivotal to the functionality and performance of Google s services. Design, develop, and maintain highly efficient, scalable, and robust systems to manage large-scale indexing challenges. Work with cross-functional teams to create innovative solutions for distributed computing problems, ensuring that Google s products remain performant and reliable at scale. Contribute to building distributed systems capable of handling immense data volumes in real-time, ensuring that Google's systems can scale with global usage. Participate in code reviews, design discussions, and the creation of detailed documentation to ensure high-quality outcomes. Qualification : Bachelor's degree in Computer Science or a related technical field, or equivalent practical experience.

Job Title: Node.js Developer Experience: 3+ Years Location: Bangalore Responsibilities Collaborate within an Agile team to develop applications and services. Contribute to team and organizational improvements in process and infrastructure. Build customer-facing UI and back-end services for payment processing administration and management. Code, test, and operate Node.js-based services. Use tools and ingenuity to identify and fix defects before they become a problem. Requirements In-depth knowledge of Node.js with serverless frameworks for AWS Lambda Server. Hands-on experience with HTML, CSS, and AJAX. Development experience in designing object-oriented JavaScript and SVG. Familiarity with web libraries and frameworks such as AngularJS, Polymer, and Closure. Knowledge of the full web stack, including protocols and web server optimization techniques. Strong analytical and problem-solving skills. Attention to detail.

Job Title: Data Engineer Location: Bengaluru, Karnataka Experience: 5 to 8 Years Work Modality: Full-time (Work from office) Job Description: We are looking for an experienced Data Engineer to join our team and take responsibility for designing, developing, and maintaining scalable ETL/ELT pipelines. This is a full-time position based in Bengaluru, Karnataka, and you will be collaborating with cross-functional teams to define data requirements and ensure data accuracy, consistency, and integrity. Your role will also involve optimizing data workflows, automating processes, and ensuring high availability and reliability of data pipelines. Key Responsibilities: ETL/ELT Pipeline Development: Design, develop, and maintain scalable ETL/ELT pipelines to support data transformation and integration processes. Data Warehouse & Data Lake Optimization: Build and optimize data warehouses, data lakes, and real-time streaming solutions to support large-scale data operations. Collaboration & Data Requirements: Collaborate with cross-functional teams, such as product, data science, and analytics teams, to define data requirements and ensure data accuracy and consistency. Database Structure & Schema Management: Develop and maintain database structures and schemas to ensure efficient data storage and retrieval. Data Workflow Optimization: Optimize data workflows for performance, reliability, and scalability, ensuring the highest level of efficiency. Data Security & Compliance: Implement data security, governance, and compliance best practices to ensure that data is handled securely and meets industry standards. Pipeline Monitoring & Troubleshooting: Monitor, troubleshoot, and improve data pipelines to ensure uptime, reliability, and smooth data processing. Process Automation: Automate data-related processes to improve efficiency and reduce manual intervention, increasing the overall speed of data flow. Required Qualifications: Experience: 5+ years of experience in data engineering or 3-4 years of experience as a Data Engineer. Technical Skills: Strong proficiency in SQL and database management systems such as PostgreSQL, MySQL, SQL Server, etc. Experience with ETL tools such as Pentaho, Talend, Cdata, and SSIS. Exposure to Python, Java, or Scala for data processing is a plus. Experience with big data technologies such as Apache Spark, Hadoop, or Kafka. Familiarity with cloud services (AWS, Azure) and data storage solutions such as S3, Redshift, Snowflake, or BigQuery. Strong knowledge of data modeling, warehousing concepts, and data architecture best practices. Soft Skills: Excellent communication skills with the ability to collaborate effectively across teams. Strong problem-solving skills and the ability to work with large, complex datasets. What We Offer: Competitive Salary: Attractive salary based on experience and expertise. Collaborative Work Environment: Work in a dynamic and fast-paced environment with a team that fosters innovation and collaboration. Growth Opportunities: Opportunities to enhance your skills and career growth in the data engineering field. Comprehensive Benefits: Benefits package designed to support work-life balance and overall employee well-being.

Our Company At Adobe, we're changing the world through digital experiences. We empower everyone from emerging artists to global brands with everything they need to design and deliver exceptional digital experiences. We're passionate about helping people create beautiful and powerful images, videos, and apps, and transform how companies interact with customers across every screen. We are on a mission to hire the very best talent and are committed to creating exceptional employee experiences where everyone is respected and has equal access to opportunity. New ideas can come from anywhere, and the next big idea could be yours! About the Team The Adobe Content Platform and Collaboration (ACPC) group is building a highly scalable and available content platform for Adobe s SaaS offerings, running on Amazon AWS. We're pushing the future of SaaS development by leveraging the latest technology stacks and design thinking. Our dedicated team of developers is building low-latency, cloud-scale foundational components such as Cache, Storage, Databases (SQL and NoSQL), Queuing, Pub/Sub, etc., to create a standard for developing, deploying, operating, and governing Adobe SaaS solutions, catering to millions of requests every day. What the Team Does: We love writing software. We love solving complex problems with simple, elegant designs. We build the platform using modern container orchestration technologies. We take a long-term architectural approach, ensuring that the platform supports both current and future needs of all Adobe services. We use best-in-class tools like GitHub, Slack, and any tools that help us stay productive. We are committed to adapting our processes to meet evolving needs. We trust each other to do our jobs exceptionally well. What You ll Do: Design and develop one or more services/components within the platform. Take responsibility for the full lifecycle of the project from user story to design, development, testing, documentation, and maintenance. As a senior team member, you will be responsible for setting code guidelines for the rest of the team. Adhere to standard software engineering practices. Collaborate in a highly dynamic, ever-evolving environment. Work with geographically distributed teams. Mentor junior team members to help them grow and improve. What You Need to Succeed: B.Tech/M.Tech in Computer Science (or equivalent experience) with an excellent academic record. 5 to 8 years of software development experience, primarily in backend systems. Proficiency in data structures and algorithms. Expertise in Java and related technologies (e.g., Spring Boot). Proven experience in designing, building, evolving, and maintaining large-scale, complex software projects. Self-starter with an ability and eagerness to learn new technologies and adapt quickly. Proven experience in developing scalable Cloud-based services on AWS/Azure. Strong programming and problem-solving skills. Good written and verbal communication skills. Strong team player who thrives in a collaborative environment. Experience with Agile development practices. Prior experience with MySQL, Redis, Cassandra, DynamoDB, Kafka will be a bonus. Pure Innovation: We celebrate critical thinkers who embrace challenges and aspire to innovate. Pure Growth: At Adobe, you have the space to grow and contribute meaningfully to something impactful. Pure Team: We build each other up and set aside egos for the greater good. Adobe is proud to be an Equal Employment Opportunity and affirmative action employer. We do not discriminate based on gender, race, color, ethnicity, national origin, age, disability, religion, sexual orientation, gender identity or expression, veteran status, or any other characteristic protected by law. Qualification : B. Tech and/or M. Tech in Computer Science with excellent academic record, or equivalent experience

Our Company At Adobe, we re transforming the world through digital experiences. We provide everyone from emerging artists to global brands with everything they need to design and deliver exceptional digital experiences. We re passionate about empowering people to create beautiful and powerful images, videos, and apps, and transform how companies interact with customers across every screen. We re on a mission to hire the very best and are committed to creating exceptional employee experiences where everyone is respected and has equal access to opportunity. We believe that new ideas can come from anywhere in the organization, and the next big idea could be yours! Responsibilities Own the development of features of medium to large complexity, applying in-depth knowledge to turn requirements into architectural and feature specifications. Contribute extensively to the analysis, design, prototyping, and implementation of new features while improving existing ones. Address broad architecture and design issues for current and future products or technologies, providing strategic direction in evaluating new technologies. Collaborate with product management and engineering leads to evaluate and determine new features to be added. Be a proactive self-starter and fast learner who can develop methods, techniques, and evaluation criteria to achieve results. Specialize in one or more platforms and be knowledgeable of cross-platform issues and customer requirements. Significantly contribute towards the development and application of sophisticated concepts, technologies, and expertise within the team. Ensure high-quality code and related documentation. What You Need to Succeed B.Tech/M.Tech from an outstanding institute with 3 to 5 years of hands-on design and development experience in software development, preferably in a product development organization. Strong programming skills in Java. Hands-on experience with REST APIs and message pub/sub models. Proven understanding of frameworks like Spring Boot and Apache Kafka. Knowledge of software fundamentals, including design & analysis of algorithms, data structure design, implementation, documentation, and unit testing. Good understanding of object-oriented design, product life cycles, and associated issues. Strong computer science fundamentals with a solid understanding of architecture, design, and performance. Ability to work proactively and independently with minimal direction. Be an excellent teammate with strong written and oral communication skills. Pure Innovation: We celebrate those who think critically, embrace challenges, and aspire to be trailblazers. Pure Growth: At Adobe, we give you the space and support to grow, contribute meaningfully, and expand your horizons. Pure Team: We build each other up and set aside egos for the greater good. Adobe is proud to be an Equal Employment Opportunity and Affirmative Action employer. We do not discriminate based on gender, race, color, ethnicity, national origin, age, disability, religion, sexual orientation, gender identity or expression, veteran status, or any other characteristic protected by law. Qualification : B.Tech/M.Tech from an outstanding institute with 3 to 5 years of hands-on design / development experience in software development, preferably in a product development organization.

Job/Position Summary: Data Engineer Responsibilities: Implement data pipelines that meet design and are efficient, scalable, and maintainable. Implement best practices including proper use of source control, participation in code reviews, data validation and testing. Timely deliveries while working on projects. Act as advisor/mentor and helps junior data engineers in their deliverables. Must Have Skills: Should have experience of at least 4+ years with Data Engineering. Strong experience of design, implementation and fine-tuning big data processing pipelines in production environment. Experience with big tools like Hadoop, Spark, Kafka, Hive, Databricks. Experience in programming at least one of with Python, Java, Scala, Shell Script. Experience with relational SQL and NO SQL databases like PostgresSQL, MYSQL, Cassandra etc. Experience with any data visualization tool (Plotly, Tableau, Power BI, Google Data Studio, Quick sight etc.). Good To Have Skills: Should have Basic Knowledge of CI/CD Pipeline. Experience in working on at least one Cloud (AWS or Azure or GCP). For AWS: - Experience with AWS Cloud services like EC2, S3, EMR, RDS, Athena, Glue, Lambda, EMR. For Azure: -Experience with Azure Cloud services like Azure Blob/Data Lake GEN2, Delta Lake, Databricks, Azure SQL, Azure DevOps, Azure Data Factory, Power BI. For GCP: - Experience with GCP Cloud services Big Query, Cloud Storage bucket, DataProc, Dataflow, Pub Sub, Cloud Function, Data Studio. Sound familiarity in Versioning tools (Git, SVN etc.). Experience Mentoring students is desirable. Knowledge of latest developments in Machine Learning, Deep Learning, Optimization in Automotive domain. Open minded approach to explore multiple algorithms to design optimal solution. History of contribution to articles/blogs/whitepapers etc. in Analytics. History of contribution to Open Source. Requirement: ESSENTIAL SKILLS /COMPETENCIES Data Engineering Hadoop Kafka CI/CD Cloud

Your Impact We are seeking a passionate Data Engineer with expertise in Python development who is excited about cloud-based data engineering using AWS services. You will be an integral part of a dynamic, multi-disciplinary team, working closely with digital product professionals, data scientists, cloud engineers, and other stakeholders. As a key member of a global team working on our generative AI initiative, you will be based in one of our European offices. McKinsey s Tech Ecosystem function is responsible for developing and delivering all technology solutions for the firm s internal use, and your role will be crucial in driving the development of data solutions to support generative AI applications. You will work with a team of data engineers to develop robust data ingestion pipelines and enhance data processing capabilities that integrate data into systems used by AI applications. Your responsibilities will include writing Python code, creating tests, developing and maintaining GitHub Action CICD pipelines, and managing AWS-based infrastructure and Docker containers. Your Growth As a member of the global team working on our generative AI initiative, you will play a key role in shaping and accelerating the delivery of McKinsey's target state data platform, which will enable AI use-cases. You will be part of our cloud-first approach, transforming data platforms and analytical applications across the firm. Working closely with multidisciplinary teams, you will contribute to building cutting-edge data solutions in a fast-paced, innovative environment. McKinsey s Tech Ecosystem function is responsible for developing all technology solutions for the firm s internal needs, and you ll have the opportunity to shape how these solutions evolve. Your Qualifications and Skills 3+ years of professional experience as a Data Engineer, with a focus on cloud-based data engineering using AWS services Expertise in Python development and a strong understanding of clean code, modularity, error handling, and test automation Extensive experience with relational databases and data pipeline performance Hands-on experience with Docker and CI/CD pipelines (e.g., GitHub Actions) Strong execution focus, with the ability to work independently in complex, fast-paced environments and deliver results Demonstrable experience in solving data pipeline performance issues and diagnostics Interest in generative AI and machine learning topics Experience with Kedro framework is a plus Opinionated and confident in sharing ideas, willing to speak up at all levels Familiarity with Agile principles and product development methodologies Excellent problem-solving skills and the ability to analyze and resolve complex data engineering challenges Strong communication and collaboration skills, with the ability to work effectively in cross-functional teams

Your Impact As a Data Engineer at QuantumBlack, you will collaborate with stakeholders, data scientists, and internal teams to develop and implement impactful data products and solutions. Your key responsibilities will include building and maintaining technical platforms for advanced analytics, designing scalable and reproducible data pipelines for machine learning, and ensuring information security within data environments. You will assess clients' data quality, map data fields to hypotheses, and prepare data for use in analytics models. Additionally, you will contribute to R&D projects, internal asset development, and participate in cross-functional problem-solving sessions with a variety of stakeholders, including C-level executives, to create innovative analytics solutions. You will be based in Gurugram, joining a global data engineering community, and work within cross-functional and Agile project teams alongside project managers, data scientists, machine learning engineers, other data engineers, and industry experts. You will collaborate directly with our clients, ranging from data owners and users to C-level executives. You will be aligned with one of our industry-focused practices: Pharma & Medical Products (PMP) or Global Energy & Materials (GEM). In these practices, you ll work on solving the most critical challenges for our clients in these sectors. PMP focuses on advancing the development and delivery of life-saving medicines and medical treatments, while GEM supports industries like chemicals, steel, mining, and energy to achieve operational excellence. GEMx and PMPx, the assetization arm of these practices, focus on creating reusable digital and analytics assets to support client work. As part of this team, you will help shape impactful solutions for large organizations, developing capabilities for sustained impact. Your Growth In this role, you will contribute to the frameworks and libraries that our teams of Data Scientists and Engineers use to progress from data to meaningful impact. You will have the opportunity to guide global companies through data science solutions, helping them transform and enhance performance across industries including healthcare, automotive, energy, and elite sports. Real-World Impact: You ll gain unique learning and development opportunities globally. Fusing Tech & Leadership: Work with the latest technologies and methodologies, with access to top-tier learning programs. Multidisciplinary Teamwork: Collaborate with data scientists, engineers, project managers, UX designers, and more to enhance performance. Innovative Work Culture: Creativity, passion, and wellness are central to our modern work environment, which includes insightful talks, training sessions, and a focus on work-life balance. Striving for Diversity: We celebrate diversity, with colleagues from over 40 nationalities, appreciating the value that diverse perspectives bring to the workplace. You are a highly collaborative individual who prioritizes impact over agenda. You enjoy learning from colleagues, challenging ideas thoughtfully, and working together to improve processes and solve problems. You believe in iterative change, experimenting with new approaches, and advancing quickly through constant learning and improvement. While we value using the right tech for the right task, our team often leverages technologies such as Python, PySpark, SQL, Airflow, Databricks, Kedro (our open-source data pipelining framework), Dask/RAPIDS, Docker, Kubernetes, and cloud solutions like AWS, GCP, and Azure. Your Role as a Data Engineer Collaboration: Work with business stakeholders, data scientists, and internal teams to create extraordinary, domain-focused data products (reusable assets) and deliver them to clients. Domain Expertise: Develop deep understanding of client industries and use creative techniques to deliver meaningful impact. Technical Platforms: Build and maintain technical platforms for advanced analytics engagements, spanning both data science and data engineering work. Data Pipelines: Design and implement robust, modular, scalable, deployable, and reproducible data pipelines for machine learning. Data Management: Ensure the security of data environments and compliance with information security standards. Data Wrangling: Assess data quality, map data fields to hypotheses, and prepare data for analytics models. Contribute to R&D: Participate in internal asset development and contribute to R&D projects to drive innovation. Cross-functional Problem Solving: Collaborate with internal teams and clients, including data owners and C-level executives, to create impactful analytics solutions. Your Qualifications and Skills Bachelor s degree in computer science or related field; Master's degree is a plus. 2-5 years of relevant work experience. Proficiency in at least one programming language such as Python, Scala, or Java. Strong experience with distributed processing frameworks (e.g., Spark, Hadoop, EMR) and SQL. Experience in commercial client-facing projects, particularly in close-knit teams. Ability to work with structured, semi-structured, and unstructured data, and identify linkages across disparate data sets. Clear communication skills to explain complex solutions effectively. Understanding of information security principles to ensure compliant handling of client data. Experience with cloud platforms (AWS, Azure, Google Cloud, Databricks) is highly desirable. Experience with CI/CD processes using GitHub Actions, CircleCI, or similar, and end-to-end pipeline development including application deployment is a plus. Qualification : Bachelors degree in computer science or related field; Master's degree is a plus.

Intuit is a global leader in financial technology, dedicated to helping individuals and businesses thrive. Our suite of products, including TurboTax, Credit Karma, QuickBooks, and Mailchimp, serves approximately 100 million customers worldwide. At Intuit, we believe in providing everyone with the tools and resources they need to achieve financial success. We are constantly innovating to make financial empowerment a reality for all. Job Overview Join the Intuit Data Platform (IDP) team as a Staff Engineer and help us transform the way we handle big data! The IDP team is responsible for the Intuit Analytics Platform, which powers real-time data ingestion, cataloging, analytics, and machine learning across the entire organization. As Intuit s customer base grows, so does the volume of data we process. Our engineering excellence ensures that we can scale and leverage this data to drive machine learning and product innovations. We re in the process of building the next-generation real-time and batch ingestion engine, capable of indexing, cataloging, and organizing data and metadata. We are passionate about using open-source technologies to solve challenges and contributing back to the community. If you're excited about building a platform that will directly impact data scientists and analysts and have a desire to shape the future of data at Intuit, then come join us! Key Responsibilities Architect & Design: Build fault-tolerant and scalable big-data platforms using open-source technologies to handle massive datasets. Data Solutions: Create architecture solutions that address complex use cases like data normalization, lineage, governance, ontology, and discoverability. Cross-Team Collaboration: Work with analysts and data scientists to understand data requirements for building operational propensity models and gaining deep customer insights. Hands-On Coding: Lead development efforts within the Hadoop ecosystem using technologies such as Java MapReduce, Spark, Scala, HBase, and Hive to build and optimize data pipelines for both real-time and batch applications. Database Management: Work with NoSQL, SQL, and in-memory databases to design high-performance data systems. Code Reviews: Ensure code quality, consistency, and adherence to best practices through regular code reviews. Architectural Alignment: Ensure alignment between enterprise architecture and business requirements. Prove Feasibility: Conduct proof-of-concept (POC) experiments for new technologies or approaches and drive them to production. Collaboration with Data Cataloging Team: Work closely with data catalog teams and architects to index and catalog all data sources at Intuit. Agile Leadership: Lead fast-paced development teams using agile methodologies and promote best practices in software development, testing, and incident response. Design & Model: Build dimension models suited for customer business use cases and ensure seamless integration of business and technical requirements. Qualifications Experience: 12+ years of relevant experience, with at least 5+ years specializing in the big data domain. Big Data Architecture: Proven experience in architecting end-to-end ecosystems for big data and analytics platforms. Expert Knowledge: Deep expertise in building fault-tolerant, scalable big data solutions, especially using the Hadoop ecosystem (Hive, HBase, Spark, Kafka, MapReduce, etc.). Programming Expertise: Mastery of Java and Scala, with a focus on building high-throughput data services. Machine Learning: Knowledge of machine learning principles and AI applications in big data. Big-Data Technologies: Familiarity with tools such as HDFS, Storm, Zookeeper, Cassandra, Redshift, GraphDB, and others. Understanding both real-time and batch processing in the Hadoop ecosystem. Communication: Strong communication skills, with an ability to explain complex technical topics to both technical and non-technical audiences. Programming Skills: Intermediate experience in Python or R for data processing. Education: BE/BTech/MS in Computer Science or a related field (or equivalent experience). Collaboration: Demonstrated ability to work cross-functionally and lead change through influence and example. At Intuit, you ll be part of a talented, passionate team working on innovative solutions that shape the future of data analytics and machine learning. As a Staff Engineer, you ll have the chance to work with cutting-edge technologies, build scalable systems, and help revolutionize how Intuit leverages data to drive product innovation. If you're looking for a dynamic environment where you can have a meaningful impact, come join us at Intuit! Qualification : BE/BTech/MS in Computer Science (or equivalent)

Job Title: Senior Python Expert Experience: 3 to 8 years Overview of the Role: This role sets the benchmark for team software development processes and deployment procedures, while actively contributing to establishing best practices and methodologies within the team. Responsibilities: Develop backend services using Python Flask or similar web frameworks. Design and maintain data and pipeline management frameworks using open-source technologies such as Hadoop, Hive, Spark, HBase, Kafka Streaming, Tableau, Airflow, and cloud-based data engineering services like S3, Redshift, Athena, Kinesis. Design and develop efficient CRUD operations for large datasets with millions of records. Collaborate with teams to build and maintain innovative, reliable, secure, and cost-effective distributed solutions. Own and deliver complex application components within defined timelines. Ensure quality delivery using best practices in API development, performance, and scalability. Participate in customer communication, presentations, and resolution of critical issues. Contribute to architecture, feature set, and design decisions. Proactively recognize and address requirement inconsistencies and project risks. Break down work effectively and estimate accurately. Serve as a technical mentor and role model within the team. Required Skills: Strong knowledge of Python and a web framework like Flask. Experience with AWS services such as EC2, S3, Lambda, Step Functions, Glue, SNS, SQS, Secret Manager, and CodeBuild/CodePipeline. Solid understanding of SQL, including query writing, optimization, and database interaction tools. Experience with API development (RESTful services, Postman, API Gateway). Strong coding, debugging, and problem-solving skills. Understanding of architectural trade-offs and data engineering principles (data acquisition, ingestion, distributed processing, high availability). Ability to independently handle delivery of complex projects. Excellent team management and individual contributor skills. Good to Have Skills: Experience with Big Data frameworks like Hadoop and Spark. Knowledge of AWS Aurora. Familiarity with pip, setuptools, etc. Education: B.E./B.Tech, MCA, M.E./M.Tech.

Job Title: Data Engineer Job Summary As a Data Engineer in the IT team, you will work on various big data challenges using the Databricks platform. You will provide data engineering, data science, and cloud technology projects which require integrating with client systems, training, and other technical tasks to help business to get most value out of their data. The Impact You Will Have You will work on a variety of impactful Big Data projects which may include building reference architectures, how-to's and production grade MVPs. Work on implementing transformational big data projects, 3rd party migrations, including end-to-end design, build and deployment of industry-leading big data and AI applications. Work on architecture and design; bootstrap or implement strategic projects. What We Look For 7+ years experience with Big Data Technologies such as Apache Spark , Kafka, Cloud Native and Data Lakes. 4+ years of experience working on Big Data Architectures independently. Preferred experience working in the Databricks ecosystem. Comfortable writing code in either Python or Scala. Experience working across Cloud Platforms (GCP / AWS / Azure). Documentation and white-boarding skills. Build skills in technical areas which support the deployment and integration of Databricks-based solutions to complete customer projects. About Databricks Databricks is the data and AI company. More than 10,000 organizations worldwide including Comcast, Cond Nast, Grammarly, and over 50% of the Fortune 500 rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics, and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark , Delta Lake, and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook. Benefits At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees. For specific details on the benefits offered in your region, please visit https://www.mybenefitsnow.com/databricks.

Description What you'll do: Develop, test, maintain, and improve our high-quality standard services. To meet the requirements and be successful, we are constantly working on increasing the level of automation of our services that must be easily accessible for the product teams of the company. Contribute to the code review and to the code quality standards with help from dedicated tools. Technical lead for some of the projects. Making architecture decisions for future developments. Creating software concepts for next projects. Use effective communication skills to align with the team internally or collaborate with cross-functional teams to achieve the desired objectives. Provide guidance and mentorship to less experienced colleagues, fostering their growth and contributing to the overall team development. What you'll bring: 5+ years of relevant working experience. Excellent coding skills with Java. Experience in dependency injection frameworks, preferably Spring (Spring Security, Spring Web MVC, Spring Data, Spring Data REST, Spring Cache). Strong knowledge in SQL / Relational Databases (e.g., PostgreSQL, MySQL) and NoSQL databases (e.g., MongoDB). Experience in Azure. Familiar with OAuth flows, REST, and Open API specifications. Familiar with Linux, Git SCM, and Git flows. Experience with feature flag concept. Have a passion for standardization, introducing processes, and fast customer feedback. Ability to drive projects and strong problem-solving skills. Fluent in English, both written and spoken. Team-oriented with excellent communication abilities.