Batch AND Stream Processing Jobs in Bengaluru
381 Jobs Found
Mts - Software Development (cloud Ai Network Security Developer)
Aviatrix Systems
MTS - Software Developer (Cloud AI Network Security Developer) Location: Bengaluru Company: Aviatrix Experience: 1 3 years About Aviatrix: Aviatrix is a cloud network security leader trusted by over 500 enterprises. We specialize in securing multi-cloud environments, offering runtime protection and advanced control for modern cloud infrastructures. Role Strategy & Impact In this role, you will build next-generation intelligent cloud network security solutions. You will focus on developing Python/Go microservices that fuse network visibility with LLM-driven insights to redefine cloud firewall capabilities. Technical Requirements Core Competencies: Development: Professional experience in Go (Golang) or Python. Cloud Networking: Fundamentals of Routing, NAT, VPNs, and Subnets. Security: Understanding of Firewall concepts (ACLs) and Zero Trust architecture. AI Integration: Experience using AI/LLM APIs (OpenAI, Vertex AI, etc.). Data Infrastructure: Workflows involving Kafka, data ingestion, and stream processing. Cloud Ecosystem: Hands-on familiarity with AWS, Azure, or GCP. Preferred Qualifications: Network Observability: Experience with NetFlow, IPFIX, or VPC Flow Logs. Modern DevOps: Hands-on with Kubernetes, Container Networking, and Terraform. Generative AI: Knowledge of Prompt Engineering or RAG-based systems. Key Responsibilities Control Plane Development: Build services for firewall rules and policy orchestration. AI Workflows: Integrate LLM-based assistants for anomaly detection and alert summarization. Telemetry Pipelines: Maintain high-performance data pipelines for security event metrics. Security Logic: Design logic for threat pattern recognition and posture scoring. Benefits & Why Join Us Global Benefits: Private medical, pension, and life assurance. Work-Life Balance: Generous holiday allowance and annual wellbeing stipend. Growth Mindset: We value diverse paths if you are passionate about AI and Security, we want to hear from you.
Staff Engineer - Software Development
Aviatrix Systems
Staff Engineer - Software Development (Cloud AI & Network Security) Location: Bengaluru Company: Aviatrix Experience Required: 7+ Years About Aviatrix: Aviatrix is a global leader in cloud network security, trusted by over 500 enterprises. We provide a specialized platform for securing multi-cloud environments, giving organizations the control and visibility needed to modernize their cloud strategies. Architectural Focus & Impact As a Staff Engineer, you will architect and deliver advanced AI-driven network security solutions. This role bridges the gap between Distributed Systems (Python/Go), Real-time Telemetry, and LLM-integrated automation to build self-learning, adaptive security infrastructures. Technical Expertise Core Software Engineering: Languages: Deep proficiency in Python and Go (Golang). Distributed Systems: Mastery of Kubernetes, Microservices, and high-scale observability (Prometheus, ELK). Data Pipelines: Experience with real-time stream processing using Kafka, Flink, Kinesis, or Pub/Sub. Networking & Security Domain: Cloud Infrastructure: Expert knowledge of VPC/VNet design, Routing, Load Balancers, and Overlays. Firewall Technologies: Hands-on with Deep Packet Inspection (DPI), NGFW/IDS/IPS, and Cloud-native firewalls (AWS, Azure, GCP). Security Frameworks: Alignment with Zero Trust, NIST CSF, and CIS Benchmarks. AI & Machine Learning Integration: Model Serving: Experience serving ML models via REST or gRPC. Generative AI: Familiarity with LLM integration, RAG (Retrieval-Augmented Generation), LangChain, and vector databases. Key Responsibilities System Architecture: Lead the design of cloud-native microservices for security control planes. AI-Driven Features: Integrate LLMs for Natural Language-to-Firewall Rule translation and automated incident summarization. Technical Leadership: Mentor junior engineers and set high standards through rigorous Design and Code Reviews. Cross-Functional Collaboration: Partner with Data Scientists and Cloud Networking teams to deliver production-grade AI features. Benefits & Why Join Us Regional Package: Comprehensive pension, private medical coverage, and life assurance. Wellbeing: Annual wellbeing stipend and generous holiday allowance. Growth Culture: We value unique career paths and prioritize candidates who are passionate about the intersection of AI and Security.
Data Engineering Lead
Fampay
Data Engineering Lead Bengaluru | Engineering | Full-Time About Fam (formerly FamPay) Fam is India s first payments app designed for everyone aged 11 and above. FamApp enables seamless online and offline payments through UPI and FamCard. Our mission is to empower over **250 million young Indians** to start their financial journey early, becoming financially aware and confident. Founded in 2019 by IIT Roorkee alumni, Fam is backed by top-tier investors including Elevation Capital, Y-Combinator, Peak XV (Sequoia Capital India), Venture Highway, and angels like Kunal Shah and Amrish Rao. About the Role We re looking for a visionary **Data Engineering Lead** to take **end-to-end ownership** of Fam s data ecosystem from data ingestion and storage to processing and delivering actionable insights. You ll **define the data strategy and architecture** that supports both batch and **real-time** use cases, ensuring scalability, reliability, and governance across the organization. You will be instrumental in enabling accurate, complete, and trusted data flow that powers business intelligence, analytics, and product decision-making. This role involves **leadership, strategic thinking**, and hands-on problem solving. What You ll Do Own the full data lifecycle: ingestion, organization, storage, processing, and presentation. Define and execute **data architecture and strategy** aligned with operational and analytical goals. Build **scalable, reliable, and observable data systems** supporting batch and near real-time processing. Ensure **data quality, governance, and compliance**, proactively resolving discrepancies. Collaborate with product, engineering, and business teams to define, track, and optimize key metrics. Anticipate data-related challenges and implement preventive solutions. Lead, mentor, and grow the data engineering team, fostering innovation and accountability. Must-Haves 10+ years experience in data engineering, including proven leadership of teams or projects. Expertise designing, building, and scaling end-to-end data pipelines and systems. Deep understanding of the data lifecycle, from ingestion through business reporting. Strong communication skills and ability to collaborate across technical and business teams. Solid knowledge of **data governance, quality assurance, and compliance standards**. Experience with observability and proactive monitoring for data systems. Proficiency in Python and SQL; familiarity with Scala or Java. Hands-on experience with streaming and batch data frameworks. Experience designing large-scale data lakes and warehouses with best practices for schema evolution and partitioning. Strong background with **cloud platforms (AWS, GCP, or Azure)**. Fintech or regulated industry experience is a plus. Good to Have Fintech-specific data experience, including regulatory compliance and reporting. Deployment experience with **real-time analytics** and event-driven architectures. Familiarity with containerization and infrastructure tools like Docker, Kubernetes, Terraform. Knowledge of data observability tools (Monte Carlo, Databand, etc.). Exposure to **ML pipelines** and model deployment. Solve challenging problems at the intersection of big data, real-time processing, and fintech. Lead impactful data initiatives at a rapidly growing startup. Collaborate with a world-class team of engineers, data scientists, and product leaders. Competitive compensation, equity, and benefits. Clear career growth opportunities in leadership and innovation. Perks That Go Beyond the Paycheck Relocation assistance for a smooth move. Free office meals (lunch & dinner). Generous leave policies (birthday, period, parental support, and more). Salary advances and loan policies for financial support. Quarterly rewards, recognition, and referral incentives. Access to the latest gadgets and tools. Comprehensive health insurance with mental health support. Tax benefits like food coupons, phone allowances, and leasing options. Retirement benefits including PF contribution, leave encashment, and gratuity. About FamApp FamApp focuses on financial inclusion for the next generation by offering UPI and card payments to users aged 11+. Our flagship product, FamX, integrates UPI and card payments seamlessly, helping users manage, save, and learn about their finances effortlessly. With over **10 million users**, FamApp is revolutionizing how young Indians transact eliminating the need to carry cash and offering customizable FamX cards with personal doodles for a fun, unique payment experience. Join Our Dynamic Team At Fam, we foster a people-first culture with flexible work schedules, generous leave, comprehensive health benefits, and mental health support. You ll be part of a passionate, talented, and fun team shaping the future of fintech for India s youth.
Data Engineer
Capital One
Data Engineer Location: Bangalore Company: Capital One India About Capital One At Capital One, we're redefining how technology solves real-world financial challenges. As a technology-driven company, we bring together talented engineers, data scientists, and designers to innovate at scale and deliver meaningful impact to millions of customers. If you're passionate about building powerful data solutions, exploring cutting-edge technologies, and working in a collaborative, fast-paced environment this is the place for you. About the Role As a Data Engineer at Capital One, you ll join a team of innovators who design and build next-generation data platforms and pipelines that power real-time decision-making. You ll collaborate across disciplines engineering, product, machine learning, and cloud infrastructure to transform how we leverage data at scale. What You ll Do Collaborate across Agile teams to design, develop, test, and deploy data-driven solutions. Build and support scalable data pipelines using modern data engineering tools and cloud services. Work on real-time and batch data processing systems that integrate with distributed microservices and ML platforms. Use programming languages such as Python, Java, or Scala with SQL, NoSQL, and cloud data warehouses like Redshift or Snowflake. Contribute to code reviews, unit testing, and performance optimization to ensure high-quality data systems. Partner with product managers and platform teams to deliver robust, cloud-native data solutions that power business decisions. Stay ahead of tech trends, share knowledge, and mentor junior engineers. Basic Qualifications Bachelor s degree in Computer Science, Engineering, or a related field. 1.5+ years of hands-on experience in application or data engineering (excluding internships). At least 1 year of experience working with big data technologies. Preferred Qualifications 3+ years of application/data engineering experience using Python, Scala, Java, or SQL. 1+ year of experience with cloud platforms (AWS, Azure, or GCP). 2+ years of experience with distributed computing tools (Spark, Hadoop, Hive, EMR, Kafka, etc.). 1+ year working on real-time streaming applications. 1+ year of experience with NoSQL databases (MongoDB, Cassandra). 1+ year of experience with data warehousing (Redshift, Snowflake). 2+ years working with Linux/Unix systems and shell scripting. Familiarity with Agile methodologies and modern DevOps practices. Why Join Capital One Work on high-impact data solutions at one of the world s most innovative financial institutions. Be part of a collaborative tech culture that values experimentation and learning. Access to top-tier tools, mentorship, and career development opportunities. Competitive compensation and benefits in a mission-driven environment. Qualification : Bachelors degree in Computer Science, Engineering, or a related field
Lead / Senior Software Engineer
Ultraviolette Automotive
Job Title: Lead / Senior Software Engineer (Microservices, IoT, Kafka, AWS) Location: Bengaluru Experience Required: 5 8 years Industry: Automotive (Electric Mobility) Employment Type: Full-time About Ultraviolette Join the Charge. Create the Future. At Ultraviolette, we re not just building electric vehicles we re redefining what mobility looks and feels like. From creating India s fastest electric motorcycle to designing the world s most advanced electric scooter, we're on a mission to engineer machines that are not only sustainable, but also exhilarating. Our team is a diverse mix of engineers, designers, dreamers, and doers, united by a bold vision to accelerate the global shift to electric mobility. If you're driven by innovation and looking to work on cutting-edge products that challenge the status quo, this is your platform to make a real impact. About the Role We are looking for a Senior/Lead Software Engineer who thrives in designing scalable, high-performance, and cloud-native microservices. This role is ideal for someone with hands-on experience in Java, Spring Boot, Kafka, and AWS, and a keen interest in IoT and real-time data architectures. You will work at the intersection of hardware and software helping us bring our vision of connected, intelligent, and high-performance electric vehicles to life. Key Responsibilities Design, develop, and deploy Java-based microservices using Spring Boot and related technologies. Architect and implement event-driven systems using Apache Kafka for real-time IoT data streaming. Build and manage RESTful APIs with AWS API Gateway for seamless service integration. Leverage AWS services such as Lambda, DynamoDB, and MemoryDB to build scalable, serverless solutions. Collaborate with cross-functional teams front-end developers, DevOps, and product managers to convert ideas into working features. Optimize application performance, troubleshoot issues, and ensure high reliability and availability. Mentor junior engineers, conduct code reviews, and enforce clean coding practices. Contribute to architecture decisions, technical roadmaps, and innovation initiatives. Stay on top of emerging technologies in cloud, microservices, and IoT ecosystems. Participate actively in Agile processes including sprint planning, stand-ups, and retrospectives. Required Qualifications Bachelor s or Master s degree in Computer Science, Engineering, or a related field. 5+ years of hands-on experience in Java development, with a strong foundation in Spring Boot. Proven experience designing and deploying microservices architectures. Strong knowledge of Kafka and real-time streaming/data pipeline architectures. Hands-on experience with Docker and Kubernetes for containerization and orchestration. Understanding of IoT protocols such as MQTT or CoAP and device connectivity. Proficiency with version control (Git) and Agile methodologies. Excellent problem-solving, communication, and collaboration skills. Nice to Have AWS Certifications (e.g., AWS Certified Developer or Solutions Architect). Experience with additional languages like Python or Rust. Familiarity with NoSQL databases such as MongoDB or Cassandra. Knowledge of DevOps practices, CI/CD pipelines, and Infrastructure as Code (e.g., Terraform). Exposure to stream processing frameworks (e.g., Apache Flink, Spark Streaming). Experience in edge computing and distributed systems. Strong understanding of cloud security best practices. Work on India s most futuristic electric vehicles at the intersection of technology, performance, and sustainability. Be part of a mission-driven company shaping the future of mobility. Collaborate with some of the most talented engineers and designers in the industry. A fast-paced environment that celebrates innovation, learning, and growth. Qualification : Bachelors or Masters degree in Computer Science, Engineering, or a related field
Manager - Operations
Cowrks
Manager Operations Location: Bengaluru Employment Type: Full-Time About the Role: We are looking for an experienced and forward-thinking Manager Operations to lead campus operations for Brookfield Properties in Bengaluru. The ideal candidate will bring over 10 years of experience in hospitality or real estate operations, with a proven track record of managing cross-functional teams, optimizing performance, and delivering best-in-class tenant experiences. As the operational lead for one or more campuses, you will oversee end-to-end property management, facilities operations, P&L oversight, vendor management, and customer-facing services. You will be expected to align operations with the Converge vision and drive strategic initiatives that enhance the campus environment, boost revenue, and ensure exceptional service delivery. Key Responsibilities: 1. Strategic Operations & Financial Management Develop and manage annual operating budgets; monitor P&L performance for campus/city-level operations. Work cross-functionally with leasing, marketing, tech, training, and transformation teams to execute activations and campus amenities. Lead cost optimization initiatives through preventative maintenance, energy management, and vendor negotiations. Explore new revenue streams through managed services, retail leasing, and partnerships. Support leasing of office and retail spaces in alignment with business plans. 2. Campus Operations & Experience Ensure seamless day-to-day campus operations in line with Converge standards. Implement strategies to enhance lobby, F&B, retail, and event experiences. Create and track SLAs, management reports, vendor agreements, AMCs, and compliance documentation. Conduct regular audits, walkthroughs, and cross-functional reviews to maintain high service and aesthetic standards. Plan and execute campus activations, including one marquee event annually. Partner with the marketing team to ensure alignment on all events and space-on-hire activities. Collaborate with procurement to ensure timely PO processing and vendor payments. 3. Team Leadership & Collaboration Lead campus teams including operations, F&B, tenant engagement, and event staff. Develop KPIs and implement SOPs to drive efficiency, service excellence, and staff performance. Champion a culture of continuous improvement, learning, and innovation. Guide onboarding/training of new staff across operational functions. Collaborate with corporate, city leadership, and transformation teams on strategic execution. 4. Tenant & Community Experience Work closely with tenants to enhance employee engagement and create a vibrant campus culture. Lead execution of tenant engagement surveys and act on feedback for service improvements. Support onboarding of tenants and amenities onto the Converge app and drive digital engagement. Design operational procedures to ensure consistent, high-quality tenant experiences. Encourage team involvement in delivering excellent customer care and personalized service. 5. Compliance, Safety & Sustainability Ensure adherence to fire safety, health, environmental, and legal compliance standards. Lead internal and external audits across all operational areas. Champion sustainability practices and social responsibility initiatives across the campus. Oversee maintenance and service infrastructure to ensure optimal functionality and safety. Who You Are: A strategic thinker with strong commercial acumen and customer-centric approach. A self-starter with the ability to lead autonomously and make sound business decisions. A strong communicator and collaborator who can manage multiple stakeholders and priorities. Resilient under pressure, detail-oriented, and driven to deliver high-impact outcomes. Passionate about elevating workplace experiences and building thriving communities. Preferred Qualifications: Bachelor's degree (minimum 3 years); specialization in Hospitality, Property, Facilities Management, or Business is preferred. 10+ years of experience in operations leadership within hospitality, real estate, or facilities sectors. Demonstrated success in leading large cross-functional teams. Strong client relationship management and vendor negotiation skills. Proven experience in driving operational excellence and process improvements. This Role Offers: The opportunity to lead operations at premier business campuses. Cross-functional exposure to marketing, leasing, digital products, and customer engagement. A high-impact role in shaping workplace experiences and community life across Brookfield campuses. Apply now to be part of a growing, dynamic team redefining the future of workspaces. Qualification : Bachelor's degree specialization in Hospitality, Property, Facilities Management, or Business is preferred
Data Architect
Acqueon
Position Title: Data Architect Department: R&D Engineering Location: Bangalore Experience: 15+ Years Industry: SaaS / Conversational Engagement / Customer Experience Technology About Acqueon: Acqueon is a leading provider of conversational engagement software that enables customer-centric enterprises to proactively engage with their customers across voice, messaging, and email channels. By leveraging a powerful data platform, predictive models, and intelligent workflows, we help brands enhance customer experience, improve collections, and drive revenue growth. With over 200 global clients, Acqueon is at the forefront of AI-powered customer engagement. Role Overview: We are seeking a visionary and technically hands-on Data Architect to lead the development of enterprise-scale data platforms and engineering solutions. You will work closely with Product Owners, Engineering Leadership, and cross-functional teams to define and execute a strategic technology roadmap aligned with Acqueon s business goals. As a key member of our R&D team, you ll lead the design and development of highly scalable, low-latency, fault-tolerant data systems, while mentoring top-tier engineering talent and driving high-impact product features. Key Responsibilities: Architect & Lead: Design and lead development of scalable data architectures and solutions supporting real-time and batch processing, analytics, and enterprise applications. Strategic Ownership: Define and implement the data strategy, technology roadmap, and long-term architecture vision for Acqueon s platforms. Leadership: Manage and mentor a team of senior developers and engineers, fostering innovation, ownership, and delivery excellence. Cross-functional Collaboration: Work with Product, Sales, Engineering, and Customer teams to align on feature development and delivery strategy. Project Management: Oversee the end-to-end delivery of complex features, ensuring adherence to timelines, scalability, and quality standards. System Design: Review architecture and design for robustness, performance, and fault tolerance, including multi-region, high-availability setups. R&D Enablement: Collaborate with international R&D teams and align development efforts across global product initiatives. Innovation & Optimization: Drive architectural decisions, recommend performance improvements, and ensure best practices for enterprise-scale data solutions. Required Skills & Experience: Education: Bachelor s or Master s in Computer Science, IT, or related field. Experience: 15+ years in software development and data architecture, with leadership experience in managing engineering teams. Architecture Expertise: Proven experience in designing scalable, concurrent, distributed, and highly available data systems. Database Proficiency: Strong in SQL/NoSQL databases Experience with MS SQL, Aerospike, DynamoDB, Snowflake In-depth knowledge of micro-partitions, cluster keys, warehouse cloning, time travel in Snowflake Strong in writing and tuning complex stored procedures ETL & Pipelines: Experience in building ETL pipelines and integrating data from S3, Kinesis Streams, APIs Cloud & DevOps: Strong understanding of Docker, AWS, and cloud-native deployment architectures Setting up multi-region resilience, disaster recovery strategies Technologies: Elasticsearch, AWS data services, container orchestration Big Data & Analytics: Exposure to analytical processing and statistical modeling is a plus Leadership: Strong project management skills, stakeholder engagement, and team mentoring experience Preferred Qualifications: Background in customer engagement, VDI, Cybersecurity, or Secure Access technologies Previous experience working with distributed R&D and product teams Knowledge of Acqueon, Citrix, VMware, Omnissa platforms is a plus Certifications in AWS, Snowflake, or similar technologies are an advantage Soft Skills & Behavioral Traits: Strong verbal and written communication skills Strategic thinking with hands-on execution ability High accountability and ownership mindset Ability to work in a fast-paced, dynamic, startup-like environment Comfortable with ambiguity and context-switching Team player with the ability to lead by influence and collaboration Be a part of a fast-growing, AI-driven SaaS company disrupting the customer engagement space Work on cutting-edge technologies with global product teams Ownership of end-to-end solutions and ability to shape the data platform of the future A culture that promotes innovation, agility, and career growth
Associate ML Ops
Mpokket Financial Services Private Limited
Job Title: Associate ML Ops Location: Bangalore Department: Data Science Employment Type: Full-time Experience: 1 2 years Job Overview We are seeking a motivated and detail-oriented Associate ML Ops to join our Data Science team. In this role, you will be responsible for supporting the deployment, monitoring, and scaling of machine learning models in production environments. You ll collaborate closely with data scientists and engineers to build robust MLOps pipelines and ensure model reliability, scalability, and performance. If you are passionate about bringing machine learning models to life and have hands-on experience in productionizing ML systems, we d love to hear from you. Key Responsibilities Deploy and maintain machine learning models in production environments using best-in-class tools like Databricks and MLflow. Collaborate with data scientists to translate experimental models into scalable, production-ready systems. Monitor model performance, accuracy, and overall health through automated tools and custom strategies. Build and maintain RESTful APIs using Python frameworks such as Flask or Django to serve ML models. Write efficient and optimized SQL and NoSQL queries for data extraction and transformation. Apply software engineering best practices, including version control, testing, and documentation, to MLOps workflows. Work with Python libraries like Pandas, PySpark, scikit-learn, SQLAlchemy, and Requests. Troubleshoot issues related to model deployment, API performance, or data integration pipelines. Minimum Qualifications Bachelor s or Master s degree in Computer Science, Statistics, Econometrics, Operations Research, or a related technical field. 1 2 years of hands-on experience in solving analytical or machine learning problems in production settings. Must-Have Technical Skills Hands-on experience with Databricks and MLflow Proven expertise in deploying ML models in real-world applications Strong understanding of data structures, algorithms, OOP, and software engineering principles Experience building and maintaining REST APIs using Python Proficiency in SQL and NoSQL Excellent Python programming and debugging skills Familiarity with core Python libraries used in ML and data processing: Pandas, scikit-learn, PySpark, SQLAlchemy, etc. Nice-to-Have Skills Exposure to Kafka for streaming and batch data processing Familiarity with Git and CI/CD pipelines Experience with Python multiprocessing or worker/queue systems Understanding of event-driven or asynchronous programming models This is an exciting opportunity to work at the intersection of data science and engineering. You ll play a key role in productionizing cutting-edge models and ensuring they deliver real business impact. Qualification : Bachelors or Masters degree in Computer Science, Statistics, Econometrics, Operations Research, or a related technical field
ML Ops Engineer
Mpokket Financial Services Private Limited
Job Title: ML Ops Engineer Location: Bangalore Department: Data Science Employee Type: Full-time Experience Required: 3 5 years Position Overview We are seeking an experienced and motivated ML Ops Engineer to join our Data Science team. In this role, you will be responsible for deploying, monitoring, and maintaining machine learning models in production environments. You will work closely with data scientists, engineers, and product teams to ensure models are scalable, reliable, and aligned with business objectives. This role is ideal for professionals who are passionate about building robust ML pipelines and bringing machine learning solutions into real-world applications at scale. Key Responsibilities Deploy and manage machine learning models in production environments, ensuring scalability, reliability, and performance. Build and maintain MLOps pipelines using platforms like Databricks and MLflow. Monitor model performance, accuracy, and health; implement alerting and diagnostics as needed. Develop and maintain RESTful APIs using Python frameworks such as Flask or Django to serve ML models. Optimize data workflows and collaborate with engineering teams to improve model integration and performance. Design strategies for automated model retraining, deployment, and version control. Write clean, maintainable, and efficient code using Python, adhering to OOP principles and best practices. Write complex queries using SQL and work with NoSQL databases to support data pipelines and feature stores. Leverage Python libraries such as PySpark, Pandas, scikit-learn, SQLAlchemy, and Requests. Minimum Qualifications Bachelor s or Master s degree in Computer Science, Statistics, Econometrics, Operations Research, or a related technical field. 3 5 years of experience in building, deploying, and monitoring machine learning solutions in production. Must-Have Skills Experience with Databricks and MLflow for model training and deployment. Proven expertise in machine learning model deployment and monitoring in live environments. Strong programming skills in Python, with solid understanding of data structures, algorithms, and OOP concepts. Experience developing RESTful APIs using Flask or Django. Proficient in SQL and NoSQL database operations. Hands-on knowledge of libraries such as Pandas, PySpark, scikit-learn, SQLAlchemy, and Requests. Strong analytical, problem-solving, and debugging skills. Good-to-Have Skills Experience with Kafka streaming and batch processing. Familiarity with CI/CD pipelines and version control systems like Git. Understanding of Python multiprocessing, worker/queue systems, and asynchronous/event-driven programming. This is a unique opportunity to work at the intersection of machine learning and DevOps. You'll play a critical role in operationalizing AI models and making them a core part of our product offerings. If you enjoy building scalable systems and solving real-world ML engineering challenges, we d love to meet you. Qualification : Bachelors or Masters degree in Computer Science, Statistics, Econometrics, Operations Research, or a related technical field
Senior Associate Data Engineering L2
Publicis Sapient
Senior Associate Data Engineering L2 Location: Bengaluru, India Department: Engineering Data Employment Type: Full-Time About the Role As a Senior Associate Data Engineering (L2) at Publicis Sapient, you will lead technical solutions that drive digital transformation by building scalable, high-performance data platforms. You ll be responsible for translating business and technical requirements into modern, data-centric solutions using Big Data technologies, cloud services (Azure), and advanced data engineering practices. Key Responsibilities Design and implement data ingestion, integration, and transformation processes from multiple heterogeneous sources in both batch and real-time. Build scalable data platforms using Hadoop stack components such as HDFS, Kafka, Spark, Hive, NiFi, Oozie, Airflow, Flink, and Storm. Develop real-time analytics, aggregation, and search features to support various data-driven applications. Collaborate closely with cross-functional teams on data infrastructure, computation frameworks, and data visualization. Apply cloud-native principles and Azure services to build and deploy data pipelines. Ensure performance optimization and data pipeline tuning. Work with NoSQL and MPP platforms like MongoDB, Cassandra, Redshift, Azure SQL DW, HBase, BigQuery. Contribute to infrastructure, automation, and DevOps for data pipelines using CI/CD practices. Ensure data governance, lineage, and cataloging using tools like Collibra or Alation. Required Qualifications 6 8 years of professional experience in software/data engineering. Minimum of 3 years hands-on experience with Big Data technologies. Strong programming expertise in Java (preferred), Scala, or Python. Expertise in the Hadoop ecosystem and real-time stream processing tools (Kafka, Pulsar, Spark Streaming, etc.). Hands-on experience with Azure data services (e.g., Data Factory, Synapse, ADLS, Databricks). Experience working with modern ETL tools (Informatica, Talend, etc.) and traditional RDBMS platforms (Oracle, PostgreSQL, SQL Server, MySQL). Bachelor's degree in Computer Science, Engineering, or related field. Nice to Have Certifications in Azure Data Engineer, GCP Big Data, or related cloud specializations. Experience with distributed messaging frameworks (ActiveMQ, RabbitMQ, Solace). Familiarity with microservices architecture and search technologies (Elasticsearch). Performance tuning of distributed data processing systems. Exposure to data governance, security, and metadata management. Benefits and Culture at Publicis Sapient Gender-neutral workplace policies 18 paid holidays annually Generous parental leave + new parent transition support Flexible work arrangements Access to Employee Assistance Programs (wellness & well-being) A dynamic culture focused on learning, creativity, and collaboration Qualification : Bachelor's degree in Computer Science, Engineering, or related field.
Senior Data Engineer
Synechron
Position Title: Senior Data Engineer Databricks, PySpark, Cloud Platforms Location: Bengaluru Bellandur (GTP) Employment Type: Full-time Job Summary Synechron is looking for a Senior Data Engineer to join our advanced analytics team in Bengaluru. In this role, you will architect and build scalable, high-performance data pipelines that power data science, analytics, and business intelligence initiatives. You ll work with modern tools including Databricks, PySpark, and cloud data platforms, while collaborating across teams to ensure high-quality, secure, and efficient data solutions. Key Responsibilities Design, develop, and maintain large-scale, secure, and efficient data pipelines using Databricks, PySpark, and cloud-native tools. Partner with data scientists, analysts, and business stakeholders to translate requirements into robust data solutions. Integrate data from various structured, semi-structured, and streaming sources. Ensure high standards for data quality, performance optimization, security, and cost efficiency. Drive data pipeline automation, orchestration, and monitoring using tools like Airflow. Lead troubleshooting efforts, performance tuning, and enhancements of existing pipelines. Stay informed about emerging data technologies and recommend adoption where relevant. Technical Skills Core Expertise Programming: Python (expert), SQL (advanced), PySpark. Platforms: Databricks (clusters, notebooks, workflows), AWS/Azure/GCP. Data Orchestration: Apache Airflow (or similar). Data Warehousing: Snowflake (preferred), data modeling, ETL/ELT pipelines. Streaming: Kafka or other stream processing tools. DevOps: CI/CD (GitLab CI, Jenkins), version control (Git), containerization (Docker/Kubernetes preferred). Security: Familiarity with encryption, access controls, and compliance best practices. Experience 8+ years of experience in data engineering or related roles. Proven expertise in developing and deploying scalable data pipelines using Databricks, PySpark, and SQL. Hands-on experience with cloud platforms (AWS, Azure, or GCP). Strong background in data warehousing, especially with Snowflake. Exposure to real-time data processing and orchestration tools. Experience implementing CI/CD pipelines for data workflows is a plus. Daily Responsibilities Build and optimize data ingestion, transformation, and storage workflows. Collaborate with cross-functional teams to align data solutions with business objectives. Monitor, troubleshoot, and continuously improve pipeline performance. Conduct data quality checks, ensure governance and compliance standards. Contribute to technical documentation, code reviews, and team knowledge sharing. Qualifications Bachelor s or Master s degree in Computer Science, IT, or related field. Relevant certifications (e.g., Databricks Certified Data Engineer, AWS Certified Data Analytics) are preferred. Professional Competencies Strong problem-solving and analytical mindset. Effective communicator with ability to collaborate across technical and non-technical teams. Time management and prioritization skills under tight deadlines. Proactive leadership and a passion for innovation. Commitment to ethical data use and data security. Diversity & Inclusion at Synechron Synechron is committed to building an inclusive, diverse, and equitable workplace. Through our global Same Difference DEI initiative, we celebrate and support people from all backgrounds, including race, gender, sexual orientation, religion, age, disability, and more. We offer flexible work arrangements, continuous learning, internal mobility, and mentoring programs to support every employee s growth. Qualification : Bachelors or Masters degree in Computer Science, IT, or related field
Data Engineer
Cognite
Data Engineer Location: Bengaluru Department: Global Strategic Services Data Engineering EMEA Type: Full-Time | Hybrid About Cognite Cognite is a global SaaS company at the forefront of industrial digital transformation. Our core platforms Cognite Data Fusion (CDF) and Cognite Atlas AI leverage AI and data to solve complex challenges across oil & gas, chemicals, pharma, manufacturing, and energy sectors. Recognized globally for innovation, Cognite helps businesses improve efficiency, reduce costs, and operate more sustainably. Our Values Impact: We focus on outcomes that truly make a difference. Ownership: We take responsibility and work inclusively to deliver success. Relentless: We are persistent problem-solvers who view challenges as growth opportunities. About the Team You ll join Cognite s Global Customer Success (GCS) organization spanning offices in Oslo, Austin, Houston, Tokyo, and now Bengaluru. The GCS team includes Project Managers, Data Engineers, Data Scientists, and Solution Architects, all working on high-impact industrial use cases across regions including MENA and SEA. About the Role As a Data Engineer, you'll play a key role in solving real-world industrial problems by designing, building, and optimizing scalable data architectures. You ll create robust pipelines to integrate and contextualize data using Cognite Data Fusion, working closely with cross-functional teams. This is also a growth role with the opportunity to step into a Tech Lead position. Key Responsibilities Design and implement scalable data solutions using Cognite Data Fusion. Integrate data sources using Cognite connectors, SQL, Python/Java, and REST APIs. Develop custom data models for cleansing, discovery, and mapping. Work with data scientists, architects, and PMs to deliver client outcomes. Conduct code reviews and apply best practices in maintainability and quality. Support customers and partners with data engineering tasks and challenges. Collaborate with Engineering and Product teams to influence product roadmap. What You ll Bring Bachelor's or Master s in Computer Science or related field. 3 5 years of experience in data-focused, customer-facing roles. Strong experience with Python, SQL, and REST APIs for production-grade pipelines. Familiarity with data from industrial domains like oil & gas or manufacturing is a plus. Experience with Kubernetes, Azure/GCP, and modern DevOps practices. Comfortable with Git, CI/CD, and cloud-native deployment workflows. A proactive, collaborative mindset with a passion for problem-solving and learning. Join a diverse global team with 70+ nationalities and a strong commitment to DEI. Work in a modern hybrid environment at our Bengaluru office (Rathi Legacy, Hoodi). Be part of a flat, transparent culture with direct access to leadership. Build cutting-edge solutions and influence how industries evolve digitally. Learn from top engineers and contribute to ambitious, high-impact projects. Enjoy a respectful, fun, and collaborative workplace where your voice matters. If you re excited to build data infrastructure that powers real change in heavy industries, Cognite offers you a platform to grow, lead, and innovate. We welcome candidates from all backgrounds and identities apply today and be part of something meaningful. Qualification : Bachelor's or Masters in Computer Science or related field.
Senior Backend Engineer - Cognite Innovation Team
Cognite
Senior Backend Engineer Innovation Team Location: Bengaluru (Rathi Legacy, Rohan Tech Park, Hoodi) Team: Global Strategic Services Industry Innovation and Solutions Employment: Full-Time | Hybrid About Cognite Cognite is a global SaaS leader using AI and data to transform industries such as Oil & Gas, Chemicals, Pharma, and Manufacturing. Our award-winning platforms include Cognite Data Fusion and Cognite Atlas AI, driving industrial digital transformation worldwide. The Team & Role You ll join Cognite s Innovation Team working on batch processing applications for a key American emulsions industry client. This role focuses on backend development and data engineering to deliver scalable, robust solutions handling complex data workflows with multidisciplinary teams and global stakeholders. Key Responsibilities Architect and develop backend systems for complex batch data processing workflows with a focus on reliability amid inconsistent/incomplete data. Build resilient workflows handling multi-source data and minimizing system impact from component failures. Develop and optimize data models for domain-specific industrial problems. Create data transformation functions to extract actionable insights from raw data. Manage scalable distributed systems and ensure operational reliability. Collaborate cross-functionally to refine requirements and deliver solutions. Work with large datasets using SQL and NoSQL technologies. Troubleshoot full application lifecycle issues, from dev to production. Stay current on tech trends to innovate team practices. Strong backend development experience, distributed systems, and complex domain modeling. Proficiency in Python, SQL, and NoSQL databases. Experience with cloud-based architectures, automated testing, version control, CI/CD pipelines. Problem-solving skills, especially when data or context is incomplete. Self-driven, taking ownership from concept to deployment. Collaborative and excellent communicator, comfortable engaging global stakeholders. Bonus: knowledge of Data Science and Machine Learning. Diverse global team of 70+ nationalities with strong DEI focus. Modern Bengaluru office with hybrid work flexibility. Flat structure with direct leader access, minimal bureaucracy. Work on ambitious, impactful projects with leading tech experts. Active community engagement with partners and customers. Excited to build next-gen industrial data solutions with Cognite? Apply today and make an impact in a fast-growing, innovative SaaS company!
Engineering Manager- Platform Engineering
Meesho
Engineering Manager Platform Engineering Location: Bangalore, Karnataka | Department: Tech About the Team At Meesho, we support 5% of Indian households with high-scale e-commerce solutions and we do it with zero downtime. We value speed over perfection, embrace failures as learning opportunities, and empower teams with a Founder s Mindset. As part of the Platform Engineering team, you ll be building resilient, low-latency, high-throughput systems that serve millions of users daily. We invest in the growth of every engineer through continuous feedback, open communication, and a supportive culture. And yes we know how to party as hard as we code. About the Role We are looking for a skilled Engineering Manager Platform Engineering to lead a team responsible for designing, scaling, and optimizing our core infrastructure. This role involves managing large-scale distributed systems, fostering engineering excellence, and collaborating across teams to drive innovation. You ll ensure technical quality, delivery speed, and scalable architecture for all projects under your ownership. What You Will Do Design and allocate technical tasks while maintaining Meesho s engineering standards. Own execution of platform projects from inception to deployment, ensuring scalability and reliability. Conduct regular 1:1s, drive feedback cycles, and support career growth of engineers. Partner closely with Product and Design teams to develop new platform capabilities. Coach engineers on best practices for architecture, performance, and scalability. Monitor project health, sprint progress, and engineering KPIs. Foster a high-performing team culture with strong engineering ownership. What You Will Need Bachelor s or Master s degree in Computer Science or a related technical field. 8+ years of professional software development experience, including 1+ year in team management. Proven experience building large-scale distributed systems. Strong coding skills in Java, Python, or Go, and multithreading expertise. Deep understanding of messaging systems (Kafka, etc.), transactional and NoSQL databases. Experience working on cloud platforms like GCP or AWS. Exceptional communication, leadership, and stakeholder management skills. Good to have: Exposure to Elasticsearch, data pipelines, or stream processing systems. About Us Meesho is India s leading e-commerce platform built for the next billion users. With 1.75M+ sellers and a customer base spread across every serviceable pin code, we are democratizing internet commerce by enabling small businesses to sell online at zero commission and with the lowest logistics costs in the industry. From affordable products that reflect local demand to a robust pan-India tech infrastructure, Meesho is transforming how India shops and sells online. Our Culture & Total Rewards At Meesho, we believe in creating a culture of impact, inclusion, and innovation. Our values reflected in 11 guiding principles or "Mantras" shape how we work, collaborate, and grow together. Why You ll Love Working Here: Compensation: Competitive salary with equity-based rewards tailored to your experience and impact. Wellness: Extensive health insurance for you and your family through our MeeCare Program, mental wellness support, gym discounts, and more. Flexibility & Leave: Generous time off, parental benefits, and relocation support. Growth & Learning: Continuous learning through workshops, internal mobility, and performance coaching. Culture of Recognition: Personalized gifts, fun rituals, and regular engagement programs celebrating wins big and small. Join us to build the platform powering the future of digital commerce in India. Apply now and be part of a tech-first, people-driven journey at Meesho. Qualification : Bachelors or Masters degree in Computer Science or a related technical field.
Sr. Data Engineer
Trellissoft Engineering Services Pvt Ltd
Job Title: Data Engineer Location: Bengaluru, Karnataka Experience: 5 to 8 Years Work Modality: Full-time (Work from office) Job Description: We are looking for an experienced Data Engineer to join our team and take responsibility for designing, developing, and maintaining scalable ETL/ELT pipelines. This is a full-time position based in Bengaluru, Karnataka, and you will be collaborating with cross-functional teams to define data requirements and ensure data accuracy, consistency, and integrity. Your role will also involve optimizing data workflows, automating processes, and ensuring high availability and reliability of data pipelines. Key Responsibilities: ETL/ELT Pipeline Development: Design, develop, and maintain scalable ETL/ELT pipelines to support data transformation and integration processes. Data Warehouse & Data Lake Optimization: Build and optimize data warehouses, data lakes, and real-time streaming solutions to support large-scale data operations. Collaboration & Data Requirements: Collaborate with cross-functional teams, such as product, data science, and analytics teams, to define data requirements and ensure data accuracy and consistency. Database Structure & Schema Management: Develop and maintain database structures and schemas to ensure efficient data storage and retrieval. Data Workflow Optimization: Optimize data workflows for performance, reliability, and scalability, ensuring the highest level of efficiency. Data Security & Compliance: Implement data security, governance, and compliance best practices to ensure that data is handled securely and meets industry standards. Pipeline Monitoring & Troubleshooting: Monitor, troubleshoot, and improve data pipelines to ensure uptime, reliability, and smooth data processing. Process Automation: Automate data-related processes to improve efficiency and reduce manual intervention, increasing the overall speed of data flow. Required Qualifications: Experience: 5+ years of experience in data engineering or 3-4 years of experience as a Data Engineer. Technical Skills: Strong proficiency in SQL and database management systems such as PostgreSQL, MySQL, SQL Server, etc. Experience with ETL tools such as Pentaho, Talend, Cdata, and SSIS. Exposure to Python, Java, or Scala for data processing is a plus. Experience with big data technologies such as Apache Spark, Hadoop, or Kafka. Familiarity with cloud services (AWS, Azure) and data storage solutions such as S3, Redshift, Snowflake, or BigQuery. Strong knowledge of data modeling, warehousing concepts, and data architecture best practices. Soft Skills: Excellent communication skills with the ability to collaborate effectively across teams. Strong problem-solving skills and the ability to work with large, complex datasets. What We Offer: Competitive Salary: Attractive salary based on experience and expertise. Collaborative Work Environment: Work in a dynamic and fast-paced environment with a team that fosters innovation and collaboration. Growth Opportunities: Opportunities to enhance your skills and career growth in the data engineering field. Comprehensive Benefits: Benefits package designed to support work-life balance and overall employee well-being.
Data Scientist
Linarc
Job Title: Data Scientist Location: Bengaluru Experience: 4+ Years About Linarc: Linarc is revolutionizing the construction industry. As the emerging leader in construction technology, we are redefining how projects are planned, executed, and delivered. Built for general contractors, construction managers, and trade partners, Linarc offers a next-generation platform that delivers unmatched collaboration, automation, and real-time intelligence to construction projects. Our mission is to eliminate inefficiencies, streamline workflows, and drive profitability, empowering teams to deliver projects faster, smarter, and with greater control. Join us in shaping the future of construction tech. If you thrive in a dynamic, fast-growing environment and want to make a significant impact, Linarc is the place for you. This is a full-time position based in Bengaluru. Roles and Responsibilities: Data Analysis & Modeling: Develop and implement machine learning models to address complex business problems. Build predictive models and algorithms to forecast customer behavior, product usage, and business outcomes. Conduct statistical analysis and hypothesis testing to uncover key insights and trends. Data Engineering & Processing: Clean, preprocess, and transform large-scale datasets from multiple sources. Design and optimize data pipelines for both real-time and batch processing. Business Insights & Reporting: Collaborate with business teams to define key performance indicators (KPIs). Develop dashboards and reports to effectively communicate findings and business performance. Present actionable insights and recommendations to stakeholders and leadership teams. Product Improvement: Partner with product managers to integrate data-driven insights into product features. Monitor model performance and continuously improve accuracy and scalability. Run A/B tests to evaluate the impact of product changes and optimize user experience. Research & Development: Stay updated with the latest developments in machine learning, artificial intelligence, and data science. Explore and implement new techniques and tools to drive innovative solutions and improve outcomes. Qualifications: Minimum Experience: 3 years of professional experience in Data Analytics, Data Science, AI, or Machine Learning. Technical Expertise: Deep analytical expertise in applying statistical solutions to business problems. Experience with exploratory data analysis of large datasets using tools like SQL, Hive, Spark, or similar. Proficiency in machine learning and deep learning tools such as Scikit-learn, TensorFlow, PyTorch, and familiarity with techniques like Neural Networks, Gradient Boosting, Logistic Regression, Decision Trees, Random Forests, Support Vector Machines, Clustering, etc. Experience working with visualization tools such as Tableau and Power BI. Business & Innovation: Demonstrated ability to innovate solutions and solve business problems using data science techniques. Desirable Additional Experience: 4+ years of experience in data science or machine learning in a product-focused environment. Experience working with relational or document databases. Proficiency in C, Python, PyTorch, Java, JavaScript, Node.js, React.js, or Vue.js. Strong knowledge of hypothesis testing and regression analysis. Experience in a SaaS product environment. Educational Qualifications: Bachelor s/Master s degree in Computer Science, Data Science, Mathematics, Statistics, or a related field.
Data Engineer: Data Warehouse
International Business Machines Corporation
Job Title: Application Developer - ETL and Data Management Introduction: In this role, you ll work in one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we provide deep technical and industry expertise to a wide range of public and private sector clients globally. Our delivery centers leverage locally-based skills to help clients drive innovation and the adoption of new technologies. A career in IBM Consulting is built on long-term relationships and close collaboration with clients across the world. You will work with leaders across industries to improve the hybrid cloud and AI journeys for some of the most innovative and valuable companies worldwide. Your ability to make a meaningful impact for clients is enabled by our strategic partner ecosystem and robust technology platforms, including Software and Red Hat. Curiosity and a constant quest for knowledge are key to success in IBM Consulting. In your role, you ll be encouraged to challenge the norm, explore new ideas, and come up with creative solutions that result in groundbreaking impact for a wide network of clients. Our culture is built on evolution and empathy, focusing on long-term career growth and development in an environment that values your unique skills and experience. Your Role and Responsibilities: ETL Workflow Development: Develop and implement ETL workflows by creating ETL jobs, and data models in datamarts using technologies such as Snowflake, DBT, Unix, and SQL. Batch Processing Redesign: Redesign Control M Batch processing for ETL job builds to run efficiently in a production environment. System Evaluation and Improvement: Study the existing system to evaluate its effectiveness and design new systems to improve workflow efficiency. Business Program Analysis & Support: Perform requirements identification, business program analysis, testing, and system enhancements while providing production support. Agile Environment: Work effectively in an Agile environment and gain familiarity with tools such as JIRA and SharePoint. Client Interaction: Good written and verbal communication skills are essential as you will interact directly with client counterparts to understand requirements and provide solutions. Required Technical and Professional Expertise: Experience: A minimum of 3 years of experience in developing ETL applications, implementing workflows, and creating data models using Snowflake, DBT, Unix, and SQL technologies. Agile Environment: Strong understanding of working in an Agile environment and proficiency in tools like JIRA and SharePoint. Problem-Solving Skills: Ability to manage change and proven time management skills. Strong interpersonal skills to contribute effectively to team efforts. Continuous Learning: Stay up-to-date with technical knowledge by attending educational workshops and reviewing relevant publications. Preferred Technical and Professional Expertise: ETL Development: Experience in developing triggers, functions, and stored procedures to support ETL workflows. Impact Analysis: Assist with impact analysis of changing upstream processes on the Data Warehouse and reporting systems. ETL & Reporting Support: Participate in the design, testing, support, and debugging of new and existing ETL and reporting processes. Data Profiling & Troubleshooting: Perform data profiling and analysis using a variety of tools, troubleshoot and support production processes, and maintain documentation. Innovation: Be part of a team that drives global change and leverages cutting-edge technologies to solve complex problems. Growth: Gain access to continuous learning and career development opportunities to further your expertise in data management and cloud technologies. Collaboration: Work with a diverse team in a collaborative environment that values new ideas and creative solutions. Global Impact: Your work will contribute to improving business operations and technological advancements for clients around the world. If you're passionate about driving innovative solutions, working with a variety of clients, and continuously evolving your skills, IBM Consulting is the perfect place for you to advance your career.
Staff Data Engineer
Intuit
Intuit is a global leader in financial technology, dedicated to helping individuals and businesses thrive. Our suite of products, including TurboTax, Credit Karma, QuickBooks, and Mailchimp, serves approximately 100 million customers worldwide. At Intuit, we believe in providing everyone with the tools and resources they need to achieve financial success. We are constantly innovating to make financial empowerment a reality for all. Job Overview Join the Intuit Data Platform (IDP) team as a Staff Engineer and help us transform the way we handle big data! The IDP team is responsible for the Intuit Analytics Platform, which powers real-time data ingestion, cataloging, analytics, and machine learning across the entire organization. As Intuit s customer base grows, so does the volume of data we process. Our engineering excellence ensures that we can scale and leverage this data to drive machine learning and product innovations. We re in the process of building the next-generation real-time and batch ingestion engine, capable of indexing, cataloging, and organizing data and metadata. We are passionate about using open-source technologies to solve challenges and contributing back to the community. If you're excited about building a platform that will directly impact data scientists and analysts and have a desire to shape the future of data at Intuit, then come join us! Key Responsibilities Architect & Design: Build fault-tolerant and scalable big-data platforms using open-source technologies to handle massive datasets. Data Solutions: Create architecture solutions that address complex use cases like data normalization, lineage, governance, ontology, and discoverability. Cross-Team Collaboration: Work with analysts and data scientists to understand data requirements for building operational propensity models and gaining deep customer insights. Hands-On Coding: Lead development efforts within the Hadoop ecosystem using technologies such as Java MapReduce, Spark, Scala, HBase, and Hive to build and optimize data pipelines for both real-time and batch applications. Database Management: Work with NoSQL, SQL, and in-memory databases to design high-performance data systems. Code Reviews: Ensure code quality, consistency, and adherence to best practices through regular code reviews. Architectural Alignment: Ensure alignment between enterprise architecture and business requirements. Prove Feasibility: Conduct proof-of-concept (POC) experiments for new technologies or approaches and drive them to production. Collaboration with Data Cataloging Team: Work closely with data catalog teams and architects to index and catalog all data sources at Intuit. Agile Leadership: Lead fast-paced development teams using agile methodologies and promote best practices in software development, testing, and incident response. Design & Model: Build dimension models suited for customer business use cases and ensure seamless integration of business and technical requirements. Qualifications Experience: 12+ years of relevant experience, with at least 5+ years specializing in the big data domain. Big Data Architecture: Proven experience in architecting end-to-end ecosystems for big data and analytics platforms. Expert Knowledge: Deep expertise in building fault-tolerant, scalable big data solutions, especially using the Hadoop ecosystem (Hive, HBase, Spark, Kafka, MapReduce, etc.). Programming Expertise: Mastery of Java and Scala, with a focus on building high-throughput data services. Machine Learning: Knowledge of machine learning principles and AI applications in big data. Big-Data Technologies: Familiarity with tools such as HDFS, Storm, Zookeeper, Cassandra, Redshift, GraphDB, and others. Understanding both real-time and batch processing in the Hadoop ecosystem. Communication: Strong communication skills, with an ability to explain complex technical topics to both technical and non-technical audiences. Programming Skills: Intermediate experience in Python or R for data processing. Education: BE/BTech/MS in Computer Science or a related field (or equivalent experience). Collaboration: Demonstrated ability to work cross-functionally and lead change through influence and example. At Intuit, you ll be part of a talented, passionate team working on innovative solutions that shape the future of data analytics and machine learning. As a Staff Engineer, you ll have the chance to work with cutting-edge technologies, build scalable systems, and help revolutionize how Intuit leverages data to drive product innovation. If you're looking for a dynamic environment where you can have a meaningful impact, come join us at Intuit! Qualification : BE/BTech/MS in Computer Science (or equivalent)
Data Engineer
Indium Software
Data Engineer Role Overview We are looking for a Data Engineer to design, develop, and maintain scalable data pipelines and ETL processes. You will work closely with data scientists, analysts, and engineers to ensure efficient data processing and storage solutions that support business intelligence and analytics needs. This role requires expertise in SQL, big data technologies, and cloud platforms to build and optimize data workflows. Key Responsibilities Data Pipeline Development & Optimization Design and build scalable ETL/ELT processes for structured and unstructured data. Develop and maintain data ingestion frameworks to handle large datasets efficiently. Ensure data integrity, consistency, and security across multiple sources. Database & Data Warehouse Management Develop, optimize, and maintain relational and NoSQL databases. Implement data modeling best practices for performance and scalability. Work with cloud-based data warehouses (e.g., Snowflake, Redshift, BigQuery). Big Data & Cloud Technologies Leverage big data tools (e.g., Spark, Hadoop, Databricks) for data processing. Work with cloud platforms (e.g., AWS, Azure, GCP) to build and optimize data solutions. Develop real-time and batch processing workflows. Collaboration & Documentation Work closely with data scientists, engineers, and business teams to understand data requirements. Document data pipelines, architecture, and workflows for scalability and maintenance. Required Qualifications & Skills Technical Expertise: Strong experience with SQL and Python for data processing. Proficiency in ETL/ELT frameworks and data integration techniques. Hands-on experience with big data tools (e.g., Apache Spark, Hadoop, Kafka). Cloud & Database Management: Expertise in cloud data platforms (Azure, AWS, GCP). Experience with data warehousing solutions (Snowflake, Redshift, BigQuery). Understanding of data governance, security, and compliance. Performance Optimization & Troubleshooting: Ability to optimize SQL queries and improve data processing efficiency. Experience troubleshooting complex data pipeline issues. Apply Now & Be Part of an Innovative Data Team!
Senior Software Engineer - Backend
Nvidia
NVIDIA is searching for a highly motivated senior software engineer for the team that is building capabilities for a next generation Network management and Telemetry system in cloud using modern design principles at internet scale.The person will be will be responsible for building distributed cloud applications. It will be a highly scalable, modern network operations toolset that provides visibility, troubleshooting, validation and telemetry for Ethernet networks. What you'll be doing: Development of distributed cloud applications, micro services and SAAS platform with high throughput and reliability. Contribute to applications like data ingestion, distributed computing, near real time analytic engines, RESTful APIs and user interfaces. Drive requirement discussions, design and product improvements. Drive improvements in areas like performance, team productivity, automation, quality, monitoring and reliability of applications. Working closely with the system architects, UI/UX and test engineers What we need to see: Bachelors/Masters Degree in Computer Science/Engineering 5+ years of experience in complex microservices based architectures. Extensive programming experience in Scala, Go, Python Fluent in coding and rapid prototyping. Strong experience in developing, maintaining, and testing of scalable distributed applications. Experience with stream processing frameworks, such as Kafka,Flink , Spark Streaming, Samza etc. Background with NoSQL databases such as Cassandra, MongoDB. Experience with orchestration/scheduling technologies like Kubernetes, SLURM, Nomad etc Ways to stand out from the crowd: Experience with public clouds like AWS. Worked in Reactive application designs (https://www.reactivemanifesto.org/). Experience in network stacks, protocols, SDN. NVIDIA is widely considered to be one of the technology world s most desirable employers. We have some of the most forward-thinking and hardworking people on the planet working for us. If you're creative, passionate and self-motivated, we want to hear from you! NVIDIA is leading the way inground-breakingdevelopments in Artificial Intelligence, High-Performance Computing and Visualization. The GPU, our invention, serves as the visual cortex of modern computers and is at the heart of our products and services. Qualification : Bachelors/Masters Degree in Computer Science/Engineering
1 - 20 of 0 jobs
* No exact matches found. Showing closest results insteadNo results found
Modify search criteria or create an alert to get relevant jobs as soon as they’re posted