Data Engineer She1Wyc9Ec at Zscaler jobs

Job Title: Senior Generative AI & Data Engineer (Python) Remote | Chandigarh/Punjab Preferred Experience Required: 5+ Years Employment Type: Full-Time Work Days: 5 Days a Week Notice: Immediate Joiners Preferred About the Role We are seeking an experienced Senior Generative AI & Data Engineer with over 5 years of expertise in Python and enterprise-level AI/data engineering. This fully remote position is ideal for professionals based in Chandigarh or Punjab looking to contribute to cutting-edge AI projects while working from home. Key Responsibilities: Design and develop complex Large Language Model (LLM) layers and modules including flow creation, supervision, retrievers, and connection management. Build and optimize scalable data models, data pipelines, and translation layers for efficient data processing. Lead enterprise-grade Python development efforts with clean, maintainable code. Drive Generative AI engineering initiatives to innovate and enhance AI capabilities. Manage data engineering tasks using PostgreSQL, Polars, and orchestration tools such as Apache Airflow (DAGs). Oversee operational data workflows using key-value stores and queuing systems. Develop robust and scalable web services using FastAPI. Collaborate closely with cross-functional teams to deliver innovative engineering solutions aligned with business needs. Required Skills & Expertise: Strong proficiency in Python for enterprise applications. Hands-on experience in Generative AI engineering and LLM module development. Expertise in data engineering with PostgreSQL and Polars. Skilled in workflow orchestration using Airflow DAGs. Proficient in building web APIs with FastAPI. Experience managing key-value stores and message queuing systems for data operations. Work remotely while contributing to enterprise-level AI and data engineering projects. Collaborate with a dynamic, innovative team passionate about cutting-edge technology. Competitive salary and opportunities for professional growth. Flexible working environment supporting work-life balance.

Sr. Big Data Engineer Location: India The Impact You Will Have: You will work on a variety of impactful customer technical Big Data projects, which may include building reference architectures, how-to's and production-grade MVPs. Guide strategic customers as they implement transformational big data projects, 3rd party migrations, including end-to-end design, build and deployment of industry-leading big data and AI applications. Consult on architecture and design; bootstrap or implement strategic customer projects that lead to a customer's successful understanding, evaluation and adoption of Databricks. Work with Engineering and Databricks Customer Support to provide product and implementation feedback and to guide rapid resolution for engagement-specific product and support issues. What We Look For: 10+ years experience with Big Data Technologies such as Apache Spark , Kafka, Cloud Native and Data Lakes in a customer-facing post-sales, technical architecture or consulting role. 6+ years of experience working on Big Data Architectures independently. Strong experience working in the Databricks ecosystem. Comfortable writing code in either Python or Scala. Experience working across Cloud Platforms (GCP / AWS / Azure). Documentation and white-boarding skills. Build skills in technical areas which support the deployment and integration of Databricks-based solutions to complete customer projects. About Databricks: Databricks is the data and AI company. More than 10,000 organizations worldwide including Comcast, Cond Nast, Grammarly, and over 50% of the Fortune 500 rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics, and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark , Delta Lake, and MLflow. Benefits: At Databricks, we strive to provide comprehensive benefits and perks that meet the needs of all of our employees.

Working at Atlassian At Atlassian, we believe in empowering our employees with the flexibility to choose where they work whether in the office, from home, or a hybrid combination of both. This approach gives Atlassians greater control over their work-life balance, family priorities, and personal goals. We are a distributed-first company, hiring people across the globe where we have legal entities, with virtual interviews and onboarding processes in place. We are currently looking for a Senior Data Engineer to join our Go-To-Market Data Engineering (GTM-DE) team. This team is responsible for building and maintaining our data lake, ensuring our big data pipelines and services are optimized, and facilitating the movement of billions of messages each day. You'll work directly with business stakeholders, platform, and engineering teams to enable growth and retention strategies at Atlassian. We re looking for someone who is passionate about building scalable services and enjoys problem-solving in a fast-paced environment. What You ll Do: Help stakeholder teams ingest data into our data lake more efficiently. Improve the performance of our data pipelines, ensuring scalability. Innovate to create self-serve data engineering capabilities within the company. Design and build micro-services that support large-scale data ingestion and processing. Architect and enable self-serve data engineering services to help scale Atlassian's growth. Work with a cloud-based AWS data lake, leveraging open-source tools like Spark and Airflow. Apply your technical skills to build reliable and scalable services to support a multi-petabyte data lake. Work collaboratively in a fast-paced environment where creativity is key to success. What We Expect on Your First Day: A BS in Computer Science or equivalent professional experience. At least 5+ years of professional experience as a Sr. Software Engineer or Sr. Data Engineer. Strong proficiency in programming (preferably Python, Java, or Scala). Experience writing SQL, structuring data, and understanding data storage practices. Familiarity with data modeling and data warehousing concepts. Hands-on experience building data pipelines, platforms, microservices, and REST APIs. Familiarity with Spark, Hive, Airflow, and other streaming technologies to process large-scale streaming data. Experience working with Amazon Web Services (AWS), particularly EMR, Kinesis, RDS, S3, SQS, and similar services. Familiarity with Agile, TDD, and CI/CD development practices. A strong focus on data quality with experience using tools or frameworks to detect data issues and anomalies. A willingness to accept failure, learn from it, and continuously improve. Open-mindedness to explore unconventional solutions. Preferred Experience: Experience building self-service tooling and platforms. Experience designing and implementing Kappa architecture platforms. Experience with Databricks, including using their APIs to build pipelines. Contributions to open-source projects, such as creating Airflow operators. Perks & Benefits: Atlassian offers a wide range of perks and benefits to support you, your family, and your engagement with the community. Our offerings include: Comprehensive health coverage. Paid volunteer days to give back to your community. Wellness resources to support your mental and physical health. And much more to help you thrive both personally and professionally. This role offers you the opportunity to work on cutting-edge technologies, innovate at scale, and contribute to the overall growth and success of Atlassian. If you're passionate about data engineering, building scalable services, and working in a collaborative, fast-paced environment, we d love to hear from you! Qualification : A BS in Computer Science or equivalent professional experience.

Working at Atlassian At Atlassian, we empower our employees by giving them the flexibility to choose where they work whether in an office, from home, or through a combination of both. This approach helps Atlassians balance their work, family, and personal goals. We are a distributed-first company, and we can hire talented people from any country where we have a legal entity. All interviews and onboarding are conducted virtually to accommodate our global workforce. About the Role: Data Engineer At Atlassian, we are seeking a Data Engineer to join our Go-To-Market Data Engineering (GTM-DE) team. This team is responsible for building and maintaining our data lake, managing big data pipelines, and facilitating the movement of billions of messages every day. You will work directly with business stakeholders and collaborate with platform and engineering teams to support Atlassian s growth and retention strategies. We are looking for a structured thinker who is passionate about building scalable services and is ready to tackle the challenge of helping Atlassian scale its operations. If you enjoy solving complex problems and working in a fast-paced environment, this is the opportunity for you. What You ll Do: Help stakeholder teams efficiently ingest data into our data lake. Optimize data pipelines for better performance and scalability. Develop microservices, design architectures, and build self-serve capabilities at scale. Work with a cloud-based AWS data lake and leverage open-source technologies like Presto, Spark, Airflow, and Hive. Apply your technical expertise to manage and orchestrate multi-petabyte scale data lakes. Work with minimal legacy, spending more time on innovation and improving the platform experience. What We Expect on Your First Day: A BS in Computer Science or equivalent experience. At least 2 years of professional experience as a Software Engineer or Data Engineer. Strong programming skills in Python, Java, or Scala (a combination of these is preferred). Experience with data modeling, data warehousing, and SQL. Familiarity with modern software development practices like Agile, TDD, and CI/CD. Hands-on experience with Spark, Hive, Airflow, and other streaming technologies to process large-scale data. Experience with Amazon Web Services (AWS), particularly EMR, Kinesis, RDS, S3, and SQS. A growth mindset with a willingness to learn from failures and continuously improve. Open to exploring innovative solutions that may seem unconventional at first. Preferred Experience: Experience building self-service tooling and platforms. Experience designing and implementing Kappa architecture platforms. A passion for building and maintaining continuous integration pipelines. Experience with Databricks and its APIs. Contributions to open-source projects, such as building Airflow operators. Our Perks & Benefits: Atlassian offers a variety of perks and benefits to support your work, family, and community engagement. Some of the benefits we provide include: Health coverage to keep you well. Paid volunteer days to give back to your community. Wellness resources to support your physical and mental health. Plus many more offerings to help you thrive in both your personal and professional life. At Atlassian, you will be part of an innovative team where you can develop and grow your skills while making a direct impact on the company's growth and success. If you're ready to tackle complex data challenges in a fast-paced environment, we want to hear from you! Qualification : A BS in Computer Science or equivalent experience.

Position Title: Senior Data Engineer, Products and Platform Data Engineering About the Team: Atlassian's Data Engineering team is a highly skilled group dedicated to building world-class data solutions and applications that power crucial business decisions throughout the organization. We manage and maintain multiple analytical data models and pipelines across Atlassian, providing a unified way of analyzing products, customers, operations, and their interconnections. We re the backbone of the data-driven culture at Atlassian, supporting teams across Finance, Growth, Product Analysis, Customer Insights, Sales, and Marketing. We are looking for a Senior Data Engineer to join our Products and Platform Data Engineering team. This role reports to the Data Engineering Manager, based in Sydney. In this role, you will help build the infrastructure that enables our data solutions to grow and scale, making sure that business stakeholders have access to accurate and actionable insights. You ll partner closely with Data Analytics, Data Scientists, and various stakeholders to deliver high-impact data solutions across Atlassian's ecosystem. What You ll Do: Collaborate with Data and Business Teams: Partner with Product Analytics, Data Scientists, and other stakeholders to understand their needs and architect/build data solutions that provide valuable insights and drive business decisions. Design and Build Data Models and Pipelines: Develop scalable data models and build data acquisition/ingestion processes that can handle large volumes of data, ensuring they meet the product and business requirements. Develop Backend Systems: Build and maintain backend data systems that help Atlassian scale its data infrastructure, enabling seamless flow and access to critical data. Lead End-to-End Data Solutions: Take full ownership of building, scaling, and improving data pipelines and processes across Atlassian. You'll handle requirements from vague concepts, iterating to refine solutions rapidly. Improve Data Quality: Drive initiatives to improve the data quality within the team by using and enhancing internal tools and frameworks that automatically detect and resolve data quality issues. Mentor and Support Team Members: Help foster a culture of learning by mentoring junior engineers and ensuring best practices are maintained across the team. What We re Looking For: BS in Computer Science or equivalent experience. 8+ years of experience as a Senior Data Engineer or in a similar role, with a deep understanding of data engineering processes and infrastructure. Strong programming skills in Python (other languages such as Java or Scala are a plus) and experience with data-centric programming. Proficiency in SQL and experience designing and optimizing relational databases for large-scale data storage and retrieval. Experience with Spark (SparkSQL) and other tools in the big data ecosystem to build scalable data pipelines (e.g., Apache Kafka, Hive, Flink, etc.). Familiarity with AWS (Amazon Web Services) or similar cloud platforms and experience with services like EMR, S3, Kinesis, and RDS. Experience with data modeling and designing efficient storage solutions for both structured and unstructured data. Hands-on experience with scheduling frameworks like Airflow or equivalent tools to orchestrate workflows. Strong experience with Agile methodologies, including TDD, CICD, and continuous iteration for improvement. Problem-solving mindset: Ability to turn vague requirements into concrete solutions and handle complex challenges with creative, scalable solutions. Nice-to-Have Skills: Experience building self-service data platforms for internal teams to access and work with data independently. Experience with real-time data processing using technologies like Apache Kafka or Flink. Familiarity with Databricks and their APIs. Knowledge of monitoring and observability tools for ensuring high data quality and reliable data systems. At Atlassian, we believe in delivering high-quality solutions through collaboration, innovation, and a focus on customer success. This is a fantastic opportunity to join a fast-growing company and help build the data systems that power key business decisions across the organization. You will have the chance to work with cutting-edge technology and shape Atlassian s data-driven culture. If you're an experienced Data Engineer with a passion for building scalable data solutions and working with teams to drive business value, we d love to hear from you! Qualification : BS in Computer Science or equivalent experience.

Key Accountabilities: Using Microsoft Azure data PaaS services, design, build, modify, and support data pipelines leveraging DataBricks and PowerBI in a medallion architecture setting. If necessary, create prototypes to validate proposed ideas and solicit input from stakeholders. Excellent grasp of and expertise with test-driven development and continuous integration processes. Analysis and Design Converts high-level design to low-level design and implements it. Collaborate with Team Leads to define/clarify business requirements, estimate development costs, and finalize work plans. Run unit and integration tests on all created code Create and run unit and integration tests throughout the development lifecycle. Benchmark application code proactively to prevent performance and scalability concerns. Collaborate with the Quality Assurance Team on issue reporting, resolution, and change management. Support and Troubleshooting Assist the Operations Team with any environmental issues that arise during application deployment in the Development, QA, Staging, and Production environments. Assist other teams in resolving issues that may develop as a result of applications or the integration of multiple components. Knowledge and Experience: Understanding of design concepts and architectural basics. Knowledge of performance engineering. Understanding of quality processes and estimate methods. Fundamental grasp of the project domain. The ability to transform functional and nonfunctional needs into system requirements. The ability to develop and code complicated applications is required. The ability to create test cases and scenarios based on specifications. Solid knowledge of SDLC and agile techniques. Knowledge of current technology and trends. Logical thinking and problem-solving abilities, as well as the capacity to collaborate. Primary skills: Cloud Platform, Azure, Databricks, ADF, ADO. Sought: SQL, Python, PowerBI. General Knowledge: PowerApps, Java. 6-10 years of experience in software development with minimum 2 years of cloud computing. Education: Bachelor of Science in Computer Science, Engineering, or related technical field. Qualification : Bachelor of Science in Computer Science, Engineering, or related technical field.

Job Description Location: Hyderabad, Telangana, India Job Type: Permanent (full-time) Experience: 6+ years No. of Openings: 2 Key Responsibilities: Tech Stack Expertise: Proficiency in Python, Pyspark, and Databricks, utilizing these tools to develop robust data pipelines, perform data transformations, and optimize data workflows within the Supply Chain realm. Innovative Problem-Solving: Demonstrate technical expertise by thinking creatively and providing innovative solutions to complex data-related challenges, pushing boundaries in Supply Chain data analytics. Communication Skills: Exceptional communication abilities to effectively convey technical concepts, collaborate with cross-functional teams, and articulate solutions to non-technical stakeholders. Autonomous Work: Ability to work independently and proactively, taking ownership of projects, and delivering high-quality solutions with minimal supervision. Guidance and Best Practices: Lead the team by setting coding standards, advocating for reusable and modularized code, implementing design patterns, and guiding on best practices in data engineering. Perks and Benefits: Strong expertise in Python, Pyspark, and Databricks, with hands-on experience in developing and optimizing data pipelines for complex data transformations. Exceptional problem-solving skills, with a demonstrated ability to think creatively and develop innovative solutions for data-related issues. Excellent communication and interpersonal skills, with the ability to collaborate effectively with diverse teams and stakeholders. Self-driven and autonomous work style, capable of taking ownership of projects and delivering high-quality results independently. Experience in guiding and mentoring teams on coding standards, design patterns, and best practices in data engineering.

Data Engineer Location: Noida Experience: 2 5 Years Employment Type: Full Time About Times Internet Times Internet is India s largest digital products company, building premium digital platforms across diverse sectors like News, Sports, Fintech, and Enterprise Solutions. Our brands such as TOI, ET, Cricbuzz, Times Prime, Willow TV, and Whatshot reach millions and are designed to simplify and enrich everyday experiences. We take pride in fostering an inclusive, innovative workplace. As an equal opportunity employer, we ve maintained gender pay parity (independently verified by Aon Hewitt) and are committed to building a diverse and equitable team. About the Business Unit ET B2B The Economic Times B2B Verticals (ETB2B) is a leading business media platform, delivering insightful content, newsletters, and premium conferences across 23+ sectors including Auto, Energy, Pharma, Retail, and HR. With a monthly reach of over 8 million professionals, ETB2B powers business decision-making and industry learning through platforms like ET Masterclass, vConfex, and the ET B2B Intent Signal Platform a cutting-edge initiative leveraging behavioral data, enrichment tools, and event-driven insights. Role Overview As a Data Engineer, you will work on building scalable data pipelines, deploying machine learning models, and extracting actionable buyer intent signals using a variety of data sources and advanced ML/NLP techniques. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines to process large-scale structured and unstructured data (web behavior, CRM, third-party APIs, etc.). Build predictive models to extract buyer intent signals, leveraging NLP, behavioral analytics, and pattern recognition. Use modern ML and NLP tools: spaCy, Transformers, Scikit-learn, TensorFlow, PyTorch, and OpenAI APIs. Collaborate with product managers, backend engineers, and data scientists to deploy models into production and optimize performance. Ensure data pipeline reliability, compliance (GDPR, SOC 2), and freshness through robust monitoring and alerting systems. Enhance pipeline and model inference efficiency for near real-time scoring applications. Document workflows, model performance, and pipeline architecture for internal transparency and reproducibility. Required Skills & Qualifications Strong experience in Python for data engineering, ML pipelines, and model integration. Proficiency with ETL tools (Airflow, Prefect), data processing frameworks (Pandas, Dask), and workflow orchestration. Knowledge of machine learning and NLP techniques (entity extraction, sentiment analysis, text classification). Familiarity with ML model deployment via Docker, Kubernetes, and cloud ML services (AWS SageMaker, GCP AI Platform, Azure ML). Experience with relational databases (PostgreSQL, MySQL) and cloud data warehouses (BigQuery, Redshift, Snowflake). Strong grasp of version control (Git), CI/CD pipelines, and agile collaboration tools. Ability to interpret, visualize, and communicate model insights effectively working alongside analysts or through dashboards. Education & Experience Bachelor s or Master s degree in Computer Science, Data Science, Statistics, or a related technical field. 2 5 years of hands-on experience in data engineering, applied ML, or software development, ideally in SaaS, Martech, or B2B analytics domains. Be part of India s largest digital ecosystem. Work on impactful, high-scale products with real-world value. Collaborate with top minds in data, AI, and product development. Enjoy a culture of innovation, inclusion, and continuous growth. Qualification : Bachelors or Masters degree in Computer Science, Data Science, Statistics, or a related technical field

Job Summary: We are seeking a skilled Data Engineer with 2-5 years of industry experience to join our dynamic team. The ideal candidate will have a strong background in data engineering and a keen interest in the drug safety or pharmacovigilance domain. You will be responsible for designing, developing, and maintaining data pipelines and infrastructure to support our data-driven initiatives. Key Responsibilities: Design, develop, and maintain scalable data pipelines and ETL processes. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions. Ensure data quality and integrity through rigorous testing and validation. Optimize and tune data processing systems for performance and scalability. Implement data governance and security best practices. Work with large datasets from various sources, including clinical trials, adverse event reports, and other pharmacovigilance data. Develop and maintain documentation for data engineering processes and systems. Qualifications: Bachelor s degree in computer science, Engineering, or a related field. 2-5 years of experience in data engineering or a related role. Proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL, Oracle RDBMS). Experience with big data technologies (e.g., Hadoop, Spark). Familiarity with data warehousing solutions (e.g., Redshift, Big Query). Strong programming skills in Python, Java, or Scala. Experience with cloud platforms (e.g., AWS, Azure, GCP). Knowledge of data modeling, ETL processes, and data integration. Experience in the drug safety or pharmacovigilance domain is a plus. Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills. Qualification : Bachelors degree in computer science, Engineering, or a related field.

Job Title: Data Engineer Location: Kolkata Employment Type: Full-time Experience: 3+ years Salary: INR 10 - 15 LPA Notice Period: 30 Days Role & Responsibilities: Participate in technical and functional design discussions for sprints, modules, or projects Design and implement end-to-end data solutions involving storage, integration, processing, and visualization using Azure Ingest data from multiple sources such as SQL Server, Oracle, Excel, SQL Azure, etc., into Azure Data Factory and Azure Data Lake Storage (ADLS) Extract data from one database and load it into another, ensuring data integrity and performance Build scalable data architecture for ingestion, processing, and surfacing large-scale data for applications Utilize various scripting languages to integrate systems effectively, understanding their nuances Research and implement innovative methods to acquire data and leverage existing data for new applications Collaborate with data architects, analysts, and scientists to prepare datasets for analysis and interpretation Perform statistical analysis and fine-tune data processes based on test results Develop and extend libraries and frameworks to support data workflows Create design documents and provide technical solutions aligned with business requirements Preferred Candidate Profile: Strong understanding of database management, OLAP, and ETL frameworks Minimum 3 years experience working with Azure, Azure Data Factory, and .NET technologies Proficient in creating manual and auto-trigger pipelines in Azure Data Factory Experience in managing ETL jobs in Azure Synapse Analytics, including creating, updating, editing, and deleting jobs Ability to replicate existing application logic within Azure Data Lake, Data Factory, SQL Database, and SQL Data Warehouse environments Solid knowledge of SQL queries, SQL Server Reporting Services (SSRS), and SQL Server Integration Services (SSIS) Proven initiative and innovative mindset Strong analytical and problem-solving skills

Job Title: Data Engineer Location: Chennai (Work from Office) Job Type: Full-time Experience: 2 to 4 years Job Description Blackstraw Technologies Pvt Ltd is seeking a skilled Data Engineer to join our dynamic team. You will play a crucial role in transforming raw data into actionable insights by designing and maintaining robust data pipelines and systems. Your work will ensure alignment between data infrastructure and business goals, helping to drive efficiency and scalability. You will leverage your analytical skills and proficiency in various programming languages to integrate, process, and optimize data from diverse sources. You will also collaborate closely with data scientists and end-users to support data-driven decision-making. Key Responsibilities Design, build, and integrate data pipelines from multiple sources for analytics and operational purposes. Perform data integration and troubleshoot production and development issues. Develop, automate, and optimize ETL pipelines and workflows. Work with big data technologies including Hive, Spark, and Hadoop; optimize performance for large-scale data processing. Write and maintain efficient Spark/PySpark or Spark/Scala programs. Utilize strong SQL skills for querying and database design. Develop shell scripts (Unix/Linux) to automate data processing tasks. Collaborate with data scientists and various business teams to understand and fulfill data needs. Maintain and deploy ETL code following Agile methodologies. Handle service delivery tickets and provide support related to ETL tools. Participate in root cause analysis and problem resolution for data pipeline issues. Required Skills & Qualifications Bachelor s degree in Computer Science, Information Technology, or related field; Master s degree is a plus. 2 to 4 years of hands-on experience in data engineering or related roles. Strong expertise in ETL development and data integration tools. Proficiency in Spark (PySpark or Scala), Hive, and Hadoop ecosystem. Advanced SQL skills and experience with relational database management systems (RDBMS). Experience with shell scripting in Unix/Linux environments. Familiarity with Java, Python, or Scala programming languages is desirable. Knowledge of Cloud platforms (Azure preferred) is a plus. In-depth understanding of Retail domain data and business processes is advantageous. Excellent analytical, organizational, and communication skills. Experience with Agile project management and teamwork. About Blackstraw Technologies Pvt Ltd Founded in 2015 and fully operational since 2018, Blackstraw Technologies is a global software products and services company specializing in Artificial Intelligence (AI) and Machine Learning (ML) solutions. Our AI platform accelerates data labeling, model training, and deployment across cloud and on-premise environments. Our 250+ member team boasts over 900 years of combined experience, focusing on areas such as computer vision, natural language processing, and deep learning annotation tools. We enable enterprises worldwide to rapidly adopt AI in a cost-effective, plug-and-play manner. Headquartered in Florida, USA, Blackstraw has operations across Canada, the USA, and India. Qualification : Bachelors degree in Computer Science, Information Technology, or related field. Masters degree is a plus.

Data Engineer Location: Bangalore Company: Capital One India About Capital One At Capital One, we're redefining how technology solves real-world financial challenges. As a technology-driven company, we bring together talented engineers, data scientists, and designers to innovate at scale and deliver meaningful impact to millions of customers. If you're passionate about building powerful data solutions, exploring cutting-edge technologies, and working in a collaborative, fast-paced environment this is the place for you. About the Role As a Data Engineer at Capital One, you ll join a team of innovators who design and build next-generation data platforms and pipelines that power real-time decision-making. You ll collaborate across disciplines engineering, product, machine learning, and cloud infrastructure to transform how we leverage data at scale. What You ll Do Collaborate across Agile teams to design, develop, test, and deploy data-driven solutions. Build and support scalable data pipelines using modern data engineering tools and cloud services. Work on real-time and batch data processing systems that integrate with distributed microservices and ML platforms. Use programming languages such as Python, Java, or Scala with SQL, NoSQL, and cloud data warehouses like Redshift or Snowflake. Contribute to code reviews, unit testing, and performance optimization to ensure high-quality data systems. Partner with product managers and platform teams to deliver robust, cloud-native data solutions that power business decisions. Stay ahead of tech trends, share knowledge, and mentor junior engineers. Basic Qualifications Bachelor s degree in Computer Science, Engineering, or a related field. 1.5+ years of hands-on experience in application or data engineering (excluding internships). At least 1 year of experience working with big data technologies. Preferred Qualifications 3+ years of application/data engineering experience using Python, Scala, Java, or SQL. 1+ year of experience with cloud platforms (AWS, Azure, or GCP). 2+ years of experience with distributed computing tools (Spark, Hadoop, Hive, EMR, Kafka, etc.). 1+ year working on real-time streaming applications. 1+ year of experience with NoSQL databases (MongoDB, Cassandra). 1+ year of experience with data warehousing (Redshift, Snowflake). 2+ years working with Linux/Unix systems and shell scripting. Familiarity with Agile methodologies and modern DevOps practices. Why Join Capital One Work on high-impact data solutions at one of the world s most innovative financial institutions. Be part of a collaborative tech culture that values experimentation and learning. Access to top-tier tools, mentorship, and career development opportunities. Competitive compensation and benefits in a mission-driven environment. Qualification : Bachelors degree in Computer Science, Engineering, or a related field

Azure Data Engineer IBM Consulting Client Innovation Center Location: Bengaluru, India Experience: 7+ Years (Minimum 4+ Years in Azure Technologies) Job Type: Full-Time Education Required: Bachelor s Degree (Master s preferred) Introduction: Join IBM Consulting at our Client Innovation Center in Bengaluru, where we deliver deep technical and industry expertise to a wide range of public and private sector clients. Our delivery centers focus on innovation, agility, and adoption of next-gen technologies to transform businesses locally and globally. Your Role & Responsibilities: Design and develop scalable data engineering solutions on Microsoft Azure. Build, optimize, and manage data pipelines and ETL workflows using tools like Azure Data Factory, ADLS, and Databricks. Write clean, efficient code using PySpark, PL/SQL, and Spark SQL. Integrate and manage data from various sources, both structured and unstructured. Work with SQL, Postgres, Cassandra, and Cosmos DB. Utilize Azure services such as Stream Analytics, SQL DW, Azure Functions, ARM Templates, and Analysis Services. Implement and maintain serverless architectures for modern data solutions. Ensure high-quality data delivery, security, and compliance across systems. Collaborate across teams and contribute to solution design, code reviews, and documentation. Required Skills & Experience: Bachelor s degree in Computer Science, Information Systems, or related technical field. 7+ years total experience in Data Engineering, with 4+ years in Azure-based data projects. Strong hands-on experience with: Azure Data Factory, Data Lake (ADLS), Databricks Programming: Python, PySpark, PL/SQL, Spark SQL Databases: SQL, Postgres, Cassandra, Cosmos DB Solid grasp of data warehousing, relational databases, and cloud-based architectures. Familiarity with version control tools like Git and CI/CD pipelines. Preferred Skills & Qualifications: Master s Degree in a relevant field. Experience with: ARM Templates, Azure Functions, Serverless Architectures Object-oriented scripting languages (Python, Scala, etc.) Excellent problem-solving and communication skills. Ability to work in a fast-paced environment and collaborate effectively with teams and stakeholders. What You ll Get: Work on high-impact, global projects with cutting-edge Azure technologies. Be part of a collaborative and forward-thinking IBM team. Opportunities for professional growth, upskilling, and certifications. A dynamic work culture focused on innovation, agility, and client success. Join us in Bengaluru and be part of IBM s journey in shaping the future of data engineering. Qualification : Bachelors degree in Computer Science, Information Systems, or related technical field.

Data Engineer Location: Bengaluru Department: Global Strategic Services Data Engineering EMEA Type: Full-Time | Hybrid About Cognite Cognite is a global SaaS company at the forefront of industrial digital transformation. Our core platforms Cognite Data Fusion (CDF) and Cognite Atlas AI leverage AI and data to solve complex challenges across oil & gas, chemicals, pharma, manufacturing, and energy sectors. Recognized globally for innovation, Cognite helps businesses improve efficiency, reduce costs, and operate more sustainably. Our Values Impact: We focus on outcomes that truly make a difference. Ownership: We take responsibility and work inclusively to deliver success. Relentless: We are persistent problem-solvers who view challenges as growth opportunities. About the Team You ll join Cognite s Global Customer Success (GCS) organization spanning offices in Oslo, Austin, Houston, Tokyo, and now Bengaluru. The GCS team includes Project Managers, Data Engineers, Data Scientists, and Solution Architects, all working on high-impact industrial use cases across regions including MENA and SEA. About the Role As a Data Engineer, you'll play a key role in solving real-world industrial problems by designing, building, and optimizing scalable data architectures. You ll create robust pipelines to integrate and contextualize data using Cognite Data Fusion, working closely with cross-functional teams. This is also a growth role with the opportunity to step into a Tech Lead position. Key Responsibilities Design and implement scalable data solutions using Cognite Data Fusion. Integrate data sources using Cognite connectors, SQL, Python/Java, and REST APIs. Develop custom data models for cleansing, discovery, and mapping. Work with data scientists, architects, and PMs to deliver client outcomes. Conduct code reviews and apply best practices in maintainability and quality. Support customers and partners with data engineering tasks and challenges. Collaborate with Engineering and Product teams to influence product roadmap. What You ll Bring Bachelor's or Master s in Computer Science or related field. 3 5 years of experience in data-focused, customer-facing roles. Strong experience with Python, SQL, and REST APIs for production-grade pipelines. Familiarity with data from industrial domains like oil & gas or manufacturing is a plus. Experience with Kubernetes, Azure/GCP, and modern DevOps practices. Comfortable with Git, CI/CD, and cloud-native deployment workflows. A proactive, collaborative mindset with a passion for problem-solving and learning. Join a diverse global team with 70+ nationalities and a strong commitment to DEI. Work in a modern hybrid environment at our Bengaluru office (Rathi Legacy, Hoodi). Be part of a flat, transparent culture with direct access to leadership. Build cutting-edge solutions and influence how industries evolve digitally. Learn from top engineers and contribute to ambitious, high-impact projects. Enjoy a respectful, fun, and collaborative workplace where your voice matters. If you re excited to build data infrastructure that powers real change in heavy industries, Cognite offers you a platform to grow, lead, and innovate. We welcome candidates from all backgrounds and identities apply today and be part of something meaningful. Qualification : Bachelor's or Masters in Computer Science or related field.

Data Engineer Experience: 5+ Years Location: Bangalore, Karnataka, India Job Type: Full Time Job Summary We are looking for a talented Data Engineer with over 5 years of experience and a strong foundation in machine learning development to join our team in Bangalore. The ideal candidate will have hands-on expertise in Python programming, machine learning basics, and computer vision techniques like custom object detection and OCR. Key Responsibilities Develop and maintain data pipelines supporting machine learning models and data analytics. Implement custom object detection algorithms and OCR solutions using computer vision techniques. Utilize Python ML libraries such as OpenCV, SciPy, NumPy, Matplotlib, Pandas, Scikit-learn, Keras, PyTorch, and TensorFlow. Collaborate with data scientists and software engineers to optimize data workflows and ML model deployment. Ensure data quality, integrity, and scalability within data infrastructure. Troubleshoot and improve existing machine learning systems and pipelines. Required Skills Minimum of 5 years of experience in data engineering or related roles. At least 2 years of hands-on experience as a Machine Learning developer. Strong programming skills in Python. Solid understanding of machine learning fundamentals. Practical experience with custom object detection and OCR applications. Proficiency in key Python libraries for machine learning and data processing. Ability to work collaboratively in cross-functional teams. Qualifications Any graduate degree from a recognized university. Work with cutting-edge machine learning technologies in a supportive, innovative environment in Bangalore. Grow your career by solving complex problems and building impactful data solutions with a passionate team. Qualification : Any graduate degree from a recognized university.

Job/Position Summary: Data Engineer Responsibilities: Implement data pipelines that meet design and are efficient, scalable, and maintainable. Implement best practices including proper use of source control, participation in code reviews, data validation and testing. Timely deliveries while working on projects. Act as advisor/mentor and helps junior data engineers in their deliverables. Must Have Skills: Should have experience of at least 4+ years with Data Engineering. Strong experience of design, implementation and fine-tuning big data processing pipelines in production environment. Experience with big tools like Hadoop, Spark, Kafka, Hive, Databricks. Experience in programming at least one of with Python, Java, Scala, Shell Script. Experience with relational SQL and NO SQL databases like PostgresSQL, MYSQL, Cassandra etc. Experience with any data visualization tool (Plotly, Tableau, Power BI, Google Data Studio, Quick sight etc.). Good To Have Skills: Should have Basic Knowledge of CI/CD Pipeline. Experience in working on at least one Cloud (AWS or Azure or GCP). For AWS: - Experience with AWS Cloud services like EC2, S3, EMR, RDS, Athena, Glue, Lambda, EMR. For Azure: -Experience with Azure Cloud services like Azure Blob/Data Lake GEN2, Delta Lake, Databricks, Azure SQL, Azure DevOps, Azure Data Factory, Power BI. For GCP: - Experience with GCP Cloud services Big Query, Cloud Storage bucket, DataProc, Dataflow, Pub Sub, Cloud Function, Data Studio. Sound familiarity in Versioning tools (Git, SVN etc.). Experience Mentoring students is desirable. Knowledge of latest developments in Machine Learning, Deep Learning, Optimization in Automotive domain. Open minded approach to explore multiple algorithms to design optimal solution. History of contribution to articles/blogs/whitepapers etc. in Analytics. History of contribution to Open Source. Requirement: ESSENTIAL SKILLS /COMPETENCIES Data Engineering Hadoop Kafka CI/CD Cloud

Job Title: Data Engineer Location: Bengaluru (Hybrid) About Couchbase: As industries race to embrace AI, traditional database solutions fall short of rising demands for versatility, performance, and affordability. Couchbase is leading the way with Capella, the developer data platform for critical applications in our AI world. By uniting transactional, analytical, mobile, and AI workloads into a seamless, fully managed solution, Couchbase empowers developers and enterprises to build and scale applications with unmatched flexibility, performance, and cost-efficiency from cloud to edge. Trusted by over 30% of the Fortune 100, Couchbase is unlocking innovation, accelerating AI transformation, and redefining customer experiences. Come join our mission! Position Overview: We are seeking a highly skilled Data Engineer with a strong background in data warehousing, advanced analytics, and AI integration. The ideal candidate will have 5+ years of experience designing and implementing data pipelines, building robust analytics platforms, and working with modern data technologies like Snowflake, Google Cloud, Databricks, Looker, PowerBI, and Tableau. As part of our Delorean team, you will play a pivotal role in driving innovation and ensuring our data infrastructure supports strategic decision-making. Key Responsibilities: Data Infrastructure Development: Design, develop, and maintain scalable data pipelines and ETL processes to support data ingestion, transformation, and storage across multiple platforms. Experience with Cloud Data Platforms: Expertise in implementing and optimizing solutions using industry-leading tools such as Snowflake, Databricks, and Google Cloud Platform. Demonstrated ability to ensure high performance, scalability, and cost-effectiveness in data infrastructure and workflows. Analytics & Visualization: Build and maintain interactive dashboards and reports using Looker, PowerBI, and Tableau to provide actionable insights to business stakeholders. AI & Advanced Analytics: Collaborate with Data Science teams to integrate AI models and machine learning workflows into data pipelines for predictive analytics and automation. Data Governance: Ensure data quality, security, and compliance with industry best practices and regulatory requirements. Collaboration: Partner with cross-functional teams, including Product, Engineering, and Business Operations, to understand data requirements and deliver tailored solutions. Continuous Improvement: Identify opportunities to improve data workflows, adopt new technologies, and drive operational efficiency. Qualifications: Education: Bachelor s or Master s degree in Computer Science, Data Engineering, or a related field. Experience: 5+ years of hands-on experience in data engineering, data warehousing, or related fields. Technical Skills: Expertise in Snowflake, Databricks, and Google Cloud Platform (Mandatory experience in at least one tool). Proficiency in ETL/ELT processes and data pipeline tools (e.g., Apache Airflow, dbt). Experience with data visualization tools like Looker, Power BI, and Tableau. Strong SQL skills and familiarity with Python or Scala for data processing. Knowledge of AI/ML concepts and frameworks for integrating predictive models. Soft Skills: Excellent problem-solving skills with a proactive mindset. Strong communication and collaboration skills to work effectively with technical and non-technical teams. Ability to manage multiple priorities in a fast-paced environment. Modern customer experiences need a flexible cloud database platform that can power applications spanning from cloud to edge and everything in between. Couchbase s mission is to simplify how developers and architects develop, deploy, and consume modern applications. With Capella, our flexible, affordable cloud platform, we empower organizations to quickly build and deliver premium customer experiences with unmatched price-performance. More than 30% of the Fortune 100 trust Couchbase to power their modern applications. Benefits at Couchbase: Generous Time Off Program: Flexibility to care for yourself and your family. Wellness Benefits: Comprehensive medical plans, dental, vision, life insurance, and employee assistance programs. Financial Planning: RSU equity program, ESPP, retirement planning, and business travel insurance. Career Growth: A Be valued, Create value approach to your career development. Fun Perks: Ergonomic office setup, food & snacks for in-office employees, and more! Qualification : Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.

Data Engineer Role Overview We are looking for a Data Engineer to design, develop, and maintain scalable data pipelines and ETL processes. You will work closely with data scientists, analysts, and engineers to ensure efficient data processing and storage solutions that support business intelligence and analytics needs. This role requires expertise in SQL, big data technologies, and cloud platforms to build and optimize data workflows. Key Responsibilities Data Pipeline Development & Optimization Design and build scalable ETL/ELT processes for structured and unstructured data. Develop and maintain data ingestion frameworks to handle large datasets efficiently. Ensure data integrity, consistency, and security across multiple sources. Database & Data Warehouse Management Develop, optimize, and maintain relational and NoSQL databases. Implement data modeling best practices for performance and scalability. Work with cloud-based data warehouses (e.g., Snowflake, Redshift, BigQuery). Big Data & Cloud Technologies Leverage big data tools (e.g., Spark, Hadoop, Databricks) for data processing. Work with cloud platforms (e.g., AWS, Azure, GCP) to build and optimize data solutions. Develop real-time and batch processing workflows. Collaboration & Documentation Work closely with data scientists, engineers, and business teams to understand data requirements. Document data pipelines, architecture, and workflows for scalability and maintenance. Required Qualifications & Skills Technical Expertise: Strong experience with SQL and Python for data processing. Proficiency in ETL/ELT frameworks and data integration techniques. Hands-on experience with big data tools (e.g., Apache Spark, Hadoop, Kafka). Cloud & Database Management: Expertise in cloud data platforms (Azure, AWS, GCP). Experience with data warehousing solutions (Snowflake, Redshift, BigQuery). Understanding of data governance, security, and compliance. Performance Optimization & Troubleshooting: Ability to optimize SQL queries and improve data processing efficiency. Experience troubleshooting complex data pipeline issues. Apply Now & Be Part of an Innovative Data Team!

Data Engineer BI Analytics & DWH Location: Pune About Calfus At Calfus, we develop cutting-edge AI agents and software products that enable businesses to unlock the full potential of AI transforming operations, boosting revenue, and driving innovation. From startups to global enterprises, our clients trust us to deliver engineering excellence. Our teams have built ERP and cloud solutions that integrate seamlessly with core business systems, streamlining manual processes and empowering teams to focus on high-impact work. As one of the fastest-growing companies in our sector, we thrive on collaboration, speed, and innovation. If you re driven by impact and love working in fast-paced environments, we want to hear from you. About the Role We re looking for a Data Engineer with 3 4 years of experience to join our Business Intelligence & Data Warehouse team. In this role, you'll be responsible for building robust data pipelines, designing scalable data models, and supporting analytics and reporting capabilities. You will work with modern tools like Azure Data Factory, Databricks, and Snowflake, and help deliver reliable, performant data infrastructure for internal and client-facing initiatives. What You ll Do ETL/ELT Development Build and maintain efficient ETL/ELT pipelines using Azure Data Factory, Databricks, or equivalent. Ingest and consolidate data from multiple sources into data warehouses or data lakes. Data Modeling & Warehousing Design and implement relational and dimensional models suited for business reporting. Work with databases such as Snowflake, Redshift, Postgres, or SQL Server. SQL & Database Engineering Write performant SQL queries, stored procedures, and scripts for data transformation. Implement complex joins and aggregations for analytical workloads. BI & Visualization Support Collaborate with business users to support reporting in Power BI or Tableau. Enable data-driven decisions through accessible and accurate data models. Performance & Data Quality Monitor pipeline jobs for failures, performance issues, and delays. Apply data validation checks to ensure the integrity and accuracy of data. Bachelor s degree in Computer Science, Data Engineering, Information Systems, or related field. 3 4 years of hands-on experience in data engineering, with strong skills in ETL/ELT, SQL, and data modeling. Experience with cloud platforms (Azure, AWS) and tools like: Azure Data Factory Databricks SQL Server, Postgres, Redshift, or Snowflake Basic understanding of Python for scripting and automation. Familiarity with file formats like JSON, Parquet, and CSV. Exposure to BI tools such as Tableau or Power BI (nice to have). Benefits @ Calfus Medical, group, and parental insurance Provident fund & gratuity Birthday leave and wellness support High-growth, high-impact environment Diversity & Inclusion Calfus is proud to be an Equal Opportunity Employer. We believe that diversity drives innovation, and we are committed to fostering an inclusive, supportive, and respectful environment for all. Qualification : Bachelors degree in Computer Science, Data Engineering, Information Systems, or related field

About Talentica Software: Talentica Software is a boutique software development company founded by industry veterans and alumni from IITB. For over 21 years, we have been helping startups build products using cutting-edge technologies. With a focus on solving real-world problems, we ve worked with over 180+ startups, primarily in the US, leading to numerous successful exits. In 2022, Great Place to Work recognized Talentica Software as one of India s Great Mid-Size Workplaces. What We re Looking For: We are seeking a Data Engineer to join the product engineering team for a SAAS-based Subscription Management Platform. If you have a passion for working with data and a drive to innovate, we encourage you to apply. What You ll Be Doing: Data Transformation: Use Python and PySpark to transform large datasets and build efficient data pipelines. Pipeline Creation: Work with Kafka and Debezium to create and deploy real-time data pipelines. SQL Expertise: Write and optimize complex SQL queries for data aggregation and transformation. Delta Lake & Databricks: Utilize Delta Lake and Databricks for efficient data storage and management. Collaboration: Work with cross-functional teams to ensure data is accessible, reliable, and well-organized for product teams. To Be Successful in This Role, You Should Have: Qualification: BE/BTech from a top-tier engineering institute. Experience: 5+ years of experience in the data engineering field. Must-Have Skills: Strong proficiency in Python and PySpark for data transformation. Experience with Kafka and Debezium for data pipeline creation and deployment. Excellent SQL skills, especially for complex data aggregation and transformation. Familiarity with Delta Lake and Databricks. A solid understanding of all stages of the SDLC and experience with Agile/SCRUM methodology. Good-to-Have Skills: Understanding of machine learning algorithms. Familiarity with ETL tools like Matillion or Google Data Studio. Experience with creating BI reports using tools like Looker or Tableau. Familiarity with writing APIs to extract data from databases. What You ll Find Here: Culture of Innovation: We focus on technology expertise and innovation, not maintenance projects. Our customers come to us for cutting-edge solutions. Endless Learning Opportunities: Stay ahead of the curve by exploring new advancements in your field and applying them to build better, faster, and simpler products. Talented Peers: Work alongside experienced graduates from India's top 20 engineering colleges, including IITs, NITs, and other top-tier institutes. Flexibility: We value work-life balance and offer flexible schedules, including remote work options. Great Culture: 82% of our employees recommend Talentica to their friends, according to Glassdoor. You'll love being part of our team! At Talentica, we don t just follow trends we lead them. If you re looking for a dynamic, fast-paced environment with opportunities to innovate, grow, and shape the future of technology, Talentica is the place for you. Join us and make a real difference! Qualification : BE/BTech from a top-tier engineering institute.