Associate Data Engineer Jobs in Pune

Principal AI Engineer - MLOps Location: Pune Overview Rapid7 is seeking a **Principal AI Engineer - MLOps** to drive the development and deployment of **scalable, production-grade AI/ML systems** in the field of cybersecurity. This role is within the **AI Center of Excellence team** and involves managing the **end-to-end design of ML production systems** from scoping and data requirements to continuous deployment and monitoring. You will play a pivotal role in mentoring junior engineers and shaping the **MLOps culture**. Key Technologies Used **Cloud/Infrastructure:** AWS (research environments, data hosting), **EKS** (Kubernetes), **Terraform** (Infrastructure as Code). **Modeling/Data:** **Python** (numpy, pandas, scikit-learn), Jupyter notebooks, Anomaly detection methods. Key Responsibilities End-to-End ML Production System Design: **Architect and manage the design and deployment** of ML systems from inception to production. Define project scopes, data requirements, modeling strategies, and **deployment pipelines** for AI/ML systems. Ensure **seamless integration** of models and AI components into the production environment, focusing on **scalability, reliability, and security**. Data Pipeline Management: **Develop and maintain data pipelines**, ensuring data consistency, integrity, and accessibility throughout the ML lifecycle. Oversee the data lifecycle, including **data preprocessing, feature engineering**, and storage. ML Guardrails & Monitoring: Implement **robust ML guardrails** to ensure model performance, fairness, and compliance. Manage the **monitoring and observability** of deployed models, ensuring timely and accurate tracking of model performance. Deployment & Service Monitoring: Develop and deploy accessible endpoints, including **web applications and REST APIs**, adhering to security best practices. Continuously monitor models and data to detect issues such as **drift, anomalies, or performance degradation**. Collaboration & Mentorship: Collaborate closely with engineering, data science, and product teams to ensure alignment on goals. **Mentor and guide junior engineers**, share knowledge, and promote best practices in MLOps. Agile Development & Iteration: Embrace **agile practices**, focusing on continuous iteration and solving complex challenges. Skills & Experience Core Technical Expertise **Extensive Software Engineering Experience:** 15+ years as a software engineer, with at least **3-5 years of expertise in MLOps** in AWS environments. **MLOps & DevOps:** Proven experience deploying **scalable AI/ML systems**, managing **CI/CD pipelines**, and utilizing cloud AI resources. Expertise with **Docker, Kubernetes, and cloud-based AI infrastructure management**. **Programming:** Strong proficiency in **Python** (Flask or FastAPI) and experience building APIs. Data & Modeling Skills Experience in designing **data pipelines** and performing **feature engineering**. Familiarity with **model risk management** strategies, including **concept drift monitoring** and hyperparameter tuning. Leadership & Communication Demonstrated ability to **collaborate across engineering, data science, and product teams**. Proven track record of **mentoring and guiding junior engineers**. Strong communication skills for conveying complex technical concepts and creating detailed system architecture documentation. Nice to Have Experience deploying resources that enable data scientists to fine-tune and experiment with **LLMs (Large Language Models)**. Understanding of AI/ML operational frameworks and associated challenges. Be a part of **cutting-edge AI/ML efforts** to strengthen cybersecurity. Mentor and grow the next generation of engineers in an inclusive, fast-paced environment. Tackle some of the most challenging security problems with a passionate and creative team.

Data Engineer BI Analytics & DWH Location: Pune About Calfus At Calfus, we develop cutting-edge AI agents and software products that enable businesses to unlock the full potential of AI transforming operations, boosting revenue, and driving innovation. From startups to global enterprises, our clients trust us to deliver engineering excellence. Our teams have built ERP and cloud solutions that integrate seamlessly with core business systems, streamlining manual processes and empowering teams to focus on high-impact work. As one of the fastest-growing companies in our sector, we thrive on collaboration, speed, and innovation. If you re driven by impact and love working in fast-paced environments, we want to hear from you. About the Role We re looking for a Data Engineer with 3 4 years of experience to join our Business Intelligence & Data Warehouse team. In this role, you'll be responsible for building robust data pipelines, designing scalable data models, and supporting analytics and reporting capabilities. You will work with modern tools like Azure Data Factory, Databricks, and Snowflake, and help deliver reliable, performant data infrastructure for internal and client-facing initiatives. What You ll Do ETL/ELT Development Build and maintain efficient ETL/ELT pipelines using Azure Data Factory, Databricks, or equivalent. Ingest and consolidate data from multiple sources into data warehouses or data lakes. Data Modeling & Warehousing Design and implement relational and dimensional models suited for business reporting. Work with databases such as Snowflake, Redshift, Postgres, or SQL Server. SQL & Database Engineering Write performant SQL queries, stored procedures, and scripts for data transformation. Implement complex joins and aggregations for analytical workloads. BI & Visualization Support Collaborate with business users to support reporting in Power BI or Tableau. Enable data-driven decisions through accessible and accurate data models. Performance & Data Quality Monitor pipeline jobs for failures, performance issues, and delays. Apply data validation checks to ensure the integrity and accuracy of data. Bachelor s degree in Computer Science, Data Engineering, Information Systems, or related field. 3 4 years of hands-on experience in data engineering, with strong skills in ETL/ELT, SQL, and data modeling. Experience with cloud platforms (Azure, AWS) and tools like: Azure Data Factory Databricks SQL Server, Postgres, Redshift, or Snowflake Basic understanding of Python for scripting and automation. Familiarity with file formats like JSON, Parquet, and CSV. Exposure to BI tools such as Tableau or Power BI (nice to have). Benefits @ Calfus Medical, group, and parental insurance Provident fund & gratuity Birthday leave and wellness support High-growth, high-impact environment Diversity & Inclusion Calfus is proud to be an Equal Opportunity Employer. We believe that diversity drives innovation, and we are committed to fostering an inclusive, supportive, and respectful environment for all. Qualification : Bachelors degree in Computer Science, Data Engineering, Information Systems, or related field

Position: Associate Data Science Engineer Location: Pune, Maharashtra Work Type: Full Time Minimum Experience: 1 2 Years Top Skills: AI Applications, Python, Machine Learning, Deep Learning, NLP/NLG, Image Processing About the Role We are looking for a self-driven Associate Data Science Engineer to join our fast-growing Data & AI Practice. You will work on cutting-edge AI technologies and cloud-native solutions, helping build intelligent, high-impact products for our global clients. If you're passionate about AI/ML, Computer Vision, NLP, Generative AI, and developing scalable AI applications, this role is for you. What You Will Work On Contribute to the design, development, and deployment of AI-powered features and solutions. Collaborate with Product Management to translate business problems into AI/ML use cases. Build applications leveraging Computer Vision, NLP, Deep Learning, and Gen AI technologies. Develop and present functional prototypes, demos, and POCs. Stay current with the latest tools, frameworks, and research in ML, DL, and Gen AI. Participate in end-to-end project lifecycles from analysis to monitoring. What You Bring (Must-Have Skills) 1+ years of experience delivering AI/ML solutions through complete project lifecycle. Strong grasp of Linear Algebra, Statistics, and Probability. Proficiency in Python and data science libraries such as Pandas, NumPy, SciPy, scikit-learn, NLTK. Hands-on experience with ML/DL/NLP frameworks: TensorFlow, Keras, PyTorch, HuggingFace, spaCy, CoreNLP. Experience with Deep Learning models for Computer Vision and Object Detection. Solid knowledge of Generative AI, LLMs, and tools such as LangChain, LlamaIndex, Prompt Engineering. Strong analytical thinking, problem-solving ability, and effective communication. Great If You Also Know Experience with RAG frameworks, Function Calling, Agentic Frameworks, or Vector Search. Familiarity with cloud services: AWS (preferred), Azure, or Google Cloud Platform. Building APIs using FastAPI, async programming, and REST principles. Understanding of CI/CD, Docker/Kubernetes, and cloud-native deployment pipelines. Exposure to MLOps / LLMOps: versioning, tuning, deployment, and monitoring of models. Comfort with data visualization tools and practices. Agile development experience (Scrum, Kanban, etc.). Startup culture, small teams, and a high degree of autonomy. Work on next-gen technologies in AI, ML, GenAI, and cloud computing. Remote working options based on experience and maturity. Competitive salary and family-friendly benefits. Clear, performance-based career growth path. About Cognologix Cognologix helps businesses reimagine and innovate through modern data and cloud-native solutions. With a strong focus on data science, automation, and intelligent applications, we partner with organizations to deliver cutting-edge products in the most efficient and scalable ways. We thrive in a fast-paced, tech-first environment where innovation meets execution.

Position: Data Science Engineer Location: Pune Work Type: Full Time Minimum Experience: 3+ Years Top Skills: AI Applications, Python, Machine Learning, Deep Learning, NLP/NLG, Computer Vision, Object Detection About the Role We re looking for an experienced Data Science Engineer to join our Data & AI Practice. You will work on building and deploying intelligent, enterprise-grade AI solutions using the latest in Machine Learning, Deep Learning, Computer Vision, and Generative AI technologies. This is a role for engineers who thrive on ownership, innovation, and collaboration with the opportunity to mentor, lead, and make a real business impact through data. What You Will Work On Lead the design and development of AI/ML models and applications. Work with Product Management to translate business use cases into technical requirements and AI solutions. Develop and deploy Computer Vision, NLP, Deep Learning, and multi-modal GenAI applications. Perform dataset selection, feature engineering, and optimize data representation techniques. Mentor junior team members and contribute to team growth. Continuously explore and adopt the latest advancements in AI/ML, CV, NLP, and LLMs. Required Skills & Experience 3+ years of full lifecycle experience in building and deploying AI/ML applications. Proficiency in Python and core libraries: Pandas, NumPy, SciPy, Scikit-learn, NLTK. Hands-on experience with ML/DL/NLP frameworks: TensorFlow, Keras, PyTorch, HuggingFace Transformers, spaCy, CoreNLP. Deep knowledge in Linear Algebra, Probability, and Statistics. Strong foundation in Computer Vision: image classification, object detection (YOLO, Mask/Faster R-CNN), segmentation (semantic, instance, panoptic). Experience in Generative AI/LLMs, including Prompt Engineering, LangChain, LlamaIndex, Vector Search, Function Calling, Agentic Frameworks, and Document Processing. Exposure to cloud platforms: AWS (preferred), Azure, or GCP. Experience building RESTful APIs using frameworks like FastAPI, with async programming. Familiarity with CI/CD, Docker/Kubernetes, and scalable cloud-native deployments. Excellent problem-solving, analytical, and communication skills. Bonus Skills (Nice to Have) Experience in Agile/Scrum methodologies. Knowledge of MLOps/LLMOps: model versioning, tuning, deployment, monitoring. Experience with data visualization tools for analytics storytelling. Startup culture with autonomy, agility, and small team dynamics. Work on emerging AI/ML technologies across diverse real-world applications. Remote work options for high-performing team members. Competitive compensation with family-friendly benefits. Structured career progression based on performance and innovation. About Cognologix At Cognologix, we help businesses redefine their digital future through innovation and data-centric solutions. Our focus on cloud-native architectures, AI-driven intelligence, and automation-first thinking empowers enterprises to achieve meaningful transformation.

Position: Senior Data Engineer Location: Pune Employment Type: Full-Time Company: vConstruct (Construction Technology Company) About vConstruct vConstruct is a Pune-based construction technology company focused on leveraging data science and analytics to transform the construction industry. Our Data Science and Analytics team is a close-knit group dedicated to supporting the business with robust data infrastructure and insightful analysis. Role Overview We are seeking a highly experienced Senior Data Engineer to lead the design, development, and maintenance of scalable data pipelines and infrastructure. This role involves working closely with cross-functional teams to manage diverse data sources, ensure data integrity, and support data-driven business initiatives. Key Responsibilities Lead the design and development of scalable, high-performance data pipelines and ETL/ELT frameworks following modern data engineering best practices. Architect complex data integration workflows that unify structured, semi-structured, and unstructured data from cloud and on-premise sources. Develop robust real-time, batch, and on-demand data pipelines equipped with monitoring, alerting, and automated error handling. Collaborate with analysts, data scientists, and business leaders to create reliable data models, quality frameworks, and SLAs. Ensure optimal pipeline performance, throughput, and proactive issue resolution. Work with platform, DevOps, and architecture teams to build secure, reusable, and CI/CD-enabled data workflows aligned with enterprise standards. Enforce best practices in source control, code reviews, testing automation, and continuous delivery. Lead root cause analysis and preventive maintenance to minimize business impact from data failures. Establish standards for data modeling, transformation logic, and governance. Design and execute comprehensive testing strategies to ensure data reliability and pipeline resilience. Optimize pipeline and query performance for scalability, reliability, and cost-efficiency. Maintain detailed technical documentation including architecture diagrams, process flows, and integration specs. Facilitate discussions with business and operational teams to prioritize data initiatives and drive strategy. Qualifications 7-10 years of hands-on experience in data engineering with a track record of building scalable and secure data platforms. 5+ years of scripting experience in Python for data processing, automation, and ETL development. 4+ years experience with Snowflake, including performance tuning, security, and advanced SQL. 5+ years experience with data integration tools like Azure Data Factory, Fivetran, or Matillion. 5+ years writing complex, optimized SQL queries on large datasets. Proven experience integrating APIs, JSON, XML, and webhooks for data acquisition. Hands-on experience with cloud platforms (Azure/AWS) and orchestration tools such as Apache Airflow. Experience with CI/CD pipelines, automated testing, and code versioning tools (e.g., Git). Familiarity with dbt or similar transformation tools and modular development practices. Experience with data visualization tools like Power BI is a plus. Strong interpersonal and communication skills to lead cross-functional technical discussions. Education Bachelor s or Master s degree in Computer Science, Information Technology, or a related field. Equivalent academic and professional experience will also be considered. Qualification : Bachelors or Masters degree in Computer Science, Information Technology, or a related field

Job Title: Associate Solution Engineer Location: Pune, Maharashtra, India Travel: Occasional business travel may be required About BMC At BMC, trust is more than a value it s how we operate. We re an award-winning, inclusive, and vibrant workplace where diverse perspectives fuel innovation and growth. We believe in balancing life and work, celebrating wins (big and small), and giving back to the communities we serve. Your voice matters here, and your individuality is celebrated. We help our customers become Autonomous Digital Enterprises by enabling them to scale, innovate, and lead in the digital era. The Opportunity Join BMC Helix s Digital Solution Engineering team as an Associate Solution Engineer a role designed for emerging tech professionals who want to build a career at the intersection of technology, customer impact, and business value. Whether you're a recent graduate or have 1 3 years of experience, this is your opportunity to dive deep into enterprise platforms, AI innovation, and next-gen service management. About BMC Helix BMC Helix is our cutting-edge, AI-powered platform for IT Service and Operations Management (ITSM, ITOM, AIOps). With a containerized, microservices-based architecture, it supports flexible deployment across cloud, hybrid, and on-prem environments. BMC Helix empowers enterprises to go from reactive support to predictive, autonomous operations using advanced capabilities like Agentic AI and cross-domain automation. What You ll Do Work with global solution engineering and sales teams to support customer engagements and design compelling, tailored solutions. Participate in discovery sessions to understand client requirements and business objectives. Deliver engaging, value-driven demos that highlight BMC Helix s strengths like intelligent automation, Agentic AI, and seamless integrations. Assist in designing proof-of-value solutions, configurations, and presentations. Translate complex technical concepts like microservices, APIs, and hybrid deployments into clear, customer-centric value propositions. Collaborate with internal teams (Product, Strategy, Sales) to ensure alignment with customer needs and market trends. Stay current on emerging trends in GenAI, LLMs, ITSM, ITOM, and enterprise digital transformation. What You Bring Bachelor s degree in Computer Science, Information Technology, Engineering, or a related field. 0 3 years of experience in a technical, consulting, or customer-facing role. Excellent communication and presentation skills; able to engage both technical and business stakeholders. Interest or hands-on exposure to Generative AI and Large Language Models (LLMs). Foundational knowledge of cloud platforms (AWS, Azure, GCP), microservices, containers (Docker, Kubernetes). Curiosity for technology and enterprise IT challenges; proactive and eager to learn. Exposure to platforms like BMC Helix, ServiceNow, Jira Service Management, or similar is a plus. Basic scripting or API understanding (Python, JavaScript, REST APIs). Familiarity with ITIL, DevOps, or service management concepts. Fluent in English; multilingual proficiency (e.g., Spanish, French, German, Italian) is a bonus. Work with industry-leading platforms and AI technologies Be part of a collaborative, global team shaping the future of digital operations Learn from experienced mentors and participate in enablement programs Thrive in a culture that values your authenticity, creativity, and growth No problem. If you're passionate about this opportunity and BMC s mission, we still encourage you to apply. We value potential just as much as experience. BMC is proud to be an equal opportunity employer. Qualification : Bachelors degree in Computer Science, Information Technology, Engineering, or a related field.

Senior ETL Engineer Location: Pune Role Overview: We are seeking a Senior ETL Engineer to design, develop, and optimize our end-to-end data infrastructure, including ETL pipelines, data lakes, and data warehouses. This role is instrumental in enabling reliable, scalable, and high-performance data operations while aligning with data governance best practices. You will be an integral part of a collaborative data team, working closely with Data Engineers, Architects, Analysts, Scientists, and other stakeholders to modernize and migrate our data systems to the cloud. Key Responsibilities: Convert SSIS stored procedures into Matillion mappings, and develop associated job schedules and audit frameworks. Design, build, and maintain efficient ETL pipelines to extract, transform, and load data from multiple sources into the data lake and data warehouse. Implement both full and incremental loads using SCD (Slowly Changing Dimensions) and CDC (Change Data Capture) methodologies for real-time and batch data processing. Develop and maintain robust error handling, job monitoring, and job audit mechanisms. Conduct data profiling, data cleansing, and root cause analysis to resolve data inconsistencies and improve data quality. Optimize data pipelines for performance, scalability, and data quality, using benchmarking and tuning techniques. Build dashboards or reports for ETL job monitoring, performance tracking, and system health checks. Support data governance practices, ensuring accuracy, availability, and security of production data. Contribute to cloud migration initiatives, transitioning data workloads from on-premise systems to modern cloud data platforms. Required Skills & Qualifications: Bachelor's degree in Computer Science, Engineering, or a related field. 5+ years of experience as an ETL Engineer, Data Engineer, or in a similar data engineering role. Experience with cloud-based data platforms such as AWS, Azure, or GCP. Proficiency with data warehousing technologies like Snowflake, Redshift, Azure Synapse, or Databricks. Hands-on experience with ETL tools such as Matillion, SSIS, Informatica, AWS Glue, Azure Data Factory, StreamSets, STRiim, Oracle ODI, Coalesce, or Talend. Solid understanding of data modeling principles (E/R and Dimensional Modeling) and tools like ER/Studio or ERwin. Proficiency with SQL and programming/scripting languages such as Python, Scala, or Java. Experience with database tools such as Microsoft SSMS or Oracle Developer. Strong analytical, problem-solving, and data troubleshooting skills. Excellent communication and collaboration abilities. Ability to work efficiently in a fast-paced, agile, and dynamic environment. Qualification : Bachelor's degree in Computer Science, Engineering, or a related field.

Position: Provider Data Operations Senior Associate Location: Pune, Maharashtra, India Our Mission: Healthcare should work for patients, but often it doesn t. Zocdoc empowers patients by offering the leading healthcare marketplace to easily find and book care across specialties and insurance plans nationwide. Our goal is to make healthcare consumer-friendly, driving quality up and prices down. Your Impact: As a Senior Provider Data Operations Associate, you will be the guardian of Zocdoc s data ecosystem managing, structuring, and analyzing large volumes of data to ensure accuracy and integrity. You'll transform raw data into strategic insights that drive decision-making and continuous improvement. By leveraging AI tools, JavaScript, and SQL, you ll optimize workflows and scale operational processes across product and operations teams. Day-to-Day Responsibilities: Use AI prompts to improve workflow efficiency. Develop and maintain automation scripts using Google Apps Script and JavaScript. Troubleshoot and optimize scripts to enhance functionality and scalability. Extract, manipulate, audit, and analyze data using SQL. Generate actionable insights from various dashboards. Support product decisions with data-backed research and cross-functional collaboration. What Makes You Successful: Bachelor s degree with good academic standing (minimum 60% in 10th/12th/graduation). 3-4+ years of relevant work experience. Proven experience using AI tools for automation. Advanced skills in Microsoft Excel and Google Sheets, including Google Apps Script. Working knowledge of JavaScript and comfort with web-based tools. Understanding of Python and expertise in SQL is a plus. Strong communication skills to translate technical and non-technical information. Ability to manage multiple tasks in a fast-paced environment with a strong sense of accountability. Experience in US healthcare or healthcare technology is a bonus. Benefits: Work with a smart, supportive, and collaborative team. Competitive salary and medical insurance. Perks such as catered lunches, Ping Pong, daycare facilities, and wellness programs. Cellphone and Wi-Fi reimbursements. Competitive parental leave and sabbatical after 5 years. Annual sponsored health check-ups. Certified Great Place to Work (2024-2025). About Zocdoc: Zocdoc is the leading digital health marketplace in the US, connecting millions of patients with nearby, in-network healthcare providers. Founded in 2007, Zocdoc s mission is to empower patients with choice and transparency. We value diversity, collaboration, and innovation, striving to build a better healthcare experience for all. Qualification : Bachelors degree with good academic standing (minimum 60% in 10th/12th/graduation).

Job Title: Staff Data / Machine Learning Engineer Company: Druva Location: Pune, Maharashtra, India About Druva: Druva, the autonomous data security company, delivers data protection on autopilot through a 100% SaaS, fully managed platform that secures and recovers data from all threats. The Druva Data Security Cloud guarantees data availability, confidentiality, and integrity providing customers with autonomous protection, rapid incident response, and guaranteed data recovery. Trusted by over 6,000 customers worldwide including 65 of the Fortune 500 Druva safeguards business-critical data in today s connected world. Backed by over $350M in venture capital, Druva protects more than 200 PB of data globally and offers a $10 million Data Resiliency Guarantee against cyber threats. The Role & Team: Join Druva s Business Intelligence team, where data drives key insights and fuels operational and strategic decisions. As a Staff Data/Analytics/ML Engineer, you will lead the development of data-to-insights recommendation engines and drive collaboration across teams to deliver impactful, data-driven solutions. This is a high-impact role working alongside expert engineers and product leaders, leveraging industry-leading data platforms and tools. What You ll Do: Bridge the gap between data engineering, analytics, and machine learning to unlock actionable business insights. Collaborate cross-functionally with Product, Engineering, GTM, and Customer Success teams to build data-driven solutions that enhance decision-making and business outcomes. Design, build, and maintain scalable data pipelines and infrastructure supporting analytics, machine learning, and operational workflows. Develop and optimize data models for efficient analytics and reporting across large datasets. Lead feature engineering, model training, and deployment pipelines for real-time and batch ML/AI applications. Drive architectural improvements in data governance, observability, and platform scalability. Evaluate and implement cutting-edge data tools and platforms including Reverse ETL, MLOps, and DataOps frameworks. Mentor and guide data engineers, analysts, and ML practitioners, fostering technical excellence and collaboration. Translate complex business requirements into technical deliverables, prioritizing initiatives that maximize impact. What We Are Looking For: Bachelor s or Master s degree in Computer Science, Data Science, Engineering, Statistics, or a related field. 7+ years experience spanning data engineering, analytics engineering, and machine learning with a strong technical foundation in all three areas. Expertise in modern data stacks: Snowflake, dbt, Airflow, Spark, and cloud platforms (AWS, GCP, Azure). Proficiency with BI and analytics tools such as Looker, Sigma, Tableau, Power BI, or equivalents. Strong skills in SQL, Python, and distributed data processing frameworks (Spark, Dask, etc.). Experience designing and maintaining data pipelines, ETL/ELT workflows, and data transformations for analytics and ML use cases. Familiarity with ML frameworks like TensorFlow, PyTorch, Scikit-learn, and MLOps lifecycle management. Deep knowledge of data modeling, architecture, and governance best practices. Proven problem-solver who thrives in ambiguity and designs robust data solutions. Experience working in Agile environments with strong prioritization and collaboration skills. Excellent communication skills, capable of simplifying complex technical topics for non-technical stakeholders. Bonus Points For: Experience with Generative AI, LLMs, and integrating AI into business workflows. Knowledge of streaming architectures (Kafka, Kinesis, Pub/Sub) and real-time analytics. Familiarity with Data Contracts, Data Quality frameworks, or Data Mesh architectures. Experience with Reverse ETL tools such as Salesforce, Census, Hightouch, or Segment for operational analytics. If you re passionate about driving data innovation at scale and want to work in a dynamic environment protecting the world s critical data, Druva invites you to join our team! Qualification : Bachelors or Masters degree in Computer Science, Data Science, Engineering, Statistics, or a related field.

Job Title: Senior Azure Data Engineer Experience: 8 - 10 years Location: Kharadi, Pune Role & Responsibilities Design, develop, and maintain scalable, high-performance data pipelines using Azure Data Services such as Azure Data Factory, Azure Databricks, and Azure Synapse Analytics. Build and optimize data models, ETL/ELT processes, and data integration workflows to support analytics and BI needs. Develop and manage data lakes, data warehouses, and real-time streaming solutions on Azure. Collaborate with data analysts, scientists, and software engineers to ensure smooth data integration and processing. Ensure data security, governance, and compliance following industry best practices and regulations. Monitor and troubleshoot data pipelines, focusing on performance and reliability enhancements. Automate data workflows and infrastructure using DevOps and CI/CD methodologies. Work closely with stakeholders to translate business requirements into robust data solutions. Required Skills & Qualifications 8-10 years of experience in data engineering, with at least 3 years focused on Azure data services. Strong expertise in Azure Data Factory (ADF), Azure Synapse Analytics, Azure Databricks, Azure Data Lake, and Azure SQL Database. Proficient in SQL, Python, or Scala for data processing and transformation. Experience with Big Data technologies like Spark, Delta Lake, and Parquet. Hands-on experience with ETL/ELT development and orchestration of data pipelines. Knowledge of Azure DevOps, CI/CD pipelines, and Infrastructure as Code tools (Terraform, ARM templates, Bicep). Familiarity with data governance, security best practices, and compliance frameworks. Experience with Power BI, Snowflake, or other BI/analytics tools is a plus.

Job Title: Senior AWS Data Engineer Experience: 5+ years Location: Kharadi, Pune Role & Responsibilities Design, develop, and optimize scalable data pipelines and ETL workflows using AWS services such as AWS Glue, Lambda, and Step Functions. Implement and maintain data lakes and data warehouses on AWS platforms including S3, Redshift, and Athena. Develop and manage both real-time and batch data processing solutions. Ensure data security, governance, and compliance following industry best practices. Collaborate with data scientists, analysts, and business stakeholders to deliver actionable, data-driven solutions. Optimize query performance and troubleshoot data issues proactively. Automate infrastructure deployment using Infrastructure as Code (IaC) tools like Terraform or CloudFormation. Implement monitoring, logging, and alerting frameworks using AWS CloudWatch, Prometheus, or equivalent tools. Mentor junior engineers and actively contribute to data engineering best practices and standards. Required Skills & Qualifications 5+ years of experience in data engineering, with at least 3 years focused on AWS cloud services. Strong hands-on experience with AWS services including Glue, Redshift, S3, Lambda, EMR, and Athena. Proficient in SQL, Python, and Spark for data processing and analytics. Experience designing and implementing ETL/ELT pipelines. Familiarity with DevOps practices, CI/CD pipelines, and automation tools such as Terraform, CloudFormation, and Jenkins. Deep understanding of data governance, security protocols, and compliance frameworks. Experience with real-time data streaming and processing technologies such as Kafka, Kinesis, or similar tools. Knowledge of database optimization techniques including indexing and partitioning.

About Talentica Software: Talentica Software is a boutique software development company founded by industry veterans and alumni from IITB. For over 21 years, we have been helping startups build products using cutting-edge technologies. With a focus on solving real-world problems, we ve worked with over 180+ startups, primarily in the US, leading to numerous successful exits. In 2022, Great Place to Work recognized Talentica Software as one of India s Great Mid-Size Workplaces. What We re Looking For: We are seeking a Data Engineer to join the product engineering team for a SAAS-based Subscription Management Platform. If you have a passion for working with data and a drive to innovate, we encourage you to apply. What You ll Be Doing: Data Transformation: Use Python and PySpark to transform large datasets and build efficient data pipelines. Pipeline Creation: Work with Kafka and Debezium to create and deploy real-time data pipelines. SQL Expertise: Write and optimize complex SQL queries for data aggregation and transformation. Delta Lake & Databricks: Utilize Delta Lake and Databricks for efficient data storage and management. Collaboration: Work with cross-functional teams to ensure data is accessible, reliable, and well-organized for product teams. To Be Successful in This Role, You Should Have: Qualification: BE/BTech from a top-tier engineering institute. Experience: 5+ years of experience in the data engineering field. Must-Have Skills: Strong proficiency in Python and PySpark for data transformation. Experience with Kafka and Debezium for data pipeline creation and deployment. Excellent SQL skills, especially for complex data aggregation and transformation. Familiarity with Delta Lake and Databricks. A solid understanding of all stages of the SDLC and experience with Agile/SCRUM methodology. Good-to-Have Skills: Understanding of machine learning algorithms. Familiarity with ETL tools like Matillion or Google Data Studio. Experience with creating BI reports using tools like Looker or Tableau. Familiarity with writing APIs to extract data from databases. What You ll Find Here: Culture of Innovation: We focus on technology expertise and innovation, not maintenance projects. Our customers come to us for cutting-edge solutions. Endless Learning Opportunities: Stay ahead of the curve by exploring new advancements in your field and applying them to build better, faster, and simpler products. Talented Peers: Work alongside experienced graduates from India's top 20 engineering colleges, including IITs, NITs, and other top-tier institutes. Flexibility: We value work-life balance and offer flexible schedules, including remote work options. Great Culture: 82% of our employees recommend Talentica to their friends, according to Glassdoor. You'll love being part of our team! At Talentica, we don t just follow trends we lead them. If you re looking for a dynamic, fast-paced environment with opportunities to innovate, grow, and shape the future of technology, Talentica is the place for you. Join us and make a real difference! Qualification : BE/BTech from a top-tier engineering institute.

Data Engineer IBM Consulting About the Role Join us at IBM Consulting in one of our Client Innovation Centers, where we deliver specialized technical and industry expertise to both public and private sector clients worldwide. Our centers bring local skills and technical capabilities to help clients drive innovation and adopt cutting-edge technologies. A career at IBM Consulting is built on fostering long-term relationships and collaborating closely with clients around the globe. You ll work with leaders across multiple industries to advance the hybrid cloud and AI journeys of some of the world s most innovative and influential companies. Your ability to create impactful change will be empowered by IBM's strategic partner ecosystem and our robust technology platforms, including IBM Software and Red Hat. At IBM, curiosity and a continual pursuit of knowledge form the cornerstone of success. In your role, you'll be encouraged to challenge norms, explore new ideas beyond your immediate responsibilities, and deliver innovative solutions that make a meaningful difference for clients. Our culture is rooted in growth and empathy, ensuring long-term career development in an environment that celebrates your unique skills and experiences. Your Role and Responsibilities As a Data Engineer at IBM, you ll unlock the power of data, revealing insightful stories and complex patterns. You ll contribute to data gathering, storage, and both batch and real-time processing, playing an essential role in defining data management systems and identifying key data for analysis. Your key responsibilities will include: Predictive Modeling: Implementing, validating, and maintaining statistical and predictive models, utilizing big data techniques along with various machine learning methodologies. Enterprise Search Applications: Designing and deploying enterprise search applications like Elasticsearch and Splunk to meet client-specific needs. Collaboration & Agile Practices: Working in an agile, team-oriented environment, collaborating with data scientists, engineers, consultants, and database administrators to bring statistical rigor to predictive behavior challenges. Data Integration & Cleansing: Developing teams or writing programs to efficiently cleanse and integrate data, build predictive models, and evaluate the results. Required Education Bachelor's Degree in Computer Science, Information Technology, Data Science, or a related field. Preferred Education Master's Degree in a relevant technical discipline. Required Technical and Professional Expertise 4+ years of experience in data modeling and data architecture. Proficiency in data modeling tools like ERwin, IBM Infosphere Data Architect, and experience with database management systems. Familiarity with various data models, including relational, dimensional, and NoSQL databases. Strong understanding of business processes and how data supports business decision-making. Expertise in database design principles, data warehousing concepts, and data governance practices. Preferred Technical and Professional Experience Exceptional analytical and problem-solving skills, with a sharp attention to detail. Ability to work collaboratively in a team-oriented environment and manage multiple projects simultaneously. Knowledge of SQL or similar programming languages. At IBM Consulting, we foster a culture of growth, innovation, and collaboration. As a Data Engineer, you ll have the opportunity to work with some of the brightest minds in the industry, all while impacting critical data-driven solutions for global clients. Join us and be a part of a diverse team committed to driving change and shaping the future of technology and analytics. Qualification : Bachelor's Degree in Computer Science, Information Technology, Data Science, or a related field.

Transform Healthcare with Data: Join Cencora as a Data Engineer II At Cencora, our team members are at the heart of everything we do. We are on a mission to create healthier futures, and every individual here plays a crucial role in fulfilling that purpose. We believe in making a difference at the intersection of healthcare, technology, and innovation. If you're ready to shape the future of health and help improve lives globally, Cencora is the place for you. We are looking for a Data Engineer II to join our dynamic team. This role is pivotal in delivering accurate and reliable data to support our Legal and Compliance departments, with a direct impact on driving data-driven decisions that improve operational efficiencies and support strategic business goals. What You ll Do: As a Data Engineer II, you will be responsible for transforming raw data into actionable insights and building solutions that enhance business operations. Your role will span across multiple platforms, using SQL, SAP BusinessObjects, and a variety of other tools, to ensure that our data remains accurate, accessible, and consistent. Enhance Analytics Solutions: Lead the design and implementation of enhancements to business analytics solutions that align with evolving business needs. You ll turn complex business challenges into accessible data access requirements. Data Mining & Analysis: Perform data mining on vast datasets to uncover growth opportunities in sales, margin, and market share. Analyze pricing activities, including discounts, allowances, programs, and contract pricing, to drive profitability. Data Quality Assurance: Validate, cleanse, and aggregate data, ensuring its accuracy and integrity. You ll identify and resolve data inaccuracies, improving the overall quality of enterprise data. Collaboration & Problem-Solving: Work cross-functionally with teams like Inside Sales, Finance, and Sourcing to develop performance forecasts, projections, and plans based on data assumptions and variables. Automated Solutions: Identify opportunities for automated solutions to enhance data quality and efficiency across the enterprise. Report Development: Collaborate with internal stakeholders to define reporting deliverables, ensuring that data is appropriately named, structured, and ready for decision-making. Data Integrity & Monitoring: Monitor and update data changes on a mass scale using tools, ensuring the highest level of data integrity is maintained. What You Bring to the Table: Experience: You have 4+ years of direct experience in healthcare data analytics or data engineering. You know your way around large datasets and understand how to unlock insights that drive business decisions. Technical Expertise: Proficient in SAP and BusinessObjects, with an extensive background in Business Intelligence and SQL. Strong understanding of the wholesale distribution process, including purchases, sales, payments, and supplier rebates. Knowledge of pharmaceutical wholesale processes and chargebacks to wholesalers is a plus. Familiar with data visualization tools like Power BI or Tableau. Analytical Mindset: You re a critical thinker with an eye for detail, always ready to dive into data to uncover trends, identify issues, and solve problems. Mentorship: You bring leadership experience, with a knack for mentoring and guiding team members. Communication Skills: Your ability to communicate complex data insights to non-technical stakeholders is top-notch. You can explain trends and patterns with clarity and confidence. Bonus Points: Familiarity with Microsoft Power BI, Tableau, or other data visualization tools. Experience with Big Data analysis tools and techniques. Background in data governance and data management. What You ll Get from Us: Meaningful Work: At Cencora, your work directly impacts the future of healthcare. Your expertise in data will help us make strategic, data-driven decisions that improve lives globally. Growth Opportunities: We invest in our team members' growth through mentorship, professional development, and the chance to shape the future of healthcare technology. Collaborative Environment: Work alongside innovative professionals in a culture that values diverse perspectives and fosters collaboration. Comprehensive Benefits: We offer a range of benefits designed to support you in every aspect of life health, wellness, work-life balance, and more. Join Cencora Today! If you re ready to leverage your data engineering expertise to drive meaningful change, we d love to hear from you! Apply now to become part of a purpose-driven company at the forefront of innovation in the healthcare sector. Cencora is committed to building a healthier future, and we want you on this journey with us.

Position Summary This role will be responsible for in-patient journey analysis and working with patient-level data to develop a robust solution for the client's teams. An expert in Patient Analytics who can guide and lead the team supporting pharma clients. Job Responsibilities Understanding & working experience towards patient analytics like patient journey, persistency, adherence, patient identification etc. Experience using various patient level data like APLD, EHR, patient registries etc. Can work across a variety of projects from advanced analytics, ad-hoc analysis and reporting. Strong in logical reasoning, structuring of analysis, asking the right questions and logical approach to analyse data, problems, and situations. Proficient in Microsoft Office suite. Teamwork and collaboration and can mentor junior associates. Articulate in communication written or verbal and able to convey ideas and information clearly to client or client facing teams. Willingness to learn and innovate. Experience in pharmaceutical sales and marketing analytics would be preferred. Contribute towards Axtria tools and capabilities as per the business requirements. Build organization capabilities by participating in Hackathon, solution design, and process automation. Effectively communicate with onshore/ client (as per business needs). Education Bachelor of Engineering in Others Bachelor Equivalent - Engineering Work Experience Overall 4-6 years of rich experience in the Pharmaceutical / Life Sciences Domain. We are looking for experts in the space of commercial pharmaceutical analytics- HCP analytics, payer analytics, and patient analytics. Relevant experience in Statistical/ modeling knowledge, ability to transform data to insights, good data visualization/ reporting skills. Exhibit expertise on pharma real-world data like APLD, EMR, Prescription data, formulary data etc. Worked on advanced analytics in the pharma domain throughout the patient journey like the line of therapy, switch analysis, source of business, segmentation, persistence & compliance, etc using various data sources. Strong communication, project management, and storyboarding skills. Technical knowledge- R/ Python/ SQL. Knowledge of self-service analytics platforms such as DataiKU/ KNIME/ Alteryx will be an added advantage. MS Excel knowledge is mandatory. Experience in Pharma commercial analytics experience will be preferred. Behavioural Competencies Teamwork & Leadership Motivation to Learn and Grow Ownership Cultural Fit Project Management Communication Technical Competencies Python SQL EXCEL Machine Learning Pharma Commercial Know How Patient Data Analytics Know How Others

Company Description Calsoft is a leading technology-first partner providing digital and product engineering services with over 25 years of experience. Role Description This is a full-time on-site role for an Associate Director / Director in Business Development for Calsoft s new Gen AI Practice. This is a new practice and an opportunity to build and be part of a team from the ground up. They will be responsible for hiring and leading an inside sales team of 2-3 people. The Associate Director will be located in Pune or Bengaluru and will have the following responsibilities: Responsibilities Responsible for business development and sales of Calsoft s AI practice. Be responsible for a revenue target at a personal and team level. Be a hands-on manager, drive the sales team, and ensure their productivity and efficiency in meeting the target. Be responsible for hands-on calling, lead generation, and closing of new accounts to meet the revenue goals. Work in collaboration with the engineering and presales teams. Activities in scope include prospecting for new business, detailed qualification of sales opportunities, management of the sales cycle, and negotiation of contracts and agreements. Develop and implement sales strategies while keeping both short- and long-term goals in perspective. Qualifications Strong leadership, management, and collaboration skills. Prior experience selling AI or ML services preferred. Excellent communication and interpersonal skills. Should have managed an inside sales team from India but also be hands-on with individual contributor experience. Experience selling into the US and Europe markets. Strong analytical and problem-solving skills. Degree in Computer Science or a related field preferred. Qualification : Degree in Computer Science or a related field preferred.

Job Overview Senior Engineer - Data Analytics in the Manufacturing domain with 3-7 years of experience. Lead data analytics projects focusing on aerospace reliability and warranty analysis. Develop solutions using Python and Power BI, creating insightful dashboards. Analyze warranty and aftermarket data for valuable insights. Collaborate across teams to address challenges. Streamline data processes using VBA and MS Excel. Drive reliability analysis initiatives for product performance. Communicate findings effectively and mentor junior team members. Demonstrate strong project management and leadership skills. Preferred: Experience in After-market data analysis, strong communication skills, proficiency in VB.Net. Education: Bachelor of Technology or Bachelor of Engineering in Computer Science coupled with an MBA in Business Analytics or Data Science. Preferred certifications: Microsoft Certified: Data Analyst Associate and Certified Reliability Engineer (CRE). Roles & Responsibilities Lead and oversee data analytics projects with a focus on reliability and warranty analysis in the aerospace industry. Utilize Python and Power BI to develop advanced data analytics solutions and create insightful Power BI dashboards for stakeholders. Conduct in-depth analysis of warranty data and aftermarket performance to provide valuable insights and recommendations. Collaborate with cross-functional teams to understand and address the unique challenges of the aerospace aftermarket business. Apply expertise in VBA and MS Excel to streamline data processes and enhance data visualization. Drive reliability analysis initiatives to ensure product performance and customer satisfaction. Communicate findings effectively through reports and presentations. Mentor junior team members and contribute to their professional development. Demonstrate strong leadership skills in project management and team coordination. Maintain a high level of professionalism and uphold ethical standards in data analysis and reporting. Our ideal candidate Extensive experience in Python, Power BI, Warranty Analysis, Aerospace V&V (Verification and Validation) processes, and Reliability Analysis 3-7 years of experience with advanced proficiency in Python for developing data analysis scripts and automation tools Expertise in Power BI for creating interactive visualizations and dashboards In-depth knowledge of Warranty Analysis and Reliability Analysis methodologies Ability to apply VBA and MS Excel for data manipulation and analysis Education: Bachelor of Technology (B.Tech) or Bachelor of Engineering (B.E) in Computer Science or Information Technology, coupled with a Master of Business Administration (MBA) in Business Analytics or Data Science Certification: Microsoft Certified: Data Analyst Associate and Certified Reliability Engineer (CRE) preferred Company overview Altimetrik delivers outcomes for our clients by rapidly enabling digital business & culture and infuse speed and agility into enterprise technology and connected solutions. We are practitioners of end-to-end business and technology transformation. We tap into an organization s technology, people, and assets to fuel fast, meaningful results for global enterprise customers across financial services, payments, retail, automotive, healthcare, manufacturing, and other industries. Founded in 2012 and with offices across the globe, Altimetrik makes industries, leaders and Fortune 500 companies more agile, empowered and successful.Altimetrik helps get companies get unstuck . We re a technology company that lives organizations a process and context to solve problems in unconventional ways. We re a catalyst for organization s talent and technology, helping teams push boundaries and challenge traditional approaches. We make delivery more bold, efficient, collaborative and even more enjoyable. Qualification : Bachelor of Technology (B.Tech) or Bachelor of Engineering (B.E) in Computer Science or Information Technology, coupled with a Master of Business Administration (MBA) in Business Analytics or Data Science

About Apexon Apexon is a digital-first technology services firm focused on accelerating business transformation and delivering human-centric digital experiences. We empower businesses to outperform their competition by meeting customers at any stage in the digital lifecycle. Apexon brings together expertise in AI, analytics, app development, cloud, commerce, CX, data, DevOps, IoT, mobile, quality engineering, and UX, along with deep domain knowledge in BFSI, healthcare, and life sciences. Backed by Goldman Sachs Asset Management and Everstone Capital, Apexon has a global footprint with 15 offices and 10 delivery centers across four continents. Position: Senior Java Developer Location: Pune, India Experience: Minimum 8 years of Java development experience Total Positions: 1 Job Description We are looking for a Senior Java Developer to join our dynamic team. The ideal candidate will have a deep understanding of Core Java, Spring Core, and React, along with experience in DevOps tools such as Docker, Kubernetes, and Git Actions. You will be responsible for designing and developing robust, scalable applications, providing technical leadership, and ensuring seamless integration and deployment. Key Responsibilities Develop and Maintain: Write clean, high-quality, high-performance, and maintainable code. Support test and deployment processes to ensure smooth delivery. Architectural Design: Design and develop robust, scalable applications using Java and Spring Core. Technical Leadership: Provide guidance and mentorship to junior developers. Collaborate with cross-functional teams to ensure optimal software solutions. Full Stack Development: Utilize your expertise in Java and React to develop both backend and frontend solutions. DevOps Integration: Utilize tools like Docker, Kubernetes, and Git Actions for continuous integration, deployment, and automation. Collaboration: Work with business units and stakeholders to gather requirements and ensure the delivery of reliable software solutions. Troubleshooting: Identify and resolve technical issues, and participate in an on-call rotation as needed. Documentation: Maintain documentation and ensure best practices are followed. Required Skills & Experience Core Java: Minimum of 8 years of hands-on experience in Java development. Spring Core: Proficiency in using the Spring framework to create enterprise-level applications. React: Solid understanding of React and its core principles. DevOps Tools: Experience with Docker, Kubernetes, and Git Actions. Software Architecture: Strong knowledge of software architecture and design patterns. Problem-Solving: Excellent problem-solving skills and the ability to work with minimal supervision. Communication: Strong written and verbal communication skills. Desirable Skills Cloud Services: Experience with cloud platforms such as AWS, Azure, or Google Cloud. Microservices: Familiarity with microservices architecture. Agile Methodologies: Understanding of Agile practices and experience working in Agile teams. Database Management: Experience with relational and non-relational databases. Our Commitment to Diversity & Inclusion Apexon is proud to be recognized as a Great Place to Work, certified in the USA, India, and the UK. We are committed to creating an inclusive and diverse workplace. We ensure equal opportunity employment for all qualified individuals and strictly prohibit any form of discrimination or harassment. At Apexon, we believe in promoting a #HumanFirstDigital culture. Perks and Benefits As an Apexon Associate, you'll enjoy: Comprehensive health benefits: Group Health Insurance (covering a family of 4), Term Insurance, and Accident Insurance. Paid Holidays & Leaves: Paid Parental Leave and earned leaves. Career Development: Learning opportunities and career advancement. Employee Wellness: Well-being programs to support you and your loved ones.

Role: Senior Architect Key Responsibilities: Platform Integration & Collaboration: Work across various platforms, collaborating with other support teams to ensure smooth integration and efficient operations. Solution Design & Development: Design, develop, test, and implement changes to solutions based on business requirements and strategic direction. Collaboration with Development Teams: Collaborate with development teams to test and maintain platforms and ensure smooth integration with other systems. Product/Project Support: Communicate regularly with Product Owners and agile Pods to provide ongoing support for products and projects. Guidance & Mentorship: Guide junior team members in development and delivery to help them grow and ensure efficient workflow. Release Support & Monitoring: Responsible for supporting production releases, conducting peer reviews, and overseeing the CI/CD pipeline implementation and service monitoring. Shift & On-Call Availability: Be flexible in working hours, ready to work in shifts and be on-call when needed. Requirements: To be successful in this role, you should meet the following criteria: Experience: 8-10 years of experience in software development, specifically in data analysis or big data projects. Technical Expertise: Strong skills in SpringBoot, Java 8, Kubernetes, Docker, Microservices, and PostgreSQL. Good understanding of CDP 7.1, HDFS filesystems, Unix, and Unix Shell Scripting. Knowledge of Elasticsearch, DevOps concepts, and related platforms. System Design: Proven experience in analyzing complex business requirements and translating them into effective system designs. Data Processing Expertise: Expertise in building and deploying production-level data processing batch systems supported by application support teams. Communication & Collaboration: Ability to work at pace, communicate clearly, and operate both independently and as part of a team. Desirable Skills: Good to have knowledge of Quantexa software, Angular, HTML, CSS, and JavaScript. Optional knowledge of cloud platforms. At HSBC, you'll have the opportunity to make a real impact. We support your career development, offer flexible working, and provide a collaborative, inclusive environment where you can thrive. Qualification : Person should have 8-10 years of experience in s/w development on data analysis or big data projects.

In this role, you will join one of IBM Consulting's Client Innovation Centers (Delivery Centers), where we deliver deep technical expertise and industry insight to both public and private sector clients globally. Our centers provide locally based skills to drive the innovation and adoption of new technologies. A career in IBM Consulting offers long-term relationships and close collaboration with clients worldwide. You ll work alongside visionaries across multiple industries to improve hybrid cloud and AI journeys for some of the world s most innovative and valuable companies. Your ability to accelerate impact and foster meaningful change is enabled by IBM's robust technology platforms, including software and Red Hat. Curiosity and a constant quest for knowledge are key to success, allowing you to challenge the norm, think creatively, and produce groundbreaking solutions for our clients. This environment promotes long-term growth, with ample career development opportunities. Your Role and Responsibilities: As a Big Data Engineer, you will be responsible for the design, maintenance, evaluation, and testing of big data solutions. Your key responsibilities include: Designing, building, optimizing, and supporting data models and ETL processes based on client business requirements. Building, deploying, and managing data infrastructure that can support the needs of a rapidly growing, data-driven organization. Coordinating data access and security to ensure seamless data access for data scientists and analysts when needed. Developing data pipelines/workflows for Source to Target and implementing solutions to address client needs. Ensuring high performance, scalability, and reliability of big data solutions. Required Technical and Professional Expertise: 3-5 years of experience working with Big Data technologies (Hadoop, Spark, HBase, Hive). Proficient in Scala and Python for data engineering tasks, including writing Pyspark programs for data analysis. Good working experience with Python to develop a custom framework for rule generation (similar to a rules engine). Developed Python code to gather data from HBase and implemented solutions using Pyspark. Strong knowledge of Apache Spark, including working with DataFrames and RDDs for business transformations. Experience using Hive Context objects for read/write operations in Hive. Preferred Technical and Professional Expertise: Understanding of DevOps principles and practices. Experience in building scalable end-to-end data ingestion and processing solutions. Familiarity with AWS services (e.g., S3, Athena, DynamoDB, Lambda, Jenkins). Proficiency in object-oriented and/or functional programming languages such as Python, Java, and Scala. As a Big Data Engineer at IBM Consulting, you will have the opportunity to work with cutting-edge technologies, influence industry transformations, and drive meaningful change for a variety of clients. IBM fosters a culture of continuous learning and career development, giving you the resources and support to grow in your role and within the company. This role offers the chance to work on complex, high-impact projects and collaborate with a global team of industry leaders.