Spark Jobs in Noida

Data Engineer Location: Noida Experience: 2 5 Years Employment Type: Full Time About Times Internet Times Internet is India s largest digital products company, building premium digital platforms across diverse sectors like News, Sports, Fintech, and Enterprise Solutions. Our brands such as TOI, ET, Cricbuzz, Times Prime, Willow TV, and Whatshot reach millions and are designed to simplify and enrich everyday experiences. We take pride in fostering an inclusive, innovative workplace. As an equal opportunity employer, we ve maintained gender pay parity (independently verified by Aon Hewitt) and are committed to building a diverse and equitable team. About the Business Unit ET B2B The Economic Times B2B Verticals (ETB2B) is a leading business media platform, delivering insightful content, newsletters, and premium conferences across 23+ sectors including Auto, Energy, Pharma, Retail, and HR. With a monthly reach of over 8 million professionals, ETB2B powers business decision-making and industry learning through platforms like ET Masterclass, vConfex, and the ET B2B Intent Signal Platform a cutting-edge initiative leveraging behavioral data, enrichment tools, and event-driven insights. Role Overview As a Data Engineer, you will work on building scalable data pipelines, deploying machine learning models, and extracting actionable buyer intent signals using a variety of data sources and advanced ML/NLP techniques. Key Responsibilities Design, develop, and maintain ETL/ELT pipelines to process large-scale structured and unstructured data (web behavior, CRM, third-party APIs, etc.). Build predictive models to extract buyer intent signals, leveraging NLP, behavioral analytics, and pattern recognition. Use modern ML and NLP tools: spaCy, Transformers, Scikit-learn, TensorFlow, PyTorch, and OpenAI APIs. Collaborate with product managers, backend engineers, and data scientists to deploy models into production and optimize performance. Ensure data pipeline reliability, compliance (GDPR, SOC 2), and freshness through robust monitoring and alerting systems. Enhance pipeline and model inference efficiency for near real-time scoring applications. Document workflows, model performance, and pipeline architecture for internal transparency and reproducibility. Required Skills & Qualifications Strong experience in Python for data engineering, ML pipelines, and model integration. Proficiency with ETL tools (Airflow, Prefect), data processing frameworks (Pandas, Dask), and workflow orchestration. Knowledge of machine learning and NLP techniques (entity extraction, sentiment analysis, text classification). Familiarity with ML model deployment via Docker, Kubernetes, and cloud ML services (AWS SageMaker, GCP AI Platform, Azure ML). Experience with relational databases (PostgreSQL, MySQL) and cloud data warehouses (BigQuery, Redshift, Snowflake). Strong grasp of version control (Git), CI/CD pipelines, and agile collaboration tools. Ability to interpret, visualize, and communicate model insights effectively working alongside analysts or through dashboards. Education & Experience Bachelor s or Master s degree in Computer Science, Data Science, Statistics, or a related technical field. 2 5 years of hands-on experience in data engineering, applied ML, or software development, ideally in SaaS, Martech, or B2B analytics domains. Be part of India s largest digital ecosystem. Work on impactful, high-scale products with real-world value. Collaborate with top minds in data, AI, and product development. Enjoy a culture of innovation, inclusion, and continuous growth. Qualification : Bachelors or Masters degree in Computer Science, Data Science, Statistics, or a related technical field

Job Summary: We are seeking a skilled Data Engineer with 2-5 years of industry experience to join our dynamic team. The ideal candidate will have a strong background in data engineering and a keen interest in the drug safety or pharmacovigilance domain. You will be responsible for designing, developing, and maintaining data pipelines and infrastructure to support our data-driven initiatives. Key Responsibilities: Design, develop, and maintain scalable data pipelines and ETL processes. Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions. Ensure data quality and integrity through rigorous testing and validation. Optimize and tune data processing systems for performance and scalability. Implement data governance and security best practices. Work with large datasets from various sources, including clinical trials, adverse event reports, and other pharmacovigilance data. Develop and maintain documentation for data engineering processes and systems. Qualifications: Bachelor s degree in computer science, Engineering, or a related field. 2-5 years of experience in data engineering or a related role. Proficiency in SQL and experience with relational databases (e.g., PostgreSQL, MySQL, Oracle RDBMS). Experience with big data technologies (e.g., Hadoop, Spark). Familiarity with data warehousing solutions (e.g., Redshift, Big Query). Strong programming skills in Python, Java, or Scala. Experience with cloud platforms (e.g., AWS, Azure, GCP). Knowledge of data modeling, ETL processes, and data integration. Experience in the drug safety or pharmacovigilance domain is a plus. Excellent problem-solving skills and attention to detail. Strong communication and collaboration skills. Qualification : Bachelors degree in computer science, Engineering, or a related field.

Gen AI Engineer Location: Noida Experience: 2+ years Openings: 2 Roles & Responsibilities: Build and optimize RAG (Retrieval-Augmented Generation) and Agentic AI workflows using LangChain, vector databases (Pinecone, FAISS, ChromaDB), and embedding models. Implement LLM fine-tuning, prompt chaining, and response ranking to enhance generative AI performance. Develop and evaluate retrieval mechanisms using semantic search and vector similarity techniques. Benchmark model performance using BLEU, ROUGE, METEOR, BERTScore, and other LLM evaluation metrics. Deploy AI models in production environments (AWS, GCP, Azure) with MLOps best practices for monitoring and retraining. Stay updated on the latest advancements in LLMs, transformers, and multi-agent AI architectures, applying new techniques to improve AI-driven applications. Required Skills: 2+ years of hands-on experience in Generative AI, RAG, and Agentic AI workflows, with expertise in LangChain, embedding models, and vector databases (Pinecone, FAISS, ChromaDB). Strong Python skills with experience in Hugging Face, PyTorch, TensorFlow, and scikit-learn for LLM fine-tuning and prompt engineering. Proficiency in benchmarking LLMs using BLEU, ROUGE, METEOR, and BERTScore, ensuring model quality and optimization. Experience in cloud-based deployments (AWS, GCP, Azure) and MLOps tools (MLflow, Weights & Biases, KubeFlow) for monitoring and retraining. Familiarity with containerized deployments (Docker, Kubernetes) and distributed computing (Ray, Spark) for scalable AI workloads. Strong problem-solving skills, with the ability to optimize LLM pipelines for latency, efficiency, and real-world applications.

Apache NiFi Developer Noida 3+ Years Experience Full-Time Location: Noida Experience Required: 3+ years Job Type: Full-Time Openings: 2 Job Overview: We are hiring a skilled and experienced Apache NiFi Developer to join our data engineering team. The ideal candidate will have hands-on expertise in developing and managing robust, scalable data pipelines using Apache NiFi, along with a strong background in system administration, data flow optimization, and secure data integration. Key Responsibilities: Design, develop, and maintain scalable data pipelines using Apache NiFi. Manage the daily operations of Apache NiFi environments, ensuring high availability and performance. Set up, configure, and maintain new NiFi environments across development, testing, and production. Monitor system health, troubleshoot issues, and ensure security and compliance. Implement and manage user access control, SSL, and LDAP integration. Optimize data flows and NiFi processors for maximum performance and reliability. Integrate NiFi with databases (SQL/NoSQL), cloud storage (S3, Azure Blob), Kafka, and SNS. Handle structured data formats like JSON, XML, CSV, Avro, and use tools like Jolt Transform. Develop and maintain backup and disaster recovery plans for NiFi environments. Document configurations, processes, troubleshooting steps, and best practices. Required Skills and Qualifications: 3+ years of experience with Apache NiFi development and administration. Deep understanding of NiFi architecture, processors, flow files, and controllers. Proficiency in designing and deploying data flows across complex systems. Experience with NiFi Toolkit, NiFi Registry, and NiFi Reporting Tasks. Familiarity with MiNiFi, shell scripting, Linux/Unix environments, and monitoring tools like Prometheus and Grafana. Working knowledge of REST APIs, SQL, and cloud platforms like AWS or Azure. Strong written and verbal communication skills. Experience with other data integration technologies like Apache Kafka or Apache Spark is a plus.

Join Adobe s Genuine Engineering Team as a Full Stack Developer! At Adobe, we re transforming the digital experience and empowering everyone from emerging artists to global brands to create and deliver exceptional content across every screen. We believe in the power of technology to unlock creativity, and we re on a mission to find the best talent to help us build the next generation of digital solutions. As part of our Genuine Engineering team, you ll play a critical role in safeguarding Adobe s intellectual property, strengthening product security, and combating piracy and fraud. You ll contribute to developing frontend C++ libraries for Adobe desktop products on Windows and macOS, while building robust backend services with Java, Spring, AWS, and Databricks. Your expertise will directly influence the security of Adobe s products, and your work will have a meaningful impact across the globe. What You ll Do: Frontend Development (C++): Develop and enhance features for the GoCart or product hardening libraries used in Adobe desktop applications. Ensure flawless integration of C++ libraries across Windows and macOS platforms. Optimize code for performance, memory usage, and stability while resolving technical challenges quickly and effectively. Backend Development (Java, Spring, AWS, Databricks): Build and optimize microservices using Java and Spring Boot to enhance backend systems. Develop cloud-based solutions with AWS services and create ETL pipelines in Databricks. Focus on improving the security, performance, and scalability of backend services. General Responsibilities: Take full ownership of your features, from design to deployment, ensuring they meet business needs and technical excellence. Work collaboratively with cross-functional teams to deliver innovative, impactful solutions. Continuously identify opportunities for improvement, and implement innovations to close system gaps. Contribute to Agile practices, including code reviews and following best practices. What You Need to Succeed: Educational Qualifications: Bachelor s or Master s degree in Computer Science, Software Engineering, or a related field (or equivalent experience). Professional Experience: 4+ years of hands-on experience in full-stack development. Technical Expertise: Frontend (C++): Proficiency in modern C++ (C++11/14/17/20) and solid understanding of object-oriented programming. Experience with cross-platform development on Windows and macOS. Ability to debug, profile, and optimize native code for performance. Backend (Java, Spring, AWS, Databricks): Expertise in Java and Spring Boot for building scalable microservices. Hands-on experience with AWS technologies such as S3, Lambda, DynamoDB, ECS, and more. Experience working with Databricks, Spark, and ETL data pipelines. Strong knowledge of SQL and NoSQL databases. General Skills: A strong understanding of computer science fundamentals, system design, and performance optimization. Demonstrated ability to analyze complex technical challenges and find effective solutions. Excellent written and verbal communication skills, with the ability to work both independently and collaboratively. Familiarity with Agile development methodologies and practices. By joining Adobe s Genuine Engineering team, you ll work on high-impact projects that directly affect Adobe s business and customers worldwide. Your contributions will help secure and protect Adobe s products, ensuring that we continue to provide cutting-edge, safe, and secure solutions to users globally. If you re passionate about working with C++ and Java technologies in a dynamic, high-impact environment, we d love to hear from you! Qualification : Bachelors or Masters degree in Computer Science, Software Engineering, or a related field, or equivalent professional experience.

Overview This is a Research Efficiency Platform team which provides leadership, direction, and accountability for Security Researchers tools and platforms, CICD, AI/ML, Virtualization and infrastructure, and secured development standards. You will help conduct business reviews and operate our production services. Strong collaboration skills will be required to work closely with other engineering teams to ensure services and systems are highly stable, performant, and meet the expectations of internal and external customers and users. We are looking for a Software Engineer to help us code and build large scale platforms, cloud services and engineering systems. You will work in collaboration with other Engineers, Researchers, Data Scientists and Architects to continue to learn and build your skill sets on how to innovate the design approach of said pipelines and services. Qualifications Qualifications Required Bachelor s degree in computer scienceor related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, SQL, C, C++, C#, Java, JavaScript, or Python OR equivalent experience. Support a 24x7 live site support model for the services the team owns. Qualifications: Other Requirements Master's Degree in Computer Scienceor related technical field AND 4+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR Bachelor's Degree in Computer Science or related technical field AND 6+ years technical engineering experience with coding in languages including, but not limited to, C, C++, C#, Java, JavaScript, or Python OR equivalent experience. Azure Cloud, DevOps and GitHub tooling Hands-on scripting knowledge in PowerShell Platform development experience for production environments Azure Data Explorer, Azure Data Lake, Azure Machine Learning, Azure Batch, HDI, Spark, Azure Synapse Excellent communication skills and ability to collaborate in a multi-disciplinary team consisting of Software Engineers, Program Managers, Security Researchers, and Data Scientists Development experience with cloud platforms such as Microsoft Azure Responsibilities As a Software Engineer, you would be responsible for: Design, develop,validateand implement the next generation of platform infrastructure that is highly secure, performant, resilient, modular, meets reliability targets, includes observability, and that enables next gen experiences. Take up end-to-end ownership of deliverables and produceviablesolutions. Collaborate with cross-functional teams to build scalable backend infrastructure services that deliver delightful customer experiences. Focus on building innovative user experiences and automating manual processes. Ensure the security, scalability, reliability, and performance of services in production. Activelyparticipatein code and design reviews tomaintainhigh-quality code and solutions. Write andmaintainunit tests and automated integration tests Responding to and mitigating any live site issuesimpactingcustomers Qualification : Bachelors degree in computer science or related technical field

Rotary Lead / Engineer Location: Noida Experience: 5 20 Years Industry: Chemical Process | Oil & Gas | Refinery | Petrochemical | Fertilizer Employment Type: Full-Time Role Overview: We are looking for a highly skilled Rotary Equipment Lead / Engineer with 5 to 20 years of experience in the detailed engineering and specification of rotary equipment for large-scale industrial projects. This role involves end-to-end engineering of equipment such as pumps, compressors, blowers, cooling towers, and agitators, primarily for the oil & gas, chemical, fertilizer, and petrochemical sectors. Key Responsibilities: Detailed Engineering of Rotary Equipment Execute detailed engineering and selection of rotary equipment including: Pumps Compressors Blowers Cooling Towers Agitators Ensure designs meet industry standards and project specifications for process plants. Documentation & Procurement Support Prepare comprehensive Mechanical Data Sheets for equipment selection and vendor communication. Create and review Material Requisitions (MR) and Purchase Requisitions (PR). Conduct Technical Bid Analysis (TBA) for vendor evaluation and selection. Required Skills & Qualifications: Bachelor s degree in Mechanical Engineering or equivalent 5 20 years of hands-on experience in rotary equipment engineering within relevant industries In-depth knowledge of rotary equipment design principles, performance characteristics, and operational considerations Experience in working with vendors and cross-disciplinary engineering teams Familiarity with industry standards (API, ASME, ISO) for rotating equipment is preferred Strong documentation, analytical, and communication skills Preferred Industry Background: EPC, Design, or Operations firms in: Oil & Gas Chemical & Petrochemical Refineries Fertilizers Other Process Plants If you re passionate about mechanical systems and ready to lead the design and execution of rotary equipment in complex industrial projects, we welcome your application. Qualification : Bachelors degree in Mechanical Engineering or equivalent