Apache Hadoop Jobs in Bengaluru

128 Jobs Found

TV

Backend Developer

Team Vunet Systems

2+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Backend Developer Location: Bengaluru, India Experience: 2 - 5 Years Job Type: Full-time About VuNet VuNet is pioneering Business Observability by combining big data and machine learning to revolutionize customer journey monitoring and enhance digital payment experiences for leading financial institutions. We foster financial inclusion nationwide by delivering innovative, scalable solutions in a collaborative, high-impact environment. Role Overview: Backend Developer Join VuNet s Platform Core OS team to build highly scalable backend systems powering critical products. As a Backend Developer, you ll have significant ownership and influence in shaping the architecture, collaborating across teams, and delivering world-class software solutions that impact millions of users. Key Responsibilities Own backend development and work closely with product, design, and business teams to architect and deliver scalable solutions Deliver major initiatives independently and on time Follow best development practices including design specs, coding standards, unit testing, and code reviews Solve complex service scalability problems with creativity and technical expertise Take complete ownership of your solutions and drive their success Communicate effectively across multidisciplinary teams Skills & Experience 2+ years of strong programming experience in Python Experience with asyncio and the Django framework Hands-on with web application development and working knowledge of SQL/NoSQL databases Experience with third-party integrations Strong communication and teamwork skills Detail-oriented with good analytical and problem-identification abilities Proven success as an individual contributor Bonus Skills: Elasticsearch, Kafka, Apache Spark, Redis, Cassandra, Hadoop Eagerness to learn new technologies and domains 100% health insurance coverage for you and your family, including parents Financial protection with disability, life, and accidental death insurance Mental wellness programs and one-on-one counseling A culture focused on delivering excellence and fostering career growth Transparent, inclusive, and collaborative work environment Continuous learning and development opportunities Apply now and be part of our trailblazing team!

Backend Developer Backend Developer Full-Time Backend Development
LA

Senior Analyst - Data Engineering

Latentview Analytics

3-6 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Role: Senior Analyst Data Engineering Location: Bengaluru, Karnataka, India Experience: 3 6 Years Employment Type: Permanent, Full-Time About the Role We are looking for a results-driven Senior Data Engineer to join our high-performing data team in Bengaluru. The ideal candidate will have 3 6 years of experience in data engineering, AI/ML implementation, and working with large-scale databases like Snowflake and Teradata. If you're passionate about driving data-powered insights, building scalable solutions, and applying advanced machine learning and AI techniques, we want to hear from you. Key Responsibilities Design, develop, and implement machine learning models to solve complex business challenges. Apply AI techniques, including generative AI, NLP, and computer vision, to improve analytics capabilities. Use Tableau, Power BI, and other tools to develop insightful, interactive data dashboards. Manage and optimize large datasets using platforms like Snowflake, Teradata, and SQL/NoSQL databases. Collaborate with business and technical teams to translate requirements into robust data engineering solutions. Guide junior data professionals and foster a culture of learning and innovation. Communicate analytical findings clearly to non-technical stakeholders. Stay current with the latest in data science, machine learning, cloud platforms, and big data technologies. Key Skills & Technologies Machine Learning & AI Techniques: Supervised & unsupervised learning, deep learning, neural networks Reinforcement learning, decision trees, random forests, clustering NLP, computer vision, GANs, transfer learning Data Visualization Tools: Tableau, Power BI, Matplotlib, Seaborn, Plotly Programming Languages & Libraries: Python (essential), R, SQL TensorFlow, PyTorch, scikit-learn, pandas, NumPy, Keras, SciPy Databases & Data Management: Snowflake, Teradata, SQL/NoSQL, ETL, data lakes, data warehousing Cloud Platforms: AWS, Azure, Google Cloud Platform (GCP) Big Data Technologies: Apache Spark, Hadoop

Senior Analyst Senior analyst Data Data analyst
CI

Data Engineer

Colan Infotech

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Data Engineer Experience: 5+ Years Location: Bangalore, Karnataka, India Job Type: Full Time Job Summary We are looking for a talented Data Engineer with over 5 years of experience and a strong foundation in machine learning development to join our team in Bangalore. The ideal candidate will have hands-on expertise in Python programming, machine learning basics, and computer vision techniques like custom object detection and OCR. Key Responsibilities Develop and maintain data pipelines supporting machine learning models and data analytics. Implement custom object detection algorithms and OCR solutions using computer vision techniques. Utilize Python ML libraries such as OpenCV, SciPy, NumPy, Matplotlib, Pandas, Scikit-learn, Keras, PyTorch, and TensorFlow. Collaborate with data scientists and software engineers to optimize data workflows and ML model deployment. Ensure data quality, integrity, and scalability within data infrastructure. Troubleshoot and improve existing machine learning systems and pipelines. Required Skills Minimum of 5 years of experience in data engineering or related roles. At least 2 years of hands-on experience as a Machine Learning developer. Strong programming skills in Python. Solid understanding of machine learning fundamentals. Practical experience with custom object detection and OCR applications. Proficiency in key Python libraries for machine learning and data processing. Ability to work collaboratively in cross-functional teams. Qualifications Any graduate degree from a recognized university. Work with cutting-edge machine learning technologies in a supportive, innovative environment in Bangalore. Grow your career by solving complex problems and building impactful data solutions with a passionate team. Qualification : Any graduate degree from a recognized university.

Data Engineer Data Engineer Full-Time Data Pipeline
ME

Data Scientist -i

Meesho

1-2 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Data Scientist - I | Meesho Careers Location: Bangalore, Karnataka | Department: Tech Join Meesho as a Data Scientist and Shape the Future of E-commerce At Meesho, our Data Scientists work on high-impact projects such as fraud detection, inventory optimization, and platform vernacularization, building intelligent systems that enhance the experience of millions of users across thousands of product categories. What You ll Do Design and develop advanced data models and run experiments to derive insights from complex datasets Enhance product usability and identify new growth opportunities through data Analyze reseller behavior to recommend relevant products and increase engagement Design and optimize discount programs to boost reseller sales Identify end-customer preferences to help resellers improve revenue Use analytics to identify supply chain bottlenecks and help suppliers meet SLA requirements Forecast seasonal demand and contribute to strategic planning by modeling key metrics Mentor junior data scientists and help foster a collaborative learning environment Bachelor's or Master s degree in Computer Science, Data Science, or related fields 1 2 years of professional experience as a Data Scientist in a fast-paced, preferably B2C environment Hands-on experience with machine learning, neural networks, and related techniques Proficiency in tools like SQL, Python, R Strong statistical foundation, with expertise in hypothesis testing and model validation Proven ability to conduct and analyze A/B testing Experience working closely with product and tech teams Bonus Points If You Have: Experience in personalization or similar ML problem spaces Familiarity with Big Data technologies such as Apache Spark, Hadoop, or Redshift About Meesho Meesho is India's top-rated e-commerce platform on Glassdoor, known for empowering millions of entrepreneurs across the country. We provide sellers with zero-commission benefits and the lowest shipping costs, serving every serviceable pincode in India. With over 1.75 million registered sellers, Meesho is helping small businesses grow using a unique business model and world-class tech infrastructure. From first-time internet users to seasoned entrepreneurs, our inclusive platform caters to a diverse customer base across India. Our Mission Democratizing Internet Commerce for Everyone Meesho (short for Meri Shop) is dedicated to enabling 100 million small businesses to succeed online, offering them an affordable, reliable, and easy-to-use platform that mirrors local market preferences. Our Culture and Total Rewards We cultivate a performance-driven, people-first culture that values learning, creativity, and excellence. Our "Mantras" guide our daily decisions, from hiring to growth discussions. Meesho Total Rewards includes: Market-leading compensation, including equity-based benefits Holistic wellness through our MeeCare Program: medical insurance, telehealth, wellness events, gym discounts, and more Work-life balance support: generous leave policies, parental benefits, and retirement plans Learning & development assistance and career mobility programs Fun workplace culture with events, personalized gifts, and team engagement activities Join Meesho to work on meaningful data problems, impact millions, and grow your career with a supportive and innovative team. Qualification : Bachelor's or Masters degree in Computer Science, Data Science, or related fields

Data Scientist Data scientist I Full-Time
ME

Data Scientist Ii

Meesho

2-4 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Data Scientist II Join the Meesho Tech Team Location: Bangalore, Karnataka | Department: Technology Empower the Future of E-Commerce with Meesho We are looking for an experienced Data Scientist II to join our Avengers-like Data Science team at Meesho. As part of our mission to revolutionize e-commerce in India, we solve complex problems related to fraud detection, inventory optimization, and localized platform experiences that impact millions of users daily. As a mid-level Data Scientist, you ll partner with stakeholders across teams to create data-driven strategies, improve platform performance, and lead initiatives that enable better business outcomes for resellers and end users across Bharat. What You ll Do Design and build predictive models and machine learning solutions using large-scale data Conduct experiments and extract insights from high-volume structured and unstructured data Enhance product usability and surface new growth opportunities through advanced analytics Analyze reseller behavior to deliver personalized product recommendations Design effective discount programs to increase reseller sales performance Identify customer preferences to help resellers better serve their buyers and drive revenue Uncover supply chain inefficiencies and support SLA adherence through data insights Forecast seasonal and regional demand to optimize key business metrics Mentor junior data scientists and contribute to team knowledge-sharing What You ll Need Bachelor s or Master s degree in Computer Science, Data Science, or a related technical field 2 4 years of experience in a Data Scientist role, ideally within a fast-moving B2C tech environment Strong knowledge of Machine Learning algorithms, Neural Networks, and data modeling techniques Hands-on experience with SQL, Python, and R for data manipulation and analysis Strong understanding of statistics, linear algebra, and A/B testing methodologies Demonstrated ability to turn business problems into data science solutions Experience working with product and engineering teams to deliver scalable systems Preferred Qualifications Prior experience solving personalization or recommendation engine problems Familiarity with Big Data technologies like Apache Spark, Hadoop, or Amazon Redshift About Meesho Meesho is India's top-rated e-commerce workplace, revolutionizing online commerce for the next billion users. We empower over 1.75 million sellers through an inclusive, technology-driven platform that offers zero-commission selling, the lowest logistics costs, and access to a massive customer base across every serviceable pincode in India. Our Mission Democratizing Internet Commerce for Everyone Meesho (Meri Shop) aims to help 100 million small businesses succeed online. With relatable merchandise and localized user experiences, we re building a truly inclusive digital shopping ecosystem for underserved markets in India. Our Culture and Benefits Meesho fosters a high-performance and people-first culture. Our workplace philosophy is shaped by 11 guiding principles ("Mantras") and practices like Reflections, Listen or Die, and a robust Internal Mobility Program. Our Total Rewards Include: Competitive compensation packages including performance-based equity Comprehensive wellness through our MeeCare Program covering medical, mental, and financial well-being Flexible leave policies and parental support benefits to ensure work-life balance Relocation support, salary advances, and learning & development assistance Fun, collaborative culture with team engagement events, gifts, and recognition programs We are not just building a company; we re creating a movement. If you re driven by data and inspired to make a difference, come be a part of Meesho s journey to redefine e-commerce in India. Apply today and let's build the future of commerce, together. Qualification : Bachelors or Masters degree in Computer Science, Data Science, or a related technical field

Data Scientist Data scientist Ii Data Scientist Ii
ME

Data Scientist III

Meesho

4-7 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Data Scientist III Advanced Analytics Role at Meesho Location: Bangalore, Karnataka | Department: Tech About the Data Science Team at Meesho Our Data Science team is the engine driving intelligent decision-making at Meesho. Known internally as the Avengers to our S.H.I.E.L.D, we tackle mission-critical challenges using cutting-edge data science to empower millions of users and small businesses across Bharat. From fraud detection and inventory optimization to platform vernacularization, we re shaping the future of e-commerce in India through data innovation. As a Data Scientist III, you'll be solving complex problems in a rapidly evolving market. You ll lead initiatives that translate data into strategic insights and scalable models, impacting business across various functions. About the Role If you thrive on solving high-impact data problems and love deriving actionable solutions from large datasets, you may be the next Data Scientist III at Meesho. Your primary responsibility will be to increase the utility of business data through experimentation, modeling, and strategic insights. This role involves significant collaboration with cross-functional teams, mentorship, and ownership of critical analytics workflows. What You ll Do Build and deploy machine learning models to extract insights from complex data Run experiments to test hypotheses and evaluate product performance Drive product usability improvements and uncover new growth opportunities Analyze reseller preferences and personalize product offerings Create effective discount strategies to support reseller success Enhance understanding of end-user behavior for improved targeting Uncover supply chain inefficiencies and improve SLA performance Predict seasonal and regional demand to guide planning decisions Mentor junior data scientists and contribute to team development What You ll Need Bachelor s or Master s degree in Computer Science, Data Science, or a related technical field 4 7 years of experience as a Data Scientist, ideally in a fast-paced B2C company Proficiency with Neural Networks, supervised/unsupervised learning, and ML frameworks Expertise in Python, SQL, and R for data analysis and model development Strong knowledge of statistics, linear algebra, and model validation techniques Experience in designing, running, and analyzing A/B tests Exceptional problem-solving and communication skills Experience working closely with product and engineering teams Bonus Points For: Work on personalization systems or recommendation algorithms Familiarity with Big Data technologies like Apache Spark, Hadoop, or Redshift About Meesho Meesho is India s leading social commerce platform, empowering over 1.75 million sellers to grow their businesses online. We offer a unique business model with zero commission fees and the lowest shipping costs, supported by a powerful tech ecosystem and pan-India logistics network. Our platform is built for first-time internet users and small business owners alike, democratizing e-commerce across every corner of the country. Our Mission Democratizing Internet Commerce for Everyone Meesho s goal is to enable 100 million small businesses to succeed online. With relatable, affordable merchandise and innovative seller support tools, we help entrepreneurs reach new customers and scale efficiently. Our Culture and Total Rewards At Meesho, we cultivate a high-performance, people-first culture supported by our 11 guiding Mantras. We prioritize continuous learning, collaboration, and transparency, and back it up with industry-leading employee experiences. Our Total Rewards Include: Market-leading compensation with both cash and equity components Access to MeeCare Program for physical, mental, financial, and social wellness Comprehensive medical coverage for employees and families Parental benefits, generous leave policies, and retirement plans Support for learning and career development through structured programs Employee engagement, flexible benefits, relocation assistance, and much more We re more than just a company we re a movement redefining online commerce in India. If you re a passionate data expert ready to lead high-impact initiatives, apply now and become a part of Meesho s incredible journey. Qualification : Bachelors or Masters degree in Computer Science, Data Science, or a related technical field

Data Scientist Data scientist Full-Time Data Scientist III
IN

Sr. Data Engineer- Aws- Big Data

Infocepts

7-10 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Sr. Data Engineer - AWS - Big Data Location:Bangalore Type of Employment: Full-Time Experience Required: 7 to 10 years Job Overview: We are seeking a highly skilled Sr. Data Engineer with expertise in AWS cloud technologies and Big Data to join our Cloud Data Architect Team at Infocepts. In this critical role, you will design and implement robust data solutions using technologies like EMR, Athena, PySpark, AWS Lambda, S3, and other AWS services. The ideal candidate will have a strong foundation in database concepts and SQL and will be responsible for building scalable data pipelines to support high-performance data processing. Key Responsibilities: Technology Assessment and Design: Study the existing technology landscape and evaluate current data integration frameworks. Assist in designing complex Big Data use cases leveraging AWS services. Documentation and Stakeholder Communication: Prepare and maintain comprehensive project documentation, adhering to quality guidelines and schedules. Work closely with Architects and Project Managers to provide accurate estimations, scoping, and scheduling assistance. Clearly communicate design decisions and conduct Proof-of-Concepts to validate new solutions before implementation. Process Improvement and Automation: Identify areas for process automation to improve efficiency and team productivity. Provide expert guidance and troubleshooting support to junior Data Engineers. Training and Knowledge Sharing: Develop and deliver technology-focused training sessions for the team, ensuring continuous knowledge sharing. Share expertise through Expert Knowledge Sharing sessions with Client Stakeholders. Essential Skills: AWS Services Expertise: In-depth knowledge of S3, EC2, EMR, Athena, AWS Glue, and Lambda. Big Data Technologies: Proficiency with Apache Spark, Databricks, and Big Data table formats such as Delta Lake (open-source). Data Warehousing: Strong understanding of data warehousing concepts and architectures. Programming Skills: Advanced programming skills in Python for building data pipelines. SQL Expertise: Strong SQL skills for data transformation, aggregation, and querying large datasets. ETL Workflow Development: Expertise in creating ETL workflows with complex transformations (e.g., SCD, deduplication, aggregation). Orchestration Tools: Familiarity with orchestration tools like Apache Airflow. MPP Databases: Experience with at least one MPP database (e.g., AWS Redshift, Snowflake, SingleStore). Cloud Databases: Exposure to cloud databases like Snowflake or AWS Aurora. Desirable Skills: Cloud Databases: Familiarity with Snowflake, AWS Aurora. Big Data Technologies: Experience with Hadoop and Hive. AWS Certification: Associate or Professional Level AWS Certification. Advanced Knowledge of Big Data Solutions: Exposure to big data tools and frameworks on cloud platforms. Qualifications: Experience: 7+ years of overall IT experience, with 5+ years specifically focused on AWS-related projects. Educational Background: Bachelor's degree in Computer Science, Engineering, or a related field (Master's degree is a plus). Technical Certifications: Demonstrated commitment to continuous learning through certifications or relevant training. Qualities: Strong analytical and problem-solving skills to deep dive into complex technical challenges.

Sr. Data Engineer Sr. engineer Data Engineer
TE

Sr. Data Engineer

Trellissoft Engineering Services Pvt Ltd

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Job Title: Data Engineer Location: Bengaluru, Karnataka Experience: 5 to 8 Years Work Modality: Full-time (Work from office) Job Description: We are looking for an experienced Data Engineer to join our team and take responsibility for designing, developing, and maintaining scalable ETL/ELT pipelines. This is a full-time position based in Bengaluru, Karnataka, and you will be collaborating with cross-functional teams to define data requirements and ensure data accuracy, consistency, and integrity. Your role will also involve optimizing data workflows, automating processes, and ensuring high availability and reliability of data pipelines. Key Responsibilities: ETL/ELT Pipeline Development: Design, develop, and maintain scalable ETL/ELT pipelines to support data transformation and integration processes. Data Warehouse & Data Lake Optimization: Build and optimize data warehouses, data lakes, and real-time streaming solutions to support large-scale data operations. Collaboration & Data Requirements: Collaborate with cross-functional teams, such as product, data science, and analytics teams, to define data requirements and ensure data accuracy and consistency. Database Structure & Schema Management: Develop and maintain database structures and schemas to ensure efficient data storage and retrieval. Data Workflow Optimization: Optimize data workflows for performance, reliability, and scalability, ensuring the highest level of efficiency. Data Security & Compliance: Implement data security, governance, and compliance best practices to ensure that data is handled securely and meets industry standards. Pipeline Monitoring & Troubleshooting: Monitor, troubleshoot, and improve data pipelines to ensure uptime, reliability, and smooth data processing. Process Automation: Automate data-related processes to improve efficiency and reduce manual intervention, increasing the overall speed of data flow. Required Qualifications: Experience: 5+ years of experience in data engineering or 3-4 years of experience as a Data Engineer. Technical Skills: Strong proficiency in SQL and database management systems such as PostgreSQL, MySQL, SQL Server, etc. Experience with ETL tools such as Pentaho, Talend, Cdata, and SSIS. Exposure to Python, Java, or Scala for data processing is a plus. Experience with big data technologies such as Apache Spark, Hadoop, or Kafka. Familiarity with cloud services (AWS, Azure) and data storage solutions such as S3, Redshift, Snowflake, or BigQuery. Strong knowledge of data modeling, warehousing concepts, and data architecture best practices. Soft Skills: Excellent communication skills with the ability to collaborate effectively across teams. Strong problem-solving skills and the ability to work with large, complex datasets. What We Offer: Competitive Salary: Attractive salary based on experience and expertise. Collaborative Work Environment: Work in a dynamic and fast-paced environment with a team that fosters innovation and collaboration. Growth Opportunities: Opportunities to enhance your skills and career growth in the data engineering field. Comprehensive Benefits: Benefits package designed to support work-life balance and overall employee well-being.

Sr. Data Engineer Sr. engineer Data Engineer
HT

Technology Leader

Hashedin Technologies Pvt. Ltd.

12-15 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Role: Senior Delivery Leader / Practice Head Experience: 12 to 15 years Education: B.E / B.Tech / M.Tech in Computer Science, Information Technology, or equivalent Key Responsibilities: Project Leadership: Lead and successfully deliver new and complex projects. Directly oversee project delivery, ensuring adherence to scope, timelines, and quality. Conduct regular project reviews to assess progress, risks, and client satisfaction. Technology Practice Leadership: Build and scale a technology practice, driving innovation and digital capability development. Lead technical initiatives and build cutting-edge solutions that create value for customers. Drive automation in project delivery and create reusable IPs. Client Relationship Management: Act as a trusted advisor to client CIOs and senior stakeholders. Manage relationships across the pre-sales and post-sales lifecycle to drive client success. Manage and deliver superior value to key accounts, ensuring client satisfaction and retention. Thought Leadership & Innovation: Identify emerging industry needs and proactively design exceptional solutions. Bring innovative ideas to the table, identifying opportunities others may miss. Author whitepapers, present at conferences, and publish technical blogs to strengthen HashedIn s brand. People Leadership & Capability Building: Coach and mentor teams, providing clear direction and ownership. Develop the architecture and technology skills of the team, fostering a high-performance culture. Help define career paths and actively support team members' growth and progression. Desired Profile: Education & Experience: Engineering graduate (CS/IT) B.E/B.Tech/M.Tech. 12-15 years of hands-on experience in Java/JEE technologies such as Struts, JSF, Spring, EJB, JMS. Key Technical Skills: Java/J2EE expertise with a strong focus on performance engineering. Solid understanding of end-to-end technical architecture for large-scale systems. Experience reviewing existing architectures and identifying risks, trade-offs, and optimizations. Architectural and Cloud Expertise: Knowledge of infrastructure sizing, capacity planning, performance modeling, and cloud computing (AWS/Azure/GCP). Performance optimization experience across popular application servers like WebLogic, WebSphere, JBoss, with deep expertise in at least one. Technical Documentation & Tools: Skilled in creating/reviewing technical documents like Architecture Blueprints, Design Specs, Deployment Architecture, and Workload Models. Experience with performance testing tools such as Apache JMeter, HP LoadRunner, or MS VSTS Load Test. Broader Technology Exposure: Exposure to Agile methodologies and CI/CD tools. Experience with Big Data/Analytics platforms like Apache Kafka, Storm, Hadoop, or libraries like Libuv (Node.js). Leadership & Business Acumen: Proven track record in building technology teams from the ground up. Strong pre-sales experience, with a key role in winning new business alongside the sales team. Consultative mindset, with the ability to understand complex enterprise environments and propose tailored solutions. Effective in working within a multicultural environment and managing diverse teams. Strong communication, presentation, and interpersonal skills. High energy, with the ability to thrive under pressure and navigate ambiguity. Industry Awareness: Up-to-date with latest technology trends and evolving industry challenges. Demonstrated ability to handle complex, multicultural projects in dynamic environments. Strong professional and personal integrity, with the assertiveness to present clear points of view.

Technology Leader Technology leader Full-Time Technical leadership
ND

Sr. Engineering Manager

Ness Digital Engineering

13-16 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Job Title: Sr. Engineering Manager - Data Engineering Level: L5 Experience: 13-16 years Overview We are seeking an experienced Engineering Manager with a strong background in Data Engineering, including ETL/ELT processes and cloud-based data platforms such as Snowflake. The ideal candidate will lead and mentor a team of data engineers, drive data architecture initiatives, and work closely with cross-functional stakeholders to ensure our data infrastructure supports evolving business needs. Key Responsibilities Team Leadership: Lead, mentor, and develop a high-performing data engineering team, fostering a culture of collaboration, innovation, and continuous learning. Data Pipeline Development: Oversee the design, development, and maintenance of robust ETL/ELT pipelines to ingest, transform, and process data at scale. Cloud Data Infrastructure: Drive the architecture and implementation of cloud-based data solutions, especially leveraging Snowflake, ensuring scalability, security, and reliability. Cross-Functional Collaboration: Partner with product managers, analysts, data scientists, and other business stakeholders to gather requirements and prioritize engineering efforts that deliver the most impact. Architecture and Design: Develop and enforce data architecture standards for high-performance data warehousing, ensuring seamless data integration across diverse sources. Performance Optimization: Identify and resolve performance bottlenecks, focusing on query optimization, cost management, and resource efficiency. Data Quality & Governance: Define and implement data quality frameworks and governance practices, ensuring data consistency and reliability across all pipelines. Innovation & Strategy: Stay informed on emerging data technologies and industry best practices, continuously improving processes and aligning solutions with long-term data strategies. Required Skills 8+ years of hands-on experience in data engineering, including 5+ years in a leadership role. Strong expertise in ETL/ELT processes and hands-on experience with tools like Talend, Informatica, or similar platforms. Deep proficiency in Snowflake or comparable cloud data platforms such as Redshift or BigQuery. Advanced SQL skills, including query optimization, performance tuning, data modeling, and schema design. Hands-on experience with Python or Java for data processing and automation. Knowledge of data governance, compliance standards, and data security best practices. Excellent communication and project management skills, with the ability to prioritize and manage multiple projects in parallel. Preferred Skills Exposure to Big Data technologies such as Spark, Hadoop, Databricks, Synapse, etc. Experience with workflow orchestration tools like Apache Airflow or AWS Step Functions. Familiarity with CI/CD pipelines and DevOps practices within data engineering. Experience working with BI tools like Tableau or Power BI, and reporting integrations.

Sr. Engineering Manager Sr. manager Engineering manager
CO

Data Engineer

Couchbase

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Job Title: Data Engineer Location: Bengaluru (Hybrid) About Couchbase: As industries race to embrace AI, traditional database solutions fall short of rising demands for versatility, performance, and affordability. Couchbase is leading the way with Capella, the developer data platform for critical applications in our AI world. By uniting transactional, analytical, mobile, and AI workloads into a seamless, fully managed solution, Couchbase empowers developers and enterprises to build and scale applications with unmatched flexibility, performance, and cost-efficiency from cloud to edge. Trusted by over 30% of the Fortune 100, Couchbase is unlocking innovation, accelerating AI transformation, and redefining customer experiences. Come join our mission! Position Overview: We are seeking a highly skilled Data Engineer with a strong background in data warehousing, advanced analytics, and AI integration. The ideal candidate will have 5+ years of experience designing and implementing data pipelines, building robust analytics platforms, and working with modern data technologies like Snowflake, Google Cloud, Databricks, Looker, PowerBI, and Tableau. As part of our Delorean team, you will play a pivotal role in driving innovation and ensuring our data infrastructure supports strategic decision-making. Key Responsibilities: Data Infrastructure Development: Design, develop, and maintain scalable data pipelines and ETL processes to support data ingestion, transformation, and storage across multiple platforms. Experience with Cloud Data Platforms: Expertise in implementing and optimizing solutions using industry-leading tools such as Snowflake, Databricks, and Google Cloud Platform. Demonstrated ability to ensure high performance, scalability, and cost-effectiveness in data infrastructure and workflows. Analytics & Visualization: Build and maintain interactive dashboards and reports using Looker, PowerBI, and Tableau to provide actionable insights to business stakeholders. AI & Advanced Analytics: Collaborate with Data Science teams to integrate AI models and machine learning workflows into data pipelines for predictive analytics and automation. Data Governance: Ensure data quality, security, and compliance with industry best practices and regulatory requirements. Collaboration: Partner with cross-functional teams, including Product, Engineering, and Business Operations, to understand data requirements and deliver tailored solutions. Continuous Improvement: Identify opportunities to improve data workflows, adopt new technologies, and drive operational efficiency. Qualifications: Education: Bachelor s or Master s degree in Computer Science, Data Engineering, or a related field. Experience: 5+ years of hands-on experience in data engineering, data warehousing, or related fields. Technical Skills: Expertise in Snowflake, Databricks, and Google Cloud Platform (Mandatory experience in at least one tool). Proficiency in ETL/ELT processes and data pipeline tools (e.g., Apache Airflow, dbt). Experience with data visualization tools like Looker, Power BI, and Tableau. Strong SQL skills and familiarity with Python or Scala for data processing. Knowledge of AI/ML concepts and frameworks for integrating predictive models. Soft Skills: Excellent problem-solving skills with a proactive mindset. Strong communication and collaboration skills to work effectively with technical and non-technical teams. Ability to manage multiple priorities in a fast-paced environment. Modern customer experiences need a flexible cloud database platform that can power applications spanning from cloud to edge and everything in between. Couchbase s mission is to simplify how developers and architects develop, deploy, and consume modern applications. With Capella, our flexible, affordable cloud platform, we empower organizations to quickly build and deliver premium customer experiences with unmatched price-performance. More than 30% of the Fortune 100 trust Couchbase to power their modern applications. Benefits at Couchbase: Generous Time Off Program: Flexibility to care for yourself and your family. Wellness Benefits: Comprehensive medical plans, dental, vision, life insurance, and employee assistance programs. Financial Planning: RSU equity program, ESPP, retirement planning, and business travel insurance. Career Growth: A Be valued, Create value approach to your career development. Fun Perks: Ergonomic office setup, food & snacks for in-office employees, and more! Qualification : Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.

Data Engineer Data Engineer Full-Time Data Engineering
IS

Data Engineer

Indium Software

Fresher | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Data Engineer Role Overview We are looking for a Data Engineer to design, develop, and maintain scalable data pipelines and ETL processes. You will work closely with data scientists, analysts, and engineers to ensure efficient data processing and storage solutions that support business intelligence and analytics needs. This role requires expertise in SQL, big data technologies, and cloud platforms to build and optimize data workflows. Key Responsibilities Data Pipeline Development & Optimization Design and build scalable ETL/ELT processes for structured and unstructured data. Develop and maintain data ingestion frameworks to handle large datasets efficiently. Ensure data integrity, consistency, and security across multiple sources. Database & Data Warehouse Management Develop, optimize, and maintain relational and NoSQL databases. Implement data modeling best practices for performance and scalability. Work with cloud-based data warehouses (e.g., Snowflake, Redshift, BigQuery). Big Data & Cloud Technologies Leverage big data tools (e.g., Spark, Hadoop, Databricks) for data processing. Work with cloud platforms (e.g., AWS, Azure, GCP) to build and optimize data solutions. Develop real-time and batch processing workflows. Collaboration & Documentation Work closely with data scientists, engineers, and business teams to understand data requirements. Document data pipelines, architecture, and workflows for scalability and maintenance. Required Qualifications & Skills Technical Expertise: Strong experience with SQL and Python for data processing. Proficiency in ETL/ELT frameworks and data integration techniques. Hands-on experience with big data tools (e.g., Apache Spark, Hadoop, Kafka). Cloud & Database Management: Expertise in cloud data platforms (Azure, AWS, GCP). Experience with data warehousing solutions (Snowflake, Redshift, BigQuery). Understanding of data governance, security, and compliance. Performance Optimization & Troubleshooting: Ability to optimize SQL queries and improve data processing efficiency. Experience troubleshooting complex data pipeline issues. Apply Now & Be Part of an Innovative Data Team!

Data Engineer Data Engineer Full-Time Data pipeline development
OK

Senior Data Engineer

Okta

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Senior Data Engineer Enterprise Data Platform Location: Bengaluru Department: Business Technology Data Engineering Experience: 5+ Years Employment Type: Full-Time About Okta Okta is The World s Identity Company. We empower people to securely use any technology, anywhere, on any device. Through our Okta and Auth0 platforms, we provide secure access, authentication, and automation placing identity at the center of security and growth for thousands of organizations. We value diverse perspectives and lifelong learners. We re not looking for someone who checks every box we re looking for someone who will make us better with their unique experiences. Team: Business Technology Data Engineering The Data Engineering team at Okta supports cross-functional partners by building scalable, secure, and high-performing platforms. These platforms power decision-making and business processes across sales, marketing, engineering, finance, product, and operations. As part of this team, you ll contribute to data solutions that fuel Okta s hyper-growth. You will have the opportunity to work with cutting-edge technologies in cloud infrastructure, data lakes, automation, and CI/CD pipelines. The Role: Senior Data Engineer As a Senior Data Engineer, you will design, build, and manage modern data pipelines, infrastructure, and automation frameworks. You ll help scale our enterprise data platform using tools such as Snowflake, dbt, Airflow, Databricks, and AWS, while ensuring security, observability, and performance. You ll also contribute to CI/CD pipelines, infrastructure as code (IaC), and secure development lifecycle practices, enabling consistent, efficient, and secure delivery of data solutions. Key Responsibilities Platform Development & Infrastructure Design and maintain scalable data pipelines and platforms using Snowflake, AWS, Databricks, dbt, and Airflow. Manage infrastructure with Terraform, enabling repeatable and consistent deployments. Develop and maintain robust CI/CD pipelines using GitHub Actions, GitLab, or Jenkins. Containerize data services using Docker for better scalability and portability. Security & Compliance Implement and enforce secure development lifecycle practices, integrating tools like DAST, SAST, SCA, and Secret Scanning into pipelines. Conduct vulnerability scanning and apply patches to ensure system integrity. Ensure data security and compliance with industry standards and regulations. Collaboration & Innovation Collaborate with data engineers, data scientists, and analysts across business units to ensure data availability and integrity. Identify opportunities for automation and optimization within the data platform. Stay updated on emerging technologies and drive adoption of best practices. Must-Have Skills Bachelor s degree in Computer Science, Engineering, or a related technical field. 5+ years of experience in data engineering, including: Advanced SQL and ETL development with Airflow and dbt. Experience with data warehouses such as Snowflake, Redshift, or BigQuery. Strong hands-on experience with AWS (S3, Lambda, EC2, EMR, EKS). 2+ years of experience managing CI/CD pipelines using tools like GitHub Actions, GitLab, Jenkins, or ArgoCD. Experience with Terraform and Docker. Proficiency in backend languages such as Python, Java, or Go. Preferred Skills Experience with lakehouse architectures like Databricks, including knowledge of Delta Lake and Apache Iceberg. Background in infrastructure security, vulnerability management, and observability tooling. High Impact: Help build and scale the data platform that powers Okta s global business. Cutting-Edge Stack: Work with best-in-class technologies like AWS, Snowflake, dbt, Terraform, and Databricks. Collaborative Culture: Join a diverse, inclusive, and globally distributed team that values knowledge sharing and continuous learning. Career Growth: Shape the future of Okta s data engineering practice while expanding your technical and leadership skills. Bring your passion for data, cloud, and automation and let s shape the future of secure, scalable enterprise data platforms together. Qualification : Bachelors degree in Computer Science, Engineering, or a related technical field

Senior Data Engineer Senior engineer Data Engineer
TV

Java Developer

Team Vunet Systems

3-6 Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Java Developer Software Engineer Experience: 3 6 years Location: Bengaluru Department: Product Development Join VuNet Build the Future of Digital Observability VuNet is pioneering a new category in Business Journey Observability, combining Big Data and Machine Learning to transform digital experiences in financial services. We monitor over 28 billion transactions monthly, supporting 300+ million users across major banks and financial institutions (e.g., UPI, real-time payment systems). VuNet is Series B funded, part of NASSCOM DeepTech Club, and recognized by Forbes, Gartner, and others for its innovation in AI and observability. About the Role We are looking for a Java Developer to join our Product Development Team, focused on building lightweight, high-performance Java agents that collect observability data from real-world production systems. Your work will directly contribute to improving digital journeys for millions of users. If you thrive in a fast-paced environment, enjoy solving complex problems, and want to be part of a cutting-edge platform built in India for the world this is your opportunity. Your Responsibilities Design and develop scalable Java-based agents to collect data from JVMs and other systems in production environments. Integrate with various systems using protocols like TCP, HTTP, and REST APIs. Maintain and enhance existing agents: debug, analyze root causes, and collaborate with customer-facing teams for fixes. Ensure clean, efficient code through unit testing, design documentation, coding standards, and peer reviews. Take full ownership of the development lifecycle from requirement analysis to production deployment. Collaborate with Product, Platform, and Support teams to build robust, real-world-ready solutions. Contribute to broader Java development efforts (e.g., platform modules, service components, custom adapters) as needed. What You Bring Must-Have Skills 3 6 years of hands-on Java development experience. Strong understanding of multithreading, file handling, and socket programming. Experience working with Java Management Extensions (JMX). Proficiency in consuming REST APIs with Java. Strong debugging skills and ability to resolve complex production issues. Excellent communication and documentation skills. Self-motivated, detail-oriented, and collaborative mindset. A growth-driven mindset with curiosity to learn new technologies and domains. Nice-to-Have Skills Experience with Java application servers like Tomcat, JBoss, or Jetty. Familiarity with Apache Kafka and event-driven architectures. Exposure to distributed systems or big data tools: Spark, Redis, Cassandra, Hadoop, MySQL. Life at VuNet We re not just building technology we re building the future of digital experiences. At VuNet, you ll be part of a collaborative, innovation-driven team where curiosity, ownership, and continuous learning are core to our culture. Whether you re exploring Generative AI, solving high-scale data challenges, or working alongside some of the sharpest minds in deep tech VuNet is where you can grow, build, and make a real-world impact. Your Benefits Health insurance for you, your dependents, and parents. Mental wellness support with 1:1 counseling. Transparent and inclusive workplace culture. Access to emerging tech (including Gen AI) and cutting-edge tools. Career development and technical learning programs. Opportunity to shape a globally recognized, Made-in-India deep tech product.

Java Developer Java developer Full-Time Spring Boot
SY

Senior Data Engineer

Synechron

8+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Position Title: Senior Data Engineer Databricks, PySpark, Cloud Platforms Location: Bengaluru Bellandur (GTP) Employment Type: Full-time Job Summary Synechron is looking for a Senior Data Engineer to join our advanced analytics team in Bengaluru. In this role, you will architect and build scalable, high-performance data pipelines that power data science, analytics, and business intelligence initiatives. You ll work with modern tools including Databricks, PySpark, and cloud data platforms, while collaborating across teams to ensure high-quality, secure, and efficient data solutions. Key Responsibilities Design, develop, and maintain large-scale, secure, and efficient data pipelines using Databricks, PySpark, and cloud-native tools. Partner with data scientists, analysts, and business stakeholders to translate requirements into robust data solutions. Integrate data from various structured, semi-structured, and streaming sources. Ensure high standards for data quality, performance optimization, security, and cost efficiency. Drive data pipeline automation, orchestration, and monitoring using tools like Airflow. Lead troubleshooting efforts, performance tuning, and enhancements of existing pipelines. Stay informed about emerging data technologies and recommend adoption where relevant. Technical Skills Core Expertise Programming: Python (expert), SQL (advanced), PySpark. Platforms: Databricks (clusters, notebooks, workflows), AWS/Azure/GCP. Data Orchestration: Apache Airflow (or similar). Data Warehousing: Snowflake (preferred), data modeling, ETL/ELT pipelines. Streaming: Kafka or other stream processing tools. DevOps: CI/CD (GitLab CI, Jenkins), version control (Git), containerization (Docker/Kubernetes preferred). Security: Familiarity with encryption, access controls, and compliance best practices. Experience 8+ years of experience in data engineering or related roles. Proven expertise in developing and deploying scalable data pipelines using Databricks, PySpark, and SQL. Hands-on experience with cloud platforms (AWS, Azure, or GCP). Strong background in data warehousing, especially with Snowflake. Exposure to real-time data processing and orchestration tools. Experience implementing CI/CD pipelines for data workflows is a plus. Daily Responsibilities Build and optimize data ingestion, transformation, and storage workflows. Collaborate with cross-functional teams to align data solutions with business objectives. Monitor, troubleshoot, and continuously improve pipeline performance. Conduct data quality checks, ensure governance and compliance standards. Contribute to technical documentation, code reviews, and team knowledge sharing. Qualifications Bachelor s or Master s degree in Computer Science, IT, or related field. Relevant certifications (e.g., Databricks Certified Data Engineer, AWS Certified Data Analytics) are preferred. Professional Competencies Strong problem-solving and analytical mindset. Effective communicator with ability to collaborate across technical and non-technical teams. Time management and prioritization skills under tight deadlines. Proactive leadership and a passion for innovation. Commitment to ethical data use and data security. Diversity & Inclusion at Synechron Synechron is committed to building an inclusive, diverse, and equitable workplace. Through our global Same Difference DEI initiative, we celebrate and support people from all backgrounds, including race, gender, sexual orientation, religion, age, disability, and more. We offer flexible work arrangements, continuous learning, internal mobility, and mentoring programs to support every employee s growth. Qualification : Bachelors or Masters degree in Computer Science, IT, or related field

Senior Data Engineer Senior engineer Data Engineer
NE

Senior Data Engineer

Neuron7.ai

4+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Senior Data Engineer Location: Bengaluru, India Employment Type: Full-time, On-site About Neuron7.ai Neuron7.ai is a rapidly growing AI-first SaaS company that is revolutionizing the world of service intelligence. Backed by leading Silicon Valley venture capitalists and a distinguished group of angel investors, we are recognized as a startup to watch. Our AI-driven platform delivers service predictions in seconds by analyzing structured and unstructured data, alongside insights from top experts. We specialize in complex service environments like high-tech devices, manufacturing, and medical devices, empowering service leaders to optimize key metrics like first-call resolution, turnaround time, and service margins. At Neuron7.ai, you will be part of a dynamic, innovative team that is pushing the boundaries of service intelligence. We value creativity, collaboration, and a relentless commitment to innovation. This is your chance to join a fast-growing startup and help redefine how AI impacts the future of service optimization. About the Role We are building a real-time analytics platform to improve uptime of connected devices and minimize maintenance costs. This involves collecting, analyzing, and leveraging large volumes of sensor data for observability and predictive analytics. As a Senior Data Engineer, you will have the opportunity to build and maintain scalable, high-performance data systems using modern data engineering tools. The ideal candidate will have hands-on experience with Python, Scala, ELT processes, Lakehouse architecture, Apache Spark, Databricks, real-time streaming systems like Kafka and Flink, and observability tools such as the ELK stack, Prometheus, and Grafana. What You ll Do: Data Model Development: Design and implement advanced data models to extract insights from complex datasets. Collaborative Problem Solving: Work with cross-functional teams to devise data-driven solutions for service optimization. Experimental Analysis: Conduct experiments and analyses to enhance service predictions and outcomes. Actionable Insights: Present findings and actionable recommendations to stakeholders, translating complex data insights into clear, understandable concepts. Mentorship: Guide and mentor junior team members, fostering a collaborative and knowledge-sharing environment. What We re Looking For: Experience: At least 4 years of hands-on experience in data engineering, with a focus on building large-scale data systems. Data Pipeline Expertise: Strong proficiency in Python and Scala for developing data pipelines. Big Data Tools: Proven experience with Apache Spark and Databricks for big data processing and analytics. Observability Tools: Hands-on experience with observability tools like Prometheus, Grafana, and the ELK stack or OpenSearch. Real-Time Data Streaming: Experience working with real-time streaming systems such as Kafka and Flink. Cloud Platforms: Familiarity with cloud platforms like Azure, GCP, or AWS. Problem-Solving Skills: Strong analytical and problem-solving abilities to work with complex data systems in fast-paced environments. Collaboration: Excellent communication and teamwork skills, with the ability to work effectively with cross-functional teams. What We Do and Value: At Neuron7.ai, we prioritize integrity, innovation, and a customer-centric approach. Our mission is to leverage advanced AI technology to enhance service decision-making and we are dedicated to delivering excellence in all aspects of our work. Company Perks & Benefits: Competitive salary, equity, and spot bonuses Paid sick leave Latest MacBook Pro for your work Comprehensive health insurance Paid parental leave Flexible work arrangements work from home or from our vibrant Bengaluru office Our Commitment to Diversity and Inclusion: Neuron7.ai is committed to fostering a diverse and inclusive workplace. We ensure equal employment opportunities without discrimination based on race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, marital status, or any other characteristic protected by law. If you re passionate about using data to drive service intelligence and want to be part of a forward-thinking team, we d love to hear from you!

Senior Data Engineer Senior engineer Data Engineer
TT

Lead Consultant Data Engineer

Thoughtworks Technologies (india) Pvt Ltd.

Fresher | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Lead Data Engineer | ThoughtWorks | Bangalore, India Location: Bangalore, India Employment Type: Full-time, Regular Industry: Information Technology About ThoughtWorks At ThoughtWorks, we're a global technology consultancy that integrates strategy, design, and engineering to drive digital innovation. For over 30 years, we've worked alongside our clients to deliver solutions that challenge the status status quo. With a diverse and inclusive team, we empower each other to grow through shared learning, fostering an environment where innovation thrives. Our commitment to a cultivation culture is key to our success, and we re looking for a Lead Data Engineer to join our Bangalore team and lead transformative projects. Job Overview As a Lead Data Engineer at ThoughtWorks, you will be responsible for designing, developing, and operating modern data architectures that meet client business objectives. You will lead and manage data engineering projects end-to-end, from strategic planning to hands-on coding, ensuring the delivery of scalable and efficient data solutions. Working with cutting-edge technologies, you ll collaborate with stakeholders, clients, and cross-functional teams to implement data-driven strategies that address complex business challenges. Key Responsibilities Project Leadership: Lead and manage data engineering projects from inception to completion, including goal-setting, scope definition, and ensuring on-time delivery in collaboration with cross-functional teams. Data Architecture & Solution Design: Collaborate with clients to design modern data architecture and implement end-to-end solutions that meet key business objectives. Create intricate data processing pipelines to address complex business problems. Stakeholder Collaboration: Work closely with stakeholders to understand business objectives and identify opportunities to leverage data and data quality improvements. Data Modeling & Governance: Develop data models using modern modeling techniques and implement them using appropriate technologies. Ensure compliance with data governance, security, and privacy requirements. Scalable Implementations: Partner with data scientists to design scalable implementations of their models, ensuring the solutions are robust and efficient. Clean, Iterative Code: Write clean, modular code based on TDD (Test-Driven Development) and implement continuous delivery practices to support and operate data pipelines. Technology Guidance: Advise clients on distributed storage and computing technologies, selecting the best options to fit their business needs. Data Quality Strategy: Define and incorporate data quality strategies into daily work processes to ensure high standards and compliance. Job Qualifications Technical Skills: Proven experience in data engineering and system design, with a focus on building Big Data architecture and data pipelines within distributed systems. Deep knowledge of data modeling and hands-on experience with modern data engineering tools and platforms. Strong programming skills, with expertise in building scalable, high-quality data pipelines using languages like Python, Java, Scala, or others. Experience with distributed storage platforms (e.g., Hadoop, Amazon S3, etc.) and distributed processing platforms (e.g., Spark, Flink). Experience working with SQL, NoSQL, data lakes, and other data storage technologies. Familiarity with data visualization techniques and ability to communicate insights effectively across varying audiences. Professional Skills: Stakeholder Management: Strong ability to liaise between clients and other key stakeholders, ensuring trust and buy-in throughout projects. Adaptability & Resilience: Comfortable handling ambiguity and finding innovative solutions to complex challenges. Leadership & Mentorship: Experienced in coaching and mentoring team members, fostering a culture of professional growth and accountability. Risk & Conflict Management: Skilled in managing risks and resolving conflicts, driving projects forward despite challenges. Relationship Building: Natural at cultivating strong relationships with clients, stakeholders, and internal teams to create new opportunities. What You Bring to the Team Leadership: A proven track record in leading high-performance teams and supporting colleagues in their professional development. Curiosity & Innovation: A passion for data and technology and a willingness to continually learn and push the boundaries of what's possible. Collaboration: Ability to work closely with cross-functional teams and stakeholders to design and implement innovative data solutions. At ThoughtWorks, we believe in giving you the autonomy to carve out your unique career path, while providing support through development programs and a vibrant culture of learning. You ll work on exciting projects with a diverse team, solve complex challenges, and make an impact at a global scale. Join ThoughtWorks and be part of a global community of innovators. Together, we turn curiosity into action and creativity into impactful solutions.

Lead Consultant Lead Consultant Data Data lead
IN

Cloud Data Engineer - AWS Big Data

Infocepts

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Position: Cloud Data Engineer AWS Big Data Location: Bangalore, India Employment Type: Full-time Experience Required: 5 to 8 years Purpose of the Position: Join the Infocepts Cloud Data Architect Team as a Cloud Data Engineer and help design and implement cutting-edge big data solutions on AWS. You will leverage your expertise in EMR, Athena, PySpark, S3, AWS Lambda, and SQL to develop robust and scalable data platforms. Key Responsibilities: Technology Assessment and Design: Assess existing technology landscape and data integration frameworks. Design complex Big Data use cases using AWS services under guidance of the Architect. Support architectural decision-making by evaluating trade-offs in cost, performance, and durability. Recommend optimizations to existing data infrastructure. Documentation and Stakeholder Communication: Create project documentation adhering to quality and delivery standards. Collaborate closely with Architects and Project Managers for scoping, estimation, and planning. Present design decisions to technical and business stakeholders clearly. Conduct PoCs and design review sessions. Process Improvement and Automation: Identify and suggest opportunities for automation and process enhancements. Mentor junior engineers and support technical problem solving. Training and Knowledge Sharing: Prepare and deliver internal training on AWS and Big Data topics. Lead client knowledge sharing sessions and contribute to case studies. Essential Skills: In-depth experience with AWS services: S3, EC2, EMR, Athena, Glue, Lambda Familiarity with MPP databases like Redshift, Snowflake, or SingleStore Proficiency in Apache Spark and Databricks Strong programming skills in Python Experience building data pipelines using AWS and Databricks Knowledge of Big Data file formats such as Delta Lake Advanced SQL skills for large-scale data manipulation Hands-on experience with Apache Airflow or similar orchestration tools Strong understanding of ETL workflows and data warehousing concepts Desirable Skills: Cloud databases: AWS Aurora, Snowflake Experience with Hadoop and Hive AWS Certifications (Associate or Professional level) are a plus Qualifications: Bachelor s degree in Computer Science, Engineering, or related field (Master s preferred) Overall 5+ years of IT experience with at least 3 years in AWS Big Data projects Ongoing learning and technical certifications are strongly encouraged Key Qualities: Strong problem-solving and analytical thinking Self-driven with a passion for emerging data technologies Excellent communication and client presentation skills Ability to work in cross-functional, agile teams Apply now to be part of a high-impact data transformation team working on large-scale cloud data projects! Qualification : Bachelors degree in Computer Science, Engineering, or related field (Masters preferred)

Cloud Data Cloud data Engineer Cloud engineer
IN

Staff Data Engineer

Intuit

12+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Intuit is a global leader in financial technology, dedicated to helping individuals and businesses thrive. Our suite of products, including TurboTax, Credit Karma, QuickBooks, and Mailchimp, serves approximately 100 million customers worldwide. At Intuit, we believe in providing everyone with the tools and resources they need to achieve financial success. We are constantly innovating to make financial empowerment a reality for all. Job Overview Join the Intuit Data Platform (IDP) team as a Staff Engineer and help us transform the way we handle big data! The IDP team is responsible for the Intuit Analytics Platform, which powers real-time data ingestion, cataloging, analytics, and machine learning across the entire organization. As Intuit s customer base grows, so does the volume of data we process. Our engineering excellence ensures that we can scale and leverage this data to drive machine learning and product innovations. We re in the process of building the next-generation real-time and batch ingestion engine, capable of indexing, cataloging, and organizing data and metadata. We are passionate about using open-source technologies to solve challenges and contributing back to the community. If you're excited about building a platform that will directly impact data scientists and analysts and have a desire to shape the future of data at Intuit, then come join us! Key Responsibilities Architect & Design: Build fault-tolerant and scalable big-data platforms using open-source technologies to handle massive datasets. Data Solutions: Create architecture solutions that address complex use cases like data normalization, lineage, governance, ontology, and discoverability. Cross-Team Collaboration: Work with analysts and data scientists to understand data requirements for building operational propensity models and gaining deep customer insights. Hands-On Coding: Lead development efforts within the Hadoop ecosystem using technologies such as Java MapReduce, Spark, Scala, HBase, and Hive to build and optimize data pipelines for both real-time and batch applications. Database Management: Work with NoSQL, SQL, and in-memory databases to design high-performance data systems. Code Reviews: Ensure code quality, consistency, and adherence to best practices through regular code reviews. Architectural Alignment: Ensure alignment between enterprise architecture and business requirements. Prove Feasibility: Conduct proof-of-concept (POC) experiments for new technologies or approaches and drive them to production. Collaboration with Data Cataloging Team: Work closely with data catalog teams and architects to index and catalog all data sources at Intuit. Agile Leadership: Lead fast-paced development teams using agile methodologies and promote best practices in software development, testing, and incident response. Design & Model: Build dimension models suited for customer business use cases and ensure seamless integration of business and technical requirements. Qualifications Experience: 12+ years of relevant experience, with at least 5+ years specializing in the big data domain. Big Data Architecture: Proven experience in architecting end-to-end ecosystems for big data and analytics platforms. Expert Knowledge: Deep expertise in building fault-tolerant, scalable big data solutions, especially using the Hadoop ecosystem (Hive, HBase, Spark, Kafka, MapReduce, etc.). Programming Expertise: Mastery of Java and Scala, with a focus on building high-throughput data services. Machine Learning: Knowledge of machine learning principles and AI applications in big data. Big-Data Technologies: Familiarity with tools such as HDFS, Storm, Zookeeper, Cassandra, Redshift, GraphDB, and others. Understanding both real-time and batch processing in the Hadoop ecosystem. Communication: Strong communication skills, with an ability to explain complex technical topics to both technical and non-technical audiences. Programming Skills: Intermediate experience in Python or R for data processing. Education: BE/BTech/MS in Computer Science or a related field (or equivalent experience). Collaboration: Demonstrated ability to work cross-functionally and lead change through influence and example. At Intuit, you ll be part of a talented, passionate team working on innovative solutions that shape the future of data analytics and machine learning. As a Staff Engineer, you ll have the chance to work with cutting-edge technologies, build scalable systems, and help revolutionize how Intuit leverages data to drive product innovation. If you're looking for a dynamic environment where you can have a meaningful impact, come join us at Intuit! Qualification : BE/BTech/MS in Computer Science (or equivalent)

Data Engineer Staff Engineer Data Engineer Full-Time
DA

Sr. Solutions Engineer

Databricks

5+ Years | Not Disclosed | Bengaluru, Karnataka, India | Full-time

Job Title: Senior Solutions Engineer (Analytics, AI, Big Data, Public Cloud) Job Summary As a Senior Solutions Engineer (Analytics, AI, Big Data, Public Cloud), you will guide the technical evaluation phase in a hands-on environment throughout the sales process. You will be a technical advisor internally to the sales team and work with the product team as an advocate of your customers in the field. You will help our customers achieve tangible data-driven outcomes through the use of our Databricks Lakehouse Platform, helping data teams complete projects and integrate our platform into their enterprise Ecosystem. You'll grow as a leader in your field while finding solutions to our customers' biggest challenges in big data, analytics, data engineering, and data science problems. You will report to the Solutions Architect (SA) Manager. The Impact You Will Have You will be a Big Data Analytics expert on aspects of architecture and design. Engage with the technical community by leading presentations, workshops, seminars, and meet-ups. Lead your clients through evaluating and adopting Databricks, including hands-on Spark programming and integration with the wider cloud ecosystem. Support your customers by authoring reference architectures, how-tos, and demo applications. Integrate Databricks with 3rd-party applications to support customer architectures. Together with your Account Executive, you will form successful relationships with clients throughout your assigned territory to provide technical and business value. What We Look For Consulting, pre-sales, or post-sales experience working with external clients across a variety of industry markets. Core strength in either data engineering or data science is advantageous. 5+ years of experience demonstrating technical concepts, including presenting and white-boarding. 4+ years of experience designing architectures within a public cloud (AWS, Azure, or GCP). 4+ years of experience with Big Data technologies, including Spark, AI, Data Science, Data Engineering, Hadoop, Cassandra, and others. Solid coding experience in Python, R, Java, Spark, or Scala. About Databricks Databricks is the data and AI company. More than 10,000 organizations worldwide including Comcast, Cond Nast, Grammarly, and over 50% of the Fortune 500 rely on the Databricks Data Intelligence Platform to unify and democratize data, analytics, and AI. Databricks is headquartered in San Francisco, with offices around the globe and was founded by the original creators of Lakehouse, Apache Spark , Delta Lake, and MLflow. To learn more, follow Databricks on Twitter, LinkedIn, and Facebook.

Sr. Solutions Engineer Sr. engineer Solutions engineer

1 - 20 of 0 jobs

* No exact matches found. Showing closest results instead
Sort by:

No results found

Modify search criteria or create an alert to get relevant jobs as soon as they’re posted

Create an alert

Continue to Save

Please login to your jobseeker account, or create a new one to save this job.

Feedback

Share Feedback