BIG Data Engineer Jobs in Bengaluru
1095 Jobs Found
Application Support Engineer I
Tekion
Application Support Engineer I Location: Bangalore Experience: Up to 2+ Years About Tekion Tekion is revolutionizing the automotive industry with the first cloud-native automotive platform, transforming an ecosystem that hasn t seen major innovation in over 50 years. Our Automotive Retail Cloud (ARC), Automotive Enterprise Cloud (AEC), and Automotive Partner Cloud (APC) connect OEMs, dealerships, consumers, and partners through one seamless platform. Leveraging big data, AI, and machine learning, Tekion delivers the best automotive retail experiences across North America, Asia, and Europe. Role Overview We are looking for an enthusiastic Application Support Engineer I who loves challenges, enjoys problem-solving, and is passionate about delivering exceptional customer experiences. You will act as a subject matter expert in your domain, supporting our dealership clients, coordinating technical issues, and ensuring the support team follows processes and delivers solutions effectively. Key Responsibilities Take ownership of customer-reported issues and ensure problems are resolved end-to-end. Research, diagnose, troubleshoot, and identify solutions to resolve system and application issues. Follow standard escalation procedures for unresolved issues and coordinate with internal teams. Understand applications both functionally and technically. Use internal databases and external resources to provide accurate technical solutions. Ensure all issues are properly logged and tracked. Prioritize and manage multiple open issues simultaneously. Prepare accurate and timely reports on support activities. Document technical knowledge in notes, manuals, and knowledge bases. Support rotational shifts as required. Required Skills & Experience 0 2+ years of experience in application, production, or product support. Advanced knowledge of SQL, MongoDB, and basic network configurations. Strong understanding of databases, APIs, and coding fundamentals. Ability to learn and master company-specific software. Strong problem-solving skills and attention to detail. Excellent written and verbal communication skills. Ability to diagnose and address application-related issues effectively. Understanding of IoT and software systems is a plus. Be the first point of contact for dealership clients, ensuring their technical issues are resolved quickly. Play a key role in improving customer satisfaction by providing accurate and timely solutions. Contribute to a knowledge-sharing culture through documentation and process improvements. Work in a collaborative, agile environment alongside cross-functional teams on challenging projects.
Customer Support Engineer I - Crm
Tekion
Customer Support Engineer I CRM Location: Bangalore About Tekion Tekion is transforming the automotive retail industry with the first cloud-native platform, including: Automotive Retail Cloud (ARC) for dealerships and retailers Automotive Enterprise Cloud (AEC) for manufacturers and large enterprises Automotive Partner Cloud (APC) for technology and industry partners The platform leverages Big Data, Machine Learning/AI, and IoT to deliver seamless, intuitive solutions connecting OEMs, dealers, and consumers. Tekion employs ~3,000 people globally and emphasizes innovation, entrepreneurial spirit, and collaboration. Role Overview As a Customer Support Engineer I CRM, you will provide end-to-end customer support for Tekion s cloud platform. You will be responsible for resolving technical issues, documenting solutions, and ensuring an excellent customer experience. Core Responsibilities Take ownership of customer-reported issues and see them through to resolution. Research, diagnose, troubleshoot, and identify solutions to system issues. Follow standard procedures for proper escalation of unresolved issues to internal teams. Understand the application both functionally and technically. Reference internal databases and external resources to provide accurate technical solutions. Ensure all issues are properly logged in the system. Prioritize and manage multiple open issues simultaneously. Prepare accurate and timely reports. Document technical knowledge in the form of notes, manuals, and guides. Required Skills Experience as an Application, Product, or Customer Support Engineer. Ability to quickly learn and master employer-specific software. Strong problem-solving skills for complex technical issues. Excellent written and verbal communication skills. Ability to diagnose and address application issues efficiently. Basic understanding of IoT and software (preferred but not mandatory). Comfortable working rotational shifts. Hands-on experience with chat or ticketing tools for customer interaction.
Lead Software Engineer - Scale & Performance
Team Vunet Systems
Lead Software Engineer - Scale & Performance Location: Bengaluru Experience: 6 12 years About VuNet VuNet is a pioneer in Business Journey Observability, using Big Data and Machine Learning to revolutionize digital experiences in the financial services industry. Our platform delivers end-to-end visibility into customer journeys, helping organizations proactively resolve issues, ensure operational resilience, and deliver superior user satisfaction. With over 28 billion digital transactions monitored every month and serving more than 300 million users globally, VuNet is shaping the future of observability for some of the largest banks and financial institutions. We are Series B funded, part of NASSCOM s DeepTech Club, and recognized by global analysts such as Gartner and Omdia. Your Role: Lead Software Engineer - Scale & Performance As a Lead Software Engineer for Scale & Performance, you ll own the performance and scalability benchmarks for VuNet s observability platform. You will work with cutting-edge technologies, design robust test frameworks, and ensure that our platform scales seamlessly to meet the demands of millions of users. Roles & Responsibilities Own performance and scalability benchmarking for key platform components (ingestion pipelines, data storage, and query services). Design and execute load, stress, soak, and capacity tests across microservices, agents, and ingestion layers. Identify and resolve performance bottlenecks in both infrastructure (CPU/memory/IO) and application layers (API latency, throughput, GC behavior). Develop and maintain performance test frameworks, preferably using Kubernetes-based environments. Collaborate with DevOps and SRE teams to optimize system configurations (Kubernetes, Postgres/TimescaleDB, ClickHouse, Kafka) for scale. Implement OpenTelemetry for service instrumentation to monitor system health and latency (p50/p95/p99 metrics). Contribute to capacity planning, scaling strategies (horizontal/vertical), and resource optimization. Analyze production incidents related to scaling issues and drive permanent fixes. Work with engineering teams to design scalable architecture patterns and define SLIs/SLOs for system performance. Document performance baselines, tuning guides, and scalability best practices for internal use. What You Bring Mandatory Skills: Strong background in performance engineering for large-scale distributed systems or SaaS platforms. Expertise in Kubernetes, container runtimes (containerd/Docker), and resource profiling in containerized environments. Solid understanding of Linux internals, CPU/memory profiling, and network stack tuning. Hands-on experience with observability tools (Prometheus, Grafana, OpenTelemetry, Jaeger, Loki, Tempo, etc.). Familiarity with observability platform datastores like ClickHouse, PostgreSQL/TimescaleDB, Elasticsearch, or Cassandra. Experience with performance benchmarking tools such as k6, Locust, JMeter, or custom Golang/Python scripts. Ability to interpret system metrics (CPU usage, memory, GC, latency) and correlate across different layers. Nice-to-Have Skills: Experience with agent benchmarking (OpenTelemetry Collector, custom data shippers). Exposure to streaming systems like Kafka, NATS, or Pulsar. Familiarity with CI/CD pipelines for performance testing and regression tracking. Knowledge of cost optimization and capacity forecasting in cloud environments (AWS/GCP/Azure). Proficiency in Go, Python, or Bash scripting for automation and data analysis. Life at VuNet: At VuNet, we're building a world-class observability platform, and we re just getting started. You ll be part of a passionate, problem-solving team that embraces collaboration, fast learning, and staying ahead of emerging technologies like Gen AI. We foster a high-trust, inclusive culture where collaboration, ownership, and innovation are central to our success. If you're looking to work on cutting-edge tech, make a real impact, and grow with a supportive team you ll fit right in at VuNet. Benefits: Comprehensive health insurance coverage for you, your parents, and dependents. Mental wellness and 1:1 counseling support. A culture that promotes continuous learning, innovation, and career growth. Transparent, inclusive, and high-trust workplace. Opportunities for skill enhancement with training programs focused on new Gen AI technologies.
Senior Qa Engineer
Team Vunet Systems
Senior QA Engineer - AI-Powered Observability Platform Location: Bengaluru Experience: 6 10 years About VuNet VuNet is at the forefront of Business Journey Observability, revolutionizing the financial services industry with Big Data and Machine Learning. Our deep-tech platform provides comprehensive visibility into customer journeys, enabling proactive issue resolution, operational resilience, and superior user experiences. We monitor over 28 billion digital transactions monthly, serving 300 million users globally, and we re powering some of the largest banks and financial institutions in India and MEA. VuNet is Series B funded, part of NASSCOM s DeepTech Club, and recognized by analysts like Gartner and Omdia. Your Role: Senior QA Engineer - AI-Powered Observability Platform As a Senior QA Engineer at VuNet, you ll play a crucial role in ensuring the quality and reliability of our VuSmartMaps Observability Platform. You ll lead the design and implementation of cutting-edge test automation, performance validation, and reliability frameworks across distributed systems that handle billions of telemetry events. Working closely with development, operations, and QA teams, you will drive quality across the entire platform and play a key role in ensuring that our systems are scalable, resilient, and performant. Roles & Responsibilities Quality Strategy Ownership: Own the end-to-end quality strategy for observability platform components (metrics, logs, tracing, alerting, dashboards, MLOps). Automated Testing: Build and maintain automated test suites for data pipelines, APIs, and integration flows involving tools like Prometheus, Grafana, Loki, Elastic, and OpenTelemetry. Performance Validation: Design and execute tests to validate high-throughput, distributed systems under real-world load conditions, ensuring performance benchmarks are met. Test Frameworks Development: Develop and maintain test frameworks and tools using Python, Go, Bash, pytest, k6, Playwright, and others. System Reliability & Alerting: Define and implement test coverage for system reliability, alerting accuracy, and visualization correctness. Collaboration: Partner with developers, SREs, and DevOps teams to shift quality left in the development lifecycle, contributing to CI/CD pipelines and automation workflows using GitOps tools. Automation Integration: Integrate automated test suites into smoke, functional, and regression pipelines using Jenkins, Spinnaker, and other CI/CD tools. Mentorship: Mentor junior QA engineers, establish best practices, and ensure consistency in the QA discipline across the team. What You Bring Mandatory Skills: Experience: Minimum 6+ years in software quality engineering, with a focus on automated testing, performance, and reliability. Scripting/Programming: Proficiency in at least one scripting or programming language (JavaScript, Python, Go). CI/CD Systems: Experience with CI/CD systems such as GitHub Actions, Jenkins, or ArgoCD. Debugging Skills: Excellent debugging skills and the ability to analyze code quality and system performance. Distributed Systems Knowledge: Familiarity with Kafka, Kafka Streams, ClickHouse DB, and distributed systems. Kubernetes & Microservices: Strong experience testing Kubernetes-native systems, Helm deployments, and microservices. Observability Tools: Knowledge of observability tools like Prometheus, Grafana, Elastic Stack, OpenTelemetry, Loki, or Jaeger. Tooling & Deployment: Proficiency in Jenkins, Spinnaker, GitOps, Kubernetes, and Docker. Testing Experience: Hands-on experience in various types of testing (functional, performance, load, etc.) and knowledge of testing tools. Documentation Skills: Ability to create clear documentation (e.g., release notes, troubleshooting guides, and migration guides). Nice-to-Have Skills: Performance Testing: Experience designing and executing performance and load testing for high-traffic applications. Web Services & Systems Design: Understanding of web services and distributed systems architecture. Cross-Functional Communication: Excellent communication skills with the ability to coordinate across multiple teams. Life at VuNet: At VuNet, we re building a world-class observability platform proudly Made in India and we re just getting started. Join a passionate team of problem-solvers who love tackling complex challenges and stay ahead of the curve with technologies like Gen AI. We offer an environment where collaboration, innovation, and learning are at the core of everything we do. You ll have the opportunity to work on cutting-edge technologies and make a real impact on a product that powers leading banks and financial institutions globally. Benefits: Comprehensive health insurance coverage for you, your parents, and dependents. Mental wellness support and 1:1 counseling. A learning culture that promotes growth, innovation, and ownership. Transparent, inclusive, and high-trust workplace culture. Exposure to Gen AI and integrated technology workspaces. Support for career development with various training programs to enhance your skills and expertise.
Mobile App And Observability Sdk Engineer
Team Vunet Systems
Mobile App and Observability SDK Engineer Experience: 3 6 Years Location: Bengaluru About VuNet VuNet is a pioneer in Business Journey Observability, revolutionizing the financial services industry with Big Data and Machine Learning. Our cutting-edge platform offers end-to-end visibility into customer journeys, driving proactive issue resolution, operational resilience, and superior user satisfaction. With over 28 billion digital transactions monitored monthly touching 400 million users worldwide we re already powering leading banks and financial institutions across India and MEA. VuNet is Series B funded, part of NASSCOM DeepTech Club, and recognized globally by analysts like Gartner and Omdia. Your Role: Mobile App and Observability SDK Engineer At VuNet, the Product Development Team is dedicated to delivering exceptional customer experiences through scalable products. We are looking for a Mobile App and Observability SDK Engineer to join this team. In this role, you ll be at the forefront of building high-quality mobile applications and advancing our Mobile Real User Monitoring (MRUM) initiatives. You ll capture and translate mobile performance data into actionable insights, helping improve the performance and user experience of mobile apps across various platforms. If you re passionate about mobile engineering, user experience, and observability this role offers a unique opportunity to merge these interests into a groundbreaking solution. Roles & Responsibilities Mobile Application Development: Design, develop, and maintain robust, high-performance mobile applications for iOS and Android using Swift, Kotlin, Flutter, or React Native. Testing & Quality Assurance: Implement unit, integration, and UI testing strategies to ensure the app s quality, stability, and regression coverage. Debugging & Profiling: Identify and resolve performance bottlenecks, ANRs, crashes, and memory leaks using tools like Android Studio Profiler, Xcode Instruments, or Flipper. Crash Analysis & Reporting: Integrate crash analytics tools and develop efficient incident tracking and resolution workflows. Performance Monitoring & Insights: Leverage telemetry, profiling, and analytics data to enhance app performance, responsiveness, and overall user experience. Observability Collaboration: Work with SRE and backend teams to export performance metrics, logs, and traces from mobile clients into centralized observability platforms. Code Quality: Write clean, modular, and well-documented code, adhering to best practices in mobile development and SDK maintenance. What You Bring Mandatory Skills: Mobile App Development: 3+ years of hands-on experience in mobile app development using Flutter, React Native, Swift, or Kotlin (experience in at least two of these). App Lifecycle & Performance: Strong understanding of mobile app lifecycle, UI rendering, asynchronous processing, state management, and performance optimization (ANRs, memory management, network latency). Debugging & Profiling Tools: Proficiency in debugging, profiling, and testing mobile applications using tools like Android Studio Profiler, Xcode Instruments, or Flipper. Crash Analytics: Experience integrating and using crash analytics and reporting tools. CI/CD & SDK Versioning: Familiarity with CI/CD pipelines, automated testing, and SDK versioning. Performance Instrumentation: Interest in observability, monitoring, and performance instrumentation with a willingness to learn OpenTelemetry and RUM concepts. Problem-Solving Mindset: Strong analytical and debugging skills, focused on enhancing performance and reliability. Nice-to-Have Skills: OpenTelemetry & SDKs: Exposure to OpenTelemetry SDKs or other instrumentation frameworks for capturing telemetry data (e.g., traces, metrics, logs). Mobile Observability: Familiarity with mobile observability backends. Session Replay & Mobile Analytics: Knowledge of session replay, user behavior tracking, or mobile analytics SDKs. SRE & Monitoring Practices: Understanding of SRE principles, monitoring best practices, and golden signals. Open Source Contributions: Contributions to open-source SDKs or mobile performance tools. Life at VuNet: At VuNet, we re building a world-class observability platform proudly Made in India. We re just getting started, and we re looking for people like you to join us in tackling some of the most complex challenges in the digital world. Our team is filled with passionate problem-solvers who thrive in a collaborative, fast-paced environment. We embrace continuous learning, adapt quickly, and stay ahead of emerging technologies like Gen AI. If you re looking to work on cutting-edge technology, make a real impact, and grow with a supportive team, you ll feel right at home here at VuNet. Benefits: Comprehensive health insurance coverage for you, your parents, and dependents. Mental wellness and 1:1 counseling support. A learning culture that promotes growth, innovation, and ownership. A transparent, inclusive, and high-trust workplace culture. Access to Gen AI and integrated technology workspaces. Supportive career development programs to expand your skills with various training opportunities.
Solution Engineer
Team Vunet Systems
Solution Engineer Experience: 3 5 Years Location: Mumbai About VuNet VuNet is a trailblazer in Business Journey Observability, leveraging Big Data and Machine Learning to transform digital experiences in the financial services sector. Our platform provides end-to-end visibility into customer journeys, driving proactive issue resolution, operational resilience, and enhanced user satisfaction. With over 28 billion digital transactions monitored monthly, we re already empowering leading banks and financial institutions to improve their digital operations, touching over 300 million users globally. VuNet is Series B funded, part of NASSCOM DeepTech Club, and recognized in Forbes DGEMS 200 and by global analysts such as Gartner and Omdia. Your Role: Solution Engineer We are looking for a Solution Engineer who will serve as a trusted advisor for our customers, understanding their business needs and translating them into actionable solutions using our VuSmartMaps platform. You will collaborate closely with customers to optimize their business processes, IT landscape, and overall customer experience by leveraging advanced observability and analytics capabilities. As a Solution Engineer, you ll work across a diverse range of technologies, from applications and networks to databases and business domains. You ll be instrumental in ensuring that our solutions deliver measurable value, maximize ROI, and help customers gain deeper insights into their infrastructure and application performance. Roles & Responsibilities End-to-End Solution Design: Work across a broad range of infrastructure domains network, compute, storage, databases, applications, transactions, and business domains to design integrated observability solutions tailored to customer needs. Leverage Big Data & AI: Use the Big Data and AI-driven features of VuSmartMaps to provide insights into application performance, enhancing business outcomes and end-user experiences. Customer Collaboration: Engage with customer teams and stakeholders to understand their pain points, gather requirements, and design custom solutions and integrations. Solution Architecture & Validation: Develop detailed design documents for solutions and collaborate with internal engineering teams to validate and refine designs. Identify Critical Metrics: Analyze customer application architectures and transaction workflows to identify key touchpoints, critical metrics, and monitoring requirements. Business Insights & Recommendations: Use analytics to provide actionable insights and business recommendations, helping customers improve system performance and optimize workflows. Platform Implementation: Oversee the planning and execution of platform implementation at customer sites, ensuring smooth and successful deployment. Product Development Feedback: Work with the product team to develop new features, identify bugs, and address solution gaps based on customer feedback and your findings. Continuous Learning: Keep up to date with emerging technologies such as Big Data, NoSQL databases, Elastic Search, MongoDB, and DevOps practices, and apply these learnings to enhance our offerings. What You Bring Experience: At least 3+ years of experience in IT infrastructure, observability, or APM (Application Performance Monitoring). Infrastructure Expertise: Hands-on experience working with large-scale IT infrastructures, including applications, databases, networks, and transactions. Monitoring & Automation Tools: Experience with monitoring tools, automation platforms, and an understanding of observability principles. Scripting Skills: Proficiency in scripting (Python, Bash, or similar languages). Technology Familiarity: Knowledge or experience in technologies like Elastic Search, MongoDB, Big Data, SQL databases, ELK stack, REST APIs, JMX, and related observability tools. Good to Have Certification: CKA (Certified Kubernetes Administrator) certification is a plus. Life at VuNet: At VuNet, we re building a world-class observability platform proudly Made in India. As part of our team, you ll tackle some of the most exciting and complex challenges in the tech world, all while staying ahead of the curve with emerging technologies like Gen AI. Collaboration & Innovation: We re a team of passionate problem-solvers who thrive on collaboration, learning, and innovation. We embrace a high-trust, transparent workplace culture where you can take ownership, grow, and make a meaningful impact. Growth & Development: If you re excited to work with cutting-edge technology, drive real-world impact, and grow in your career, VuNet is the place for you. Benefits for You Comprehensive health insurance coverage for you, your parents, and dependents. Mental wellness and 1:1 counseling support. A learning culture that promotes growth, innovation, and ownership. Transparent and inclusive workplace culture that values high trust. Access to Gen AI and integrated technology workspaces. Career development programs to enhance your skills through continuous training and support.
Lead Platform Engineer
Team Vunet Systems
Lead Platform Engineer Observability Solutions Location: Bengaluru Experience: 6 10 Years Function: Observability Engineering | Platform Architecture | SRE Enablement Join VuNet Redefining Digital Observability at Scale VuNet is transforming the future of digital experiences through Business Journey Observability, combining Big Data and AI/ML to empower real-time visibility across payments, banking, and financial services. Monitoring 28+ billion transactions/month, our platform is trusted by top financial institutions and powers over 300 million users. Backed by Series B funding and recognized by Gartner, NASSCOM, and Forbes, we are leading the charge in building a new category of observability, proudly Made in India for global impact. Your Role: Lead Platform Engineer As the Lead Platform Engineer, you will architect and drive the development of packaged observability solutions across 100+ infrastructure and application technologies. You will define **golden signals**, build **data collection strategies**, and lead the standardization of alerts, dashboards, and RCA workflows for platforms like **Kubernetes, Oracle DB, and Tomcat**. This is a cross-functional leadership role that sits at the intersection of product, platform, DevOps, and SRE. You will **lead a team** and influence how observability is delivered, scaled, and adopted across complex environments. Key Responsibilities Observability Solution Development Design and lead the delivery of observability packages for databases, middleware, cloud-native, and legacy platforms. Define and implement data collection pipelines, including agents, APIs, logs, metrics, traces, and service discovery. Establish **golden signals, SLIs/SLOs**, and health KPIs for performance, availability, and anomaly detection. Dashboards, Alerts & RCA Develop standardized, reusable dashboards, alerts, reports, and troubleshooting playbooks. Automate **RCA workflows** to improve MTTR and reduce alert fatigue. Platform Enablement & Integration Work with engineering to enhance agent capabilities and support new data sources/formats. Guide implementation of platform features for better observability at scale. Team Leadership & Governance Lead and mentor a team of observability engineers and specialists. Define design patterns, reusable modules, and version-controlled libraries. Stakeholder Collaboration Partner with product managers, DevOps, SREs, and customer teams to gather requirements, align priorities, and validate use cases. Ensure deliverables are scalable, well-documented, and production-ready. What You Bring Must-Have Skills 6 10 years of experience in observability, platform engineering, or SRE roles. Hands-on with tools like Prometheus, Grafana, OpenTelemetry, ELK/EFK, Datadog, Splunk. Strong understanding of logs, metrics, traces, profiling, and collection strategies. Experience developing solutions for platforms like Kubernetes, Oracle, PostgreSQL, Tomcat, etc. Proficient in Python, Shell scripting, APIs, and automation tools (**Terraform**, etc.). Familiar with alert fatigue mitigation, anomaly detection, and RCA frameworks. Excellent communication, technical leadership, and documentation skills. Nice to Have Experience managing an observability marketplace or solution catalog. Contributions to open-source observability projects. Certifications in Kubernetes, Observability platforms, or cloud providers (AWS/GCP/Azure). Background in ITSM tools, CMDBs, or incident workflow automation. At VuNet, you ll help build a category-defining observability platform that s already transforming critical infrastructure for leading financial institutions. You ll work with passionate engineers, push technical boundaries, and grow in a high-trust, high-impact environment. What You ll Experience: Ownership of key observability initiatives impacting 300M+ users. Collaboration with SRE, DevOps, and product teams across real-time financial systems. Opportunity to experiment with and shape Gen AI, ML, and emerging telemetry trends. Perks & Benefits Health insurance for you, your parents, and dependents. 1:1 mental wellness support. Training programs, certifications, and career growth opportunities. Transparent, inclusive, and high-trust work culture. Access to cutting-edge technology and Gen AI-powered workspaces.
Sr. Data Science Engineer
Scaledge
Job Title: Sr. Data Science Engineer Location: Bangalore Experience: 5 8+ Years Job Description As a Senior Data Science Engineer, you will develop and maintain scalable data pipelines and manage API integrations to support growing data volume and complexity. You will play a key role in ensuring data quality and enabling accurate AI model development through effective data handling and automation. Responsibilities Monitor data quality and implement processes for data cleansing and validation. Analyze data to troubleshoot and resolve data-related issues promptly. Develop automation workflows for efficient data labeling, preparation, and augmentation to improve AI model accuracy and utility. Requirements Proven experience as a Data Scientist, Data Analyst, or related role, with experience in data mining. Strong proficiency in data manipulation using Python or R; familiarity with Scala, Java, or C++ for raw data processing is a plus. Experience with business intelligence tools (e.g., Tableau) and big data frameworks like Hadoop and Spark. Strong mathematical foundation, especially in statistics and algebra. Advanced SQL skills with experience in database development, data migration, and integration. Expertise in exploratory data analysis and familiarity with common data science toolkits. Ability to communicate complex data insights clearly and effectively to non-technical stakeholders. Familiarity with data management tools and experience applying machine learning and AI techniques.
Software Development Manager
Kaleidofin Private Limited
Software Development Manager Experience: 8+ Years Location: Bangalore Company: Kaleidofin About Us Kaleidofin is a mission-driven fintech platform committed to delivering financial solutions for underserved communities. Our proprietary ki credit platform combines advanced credit health analytics, middleware integration, risk dashboards, and debt capital structuring empowering financial institutions to serve low-income customers, informal sector businesses, and MSMEs more effectively. Operating across India, Bangladesh, and Kenya, we ve already impacted over 6 million customers, enabling over $3 billion in debt capital for underserved communities. Backed by global investors including the Gates Foundation, Omidyar Network, and Blume Ventures, Kaleidofin is recognized as a leader in inclusive finance. Featured on Forbes Asia's "100 to Watch" Winner of RBI s Swanari TechSprint 2023 Winner of G20 TechSprint 2024 by BIS Certified as a Great Place to Work Role Overview As a Software Development Manager, you will play a key leadership role in shaping the technical roadmap and executing high-impact fintech products. You ll lead a team of engineers to build scalable, secure, and robust systems. This is a hands-on leadership role requiring active involvement in design, architecture, and code-level decisions. What You ll Do Own the product s engineering roadmap, architecture, scalability, and code quality. Guide system design decisions and lead architecture/code reviews. Collaborate closely with Product and Business teams to align on product strategy and feature development. Provide hands-on technical leadership especially in solving complex problems in fintech and lending domains. Mentor and develop engineering talent; foster a culture of ownership, innovation, and quality. Champion engineering best practices including test automation, CI/CD, and DevOps. Lead the adoption of data-driven practices including Big Data, PySpark, and data pipeline architectures. Balance short-term deliverables with long-term technical sustainability and scalability. Support team growth through career development, learning plans, and succession planning. 8+ years of software development experience, with 1 2+ years in people/team management roles. Strong fundamentals in computer science, system design, and problem-solving. Bachelor's or Master's degree in CS or related field from a top engineering institution (IITs, NITs, etc.). Proven experience building distributed, scalable, and secure systems. Strong hands-on experience with Python, Java, Spring Boot, and microservice architectures. Proficient in SQL/NoSQL, Kafka, CI/CD pipelines, and DevOps tools (Docker, Kubernetes, Jenkins, etc.). Familiarity with AI/ML model integration and credit scoring systems is a strong plus. Experience with data engineering: Big Data, Spark, Airflow, AWS stack (S3, Glue, Athena), etc. Knowledge of front-end and mobile (Android) development is a plus. Experience working in fintech, lending tech, or high-growth startup environments preferred. Excellent leadership, communication, and cross-functional collaboration skills. Be part of a high-impact team driving financial inclusion at scale. Work on cutting-edge technology in fintech and data infrastructure. Grow with a company that s winning global recognition and delivering real-world impact. Contribute to a people-first culture that values learning, innovation, and work-life balance. Qualification : Bachelor's or Master's degree in CS or related field from a top engineering institution
Data Engineer
Capital One
Data Engineer Location: Bangalore Company: Capital One India About Capital One At Capital One, we're redefining how technology solves real-world financial challenges. As a technology-driven company, we bring together talented engineers, data scientists, and designers to innovate at scale and deliver meaningful impact to millions of customers. If you're passionate about building powerful data solutions, exploring cutting-edge technologies, and working in a collaborative, fast-paced environment this is the place for you. About the Role As a Data Engineer at Capital One, you ll join a team of innovators who design and build next-generation data platforms and pipelines that power real-time decision-making. You ll collaborate across disciplines engineering, product, machine learning, and cloud infrastructure to transform how we leverage data at scale. What You ll Do Collaborate across Agile teams to design, develop, test, and deploy data-driven solutions. Build and support scalable data pipelines using modern data engineering tools and cloud services. Work on real-time and batch data processing systems that integrate with distributed microservices and ML platforms. Use programming languages such as Python, Java, or Scala with SQL, NoSQL, and cloud data warehouses like Redshift or Snowflake. Contribute to code reviews, unit testing, and performance optimization to ensure high-quality data systems. Partner with product managers and platform teams to deliver robust, cloud-native data solutions that power business decisions. Stay ahead of tech trends, share knowledge, and mentor junior engineers. Basic Qualifications Bachelor s degree in Computer Science, Engineering, or a related field. 1.5+ years of hands-on experience in application or data engineering (excluding internships). At least 1 year of experience working with big data technologies. Preferred Qualifications 3+ years of application/data engineering experience using Python, Scala, Java, or SQL. 1+ year of experience with cloud platforms (AWS, Azure, or GCP). 2+ years of experience with distributed computing tools (Spark, Hadoop, Hive, EMR, Kafka, etc.). 1+ year working on real-time streaming applications. 1+ year of experience with NoSQL databases (MongoDB, Cassandra). 1+ year of experience with data warehousing (Redshift, Snowflake). 2+ years working with Linux/Unix systems and shell scripting. Familiarity with Agile methodologies and modern DevOps practices. Why Join Capital One Work on high-impact data solutions at one of the world s most innovative financial institutions. Be part of a collaborative tech culture that values experimentation and learning. Access to top-tier tools, mentorship, and career development opportunities. Competitive compensation and benefits in a mission-driven environment. Qualification : Bachelors degree in Computer Science, Engineering, or a related field
Data Architect
Growtharc Technologies
Position: Data Architect Location: Remote/Hybrid | Bengaluru, IND We're searching for a highly skilled and experienced Data Architect to join our team. If you have a deep understanding of big data technologies and extensive experience with Hadoop, Python, Snowflake, and Databricks, you're the ideal candidate. You'll be responsible for designing, implementing, and managing complex data architectures that support our critical business needs and objectives. What You'll Do: Design & Architecture Leadership: Design scalable and efficient data architecture solutions that meet current and future business data needs. Lead the development of data models, schemas, and databases, ensuring alignment with business requirements. Architect and implement robust data solutions on leading cloud platforms (AWS, Azure, or GCP). Data Management & Governance: Develop and maintain robust data pipelines and ETL processes using Hadoop, Databricks, and other essential tools. Oversee data integration and quality efforts to ensure consistency and reliability across the organization. Implement data governance best practices, focusing on data security, privacy, and compliance. Collaboration & Mentorship: Work closely with data engineers, data scientists, and business stakeholders to translate data requirements into effective technical solutions. Provide technical leadership and mentorship to junior data engineers and architects. Collaborate with cross-functional teams to ensure data solutions align perfectly with overall business goals. Optimization & Innovation: Optimize existing data architectures for peak performance, scalability, and cost-efficiency. Monitor and troubleshoot data systems to ensure high availability and reliability. Continuously evaluate and recommend new tools and technologies to improve our data architecture. What You'll Bring: Experience: 10+ years in data architecture, data engineering, or a related field. Big Data Expertise: Proven experience with Hadoop ecosystems (HDFS, MapReduce, Hive, HBase). Programming Prowess: Strong programming skills in Python for data processing and automation. Data Platform Mastery: Hands-on experience with Snowflake for data warehousing and Databricks for analytics. Cloud Fluency: Extensive experience with cloud platforms (AWS, Azure, GCP) and their data services. Data Modeling: Familiarity with data modeling tools and methodologies. Core Skills: Deep understanding of big data technologies and distributed computing. Strong problem-solving skills to design solutions for complex data challenges. Excellent communication skills, able to explain complex technical concepts clearly to diverse audiences. Proficient in SQL and database performance tuning. Experience with CI/CD pipelines and automation in data environments. Education: Bachelor's degree in Computer Science, Information Technology, or a related field. Preferred Qualifications: Advanced Degree: A Master's degree in a related field. Cloud Certifications: Certifications like AWS Certified Data Analytics, Google Professional Data Engineer, or Microsoft Certified: Azure Data Engineer Associate. Additional Languages: Experience with other programming languages like Java or Scala. Machine Learning Integration: Knowledge of machine learning frameworks and their integration with data pipelines.
Senior Associate Data Engineering L2
Publicis Sapient
Senior Associate Data Engineering L2 Location: Bengaluru, India Department: Engineering Data Employment Type: Full-Time About the Role As a Senior Associate Data Engineering (L2) at Publicis Sapient, you will lead technical solutions that drive digital transformation by building scalable, high-performance data platforms. You ll be responsible for translating business and technical requirements into modern, data-centric solutions using Big Data technologies, cloud services (Azure), and advanced data engineering practices. Key Responsibilities Design and implement data ingestion, integration, and transformation processes from multiple heterogeneous sources in both batch and real-time. Build scalable data platforms using Hadoop stack components such as HDFS, Kafka, Spark, Hive, NiFi, Oozie, Airflow, Flink, and Storm. Develop real-time analytics, aggregation, and search features to support various data-driven applications. Collaborate closely with cross-functional teams on data infrastructure, computation frameworks, and data visualization. Apply cloud-native principles and Azure services to build and deploy data pipelines. Ensure performance optimization and data pipeline tuning. Work with NoSQL and MPP platforms like MongoDB, Cassandra, Redshift, Azure SQL DW, HBase, BigQuery. Contribute to infrastructure, automation, and DevOps for data pipelines using CI/CD practices. Ensure data governance, lineage, and cataloging using tools like Collibra or Alation. Required Qualifications 6 8 years of professional experience in software/data engineering. Minimum of 3 years hands-on experience with Big Data technologies. Strong programming expertise in Java (preferred), Scala, or Python. Expertise in the Hadoop ecosystem and real-time stream processing tools (Kafka, Pulsar, Spark Streaming, etc.). Hands-on experience with Azure data services (e.g., Data Factory, Synapse, ADLS, Databricks). Experience working with modern ETL tools (Informatica, Talend, etc.) and traditional RDBMS platforms (Oracle, PostgreSQL, SQL Server, MySQL). Bachelor's degree in Computer Science, Engineering, or related field. Nice to Have Certifications in Azure Data Engineer, GCP Big Data, or related cloud specializations. Experience with distributed messaging frameworks (ActiveMQ, RabbitMQ, Solace). Familiarity with microservices architecture and search technologies (Elasticsearch). Performance tuning of distributed data processing systems. Exposure to data governance, security, and metadata management. Benefits and Culture at Publicis Sapient Gender-neutral workplace policies 18 paid holidays annually Generous parental leave + new parent transition support Flexible work arrangements Access to Employee Assistance Programs (wellness & well-being) A dynamic culture focused on learning, creativity, and collaboration Qualification : Bachelor's degree in Computer Science, Engineering, or related field.
Data Scientist
Colan Infotech
Data Scientist Experience: 5+ Years Location: Bangalore, Karnataka, India Job Type: Full Time Job Summary We are seeking an experienced Data Scientist with over 5 years of expertise to join our innovative team in Bangalore. The ideal candidate will have hands-on experience in advanced statistical methods, machine learning, and deep learning frameworks, combined with strong skills in deploying models at scale. Key Responsibilities Apply advanced Statistics and Operations Research methods to solve complex business problems. Develop and deploy predictive and machine learning models using frameworks such as PyTorch, TensorFlow, Keras, and XGBoost. Utilize data engineering and big data tools including PySpark, Databricks, and Flask for building scalable data solutions. Implement and fine-tune NLP models like RNN, LSTM, and attention-based architectures, leveraging pre-trained models from Stanford, IBM, Azure, and OpenAI. Design and optimize efficient SQL queries to extract data from large databases. Work with visualization tools like D3.js, Dash Plotly, and deploy interactive dashboards using Streamlit. Utilize graph databases such as Neo4j for complex data relationships. Manage version control with tools like GitHub or Bitbucket. Deploy machine learning models into production environments using MLOps practices on cloud platforms such as AWS or Azure. Required Skills 5+ years of professional experience as a Data Scientist or similar role. Strong foundation in statistics, machine learning, and deep learning techniques. Proficient in Python-based data science libraries and frameworks. Hands-on experience with cloud-based MLOps and deployment. Excellent SQL skills for data extraction and manipulation. Experience with version control systems and collaborative development. Familiarity with NLP techniques and modern pre-trained language models. Strong problem-solving and communication skills. Qualifications Any graduate degree in Computer Science, Statistics, Mathematics, Engineering, or related field. Join a forward-thinking company in Bangalore, where you will work with cutting-edge technologies to create impactful data-driven solutions. Grow your career in an environment that values innovation, diversity, and continuous learning.
Engineering Manager- Data
Meesho
Engineering Manager Data Location: Bangalore, Karnataka | Department: Tech About the Team At Meesho, we power millions of orders daily with zero downtime thanks to our cutting-edge data and backend systems. Our engineering culture is built on speed, ownership, and a growth mindset. We embrace failures as learning opportunities and foster a 'Founder s Mindset' across all engineering teams. We're passionate technologists, team players, and storytellers just as comfortable building robust data platforms as we are bonding over books, games, or chai. If you're ready to lead a team building high-impact, real-time data systems that touch millions of users, come join us. About the Role We are seeking an experienced Engineering Manager Data to lead our data engineering initiatives. You ll be responsible for building scalable data platforms, managing distributed systems, and ensuring data accuracy, accessibility, and real-time analytics capabilities. This role combines technical depth, strategic thinking, and team leadership to build next-gen data systems powering Meesho s mission. What You Will Do Design and allocate technical tasks based on Meesho s data engineering standards and architecture. Lead and mentor a team of data engineers; conduct regular performance reviews and provide growth feedback. Oversee development of scalable, distributed data systems across streaming and batch pipelines. Collaborate with Product, Design, and Sales teams to create innovative data-driven products and features. Monitor project progress, drive sprint planning, and report on key metrics using OKRs. Promote engineering best practices, data quality standards, and cloud-native development. What You Will Need Bachelor s or Master s in Computer Science or a related field. 8+ years of overall experience, with 4+ years managing software/data engineering teams. Hands-on expertise in Java, Python, or Go and multithreading. Experience with distributed data systems, including Apache Spark, Kafka, and streaming architectures. Deep understanding of relational and NoSQL databases. Proficient in cloud platforms (preferably AWS) and containerized deployments. Experience in data modeling, pipeline architecture, and data quality frameworks. Familiarity with BI tools like Tableau, Metabase, or Superset. Good to have: Knowledge of Trino, Flink, Presto, Druid, Pinot, and real-time analytics platforms. Strong project and stakeholder management; ability to lead cross-functional teams. About Us Meesho is India s fastest-growing e-commerce platform on a mission to democratize internet commerce for every Indian. With 1.75M+ sellers and millions of active customers, we offer zero commission, the lowest shipping costs, and unparalleled reach across every serviceable pincode in India. We're building Meesho to be Meri Shop for the next billion users empowering small businesses and first-time entrepreneurs to sell and thrive online. Our Culture & Total Rewards Our culture is rooted in impact, innovation, and inclusivity. Guided by 11 core principles or "Mantras", we empower our teams to think big, move fast, and grow constantly. Daily rituals like Reflections and Listen or Die ensure everyone has a voice and a growth path. What We Offer: Competitive Compensation including salary, performance bonuses, and equity. Wellness & Insurance via our MeeCare program, with extensive coverage for employees and families, plus mental health and wellness initiatives. Work-Life Balance with flexible work hours, generous leave policies, and parental support. Learning & Development with courses, mentorship, and career mobility through our Internal Mobility Program. Perks & Benefits including salary advances, relocation support, recreational discounts, and employee gifts. Join us to lead the data transformation at Meesho and help shape the future of commerce for India s next billion users. Apply now to make your mark! Qualification : Bachelors or Masters in Computer Science or a related field.
Data Engineer, Product Analytics
Meta Careers
Data Engineer, Product Analytics Location: Bangalore, India Full Time Company: Meta Meta is looking for an experienced Data Engineer to join our Product Analytics team. In this role, you will work on some of the most extensive data sets in the world, crafting experiences for billions of people and millions of businesses. Your technical expertise and analytical mindset will help shape data solutions across Meta to drive user experience optimization, growth, and strategy. You'll work alongside software engineers, data scientists, and product managers to tackle the most interesting data challenges at a scale few companies can match. Key Responsibilities: Conceptualize and own the data architecture for multiple large-scale projects, balancing design and operational cost-benefit tradeoffs. Design and contribute to frameworks that improve logging data efficacy, working with data infrastructure teams to resolve issues. Collaborate with engineers, product managers, and data scientists to understand data needs and represent insights meaningfully. Define and manage Service Level Agreements (SLAs) for all data sets in allocated areas of ownership. Implement and maintain security models based on privacy requirements and ensure safeguards are followed to ensure data quality. Design, build, and launch sophisticated data models and visualizations that support multiple use cases across various products. Address complex data integration challenges using optimal ETL patterns, frameworks, and techniques from structured and unstructured data sources. Optimize existing processes, pipelines, dashboards, and systems to enhance the development of data artifacts. Influence product and cross-functional teams to identify data opportunities that will drive impact. Mentor junior team members by providing constructive feedback and receiving feedback to grow as a team. Minimum Qualifications: Bachelor s degree in Computer Science, Computer Engineering, or a relevant technical field, or equivalent practical experience. 4+ years of experience in roles focused on data engineering, data analysis, or similar positions, working with SQL, ETL, data modeling, and programming languages (e.g., Python, C++, Scala). Experience with SQL, ETL processes, and data modeling, along with at least one programming language (e.g., Python, C++, C#, Scala). Preferred Qualifications: Master's or Ph.D. degree in a STEM field (Science, Technology, Engineering, Mathematics). About Meta: Meta builds technologies that help people connect, find communities, and grow businesses. Since launching Facebook in 2004, we have revolutionized how people connect. Our apps, including Messenger, Instagram, and WhatsApp, empower billions worldwide. Now, Meta is moving beyond 2D screens towards immersive experiences like augmented and virtual reality, building the next evolution in social technology. When you join Meta, you help shape the future of digital connection, taking us beyond screens, distance, and even the rules of physics. Equal Employment Opportunity: Meta is proud to be an Equal Employment Opportunity employer. We do not discriminate based on race, religion, color, national origin, sex (including pregnancy, childbirth, reproductive health decisions, or related medical conditions), sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, political views or activity, or other applicable legally protected characteristics. Qualification : Bachelors degree in Computer Science, Computer Engineering, or a relevant technical field, or equivalent practical experience.
Lead Consultant Data Engineer
Thoughtworks Technologies (india) Pvt Ltd.
Lead Data Engineer | ThoughtWorks | Bangalore, India Location: Bangalore, India Employment Type: Full-time, Regular Industry: Information Technology About ThoughtWorks At ThoughtWorks, we're a global technology consultancy that integrates strategy, design, and engineering to drive digital innovation. For over 30 years, we've worked alongside our clients to deliver solutions that challenge the status status quo. With a diverse and inclusive team, we empower each other to grow through shared learning, fostering an environment where innovation thrives. Our commitment to a cultivation culture is key to our success, and we re looking for a Lead Data Engineer to join our Bangalore team and lead transformative projects. Job Overview As a Lead Data Engineer at ThoughtWorks, you will be responsible for designing, developing, and operating modern data architectures that meet client business objectives. You will lead and manage data engineering projects end-to-end, from strategic planning to hands-on coding, ensuring the delivery of scalable and efficient data solutions. Working with cutting-edge technologies, you ll collaborate with stakeholders, clients, and cross-functional teams to implement data-driven strategies that address complex business challenges. Key Responsibilities Project Leadership: Lead and manage data engineering projects from inception to completion, including goal-setting, scope definition, and ensuring on-time delivery in collaboration with cross-functional teams. Data Architecture & Solution Design: Collaborate with clients to design modern data architecture and implement end-to-end solutions that meet key business objectives. Create intricate data processing pipelines to address complex business problems. Stakeholder Collaboration: Work closely with stakeholders to understand business objectives and identify opportunities to leverage data and data quality improvements. Data Modeling & Governance: Develop data models using modern modeling techniques and implement them using appropriate technologies. Ensure compliance with data governance, security, and privacy requirements. Scalable Implementations: Partner with data scientists to design scalable implementations of their models, ensuring the solutions are robust and efficient. Clean, Iterative Code: Write clean, modular code based on TDD (Test-Driven Development) and implement continuous delivery practices to support and operate data pipelines. Technology Guidance: Advise clients on distributed storage and computing technologies, selecting the best options to fit their business needs. Data Quality Strategy: Define and incorporate data quality strategies into daily work processes to ensure high standards and compliance. Job Qualifications Technical Skills: Proven experience in data engineering and system design, with a focus on building Big Data architecture and data pipelines within distributed systems. Deep knowledge of data modeling and hands-on experience with modern data engineering tools and platforms. Strong programming skills, with expertise in building scalable, high-quality data pipelines using languages like Python, Java, Scala, or others. Experience with distributed storage platforms (e.g., Hadoop, Amazon S3, etc.) and distributed processing platforms (e.g., Spark, Flink). Experience working with SQL, NoSQL, data lakes, and other data storage technologies. Familiarity with data visualization techniques and ability to communicate insights effectively across varying audiences. Professional Skills: Stakeholder Management: Strong ability to liaise between clients and other key stakeholders, ensuring trust and buy-in throughout projects. Adaptability & Resilience: Comfortable handling ambiguity and finding innovative solutions to complex challenges. Leadership & Mentorship: Experienced in coaching and mentoring team members, fostering a culture of professional growth and accountability. Risk & Conflict Management: Skilled in managing risks and resolving conflicts, driving projects forward despite challenges. Relationship Building: Natural at cultivating strong relationships with clients, stakeholders, and internal teams to create new opportunities. What You Bring to the Team Leadership: A proven track record in leading high-performance teams and supporting colleagues in their professional development. Curiosity & Innovation: A passion for data and technology and a willingness to continually learn and push the boundaries of what's possible. Collaboration: Ability to work closely with cross-functional teams and stakeholders to design and implement innovative data solutions. At ThoughtWorks, we believe in giving you the autonomy to carve out your unique career path, while providing support through development programs and a vibrant culture of learning. You ll work on exciting projects with a diverse team, solve complex challenges, and make an impact at a global scale. Join ThoughtWorks and be part of a global community of innovators. Together, we turn curiosity into action and creativity into impactful solutions.
Sr. Data Engineer- Aws- Big Data
Infocepts
Sr. Data Engineer - AWS - Big Data Location:Bangalore Type of Employment: Full-Time Experience Required: 7 to 10 years Job Overview: We are seeking a highly skilled Sr. Data Engineer with expertise in AWS cloud technologies and Big Data to join our Cloud Data Architect Team at Infocepts. In this critical role, you will design and implement robust data solutions using technologies like EMR, Athena, PySpark, AWS Lambda, S3, and other AWS services. The ideal candidate will have a strong foundation in database concepts and SQL and will be responsible for building scalable data pipelines to support high-performance data processing. Key Responsibilities: Technology Assessment and Design: Study the existing technology landscape and evaluate current data integration frameworks. Assist in designing complex Big Data use cases leveraging AWS services. Documentation and Stakeholder Communication: Prepare and maintain comprehensive project documentation, adhering to quality guidelines and schedules. Work closely with Architects and Project Managers to provide accurate estimations, scoping, and scheduling assistance. Clearly communicate design decisions and conduct Proof-of-Concepts to validate new solutions before implementation. Process Improvement and Automation: Identify areas for process automation to improve efficiency and team productivity. Provide expert guidance and troubleshooting support to junior Data Engineers. Training and Knowledge Sharing: Develop and deliver technology-focused training sessions for the team, ensuring continuous knowledge sharing. Share expertise through Expert Knowledge Sharing sessions with Client Stakeholders. Essential Skills: AWS Services Expertise: In-depth knowledge of S3, EC2, EMR, Athena, AWS Glue, and Lambda. Big Data Technologies: Proficiency with Apache Spark, Databricks, and Big Data table formats such as Delta Lake (open-source). Data Warehousing: Strong understanding of data warehousing concepts and architectures. Programming Skills: Advanced programming skills in Python for building data pipelines. SQL Expertise: Strong SQL skills for data transformation, aggregation, and querying large datasets. ETL Workflow Development: Expertise in creating ETL workflows with complex transformations (e.g., SCD, deduplication, aggregation). Orchestration Tools: Familiarity with orchestration tools like Apache Airflow. MPP Databases: Experience with at least one MPP database (e.g., AWS Redshift, Snowflake, SingleStore). Cloud Databases: Exposure to cloud databases like Snowflake or AWS Aurora. Desirable Skills: Cloud Databases: Familiarity with Snowflake, AWS Aurora. Big Data Technologies: Experience with Hadoop and Hive. AWS Certification: Associate or Professional Level AWS Certification. Advanced Knowledge of Big Data Solutions: Exposure to big data tools and frameworks on cloud platforms. Qualifications: Experience: 7+ years of overall IT experience, with 5+ years specifically focused on AWS-related projects. Educational Background: Bachelor's degree in Computer Science, Engineering, or a related field (Master's degree is a plus). Technical Certifications: Demonstrated commitment to continuous learning through certifications or relevant training. Qualities: Strong analytical and problem-solving skills to deep dive into complex technical challenges.
Cloud Data Engineer - AWS Big Data
Infocepts
Position: Cloud Data Engineer AWS Big Data Location: Bangalore, India Employment Type: Full-time Experience Required: 5 to 8 years Purpose of the Position: Join the Infocepts Cloud Data Architect Team as a Cloud Data Engineer and help design and implement cutting-edge big data solutions on AWS. You will leverage your expertise in EMR, Athena, PySpark, S3, AWS Lambda, and SQL to develop robust and scalable data platforms. Key Responsibilities: Technology Assessment and Design: Assess existing technology landscape and data integration frameworks. Design complex Big Data use cases using AWS services under guidance of the Architect. Support architectural decision-making by evaluating trade-offs in cost, performance, and durability. Recommend optimizations to existing data infrastructure. Documentation and Stakeholder Communication: Create project documentation adhering to quality and delivery standards. Collaborate closely with Architects and Project Managers for scoping, estimation, and planning. Present design decisions to technical and business stakeholders clearly. Conduct PoCs and design review sessions. Process Improvement and Automation: Identify and suggest opportunities for automation and process enhancements. Mentor junior engineers and support technical problem solving. Training and Knowledge Sharing: Prepare and deliver internal training on AWS and Big Data topics. Lead client knowledge sharing sessions and contribute to case studies. Essential Skills: In-depth experience with AWS services: S3, EC2, EMR, Athena, Glue, Lambda Familiarity with MPP databases like Redshift, Snowflake, or SingleStore Proficiency in Apache Spark and Databricks Strong programming skills in Python Experience building data pipelines using AWS and Databricks Knowledge of Big Data file formats such as Delta Lake Advanced SQL skills for large-scale data manipulation Hands-on experience with Apache Airflow or similar orchestration tools Strong understanding of ETL workflows and data warehousing concepts Desirable Skills: Cloud databases: AWS Aurora, Snowflake Experience with Hadoop and Hive AWS Certifications (Associate or Professional level) are a plus Qualifications: Bachelor s degree in Computer Science, Engineering, or related field (Master s preferred) Overall 5+ years of IT experience with at least 3 years in AWS Big Data projects Ongoing learning and technical certifications are strongly encouraged Key Qualities: Strong problem-solving and analytical thinking Self-driven with a passion for emerging data technologies Excellent communication and client presentation skills Ability to work in cross-functional, agile teams Apply now to be part of a high-impact data transformation team working on large-scale cloud data projects! Qualification : Bachelors degree in Computer Science, Engineering, or related field (Masters preferred)
Senior Data Scientist
Infocepts
Position: Senior Data Scientist Location: Bangalore Employment Type: Full-time Experience Required: 5 to 7 years Purpose of the Position: The Senior Data Scientist will play a key role in designing, developing, and implementing machine learning models and algorithms to solve complex business problems. The role involves collaborating with data scientists, software engineers, and business stakeholders to deliver scalable, efficient machine learning solutions that drive innovation and improve business outcomes. Key Responsibilities: Model Development and Optimization: Develop, train, and optimize machine learning models to meet business objectives. Ensure models are accurate, efficient, and scalable. Data Pipeline and Infrastructure: Design and maintain robust data pipelines to support machine learning workflows. Ensure data quality and integrity throughout the data lifecycle. Deployment and Monitoring: Deploy machine learning models into production. Monitor model performance and implement improvements as needed. Collaboration and Communication: Work with cross-functional teams to understand business requirements and translate them into technical solutions. Communicate complex technical concepts to non-technical stakeholders. Research and Innovation: Stay up to date with the latest advancements in machine learning and AI. Experiment with new techniques and technologies to enhance the organization's capabilities. Essential Skills: Machine Learning Frameworks: Experience with ML frameworks such as TensorFlow, PyTorch, and Scikit-learn. Programming Languages: Strong skills in Python and R. Data Processing: Experience with tools like Pandas, NumPy, and Spark. Model Deployment: Experience with Docker, Kubernetes, MLflow, and FastAPI. Databases: Strong experience in databases like Snowflake, MongoDB, and Cosmo DB. Statistical Analysis: Strong foundation in statistics and probability. Model Management: Experience establishing model management, monitoring, support, and maintenance frameworks. ML Lifecycle: Experience setting up ML lifecycle processes and governance policies, as well as providing L0, L1, and L2 support. Desired Skills: Deep Learning: Familiarity with deep learning frameworks like TensorFlow or PyTorch. Natural Language Processing (NLP): Understanding of NLP techniques and applications. Cloud Computing: Experience with cloud platforms like AWS, Azure, or Google Cloud. Big Data Technologies: Knowledge of big data technologies such as Hive, Pig, or Cassandra. Software Engineering: Proficiency in software development and version control systems like Git. Qualifications: Education: Master's degree or Ph.D. in Computer Science, Data Science, Statistics, Mathematics, or a related field. Experience: Over 5 years of experience in machine learning, with a proven track record of developing and deploying models. Certifications: Relevant certifications in machine learning or data science are a plus. Key Qualities: Analytical Thinking: Strong analytical and problem-solving skills. Communication: Excellent verbal and written communication skills. Team Player: Ability to work effectively in a collaborative team environment. Adaptability: Flexibility to adapt to changing business needs and technologies. Innovation: A proactive approach to exploring new ideas and technologies. Apply now to join our dynamic team and lead innovation through machine learning and AI! Qualification : Master's degree or Ph.D. in Computer Science, Data Science, Statistics, Mathematics, or a related field.
Data Engineer
Kpit Technologies
Job/Position Summary: Data Engineer Responsibilities: Implement data pipelines that meet design and are efficient, scalable, and maintainable. Implement best practices including proper use of source control, participation in code reviews, data validation and testing. Timely deliveries while working on projects. Act as advisor/mentor and helps junior data engineers in their deliverables. Must Have Skills: Should have experience of at least 4+ years with Data Engineering. Strong experience of design, implementation and fine-tuning big data processing pipelines in production environment. Experience with big tools like Hadoop, Spark, Kafka, Hive, Databricks. Experience in programming at least one of with Python, Java, Scala, Shell Script. Experience with relational SQL and NO SQL databases like PostgresSQL, MYSQL, Cassandra etc. Experience with any data visualization tool (Plotly, Tableau, Power BI, Google Data Studio, Quick sight etc.). Good To Have Skills: Should have Basic Knowledge of CI/CD Pipeline. Experience in working on at least one Cloud (AWS or Azure or GCP). For AWS: - Experience with AWS Cloud services like EC2, S3, EMR, RDS, Athena, Glue, Lambda, EMR. For Azure: -Experience with Azure Cloud services like Azure Blob/Data Lake GEN2, Delta Lake, Databricks, Azure SQL, Azure DevOps, Azure Data Factory, Power BI. For GCP: - Experience with GCP Cloud services Big Query, Cloud Storage bucket, DataProc, Dataflow, Pub Sub, Cloud Function, Data Studio. Sound familiarity in Versioning tools (Git, SVN etc.). Experience Mentoring students is desirable. Knowledge of latest developments in Machine Learning, Deep Learning, Optimization in Automotive domain. Open minded approach to explore multiple algorithms to design optimal solution. History of contribution to articles/blogs/whitepapers etc. in Analytics. History of contribution to Open Source. Requirement: ESSENTIAL SKILLS /COMPETENCIES Data Engineering Hadoop Kafka CI/CD Cloud
1 - 20 of 0 jobs
* No exact matches found. Showing closest results insteadNo results found
Modify search criteria or create an alert to get relevant jobs as soon as they’re posted