Hadoop Jobs in Chennai

Job Title: Data Engineer Location: Chennai (Work from Office) Job Type: Full-time Experience: 2 to 4 years Job Description Blackstraw Technologies Pvt Ltd is seeking a skilled Data Engineer to join our dynamic team. You will play a crucial role in transforming raw data into actionable insights by designing and maintaining robust data pipelines and systems. Your work will ensure alignment between data infrastructure and business goals, helping to drive efficiency and scalability. You will leverage your analytical skills and proficiency in various programming languages to integrate, process, and optimize data from diverse sources. You will also collaborate closely with data scientists and end-users to support data-driven decision-making. Key Responsibilities Design, build, and integrate data pipelines from multiple sources for analytics and operational purposes. Perform data integration and troubleshoot production and development issues. Develop, automate, and optimize ETL pipelines and workflows. Work with big data technologies including Hive, Spark, and Hadoop; optimize performance for large-scale data processing. Write and maintain efficient Spark/PySpark or Spark/Scala programs. Utilize strong SQL skills for querying and database design. Develop shell scripts (Unix/Linux) to automate data processing tasks. Collaborate with data scientists and various business teams to understand and fulfill data needs. Maintain and deploy ETL code following Agile methodologies. Handle service delivery tickets and provide support related to ETL tools. Participate in root cause analysis and problem resolution for data pipeline issues. Required Skills & Qualifications Bachelor s degree in Computer Science, Information Technology, or related field; Master s degree is a plus. 2 to 4 years of hands-on experience in data engineering or related roles. Strong expertise in ETL development and data integration tools. Proficiency in Spark (PySpark or Scala), Hive, and Hadoop ecosystem. Advanced SQL skills and experience with relational database management systems (RDBMS). Experience with shell scripting in Unix/Linux environments. Familiarity with Java, Python, or Scala programming languages is desirable. Knowledge of Cloud platforms (Azure preferred) is a plus. In-depth understanding of Retail domain data and business processes is advantageous. Excellent analytical, organizational, and communication skills. Experience with Agile project management and teamwork. About Blackstraw Technologies Pvt Ltd Founded in 2015 and fully operational since 2018, Blackstraw Technologies is a global software products and services company specializing in Artificial Intelligence (AI) and Machine Learning (ML) solutions. Our AI platform accelerates data labeling, model training, and deployment across cloud and on-premise environments. Our 250+ member team boasts over 900 years of combined experience, focusing on areas such as computer vision, natural language processing, and deep learning annotation tools. We enable enterprises worldwide to rapidly adopt AI in a cost-effective, plug-and-play manner. Headquartered in Florida, USA, Blackstraw has operations across Canada, the USA, and India. Qualification : Bachelors degree in Computer Science, Information Technology, or related field. Masters degree is a plus.

Job Overview Senior Engineer - Data Engineering in the Automotive domain with 3-7 years of experience Extensive expertise in pyspark, hadoop, and gcp for designing and implementing data pipelines Proficient in tensorflow, python, and SQL for automation, data quality, and optimization Skilled in big data technologies like numpy, pandas, and data manipulation Responsible for architecting scalable data pipelines, data transformation, performance optimization, monitoring, and collaboration with cross-functional teams Bachelor's degree in Computer Science or equivalent; preferred certifications include Professional Data Engineer and Google Cloud Certified - Professional Data Engineer Good to have experience with CI/CD practices, data processing libraries, data quality, Pub/Sub architectures, and Tekton pipelines management Roles & Responsibilities Architect and Develop Data Pipelines: Lead the design, development, and maintenance of scalable and efficient data pipelines using GCP services like Pub/Sub for real-time data ingestion and BigQuery for storage and analysis. Data Transformation and Processing: Implement data transformation processes to cleanse, enrich, and aggregate raw data from diverse sources, ensuring data quality and consistency. Performance Optimization: Fine-tune data pipelines and queries to enhance performance, reduce latency, and ensure timely data access for stakeholders. Monitoring and Maintenance: Implement monitoring solutions for tracking pipeline performance, addressing issues proactively, and conducting regular maintenance tasks to uphold data infrastructure reliability. Collaboration: Collaborate with Data Scientists, Software Engineers, and Business Analysts to understand data requirements and provide technical solutions to meet business needs. Documentation: Document data pipelines, processes, and best practices to facilitate knowledge sharing and maintain a comprehensive understanding of data architecture. Our ideal candidate Extensive knowledge and proficiency in pyspark, hadoop, and gcp Advanced skills in designing and implementing data pipelines, data modeling, and data warehousing solutions Proficiency in tensorflow and python for automation scripts and data quality Strong SQL skills for querying and optimizing data warehouse operations Expertise in big data technologies like numpy, pandas, and libraries for data manipulation and analysis Monitoring data processes, utilizing GCP services, and implementing data engineering best practices Prioritizing skills in GCP, BigQuery, Python, Data Modeling, SQL, and Data Warehousing Bachelor of Technology (B.Tech) or Bachelor of Engineering (B.E) in Computer Science or Information Technology, or a Master of Technology (M.Tech) in Data Science or Big Data Analytics Certifications such as Professional Data Engineer and Google Cloud Certified - Professional Data Engineer preferred Company overview Altimetrik delivers outcomes for our clients by rapidly enabling digital business & culture and infuse speed and agility into enterprise technology and connected solutions. We are practitioners of end-to-end business and technology transformation. We tap into an organization s technology, people, and assets to fuel fast, meaningful results for global enterprise customers across financial services, payments, retail, automotive, healthcare, manufacturing, and other industries. Founded in 2012 and with offices across the globe, Altimetrik makes industries, leaders and Fortune 500 companies more agile, empowered and successful.Altimetrik helps get companies get unstuck . We re a technology company that lives organizations a process and context to solve problems in unconventional ways. We re a catalyst for organization s talent and technology, helping teams push boundaries and challenge traditional approaches. We make delivery more bold, efficient, collaborative and even more enjoyable. Qualification : Bachelor of Technology (B.Tech) or Bachelor of Engineering (B.E) in Computer Science or Information Technology, or a Master of Technology (M.Tech) in Data Science or Big Data Analytics

We are seeking a highly experienced Senior Big Data Software Engineer to join our dynamic team and tackle challenging projects that will enhance your skills and career. As a Senior Engineer, your contributions will be critical in designing and implementing Big Data solutions across a variety of projects. The ideal candidate will possess deep experience in Big Data and associated technologies, with a strong emphasis on Apache Spark, Python, Azure and AWS. Technologies Hadoop Hive Responsibilities Develop and execute end-to-end Big Data solutions to meet complex business needs Work collaboratively with interdisciplinary teams to comprehend project needs and deliver superior software solutions Apply your expertise in Apache Spark, Python, Azure and AWS to create scalable and efficient data processing systems Maintain and enhance the performance, security, and scalability of Big Data applications Keep abreast of industry trends and technological advancements to foster continuous improvement in our development practices Requirements 5-8 years of direct experience in Big Data and related technologies Advanced knowledge and hands-on experience with Apache Spark High-level proficiency with Hadoop and Hive Proficiency in Python Prior experience with AWS and Azure native Cloud data services We offer Opportunity to work on technical challenges that may impact across geographies Vast opportunities for self-development: online university, knowledge sharing opportunities globally, learning opportunities through external certifications Opportunity to share your ideas on international platforms Sponsored Tech Talks & Hackathons Unlimited access to LinkedIn learning solutions Possibility to relocate to any EPAM office for short and long-term projects Focused individual development Benefit package: Health benefits Retirement benefits Paid time off Flexible benefits Forums to explore beyond work passion (CSR, photography, painting, sports, etc.) Qualification : 5-8 years of direct experience in Big Data and related technologies

Position: Data Scientist Experience: 2+ Years Type: Full-time Location: Chennai About the Role As a Data Scientist at FinnUp, you will transform raw data into actionable insights that drive strategic decisions. You will work with complex datasets, build predictive models, and collaborate closely with cross-functional teams. Your expertise in analytics engineering, combined with your proficiency in Python especially pandas data frames and dictionaries will be critical in optimizing data workflows and delivering impactful results. Key Responsibilities Analyze large, complex datasets to extract meaningful insights that support business decision-making. Develop, implement, and maintain predictive models and machine learning algorithms to solve key business problems. Use pandas extensively for data manipulation, analysis, and visualization. Work closely with product, marketing, and business teams to understand requirements and provide data-driven solutions. Design and implement scalable data pipelines and ELT processes to ensure data integrity and accuracy. Create insightful dashboards and reports using Metabase to communicate findings to stakeholders. Stay current with the latest trends, tools, and advancements in data science, AI, and analytics engineering. Required Qualifications Bachelor s or Master s degree in Computer Science, Statistics, Mathematics, or related field. 2-3 years of experience as a Data Scientist or in a similar analytical role. Strong proficiency in Python with deep experience in pandas, dictionaries, and other core data structures. Experience with data visualization tools, especially Metabase. Solid understanding of statistical analysis and machine learning methods. Ability to write efficient SQL queries for data extraction and manipulation. Excellent problem-solving skills with high attention to detail. Strong communication and collaboration skills. Preferred Qualifications Hands-on experience building and deploying machine learning models in production environments. Familiarity with cloud platforms such as AWS or Google Cloud. Knowledge of additional programming languages like R or Julia. Experience with other data visualization tools and libraries. Competitive salary and benefits package. Opportunities for professional growth and skill development. A collaborative, innovative, and supportive work environment. Qualification : Bachelors or Masters degree in Computer Science, Statistics, Mathematics, or related field

Job Title: Data Scientist Company: Blackstraw.ai Job Location: Chennai (Work from Office) Job Type: Full-time Experience: 2 10 Years Job Summary: We are looking for Data Scientists to analyze large amounts of raw information to find patterns that will help improve our clients business. We will rely on you to build data products to extract valuable business insights. You will be part of our AI team in developing AI based products through the latest cutting edge technology. Key Responsibilities: Identify valuable data sources and automate collection processes Undertake preprocessing of structured and unstructured data Analyze large amounts of information to discover trends and patterns Build predictive models and machine-learning algorithms Combine models through ensemble modeling Present information using data visualization techniques Propose solutions and strategies to business challenges Collaborate with engineering and product development teams Communicate your findings to the appropriate teams through visualisations Collaborate and communicate findings to diverse stakeholders Provide solutions but not limited to: Object detection/Image recognition, natural language processing, Sentiment Analysis, Topic Modeling, Concept Extraction, Recommender Systems, Text Classification, Clustering, Customer Segmentation & Targeting, Propensity Modeling, Churn Modeling, Lifetime Value Estimation, Forecasting, Modeling Response to Incentives, Marketing Mix Optimization, Price Optimization. Follow/maintain an agile methodology for delivering on project milestones. Excellent oral, presentation, and written communication skills. Preferred Qualifications: Bachelors in Math, Computer Science, Information Systems, Machine Learning, Statistics, Econometrics, Applied Mathematics, Operations Research or related technical degree with ability to break complex business problems down to smaller more solvable parts. Minimum of 3+ years of experience in a related position, as a data scientist building predictive analytics or NLP or CV solutions for various types of business problems. Working knowledge of statistical techniques, NLP, machine learning algorithms and deep learning frameworks like Tensorflow, Pytorch, PySpark. Programming background and expertise in building models using at least one of the following languages: Python, R, C, C++, Spark, Scala. Good knowledge in the implementation of deep learning models for image classification, Document classification models, object detection, logo detection. Self-motivated and driven to deliver agreed results on-time. Company Profile: Conceptualized as far back as 2015, and commencing full-time operations in 2018, Blackstraw technologies Pvt Ltd. is a software products and services company specializing in Artificial Intelligence (AI) and Machine Learning solutions for various industries. We support businesses around the world, including North America, Europe and Asia, working to simplify AI implementation through our platform that expedites data labelling, AI model-training, and cloud or on-premise deployments. With more than 900 years of combined work-experience, the 250+-strong Blackstraw Team comprises various experts in the AI value chain. We are a fast-moving team that prides ourselves in rapidly identifying different use-cases and fine-tuning our products to suit specific business needs. We are focused on providing solutions related to computer vision, natural language processing, Data annotation tools for deep learning models, etc. To stay competitive in business, it is key for organizations to adopt and implement smart AI solutions and service offerings. However, most companies are unable to implement AI rapidly due to the complexity of existing solutions, inadequate data and cost implications. Our mission is to enable enterprises to adopt AI in an easier, cost-effective and time-efficient manner with a plug-and-play approach to their data. Blackstraw operations are based out of Canada, USA & India. Our head office is based in Florida. Qualification : Bachelors in Math, Computer Science, Information Systems, Machine Learning, Statistics, Econometrics, Applied Mathematics, Operations Research or related technical degree with ability to break complex business problems down to smaller more solvable parts.

Role: Assistant Manager Data Science Location: Chennai, Tamil Nadu, India Experience: 6 8 Years Employment Type: Permanent, Full-Time About the Role We are hiring an experienced Assistant Manager Data Science to join our dynamic analytics team in Chennai. This role requires 6 8 years of hands-on experience in data science, applied analytics, machine learning (ML), and statistical modeling. The ideal candidate will have a strong analytical mindset, excellent problem-solving abilities, and the ability to transform data into strategic business insights. Key Responsibilities Design and develop machine learning models and data analytics frameworks to solve real-world business problems. Apply statistical techniques to analyze structured and unstructured data. Write complex and optimized SQL queries to manage and extract data from large datasets. Build insightful and interactive dashboards and visualizations using tools like Tableau. Collaborate with cross-functional stakeholders to understand requirements and deliver data-driven solutions. Translate business challenges into analytical problems and structure approaches for resolution. Keep up-to-date with the latest trends in data science, artificial intelligence (AI), ML, and data visualization. Required Skills & Qualifications 5+ years of experience in data science, applied analytics, or machine learning. Strong command of Advanced SQL and hands-on experience with large-scale datasets. Proficient in Python for ML and statistical analysis. Expertise in data visualization, preferably with Tableau. Solid understanding of statistical methods, predictive modeling, and data interpretation. Excellent communication and stakeholder management skills. Analytical, detail-oriented, and innovative approach to problem-solving. Work on impactful and high-visibility data science projects. Be part of a collaborative and forward-thinking analytics team. Gain exposure to cutting-edge technologies and business strategies. Competitive salary and benefits package.

Job Title: Python Developer Company: Blackstraw.ai Location: Chennai (Work from Office) Job Type: Full-time Experience: 3 to 7 years Job Summary Blackstraw.ai is looking for a skilled and motivated Python Developer to join our growing engineering team. In this role, you will be responsible for server-side logic development, system integration, and ensuring high performance and responsiveness to requests from the front-end. A strong command of Python and its frameworks, along with familiarity with front-end technologies, will be essential. You ll work closely with front-end developers, data scientists, and DevOps to build and scale enterprise-grade AI and web-based applications. Key Responsibilities Develop and maintain robust, scalable backend systems using Python and relevant web frameworks (e.g., Django, Flask). Integrate multiple data sources and databases into unified systems. Implement data storage solutions and create efficient database schemas. Write reusable, testable, and efficient code while following best coding practices and quality standards. Collaborate with front-end developers to integrate user-facing elements using server-side logic. Ensure the security, accessibility, and compliance of applications. Manage authentication and authorization across multiple systems and environments. Optimize application performance for different delivery platforms (desktop, mobile, etc.). Work with version control tools like Git and follow Agile/Scrum methodologies. Promote knowledge sharing within the team and contribute to documentation. Debug, profile, and unit-test code to ensure reliability and maintainability. Required Skills 3 7 years of hands-on Python development experience. Proficiency with one or more Python web frameworks (Django, Flask, etc.). Experience with ORM libraries (e.g., SQLAlchemy, Django ORM). Strong understanding of RESTful APIs, server-side templating (e.g., Jinja2), and multi-threading/multi-processing. Familiarity with front-end technologies such as JavaScript, HTML5, and CSS3. Understanding of user authentication/authorization protocols (OAuth, JWT). Ability to work with Git, Mercurial, or SVN version control systems. Knowledge of scalable application design and event-driven programming in Python. Excellent problem-solving, organizational, and communication skills. Preferred Qualifications Bachelor s or Master s degree in Computer Science, Engineering, or related field. 5+ years of experience in building web, mobile, or ML-based applications. Experience with: JavaScript (ES6), React, Node.js MongoDB, Elasticsearch Docker, Kubernetes, Azure Microservices architecture Redis, Kafka, SQS, or similar tools Exposure to Big Data technologies (e.g., Hadoop) is a plus. Strong understanding of scalable software systems and system design principles. About Blackstraw.ai Founded in 2015 and operational since 2018, Blackstraw Technologies Pvt Ltd is a global leader in AI and ML solutions. We specialize in simplifying AI adoption for enterprises through a proprietary platform that accelerates data labeling, model training, and deployment. With offices across the USA, Canada, and India, and a 250+ strong team with over 900 years of collective experience, we deliver cutting-edge solutions in Computer Vision, Natural Language Processing, and AI Data Annotation. Our mission is to make AI adoption faster, easier, and more cost-effective for enterprises around the world. Qualification : Bachelors or Masters degree in Computer Science, Engineering, or related field.

About Comcast: Comcast is a Fortune 50 leader at the intersection of media and technology. With a focus on innovation, we create groundbreaking entertainment and online experiences for millions of customers worldwide. If you're passionate about teamwork, revolutionizing industries, and leading the future of media and technology, Comcast is the place to grow your career. Job Summary: The Data Analyst will work cross-functionally to collect, analyze, and model data to uncover trends and provide actionable insights. You'll support key business decisions through exploratory analysis, forecasting, and optimization. This role requires in-depth technical expertise, independent prioritization, and clear communication of findings to stakeholders. Core Responsibilities: Data Collection & Analysis: Work with business leaders to understand requirements and develop technical specifications. Retrieve, analyze, and summarize complex data from platforms like Teradata, SQL Server, Hadoop, and AWS. Identify relevant data sources and combine disparate datasets for actionable insights. Exploratory & Predictive Analysis: Conduct exploratory data analysis and hypothesis testing to uncover trends and drivers. Develop forecasting models for key business metrics such as sales and demand. Measure the impact of campaigns, features, or initiatives and identify opportunities for improvement. Visualization & Communication: Create narrative-driven presentations and effective data visualizations for executives and decision-makers. Communicate insights clearly and concisely to influence strategic business decisions. Innovation & Collaboration: Stay updated on trends in analytics, statistics, and machine learning, and apply them to solve business challenges. Promote a culture of knowledge sharing, healthy skepticism, and teamwork within the organization. Customer-Focused Approach: Drive results by integrating feedback from customers and employees via Comcast s Net Promoter System (NPS). Champion the customer experience and advocate for seamless digital interactions. Compliance & Accountability: Adhere to Comcast s Operating Principles and ensure regular, consistent, and punctual attendance. Fulfill additional duties as needed. Qualifications: Must-Have: Proven experience with big data platforms (e.g., Teradata, SQL Server, Hadoop, AWS). Strong proficiency in data analysis, hypothesis testing, and forecasting techniques. Ability to develop clear and compelling data visualizations and presentations. Exceptional problem-solving and independent judgment skills. Preferred: Knowledge of emerging analytics trends, statistics, and machine learning. Experience with business operating frameworks and a results-driven approach. Familiarity with Net Promoter System (NPS) feedback integration. Diversity & Inclusion: We believe diverse perspectives drive innovation and success. Growth-Oriented Culture: We foster an environment of learning and collaboration. Employee Benefits: Comprehensive benefits package, flexible schedules, and career development opportunities. Impactful Work: Be part of an organization that shapes the future of media and technology. Disclaimer: This description provides a general overview of the role. It is not intended to encompass all responsibilities or qualifications. Comcast is proud to be an equal-opportunity workplace, considering all applicants regardless of race, religion, gender, disability, or veteran status.