Senior Data Engineer Job in Coffeebeans Consulting

Senior Data Engineer

Bangalore, India

About Us

CoffeeBeans Consulting is a fast-growing tech company that offers services and productspowered by AI and Big Data - founded by IITM and BITS Pilani alums.

Technologists at CoffeeBeans work on end to end product features and release them to thousands of users all over the world. We care about the code that we ship. We care if a code manages to clock O(nlogn) rather than O(n2). You will get to see what good engineering practices look like and how to build and ship fast.

Our consulting clients, onshore and abroad, offer a dynamic arena for professionals to hone their consulting/coaching/mentoring skills. As Consultants, we play a key role in establishing core Agile practices and making available the latest tech stack. We have been identified as among the Top 20 Machine Learning Startups in India 2019 . (https://machine-learning.cioreviewindia.com/vendors/2019/0 )

People join us because they want to be apart of an organization where cutting edge technology is played around with. They join us because we hire attitude and potential. We believe in the innate capability of inspired individuals in managing their own work and ambitions, enabling the leadership to focus on the Company s vision.

We value attitude and potential and it would be great if you also have these skill sets.

What you will be doing...

• Designing and implementing data processing pipelines, as part of diverse, high energy teams
• Working with our data scientists to take our ML\AI models to production
• Hands-on programming in Python, Java, Scala
• Deploying data pipelines in production based on Continuous Delivery practices
• Recommending the right distributed storage and computing technologies to our clients from a large number of options available in the ecosystem
• Ensuring that the data is available to the consumers in a reliable, trustworthy and predictable manner
• Evolving the data platform to make it more robust, scalable

What we are looking for....

• 6-8 years of experience working as a data engineer
• Proficient understanding of distributed computing principles
• Proficiency HDFS, Spark, Kafka
• Experience with integration of data from multiple data sources
• Experience with NoSQL databases Redis, MongoDB, ElasticSearch
• Knowledge of various ETL techniques and frameworks
• Knowledge of how to create and maintain optimal data pipeline architecture,
• Good understanding of Lambda Architecture, along with its advantages anddrawbacks

Great to have:

• Ability to solve any ongoing issues with operating the cluster
• Experience with building stream-processing systems, using solutions such as Flink or Spark-Streaming
• Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala
• Experience with Cloudera/MapR/Hortonworks