Sde - Data Job in Amazon

DESCRIPTION

At Amazon Music, big data is fueling our organizations growth. To help make sense of it all, the Music Data Experience team provides the self-service data platform enabling the organization to go big with data. Our platform is used by 45+ teams in music including BI, ML, and Data Engineering and enables repeatable, easy, in depth analysis of music customer behaviors. We reduce the cost in time and effort of analysis, data set building, model building, and user segmentation and provide secure and compliant tooling that scales. Our goal is to empower all teams at Amazon Music to make data driven decisions and effectively measure their results by providing a best in class data platform.

If you love the challenges that come with big data platform development then this role is for you. We're developing our next generation platform to build Data as a Service solutions and Data Lake on Cloud technologies to support petabyte scale data. Our current infrastructure collects billions of events a day, manages petabyte scale data on Redshift and S3, and has data pipelines in Spark/Scala EMR, SQL based ETL, and Java services.

You are a talented, enthusiastic, and detail-oriented Big Data Engineer, or Software Development who knows how to take on big data challenges in an agile way. Duties include designing and building big data architecture to scale, data modeling, and development, deployment, and operations of big data pipelines both real time and batch. You will also help hire, mentor, and develop peers in the the Music Data Experience team including Data Scientists, Data Engineers, and Software Engineers. You'll help build Amazon Music's most important data pipelines and data sets, by building our data lake platform, writing APIs and enabling processing, publishing, sharing of data in realtime and batch.

This role requires you to live at the cross section of data and engineering. You have a deep understanding of data, analytical techniques, and how to connect insights to the business, and you have practical experience in big data architecture. With our Amazon Music Unlimited and Prime Music services, and our top music provider spot on the Alexa platform, providing high quality, high availability data to our internal customers is critical to our customer experiences.

In 2020 your team will build out Amazon Music's data lake storage and EMR/Spark processing layers, automate the management of our Multi-Cluster Redshift Enterprise Data Warehouse, and further develop our platform for real time capabiblities and provide data as a service to ML teams. If these sounds like challenges you'd love, please apply today!

BASIC QUALIFICATIONS

2+ years of non-internship professional software development experience
Programming experience with at least one modern language such as Java, C++, or C# including object-oriented design
1+ years of experience contributing to the architecture and design (architecture, design patterns, reliability and scaling) of new and current systems.
ETL Design & Distributed Computing AWS/Azure.

Scripting - Python/Java/Shell


Big Data - Spark/Scala/Pyspark

PREFERRED QUALIFICATIONS

3+ years of experience as a software/big data developer
Experience in Building Real Time Data Pipelines
Experience building flexible data APIs that consumers use to power other parts of the business
Knowledge of orchestration tools like Airflow/AWS Step Function
Experience working with predictive analytics/decision models/data mining libraries as well as the tools for developing it
Experience with Agile Development
Knowledge of professional software engineering practices and best practices for the full software development life cycle, including coding standards, code reviews, source control management, build processes, testing, and operations.
Experience with serverless, big data, ETL technologies at scale.
Familiarity with JDBC/ODBC, SQL query processing, and distributed query engines
Love for Data