Lead Data Engineer Job in Beaconstac

What you'll do at Beaconstac: Create and maintain optimal data pipeline architecture, Assemble large, complex data sets that meet product and business requirements. Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources such as RedShift, OLAP data stores, and DataLakes. Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other product and business performance metrics. Work with stakeholders, including the Product, Data, and Design teams, to assist with data-related technical issues and support their data infrastructure needs. Keep data separated and secure across national boundaries through multiple data centers and AWS regions. Work with internal business customers and partner with technical teams to gather and document requirements for data publishing and consumption Desired skills and experience: Bachelor's degree in Computer Science, Engineering, or a related technical discipline 4-6 years of industry experience in the Data Engineering related field with experience storing, manipulating, processing, and extracting value from large datasets using ElasticSearch, Redshift, and data lakes. Experience in building up data architectures for storing and efficiently querying large amounts of data. Experience with relevant AWS services such as S3, EMR, Lambda, Glue, and Redshift. Experience with Apache Spark, Airflow, SQL, Python Experience in Apache Pinot and Kubernetes is good to have skill. Ability to write high-quality, maintainable, and robust code. Willingness to learn and ability to flourish in a dynamic, high-growth, entrepreneurial environment collaborate with teams across the company to deliver thoughtful analyses, develop scalable data visualization solutions and set up the data roadmap. Hands-on, self-starter, capable of working independently True love for technology and what you do Maniacal attention to detail