Data Engineer Job in Aptus Data Labs

Proven experience with deploying and tuning Open Source components into enterprise ready production tooling Experience with datacentre (Metal as a Service MAAS) and cloud deployment technologies (AWS or GCP Architect certificates required)

Deep understanding of Linux from kernel mechanisms through user space management

Experience on CI/CD (Continuous Integrations and Deployment) system solutions (Jenkins).

Using Monitoring tools (local and on public cloud platforms) Nagios, Prometheus, Sensu, ELK, Cloud Watch, Splunk, New Relic etc. to trigger instant alerts, reports and dashboards. Work closely with the development and infrastructure teams to analyze and design solutions with four nines (99.99%) up-time, globally distributed, clustered, production and non-production virtualized infrastructure.

Wide understanding of IP networking as well as data centre infrastructure

Expert with software development tools and source code management, understanding, managing issues, code changes and grouping them into deployment releases in a stable and measurable way to maximize production Must be expert at developing and using ansible roles and configuring deployment templates with jinja2.

Solid understanding of data collection tools like Flume, Filebeat , Metricbeat, JMX Exporter agents.

Extensive experience operating and tuning the kafka streaming data platform, specifically as a message queue for big data processing

Strong understanding and must have experience:

Apache spark framework, specifically spark core and spark streaming,

Orchestration platforms, mesos and kubernetes,

Data storage platforms, elasticstack, carbon, clickhouse, cassandra, ceph, hdfs

Core presentation technologies kibana, and grafana.

Excellent scripting and programming skills (bash, python, java, go, rust). Must have previous experience with rust in order to support, improve in house developed products