Databricks Jobs in Chennai

Job Overview Senior Engineer - Data Engineering in the Automotive domain with 3-7 years of experience Extensive expertise in pyspark, hadoop, and gcp for designing and implementing data pipelines Proficient in tensorflow, python, and SQL for automation, data quality, and optimization Skilled in big data technologies like numpy, pandas, and data manipulation Responsible for architecting scalable data pipelines, data transformation, performance optimization, monitoring, and collaboration with cross-functional teams Bachelor's degree in Computer Science or equivalent; preferred certifications include Professional Data Engineer and Google Cloud Certified - Professional Data Engineer Good to have experience with CI/CD practices, data processing libraries, data quality, Pub/Sub architectures, and Tekton pipelines management Roles & Responsibilities Architect and Develop Data Pipelines: Lead the design, development, and maintenance of scalable and efficient data pipelines using GCP services like Pub/Sub for real-time data ingestion and BigQuery for storage and analysis. Data Transformation and Processing: Implement data transformation processes to cleanse, enrich, and aggregate raw data from diverse sources, ensuring data quality and consistency. Performance Optimization: Fine-tune data pipelines and queries to enhance performance, reduce latency, and ensure timely data access for stakeholders. Monitoring and Maintenance: Implement monitoring solutions for tracking pipeline performance, addressing issues proactively, and conducting regular maintenance tasks to uphold data infrastructure reliability. Collaboration: Collaborate with Data Scientists, Software Engineers, and Business Analysts to understand data requirements and provide technical solutions to meet business needs. Documentation: Document data pipelines, processes, and best practices to facilitate knowledge sharing and maintain a comprehensive understanding of data architecture. Our ideal candidate Extensive knowledge and proficiency in pyspark, hadoop, and gcp Advanced skills in designing and implementing data pipelines, data modeling, and data warehousing solutions Proficiency in tensorflow and python for automation scripts and data quality Strong SQL skills for querying and optimizing data warehouse operations Expertise in big data technologies like numpy, pandas, and libraries for data manipulation and analysis Monitoring data processes, utilizing GCP services, and implementing data engineering best practices Prioritizing skills in GCP, BigQuery, Python, Data Modeling, SQL, and Data Warehousing Bachelor of Technology (B.Tech) or Bachelor of Engineering (B.E) in Computer Science or Information Technology, or a Master of Technology (M.Tech) in Data Science or Big Data Analytics Certifications such as Professional Data Engineer and Google Cloud Certified - Professional Data Engineer preferred Company overview Altimetrik delivers outcomes for our clients by rapidly enabling digital business & culture and infuse speed and agility into enterprise technology and connected solutions. We are practitioners of end-to-end business and technology transformation. We tap into an organization s technology, people, and assets to fuel fast, meaningful results for global enterprise customers across financial services, payments, retail, automotive, healthcare, manufacturing, and other industries. Founded in 2012 and with offices across the globe, Altimetrik makes industries, leaders and Fortune 500 companies more agile, empowered and successful.Altimetrik helps get companies get unstuck . We re a technology company that lives organizations a process and context to solve problems in unconventional ways. We re a catalyst for organization s talent and technology, helping teams push boundaries and challenge traditional approaches. We make delivery more bold, efficient, collaborative and even more enjoyable. Qualification : Bachelor of Technology (B.Tech) or Bachelor of Engineering (B.E) in Computer Science or Information Technology, or a Master of Technology (M.Tech) in Data Science or Big Data Analytics

Job Title: Senior Technical Engineer Cloud Location: Chennai Experience: 3 5 Years Position Overview: We are looking for a skilled and self-driven Senior Technical Engineer Cloud to design, maintain, and optimize cloud-based infrastructure across various environments including Development, Testing, UAT, and Production. The ideal candidate will have hands-on experience with AWS, Azure, and GCP, solid system administration skills, and a strong grasp of cloud-native design patterns, automation, and DevOps best practices. Key Responsibilities: Design and implement best-practice cloud architectures for various deployment environments (Dev, Test, UAT, Prod). Manage and maintain cloud infrastructure and shared services across AWS, Azure, and GCP platforms. Provide system administration support (Linux/Windows), including configuration, troubleshooting, and automation. Take ownership of production environments, including upgrades, releases, and incident management. Troubleshoot issues across cloud infrastructure, networking, shared services, and application stacks. Participate in on-call rotations to ensure high availability and performance of cloud infrastructure. Manage licensing, monitoring, and performance of various cloud-based services. Develop and manage automation scripts using tools such as Terraform and PowerShell to improve efficiency and reduce manual work. Provision and manage key AWS resources: EC2, Load Balancers, Route 53, EBS, VPCs, along with Linux/Windows servers, IIS, and firewall rules. Propose and implement performance, cost, and security optimizations for cloud environments. Administer Kubernetes clusters, especially AKS, including setup of HPA (Horizontal Pod Autoscaler) and VPA (Vertical Pod Autoscaler). Implement CI/CD pipelines and workflows using industry-standard tools. Integrate monitoring and logging tools such as Prometheus and Grafana for real-time metrics and analysis. Develop and maintain infrastructure for Azure Synapse, Databricks, Azure Data Factory, including performance tuning. Automate scheduling of services (e.g., Azure VM, Synapse, SQL Pools) using Azure Automation Accounts. Create and manage alerts and diagnostic settings via scripts for continuous monitoring of all resources. Desired Candidate Profile: Bachelor's degree in Computer Science, Engineering, or a related field. 3 5 years of hands-on experience in cloud infrastructure management. Strong expertise in AWS, Azure, and optionally GCP. Proficiency in Linux/Windows administration, cloud automation, and scripting. Practical experience with Terraform, PowerShell, Kubernetes, and CI/CD tools. Knowledge of cloud security, cost optimization, and performance tuning. Familiarity with monitoring tools and log analytics platforms. Excellent problem-solving skills and a proactive mindset. Strong communication and collaboration skills. Qualification : Bachelor's degree in Computer Science, Engineering, or a related field