Hdfs Jobs in Hyderabad

Job Title: Senior Data Engineer Location: Hyderabad, Telangana, India Job Type: Full-Time | Permanent Experience Required: 6+ Years Open Positions: 2 About the Role: We are seeking a highly skilled and experienced Data Engineer to join our team in Hyderabad. As a key member of our data engineering unit, you will be responsible for designing and optimizing end-to-end data pipelines using Python, PySpark, and Databricks. This role requires strong technical expertise, a passion for solving complex data challenges, and the ability to lead best practices within the Supply Chain data domain. Key Responsibilities: Develop and maintain scalable data pipelines using Python, PySpark, and Databricks for complex data transformation within Supply Chain operations. Implement advanced data engineering solutions that support real-time and batch data processing for analytics and business intelligence. Analyze, optimize, and refactor existing data workflows to improve performance and reliability. Collaborate with cross-functional teams to understand data requirements and translate them into robust technical solutions. Take ownership of projects and deliver high-quality outcomes with minimal supervision. Provide leadership and guidance on coding standards, design patterns, and data engineering best practices. Clearly communicate technical ideas and solutions to both technical and non-technical stakeholders. Required Skills & Qualifications: 6+ years of hands-on experience in data engineering. Proficient in Python, PySpark, and Databricks for building and maintaining data pipelines. Strong problem-solving skills with a proven ability to create innovative solutions for complex data challenges. Excellent communication and interpersonal skills; able to work effectively with cross-functional teams. Experience leading and mentoring team members on technical best practices and architecture decisions. Ability to work independently, manage priorities, and drive projects to completion with minimal oversight. Perks & Benefits: Competitive compensation package Exposure to cutting-edge technologies in cloud-based data engineering Opportunity to work on large-scale Supply Chain data solutions Collaborative, innovation-driven work environment Career growth and mentorship opportunities

Introduction In this role, you'll be part of one of our IBM Consulting Client Innovation Centers (Delivery Centers), where we deliver deep technical and industry expertise to a wide range of public and private sector clients globally. Our centers provide locally-based skills and technical expertise to drive innovation and the adoption of new technologies. A career at IBM Consulting involves building long-term relationships and collaborating closely with clients across industries. You'll work alongside visionaries to help them on their hybrid cloud and AI journeys, enhancing the value and impact of some of the most innovative companies in the world. With access to IBM s strategic partner ecosystem and cutting-edge technology platforms, including IBM Software and Red Hat, you will be empowered to accelerate client transformations and create meaningful change. Curiosity and a constant quest for knowledge serve as the foundation for success at IBM. In your role, you'll be encouraged to challenge the status quo, explore ideas beyond your current role, and come up with creative solutions that will make a significant impact across a wide network of clients. Our culture of evolution and empathy is centered on long-term career growth and development opportunities in an environment that celebrates your unique skills and experience. Your Role and Responsibilities As a Data Engineer at IBM, you will be a critical player in designing and developing applications, while also providing guidance to project teams on complex coding, troubleshooting, and execution. Your key responsibilities include: Solution Design and Construction: Lead the design and construction of new data solutions using the latest technologies, ensuring business value and meeting user requirements. Continuous Improvement: Advocate for continuous improvement by testing solutions and working within an Agile framework to enhance and iterate the developed solutions. Innovation: Stay current with the latest technology trends, discovering and implementing creative solutions to solve business problems. Collaboration: Work with diverse teams to deliver impactful data solutions while ensuring scalability, quality, and performance. Required Education Bachelor s Degree in a relevant field Required Technical and Professional Expertise Apache Spark (PySpark): In-depth knowledge of Spark s architecture, core APIs, and using PySpark for distributed data processing. Big Data Technologies: Familiarity with Hadoop, HDFS, Kafka, and other big data tools. Data Engineering Skills: Strong understanding of ETL pipelines, data modeling, and data warehousing concepts. Proficiency in Python: Expertise in Python programming, particularly focused on data processing and manipulation. Data Processing Frameworks: Experience with Pandas, NumPy, and other data processing libraries. SQL Proficiency: Experience writing optimized SQL queries for large-scale data analysis and transformation. Cloud Platforms: Experience working with cloud platforms such as AWS, Azure, or GCP, including working with cloud storage systems. Preferred Technical and Professional Experience Architecture Strategy: Experience in defining, driving, and implementing architecture strategies and standards for end-to-end monitoring of data solutions. Cross-Team Collaboration: Partnering with technology teams, including application development, enterprise architecture, testing services, and network engineering. Security: Familiarity with detection and prevention tools for company products and customer-facing platforms. How You ll Grow At IBM, we offer opportunities for career development and provide an environment where you are encouraged to continuously grow, learn, and expand your expertise. Working with cutting-edge technologies and a diverse set of clients, you will be able to stay ahead of industry trends and make meaningful contributions that have a real impact. About IBM Consulting IBM Consulting helps organizations tackle their most pressing challenges through innovative and agile solutions. By partnering with clients, we drive technology transformations, helping them optimize their operations and leverage new technologies like AI, cloud, and data. Our team s expertise spans across many industries, delivering solutions that enable clients to navigate digital transformations. This role offers an exciting opportunity to work with leading technologies while contributing to client success in a highly dynamic environment. If you re passionate about solving complex data problems and delivering impactful solutions, IBM Consulting is the place to grow your career. Qualification : Bachelors Degree in a relevant field

Job Description We are looking for a passionate engineer interested in working at an early-stage startup building innovative cloud-native data protection solutions. The ideal candidate must be comfortable learning and scoping business requirements to develop effective solutions. This is a great opportunity for engineers who are eager to step out of their comfort zone and solve impactful modern data security concerns. Desired Background and Experience: Must be a problem solver with the ability to think critically and creatively. Minimum of 8 years of experience building products using Java and/or Python. Prior experience in startups is a huge plus. Comfortable with rapid prototyping and implementation of products in an agile team environment. Data engineering experience with implementation of data ingestion, ETL, data pipeline/workflows. Experience implementing database views and UDFs to control access to data using row and column filters. Working experience in databases and data warehouses. Proficient in one or more of the following: AWS RDS, AWS Data Lake, Redshift, Athena, Databricks, and Snowflake. Experience implementing data solutions using Spark. Ability to write and tune complex SQL queries.

Job Description Responsibilities Identify projects and initiatives to deliver tangible end user operational improvements across the internal and external IT ecosystem. Collaborate with IT teams to ensure seamless integration with existing systems Engineer and optimize systems to operate within defined SLAs and achieve internal KPIs related to IAM efficiency, security, and performance. Feedback from users on IAM processes (i.e. access request/revoke, access lock down, etc.). Number of help desk tickets related to IAM issues (e.g., login problems, access issues). Implement automation and process improvements to streamline IAM workflows (e.g., automated role definitions, workflow optimization), increasing operational efficiency. Tasks Design and engineer IAM solutions to support the full lifecycle of client identities, from onboarding to offboarding, ensuring secure and efficient processes. Manage and execute changes to IAM systems (e.g., SailPoint, Active Directory, Entra ID), driving system improvements while minimizing disruptions and adhering to change management protocols for security and system stability. Conduct periodic reviews of user access rights and support in forensic analysis and compliance reports. Assist in preparing for internal and external IAM audits, ensuring proper documentation and compliance. Support the implementation of Single Sign-On (SSO) systems across enterprise applications. Troubleshoot authentication issues related to login and access. Ordinary and Extraordinary administration and optimization if IAM systems (e.g., SailPoint, Active Directory, Entra ID), focusing on improving system performance, scalability, and security. Collaborate with help desk and support teams to escalate and manage complex cases. Develop, maintain, and update technical documentation related to system configurations, processes, troubleshooting steps, and resolutions, ensuring high-quality knowledge management. Working with Windows NTFS and DFS services Key competencies for position and level In addition to demonstrating our Global Values (Authentic, Bold, and Collaborative), the role holder will be expected to demonstrate the following: Be a strong team player. Proactive mindset to always look to make improvements. Confident communication skills with the ability to speak to business stakeholders / application owners technical individuals to elicit requirements. Willing to continually learn new skills and technologies. Work well under pressure. Qualifications Education / professional qualifications Graduate degree, preferably in IT related disciplines Microsoft Certifications would be preferred Background experience Hands-on experience with IAM Active Directory and Entra ID on hybrid environment Strong verbal and written communication skills. Knowledge of configuring, troubleshooting, and maintaining IAM tools and systems. Experience of Active Directory, Windows Servers and NTFS permissions. Knowledge of VMware, Azure hosting and SailPoint platform is nice to have. Company, product, and market knowledge Experience of working in a complex, multi-country professional services, financial services or BPO organisation with complex processing requirements preferred but not essential. Multi-country experience and demonstrates an ability to work in a multi-cultural, talented, and demanding team environment. Excellent communication skills in both written and oral form, with staff members, customers, suppliers, and the management team with the ability to make decisions, act and get results. Passion, dynamism, and drive Personal presence, integrity, and credibility Ability to solve problems either independently or by utilising other members of the team where necessary. Ability to focus on priority, when business impact required. Languages Fully proficient spoken and written English, additional European languages will be an asset Company description IQEQ is a preeminent service provider to the alternative asset industry. IQEQ works with managers in multiple capacities ranging from hedge fund, private equity fund, and mutual fund launches; private equity fund administration; advisory firm set-up, regulatory registration and infrastructure design; ongoing regulatory compliance (SEC, CFTC, and 40 Act); financial controls and operational support services; compliance and operational related projects and reviews; and outsourced CFO/controller and administration services to private equity fund investments portfolio companies, real estate assets and energy assets. Our client base is growing, and our existing clients are engaging the firm across the spectrum of our services offerings. Qualification : Graduate degree, preferably in IT related disciplines Microsoft Certifications would be preferred

About HSBC If you re looking for a career that will help you stand out, join HSBC and fulfill your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support, and rewards that will take you further. As one of the largest banking and financial services organizations in the world, with operations in 64 countries and territories, HSBC aims to be where the growth is, enabling businesses to thrive and economies to prosper, and ultimately, helping people fulfill their hopes and realize their ambitions. The Role We are seeking an experienced professional to join our team as a Consultant Specialist. In this role, you will play an instrumental part in ensuring the smooth execution of projects by participating in feasibility studies, project planning, technical design, and coding. You will guide and mentor the team, ensure proper communication across the project, and interact directly with clients to resolve issues related to the delivery of projects. Principal Responsibilities Feasibility Study: Participate in the feasibility study to assess the viability of projects. Project Planning: Contribute to the estimation and planning of projects. Requirement Analysis: Analyze project requirements and translate them into actionable plans. Technical Design: Develop or review technical designs to ensure they meet business objectives and technical specifications. Coding & Development: Guide and mentor team members on code development and contribute where required. Reviews: Conduct thorough reviews of design, code, test plans, and test results to ensure quality and alignment with project requirements. Testing: Oversee test planning, strategy, validation, and conduct random testing to ensure deliverables meet expectations. Post-Development Support: Coordinate post-development support to address issues and ensure smooth deployment. Team Management: Ensure effective communication and coordination within the team to foster collaboration and smooth project delivery. Client Interaction: Communicate directly with clients to provide progress updates and resolve any project delivery-related issues. Requirements To be successful in this role, you should meet the following qualifications: In-depth technical/functional knowledge of Core Banking RPS UK (Retail Processing System) and UK banking products. Job Control Language (JCL): Strong experience with IN Stream & Catalogue Procedures, Control Cards, SDFS (Spool), SPICE, SAR & MVS JCL utilities. COBOL: Strong understanding of COBOL programs, including working storage, copy books, subroutines, and Z/OS concepts. DB2 (SQL): Proficiency in relational database concepts and advanced SQL, including error handling and SQL error codes. CICS: Knowledge of infrastructure setup (PCP, PIP, KCP, SCP) and experience in writing CICS programs linked with front-end systems. TSO, ISPF: Hands-on experience with TSO and ISPF. Programming Skills: Experience with REXX, EZYTRIEVE, and SELCOPY. Tools Knowledge: Familiarity with tools like Endevor, File Manager, Road Runner, CLARITY, Fault Analyser, R2DS, RDz, RTC, SONAR. Communication: Strong verbal and written communication skills for effective team and client interaction. Self-sufficiency & Initiative: Demonstrated ability to work independently and take initiative in solving issues, addressing the root causes, and providing solutions. Additional Desired Skills Core Java, SQL Database Knowledge, Spring Boot/Spring Data, JPA/J2EE/Servlets. Experience with Web Services (REST, SOAP) and Microservices. Knowledge of Google Cloud Platform (GCP) / Amazon Web Services (AWS) (certification is a plus). Familiarity with tools such as Maven, GIT, Jenkins, Any point Studio, Spring Tool Suite (STS). Global Presence: Join one of the largest financial institutions in the world with a presence in 64 countries and territories. Career Growth: Benefit from HSBC s opportunities for professional development and career advancement. Impactful Work: Play an integral role in ensuring the smooth operation of high-importance projects. You ll achieve more when you join HSBC.