Job Description & Scope
Job description Some careers shine brighter than others. If you re looking for a career that will help you stand out, join HSBC and fulfil your potential. Whether you want a career that could take you to the top, or simply take you in an exciting new direction, HSBC offers opportunities, support and rewards that will take you further. HSBC is one of the largest banking and financial services organisations in the world, with operations in 64 countries and territories. We aim to be where the growth is, enabling businesses to thrive and economies to prosper, and, ultimately, helping people to fulfil their hopes and realise their ambitions. We are currently seeking an experienced professional to join our team in the role of Marketing Title. In this role, you will: 1. Spark Application Development Design, develop, and optimize distributed data processing pipelines using Apache Spark. Implement ETL processes for large-scale data ingestion, transformation, and storage. Write efficient, maintainable, and scalable Spark jobs in PySpark, Scala, or Java. Collaborate with data engineers and analysts to ensure data quality and reliability. 2. DevOps Leadership: Design and implement CI/CD pipelines for Spark applications and big data workflows. Automate deployment, monitoring, and scaling of Spark jobs in cloud or on-premises environments. Manage infrastructure as code (IaC) using tools like Terraform, Ansible, or CloudFormation. Ensure system reliability, availability, and performance through proactive monitoring and alerting. 3. Technical Leadership Lead a team of developers and DevOps engineers, providing technical guidance and mentorship. Define best practices for Spark development, DevOps, and cloud-native architectures. Conduct code reviews, enforce coding standards, and ensure adherence to project timelines. Collaborate with stakeholders to gather requirements and translate them into technical solutions. 4. Cloud and Big Data Ecosystem Work with cloud platforms (AWS, Azure, GCP) to deploy and manage Spark applications. Integrate Spark with big data tools like Hadoop, Hive, Kafka, and Delta Lake. Optimize resource usage and cost efficiency in cloud-based Spark clusters. Requirements To be successful in this role, you should meet the following requirements: 1)- Technical Expertise: Strong experience with Apache Spark 3. x, Delta Lake (PySpark, Scala, or Java). Proficiency in big data technologies (Hadoop, Hive, Kafka, etc. ). Hands-on experience with CI/CD tools (Jenkins, GitHub, Ansible CI/CD, etc. ). Knowledge of containerization and orchestration (Docker, Kubernetes). Experience with cloud platforms (AWS EMR, Azure Databricks, GCP DataProc). 2) DevOps Skills: Expertise in infrastructure automation tools (Terraform, Ansible, etc. ). Strong understanding of monitoring and logging tools (Prometheus, Grafana, Splunk ). Experience with version control systems (Git) and branching strategies. 3) Leadership and Communication: o Proven experience leading development and DevOps teams. o Strong problem-solving skills and ability to make technical decisions. o Excellent communication and collaboration skills. Preferred Qualifications: o Experience with real-time data processing frameworks (e. g. , Spark Streaming, Flink). o Knowledge of data lake architectures and Delta Lake. o Certifications in cloud platforms (AWS, Azure, GCP). o Familiarity with Agile/Scrum methodologies. ? Education and Experience: o Bachelor s or Master s degree in Computer Science, Engineering, or a related field. o 10+ years of experience in Spark application development and DevOps. o 3+ years of experience in a technical leadership role. You ll achieve more when you join HSBC. . Role: DevOps Consultant / Architect Industry Type: Consumer Electronics & Appliances Department: Engineering - Software & QA Employment Type: Full Time, Permanent Role Category: DevOps Education UG: Any Graduate PG: Any Postgraduate Key Skills Computer scienceVersion controlCodingGCPTechnical leadershipData processingApplication developmentData qualityMonitoringFinancial services
