Job Description

Responsibilities:

• Develop, maintain, and optimize data processing pipelines using PySpark on Azure Cloud.
• Collaborate with cross-functional teams to understand data requirements and translate business needs into technical specifications.
• Design and implement scalable data models and data architecture to support analytical and reporting needs.
• Perform data cleansing, transformation, and enrichment processes to enhance data quality and usability.
• Monitor and troubleshoot data processing jobs, ensuring high availability and performance.
• Implement best practices for data governance and security within Azure Cloud.
• Stay up-to-date with emerging technologies in big data, cloud computing, and the financial services industry.

Requirements:

• Minimum of 9 years of experience in PySpark development.
• Proven experience working with Azure Cloud services (Azure Data Factory, Azure Databricks, etc.).
• Prior experience in the banking, investment banking, or financial services sector is mandatory.
• Proficient in PySpark for data processing and ETL tasks.
• Experience with SQL and NoSQL databases.
• Familiarity with data warehousing concepts and tools.
• Knowledge of Azure services related to big data and analytics.

Preferred, but not required:

• Experience with containerization technologies (e.g., Docker, Kubernetes).
• Familiarity with machine learning frameworks (e.g., MLlib, TensorFlow).

Job Tags

Similar Jobs