27 Etl Processes jobs in the United Arab Emirates

As the Head of Data Engineering, you will be responsible for designing, implementing, and maintaining a robust, scalable, and compliant data architecture that supports our exchange’s operations, analytics, and regulatory reporting requirements. You will lead a team of data engineers, ensuring high availability, security, and performance of our data infrastructure. You will work closely with stakeholders across technology, compliance, risk, and product teams to develop data pipelines, warehouses, and real-time analytics capabilities.

This is a strategic yet hands-on role where you will drive data engineering best practices, scalability, and automation while ensuring compliance with regulatory data requirements for a financial services entity.

Data Architecture & Strategy :

• Define and own the data architecture strategy for the exchange, ensuring it is scalable, secure, and regulatory-compliant.
• Design and implement data modeling, governance, and security frameworks to meet financial and regulatory requirements.
• Architect real-time and batch processing data pipelines to handle trade data, order books, user activity, and market analytics.
• Optimize data storage and retrieval for performance and cost efficiency, leveraging cloud-based and hybrid solutions.
• Build and maintain ETL / ELT data pipelines for operational, analytical, and compliance-related data.
• Ensure high data quality, reliability, and availability through robust monitoring, alerting, and data validation techniques.
• Manage and enhance data warehouses, data lakes, and streaming platforms to support business intelligence and machine learning use cases.
• Oversee database design and optimization for transactional and analytical workloads (e.g., Aurora, Redis, Kafka).
• Implement data lineage, metadata management, and access control mechanisms in line with compliance requirements.

Compliance, Security & Risk Management :

• Work closely with compliance and risk teams to ensure data retention policies, audit trails, and reporting mechanisms meet regulatory requirements (e.g., FATF, AML, GDPR, MiCA).
• Implement encryption, anonymization, and access control policies to safeguard sensitive user and transaction data.
• Support fraud detection and risk monitoring through data analytics and alerting frameworks.

Leadership & Team Management :

• Lead, mentor, and grow a small team of data engineers, fostering a culture of collaboration, innovation, and accountability.
• Drive best practices in data engineering, DevOps for data, and CI / CD automation for analytics infrastructure.
• Collaborate with software engineers, DevOps, data analysis, and product teams to integrate data solutions into the broader exchange ecosystem.

Technical competencies and skills :

• Proven experience in data architecture and engineering, preferably within a regulated financial or crypto environment.
• Strong proficiency in SQL, Python, or Scala for data engineering.
• Experience with cloud-based data platforms (AWS, GCP, or Azure) and orchestration tools (Airflow, Prefect, Dagster).
• Hands-on experience with real-time data processing (Kafka, Pulsar, Flink, Spark Streaming).
• Expertise in data warehousing solutions (Snowflake, BigQuery, Redshift, Databricks).
• Strong understanding of database design, indexing strategies, and query optimization.
• Experience implementing data governance, lineage, and cataloging tools.
• Familiarity with blockchain / crypto data structures and APIs is a plus.

Leadership & Strategic Skills :

• Experience leading and mentoring a team of data engineers.
• Ability to design data strategies that align with business goals and regulatory requirements.
• Strong cross-functional collaboration skills with compliance, risk, and technology teams.
• Ability to work in a fast-paced, high-growth startup environment with a hands-on approach.

Industry & Compliance Knowledge :

• Experience in regulated financial markets, fintech, or crypto is highly preferred.
• Familiarity with financial data standards, KYC / AML reporting, and regulatory requirements related to data handling.

Preferred Qualifications :

• Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.
• Certifications in cloud data engineering (AWS / GCP / Azure), data governance, or security are a plus.
• Experience working in a crypto exchange, trading platform, or high-frequency trading environment is an advantage.

At M2, we believe in a workplace where talent, dedication, and passion are the only factors that count, regardless of gender, background, age, and other characteristics. We embrace diversity because we know that it fuels innovation, fosters creativity, and drives success. So, if you're ready to join a team where your potential is truly valued, welcome aboard!

Direct message the job poster from Edison Smart

Position: Head of Data Engineering Location: Dubai/Abu Dhabi (Relocation support available)
Employment Type: Full-Time | Senior Leadership
We are conducting a confidential executive search for a Head of Data Engineering to join one of the world’s fastest-growing digital asset trading platforms, serving millions of users globally with multi-billion daily trading volumes. This is a rare opportunity to lead a world-class data organization, driving scalability, reliability, and innovation in a high-performance environment.
Key Responsibilities:

• Lead and scale the Data Engineering function, overseeing global teams responsible for data pipelines, storage, and real-time processing at massive scale.
• Design and implement a robust, future-proof data architecture to support analytics, AI/ML initiatives, risk management, and next-generation trading products.
• Collaborate with Product, Engineering, Quant, and Business teams to ensure data availability, integrity, and accessibility across all platforms.
• Build and optimize large-scale distributed data systems, ensuring ultra-low latency and high fault tolerance.
• Drive data strategy and best practices, focusing on governance, lineage, and security across all datasets.
• Manage global stakeholders, providing thought leadership in data engineering, tooling, and emerging technologies.
• Hire, mentor, and inspire top-tier data engineering talent to support exponential growth.

• Proven track record leading large-scale Data Engineering organizations (100+ professionals), ideally in fintech, trading, or high-frequency transaction environments.
• Expertise in big data technologies (Spark, Flink, Kafka, Hadoop, Airflow, etc.) and modern data platforms (Snowflake, Databricks, Delta Lake).
• Strong knowledge of data streaming, real-time processing, ETL pipelines, and large-scale distributed systems.
• Deep understanding of cloud infrastructures (AWS, GCP, Azure) and data security practices in highly regulated environments.
• Exceptional leadership skills with experience building global data teams in high-growth, fast-paced organizations.
• Ability to solve complex technical challenges while aligning data strategy with long-term business goals.
• Experience in digital assets, trading platforms, or financial services preferred but not mandatory.
  

• Seniority levelExecutive

• Employment typeFull-time

• Job functionFinance
• IndustriesBanking, Financial Services, and IT Services and IT Consulting

Referrals increase your chances of interviewing at Edison Smart by 2x

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 5 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 days ago

Abu Dhabi Emirate, United Arab Emirates 1 week ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 5 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 month ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 3 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 2 months ago

Abu Dhabi Emirate, United Arab Emirates 1 week ago

Abu Dhabi Emirate, United Arab Emirates 1 week ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 6 days ago

Al Zahiyah, Abu Dhabi Emirate, United Arab Emirates 2 days ago

Abu Dhabi Emirate, United Arab Emirates 1 week ago

Abu Dhabi Emirate, United Arab Emirates 4 weeks ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 5 days ago

Abu Dhabi Emirate, United Arab Emirates 3 days ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

Join to apply for the Head of Data Engineering role at M2

3 weeks ago Be among the first 25 applicants

Join to apply for the Head of Data Engineering role at M2

Get AI-powered advice on this job and more exclusive features.

As the Head of Data Engineering, you will be responsible for designing, implementing, and maintaining a robust, scalable, and compliant data architecture that supports our exchange’s operations, analytics, and regulatory reporting requirements. You will lead a team of data engineers, ensuring high availability, security, and performance of our data infrastructure. You will work closely with stakeholders across technology, compliance, risk, and product teams to develop data pipelines, warehouses, and real-time analytics capabilities.

This is a strategic yet hands-on role where you will drive data engineering best practices, scalability, and automation while ensuring compliance with regulatory data requirements for a financial services entity.

Key Responsibilities

Data Architecture & Strategy :

• Define and own the data architecture strategy for the exchange, ensuring it is scalable, secure, and regulatory-compliant.
• Design and implement data modeling, governance, and security frameworks to meet financial and regulatory requirements.
• Architect real-time and batch processing data pipelines to handle trade data, order books, user activity, and market analytics.
• Optimize data storage and retrieval for performance and cost efficiency, leveraging cloud-based and hybrid solutions.

Data Engineering & Infrastructure :

Compliance, Security & Risk Management :

Leadership & Team Management :

Technical competencies and skills :

Leadership & Strategic Skills :

Industry & Compliance Knowledge :

Preferred Qualifications :

At M2, we believe in a workplace where talent, dedication, and passion are the only factors that count, regardless of gender, background, age, and other characteristics. We embrace diversity because we know that it fuels innovation, fosters creativity, and drives success. So, if you're ready to join a team where your potential is truly valued, welcome aboard!

Seniority level

Seniority level

Not Applicable

Employment type

Employment type

Full-time

Job function

Job function

Other

Referrals increase your chances of interviewing at M2 by 2x

Get notified about new Head of Engineering jobs in Dubai, Dubai, United Arab Emirates .

Dubai, Dubai, United Arab Emirates 21 hours ago

Director : Engineering Ports and Terminals - SSA Region

Head of Engineering (Relocate to Bangkok)

Dubai, Dubai, United Arab Emirates 7 hours ago

Dubai, Dubai, United Arab Emirates 7 hours ago

Head of Engineering, Architecture, and Delivery

Head of Engineering - Linux Platform & Automation

Dubai, Dubai, United Arab Emirates 7 hours ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

J-18808-Ljbffr

As the Head of Data Engineering, you will be responsible for designing, implementing, and maintaining a robust, scalable, and compliant data architecture that supports our exchange's operations, analytics, and regulatory reporting requirements. You will lead a team of data engineers, ensuring high availability, security, and performance of our data infrastructure. You will work closely with stakeholders across technology, compliance, risk, and product teams to develop data pipelines, warehouses, and real-time analytics capabilities.

This is a strategic yet hands-on role where you will drive data engineering best practices, scalability, and automation while ensuring compliance with regulatory data requirements for a financial services entity.

Data Architecture & Strategy :

1. Define and own the data architecture strategy for the exchange, ensuring it is scalable, secure, and regulatory-compliant.
2. Design and implement data modeling, governance, and security frameworks to meet financial and regulatory requirements.
3. Architect real-time and batch processing data pipelines to handle trade data, order books, user activity, and market analytics.
4. Optimize data storage and retrieval for performance and cost efficiency, leveraging cloud-based and hybrid solutions.
5. Build and maintain ETL / ELT data pipelines for operational, analytical, and compliance-related data.
6. Ensure high data quality, reliability, and availability through robust monitoring, alerting, and data validation techniques.
7. Manage and enhance data warehouses, data lakes, and streaming platforms to support business intelligence and machine learning use cases.
8. Oversee database design and optimization for transactional and analytical workloads (e.g., Aurora, Redis, Kafka).
9. Implement data lineage, metadata management, and access control mechanisms in line with compliance requirements.

Compliance, Security & Risk Management :

1. Work closely with compliance and risk teams to ensure data retention policies, audit trails, and reporting mechanisms meet regulatory requirements (e.g., FATF, AML, GDPR, MiCA).
2. Implement encryption, anonymization, and access control policies to safeguard sensitive user and transaction data.
3. Support fraud detection and risk monitoring through data analytics and alerting frameworks.

Leadership & Team Management :

1. Lead, mentor, and grow a small team of data engineers, fostering a culture of collaboration, innovation, and accountability.
2. Drive best practices in data engineering, DevOps for data, and CI / CD automation for analytics infrastructure.
3. Collaborate with software engineers, DevOps, data analysis, and product teams to integrate data solutions into the broader exchange ecosystem.

Technical competencies and skills :

1. Proven experience in data architecture and engineering, preferably within a regulated financial or crypto environment.
2. Strong proficiency in SQL, Python, or Scala for data engineering.
3. Experience with cloud-based data platforms (AWS, GCP, or Azure) and orchestration tools (Airflow, Prefect, Dagster).
4. Hands-on experience with real-time data processing (Kafka, Pulsar, Flink, Spark Streaming).
5. Expertise in data warehousing solutions (Snowflake, BigQuery, Redshift, Databricks).
6. Strong understanding of database design, indexing strategies, and query optimization.
7. Experience implementing data governance, lineage, and cataloging tools.
8. Familiarity with blockchain / crypto data structures and APIs is a plus.

Leadership & Strategic Skills :

1. Experience leading and mentoring a team of data engineers.
2. Ability to design data strategies that align with business goals and regulatory requirements.
3. Strong cross-functional collaboration skills with compliance, risk, and technology teams.
4. Ability to work in a fast-paced, high-growth startup environment with a hands-on approach.

Industry & Compliance Knowledge :

1. Experience in regulated financial markets, fintech, or crypto is highly preferred.
2. Familiarity with financial data standards, KYC / AML reporting, and regulatory requirements related to data handling.

Preferred Qualifications :

1. Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.
2. Certifications in cloud data engineering (AWS / GCP / Azure), data governance, or security are a plus.
3. Experience working in a crypto exchange, trading platform, or high-frequency trading environment is an advantage.

At M2, we believe in a workplace where talent, dedication, and passion are the only factors that count, regardless of gender, background, age, and other characteristics. We embrace diversity because we know that it fuels innovation, fosters creativity, and drives success. So, if you're ready to join a team where your potential is truly valued, welcome aboard

As an AI/Data Engineer, you will be responsible for designing and implementing scalable data pipelines to optimize our Large Language Models for real-world applications.

• Build ingestion pipelines for structured and unstructured data using Python.
• Clean, normalize, and prepare data formats suitable for Large Language Model (LLM) finetuning.
• Create high-quality, task-specific datasets for training and evaluation.
• Apply versioning to datasets using DVC or LakeFS for reproducibility.
• Generate embeddings using HuggingFace or Sentence Transformers.
• Manage vector indexes (FAISS, Weaviate) and optimize retrieval workflows.
• Tokenize and chunk long-form data for context window optimization.

At General Motors we pride ourselves on designing, building and selling the world's best vehicles. We are seeking a new generation of visionaries to help launch bold engineering and business initiatives, and shape new directions for General Motors. As an intern you will gain hands-on career specific experiences to maximize your real-world potential.

Work Arrangement: This role is categorized as hybrid. This means the successful candidate is expected to report to the office at minimum three times per week or other frequency dictated by the business and two times per week remote.

Your Role:

As a Data Engineering and Analysis Intern, you’ll have the opportunity to gain hands-on experience in data-driven decision-making and contribute to our global sales operations. You will collaborate with senior team members, work on data collection, analysis, and reporting, and help optimize business processes using data science techniques.

What You'll Do:

• Assist in gathering and preparing data from various sources
• Clean, transform, and analyze data to ensure quality and accuracy
• Support data analysis, hypothesis testing, and exploratory data analysis (EDA)
• Help build and validate predictive models using basic machine learning techniques
• Generate reports and create data visualizations to highlight business insights
• Collaborate with cross-functional teams (Sales, Finance, CRM, Omnichannel) to support data-driven solutions
• Contribute to business performance tracking, including dashboards and KPIs
• Participate in training and mentorship opportunities to enhance your technical skills

Your Skills & Abilities:

• Currently undergoing bachelor's or master's degree in data science, Data Engineering, Business Analytics, or a related field
• Basic knowledge of programming languages (Python preferred) and data analysis tools (e.g., SQL, Excel)
• Basic knowledge of data visualization tools (Power BI, Tableau) is a plus
• Completed coursework in AI, machine learning, deep learning, and natural language processing (NLP)
• Worked on academic projects involving generative models (like GPT-3, GANs) or reinforcement learning

What We Offer:

• Mentorship from experienced professionals in business and data
• Exposure to real-world data challenges and solutions
• Opportunity to contribute to meaningful projects that impact business outcomes

In this pivotal role, you'll embark on an 18-month journey that empowers, challenges, and shapes you into a future leader.

You will explore the business through three tailored 3-month rotations, gaining multi-dimensional exposure and practical insights. In Phase 2, you'll dive deeper into a final 9-month strategic assignment, driving meaningful impact and solving real-world challenges.

This track immerses you in the critical domain of data management and engineering. Through rotations in Data Engineering, Data Products, and Data Integrity, you will develop expertise in designing, building, and maintaining robust data infrastructures and solutions.

• Key Requirements:
• Academic background in IT, Information Security, or Data Engineering
• Strong analytical and technical problem-solving skills
• Knowledge of data architecture, pipelines, and governance
• Attention to detail and commitment to data quality

Upon successful completion of the program, you'll be offered a permanent role at the next career level – the next phase of our Future Leader Journey.

• Enriching, real-world assignments
• Leadership exposure and mentorship
• Development opportunities tailored to your journey
• Access to a competitive benefits package including remote work flexibility and exclusive employee discounts

Job Summary

As the chief architect of data engineering, you will be responsible for designing and implementing a robust and scalable data architecture that supports operations, analytics, and regulatory reporting requirements.

You will lead a team of data engineers, ensuring high availability, security, and performance of our data infrastructure. You will work closely with stakeholders across technology, compliance, risk, and product teams to develop data pipelines, warehouses, and real-time analytics capabilities.

This is a strategic yet hands-on role where you will drive data engineering best practices, scalability, and automation while ensuring compliance with regulatory data requirements.

• Define and Own Data Architecture Strategy : Design and implement a data architecture strategy that ensures scalability, security, and regulatory compliance.
• Design and Implement Data Modeling, Governance, and Security Frameworks : Develop data modeling, governance, and security frameworks to meet financial and regulatory requirements.
• Architect Real-Time and Batch Processing Data Pipelines : Architect real-time and batch processing data pipelines to handle trade data, order books, user activity, and market analytics.
• Optimize Data Storage and Retrieval : Optimize data storage and retrieval for performance and cost efficiency, leveraging cloud-based and hybrid solutions.
• Build and Maintain ETL/ELT Data Pipelines : Build and maintain ETL/ELT data pipelines for operational, analytical, and compliance-related data.
• Ensure High Data Quality and Availability : Ensure high data quality, reliability, and availability through robust monitoring, alerting, and data validation techniques.
• Manage and Enhance Data Warehouses, Data Lakes, and Streaming Platforms : Manage and enhance data warehouses, data lakes, and streaming platforms to support business intelligence and machine learning use cases.
• Oversee Database Design and Optimization : Oversee database design and optimization for transactional and analytical workloads.
• Implement Data Lineage, Metadata Management, and Access Control Mechanisms : Implement data lineage, metadata management, and access control mechanisms in line with compliance requirements.

• Work Closely with Compliance and Risk Teams : Work closely with compliance and risk teams to ensure data retention policies, audit trails, and reporting mechanisms meet regulatory requirements.
• Implement Encryption, Anonymization, and Access Control Policies : Implement encryption, anonymization, and access control policies to safeguard sensitive user and transaction data.
• Support Fraud Detection and Risk Monitoring : Support fraud detection and risk monitoring through data analytics and alerting frameworks.

• Lead, Mentor, and Grow a Team of Data Engineers : Lead, mentor, and grow a small team of data engineers, fostering a culture of collaboration, innovation, and accountability.
• Drive Best Practices in Data Engineering : Drive best practices in data engineering, DevOps for data, and CI/CD automation for analytics infrastructure.
• Collaborate with Cross-Functional Teams : Collaborate with software engineers, DevOps, data analysis, and product teams to integrate data solutions into the broader exchange ecosystem.

• Proven Experience in Data Architecture and Engineering : Proven experience in data architecture and engineering, preferably within a regulated financial or crypto environment.
• Strong Proficiency in SQL, Python, or Scala : Strong proficiency in SQL, Python, or Scala for data engineering.
• Experience with Cloud-Based Data Platforms and Orchestration Tools : Experience with cloud-based data platforms (AWS, GCP, or Azure) and orchestration tools (Airflow, Prefect, Dagster).
• Hands-On Experience with Real-Time Data Processing : Hands-on experience with real-time data processing (Kafka, Pulsar, Flink, Spark Streaming).
• Expertise in Data Warehousing Solutions : Expertise in data warehousing solutions (Snowflake, BigQuery, Redshift, Databricks).
• Strong Understanding of Database Design, Indexing Strategies, and Query Optimization : Strong understanding of database design, indexing strategies, and query optimization.
• Experience Implementing Data Governance, Lineage, and Cataloging Tools : Experience implementing data governance, lineage, and cataloging tools.
• Familiarity with Blockchain/Crypto Data Structures and APIs : Familiarity with blockchain/crypto data structures and APIs is a plus.

• Experience Leading and Mentoring a Team of Data Engineers : Experience leading and mentoring a team of data engineers.
• Ability to Design Data Strategies that Align with Business Goals and Regulatory Requirements : Ability to design data strategies that align with business goals and regulatory requirements.
• Strong Cross-Functional Collaboration Skills : Strong cross-functional collaboration skills with compliance, risk, and technology teams.
• Ability to Work in a Fast-Paced, High-Growth Startup Environment : Ability to work in a fast-paced, high-growth startup environment with a hands-on approach.

• Experience in Regulated Financial Markets, Fintech, or Crypto : Experience in regulated financial markets, fintech, or crypto is highly preferred.
• Familiarity with Financial Data Standards, KYC/AML Reporting, and Regulatory Requirements : Familiarity with financial data standards, KYC/AML reporting, and regulatory requirements related to data handling.

Join to apply for the Manager of Data Engineering and Functional Consulting role at Dicetek LLC .

Get AI-powered advice on this job and more exclusive features.

• Bachelor's or Master's degree in Computer Science, Information Systems, Data Analytics, or a related field.
• Strong understanding of end-to-end data warehousing concepts, including data modeling, ETL/ELT, and data integration.
• Proficient in Azure Synapse, Databricks, Azure Data Factory, Snowflake, and Power BI for modern cloud-based data solutions.
• Skilled in handling structured, semi-structured (JSON, XML), and unstructured data across Azure and AWS platforms.
• Expertise in Power Query and Power M for advanced data transformation and reporting automation.
• Solid grasp of Agile methodologies, functional requirement gathering, and stakeholder communication.
• Familiar with metadata management, data lineage, and data quality principles to ensure trusted analytics.

Experience

• 10+ years of experience in designing, implementing, and optimizing end-to-end data warehouse solutions across cloud and on-premise ecosystems.
• Orchestrated enterprise DWH solutions using Azure Synapse, Databricks, and ADF, integrating structured and semi-structured data sources while ensuring lineage, auditability, and performance at scale.
• Translated business needs into scalable dimensional models and Power BI solutions, leveraging Power Query (M) scripting and DAX for self-service analytics.
• Designed and governed hybrid data pipelines across Azure and AWS, ensuring data consistency, compliance, and traceability from ingestion to visualization.
• Collaborated with stakeholders and engineering teams to define KPIs, automate reconciliations, and embed validation rules across ETL and reporting layers.
• Led multi-cloud integration of systems into a unified analytics layer, optimizing transformations with Databricks notebooks, ADF workflows, and parameterized datasets.
• Enabled real-time insights by architecting incremental refresh in Power BI and optimizing semantic models with measures, security, and models.
• Implemented data observability frameworks to proactively detect anomalies and ensure SLA adherence across reporting layers.

• Not Applicable

• Contract

• Engineering and Information Technology

• IT Services and IT Consulting

Referrals increase your chances of interviewing at Dicetek LLC by 2x.