203 Data Driven Products jobs in the United Arab Emirates

We are seeking a skilled data engineering specialist to join our team.

• This role involves designing and building a data early warning system, monitoring and optimizing pricing related thresholds, and supporting pricing algorithm changes.

• Design and develop data quality rules to monitor data inputs for anomalous data.
• Develop rules to monitor data drift and optimize thresholds.
• Set up alerts for the data quality rules.
• In-depth data investigation for any incidents.
• Propose and develop pricing algorithm changes as needed.

• Bachelor or Master's degree in Computer Science Engineering or related fields.
• 2-3 years of experience in development using Python.
• Working proficiency with Python, SQL, and/or Java.
• Experience in using statistical concepts to identify anomalies.
• Experience with cache databases (E.g. Redis, MongoDB).
• Experience with job schedulers (E.g. Airflow).
• Familiar with message queue systems (E.g. Kafka, Pub/Sub).
• Familiar with Kubernetes.
• Proficiency in English.

• Experience in using Cloud services (GCP, AWS, or Azure).
• Experience with Quality Assurance is desirable.
• Experience with Recommendation Systems is a plus.
• Knowledge of the cryptocurrency ecosystem and on-chain data is a plus.

• Not Applicable

• Full-time

• Engineering and Information Technology

Lead – Data Engineering role with experience in data engineering and cloud transformation. This position requires strong Azure Data Engineering and Databricks experience, with enterprise-level data warehouse workload expertise and PySpark development.

• Overall 5-7 years of experience in data engineering and transformation on Cloud
• 3+ years of strong experience in Azure Data Engineering, Databricks
• Expertise in supporting/developing data warehouse workloads at enterprise level
• Experience in PySpark is required – developing and deploying workloads to run on the Spark distributed computing
• Candidate must possess at least a Graduate or bachelor's degree in Computer Science/Information Technology, Engineering (Computer/Telecommunication) or equivalent
• Cloud deployment: Preferably Microsoft Azure
• Experience in implementing the platform and application monitoring using Cloud native tools

• Not Applicable

• Full-time

• Information Technology

• IT Services and IT Consulting

Data Engineering Specialist

A seasoned professional in data engineering with a strong background in designing and implementing end-to-end data pipelines using cloud platforms.

• Design, develop, and optimize Extract, Transform, Load (ETL) workflows for effective migration and transformation.
• Maintain data ingestion processes from diverse sources such as Salesforce, Oracle databases, PostgreSQL, and MySQL.
• Implement and maintain ETL processes and data pipelines to ensure efficient data extraction, transformation, and loading.
• Leverage Snowflake as the data warehouse solution for managing large volumes of structured and unstructured data.

This is an opportunity to work on complex data engineering projects, collaborating with cross-functional teams to deliver high-quality solutions.

**Job Title:** Data Engineering Specialist

The Role

As a Data Engineering Specialist, you will play a pivotal role in transforming complex data landscapes into actionable business insights. Your expertise will enable you to rapidly convert diverse new data sources into datasets that support our overarching business intelligence and analytics goals.

Main Responsibilities:

• Evaluate new data sources from SAP and Non-SAP systems
• Rapidly convert these data sources into datasets
• Develop and support the integration of new data streams into our Azure Data platform
• Ensure data is accessible, reliable, and structured for business insight generation

Required Skills and Qualifications:

• Expertise in data engineering and technical integration
• Ability to evaluate diverse data sources and convert them into usable datasets
• Familiarity with Azure Data platform and its features
• Excellent analytical and problem-solving skills

Benefits:

You will directly impact our ability to make data-driven decisions and maintain our competitive edge in the marketplace. Your contributions will be invaluable in driving business growth and success.

Others:

We offer a dynamic work environment and opportunities for professional growth and development.

Description:

We are seeking an experienced Data Engineering Expert to lead the development of our big data architecture. The successful candidate will design, implement, and maintain scalable and secure data pipelines using Spark, Hive, and Python on Cloudera.

Key Responsibilities:

• Develop real-time data workflows using Kafka.
• Design and develop APIs for data access and integration.
• Utilize Hue, Oozie, and other Cloudera tools for job orchestration and data access.
• Deploy solutions on cloud platforms such as AWS, Azure, or GCP.

Requirements:

• Over 7 years of experience in Big Data engineering.
• Hands-on experience with Cloudera Spark, Hive, Kafka, Python, Hue, and Ranger.
• Strong understanding of distributed systems and cloud data services.
• Proficient in API development and data security controls.

About this Opportunity:

This is a unique opportunity to join a rapidly growing IT services company with a presence in multiple countries, working with top clients in various sectors. Our team values teamwork, quality of life, and professional growth, offering a global community of professionals committed to your development.

What We Offer:

• A competitive salary and benefits package.
• Ongoing training and development opportunities.
• A dynamic and supportive work environment.
• The chance to work on exciting projects with cutting-edge technologies.

How to Apply:

If you are a motivated and experienced data engineer looking for a new challenge, please submit your application, including your resume and a cover letter outlining your qualifications and experience.

As the Head of Data Engineering, you will be responsible for designing, implementing, and maintaining a robust, scalable, and compliant data architecture that supports our exchange’s operations, analytics, and regulatory reporting requirements. You will lead a team of data engineers, ensuring high availability, security, and performance of our data infrastructure. You will work closely with stakeholders across technology, compliance, risk, and product teams to develop data pipelines, warehouses, and real-time analytics capabilities.

This is a strategic yet hands-on role where you will drive data engineering best practices, scalability, and automation while ensuring compliance with regulatory data requirements for a financial services entity.

Data Architecture & Strategy :

1. Define and own the data architecture strategy for the exchange, ensuring it is scalable, secure, and regulatory-compliant.
2. Design and implement data modeling, governance, and security frameworks to meet financial and regulatory requirements.
3. Architect real-time and batch processing data pipelines to handle trade data, order books, user activity, and market analytics.
4. Optimize data storage and retrieval for performance and cost efficiency, leveraging cloud-based and hybrid solutions.
5. Build and maintain ETL / ELT data pipelines for operational, analytical, and compliance-related data.
6. Ensure high data quality, reliability, and availability through robust monitoring, alerting, and data validation techniques.
7. Manage and enhance data warehouses, data lakes, and streaming platforms to support business intelligence and machine learning use cases.
8. Oversee database design and optimization for transactional and analytical workloads (e.g., Aurora, Redis, Kafka).
9. Implement data lineage, metadata management, and access control mechanisms in line with compliance requirements.

Compliance, Security & Risk Management :

1. Work closely with compliance and risk teams to ensure data retention policies, audit trails, and reporting mechanisms meet regulatory requirements (e.g., FATF, AML, GDPR, MiCA).
2. Implement encryption, anonymization, and access control policies to safeguard sensitive user and transaction data.
3. Support fraud detection and risk monitoring through data analytics and alerting frameworks.

Leadership & Team Management :

1. Lead, mentor, and grow a small team of data engineers, fostering a culture of collaboration, innovation, and accountability.
2. Drive best practices in data engineering, DevOps for data, and CI / CD automation for analytics infrastructure.
3. Collaborate with software engineers, DevOps, data analysis, and product teams to integrate data solutions into the broader exchange ecosystem.

Technical competencies and skills :

1. Proven experience in data architecture and engineering, preferably within a regulated financial or crypto environment.
2. Strong proficiency in SQL, Python, or Scala for data engineering.
3. Experience with cloud-based data platforms (AWS, GCP, or Azure) and orchestration tools (Airflow, Prefect, Dagster).
4. Hands-on experience with real-time data processing (Kafka, Pulsar, Flink, Spark Streaming).
5. Expertise in data warehousing solutions (Snowflake, BigQuery, Redshift, Databricks).
6. Strong understanding of database design, indexing strategies, and query optimization.
7. Experience implementing data governance, lineage, and cataloging tools.
8. Familiarity with blockchain / crypto data structures and APIs is a plus.

Leadership & Strategic Skills :

1. Experience leading and mentoring a team of data engineers.
2. Ability to design data strategies that align with business goals and regulatory requirements.
3. Strong cross-functional collaboration skills with compliance, risk, and technology teams.
4. Ability to work in a fast-paced, high-growth startup environment with a hands-on approach.

Industry & Compliance Knowledge :

1. Experience in regulated financial markets, fintech, or crypto is highly preferred.
2. Familiarity with financial data standards, KYC / AML reporting, and regulatory requirements related to data handling.

Preferred Qualifications :

1. Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.
2. Certifications in cloud data engineering (AWS / GCP / Azure), data governance, or security are a plus.
3. Experience working in a crypto exchange, trading platform, or high-frequency trading environment is an advantage.

At M2, we believe in a workplace where talent, dedication, and passion are the only factors that count, regardless of gender, background, age, and other characteristics. We embrace diversity because we know that it fuels innovation, fosters creativity, and drives success. So, if you're ready to join a team where your potential is truly valued, welcome aboard!

Direct message the job poster from Edison Smart

Web3, Crypto & Blockchain Recruiter | Helping Leading Companies Hire Top Talent | Retired Professional Rugby Player

Position : Head of Data Engineering Location : Dubai / Abu Dhabi (Relocation support available)

Employment Type : Full-Time | Senior Leadership

We are conducting a confidential executive search for a Head of Data Engineering to join one of the world's fastest-growing digital asset trading platforms, serving millions of users globally with multi-billion daily trading volumes. This is a rare opportunity to lead a world-class data organization, driving scalability, reliability, and innovation in a high-performance environment.

Key Responsibilities :

• Lead and scale the Data Engineering function, overseeing global teams responsible for data pipelines, storage, and real-time processing at massive scale.
• Design and implement a robust, future-proof data architecture to support analytics, AI / ML initiatives, risk management, and next-generation trading products.
• Collaborate with Product, Engineering, Quant, and Business teams to ensure data availability, integrity, and accessibility across all platforms.
• Build and optimize large-scale distributed data systems, ensuring ultra-low latency and high fault tolerance.
• Drive data strategy and best practices, focusing on governance, lineage, and security across all datasets.
• Manage global stakeholders, providing thought leadership in data engineering, tooling, and emerging technologies.
• Hire, mentor, and inspire top-tier data engineering talent to support exponential growth.

Key Requirements :

About Edison Smart

Edison Smart is a global provider of specialized recruitment solutions, supporting the technology industry's most innovative companies. Headquartered in the UK, we connect talent with opportunity, driving the 'Industry 4.0' revolution. With over 30 years of experience, Edison Smart partners with forward-thinking organizations to help achieve their strategic growth goals while fostering groundbreaking technological advancements.

Seniority level

Seniority level

Executive

Employment type

Employment type

Full-time

Job function

Job function

Finance

Industries

Banking, Financial Services, and IT Services and IT Consulting

Referrals increase your chances of interviewing at Edison Smart by 2x

Sign in to set job alerts for "Head of Engineering" roles.

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 5 days ago

Head of Engineering - Linux Platform & Automation

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 days ago

Abu Dhabi Emirate, United Arab Emirates 1 week ago

Director of Engineering - LXR Eastern Mangroves Abu Dhabi (Pre-Opening)

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 5 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 1 month ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 3 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 2 months ago

Senior Manager - Software Quality Assurance

Abu Dhabi Emirate, United Arab Emirates 1 week ago

Abu Dhabi Emirate, United Arab Emirates 1 week ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 6 days ago

Al Zahiyah, Abu Dhabi Emirate, United Arab Emirates 2 days ago

Abu Dhabi Emirate, United Arab Emirates 1 week ago

Abu Dhabi Emirate, United Arab Emirates 4 weeks ago

Head of Engineer- Civil & Structural (Substation)

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 months ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 4 days ago

Abu Dhabi, Abu Dhabi Emirate, United Arab Emirates 5 days ago

Field Protection Engineer- Abu Dhabi, UAE

Abu Dhabi Emirate, United Arab Emirates 3 days ago

Lead Onshore Commissioning Engineer (Offshore WHT's , Subsea Cable & Piping)

We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

J-18808-Ljbffr

As the Head of Data Engineering, you will be responsible for designing, implementing, and maintaining a robust, scalable, and compliant data architecture that supports our exchange's operations, analytics, and regulatory reporting requirements. You will lead a team of data engineers, ensuring high availability, security, and performance of our data infrastructure. You will work closely with stakeholders across technology, compliance, risk, and product teams to develop data pipelines, warehouses, and real-time analytics capabilities.

This is a strategic yet hands-on role where you will drive data engineering best practices, scalability, and automation while ensuring compliance with regulatory data requirements for a financial services entity.

Data Architecture & Strategy :

1. Define and own the data architecture strategy for the exchange, ensuring it is scalable, secure, and regulatory-compliant.
2. Design and implement data modeling, governance, and security frameworks to meet financial and regulatory requirements.
3. Architect real-time and batch processing data pipelines to handle trade data, order books, user activity, and market analytics.
4. Optimize data storage and retrieval for performance and cost efficiency, leveraging cloud-based and hybrid solutions.
5. Build and maintain ETL / ELT data pipelines for operational, analytical, and compliance-related data.
6. Ensure high data quality, reliability, and availability through robust monitoring, alerting, and data validation techniques.
7. Manage and enhance data warehouses, data lakes, and streaming platforms to support business intelligence and machine learning use cases.
8. Oversee database design and optimization for transactional and analytical workloads (e.g., Aurora, Redis, Kafka).
9. Implement data lineage, metadata management, and access control mechanisms in line with compliance requirements.

Compliance, Security & Risk Management :

1. Work closely with compliance and risk teams to ensure data retention policies, audit trails, and reporting mechanisms meet regulatory requirements (e.g., FATF, AML, GDPR, MiCA).
2. Implement encryption, anonymization, and access control policies to safeguard sensitive user and transaction data.
3. Support fraud detection and risk monitoring through data analytics and alerting frameworks.

Leadership & Team Management :

1. Lead, mentor, and grow a small team of data engineers, fostering a culture of collaboration, innovation, and accountability.
2. Drive best practices in data engineering, DevOps for data, and CI / CD automation for analytics infrastructure.
3. Collaborate with software engineers, DevOps, data analysis, and product teams to integrate data solutions into the broader exchange ecosystem.

Technical competencies and skills :

1. Proven experience in data architecture and engineering, preferably within a regulated financial or crypto environment.
2. Strong proficiency in SQL, Python, or Scala for data engineering.
3. Experience with cloud-based data platforms (AWS, GCP, or Azure) and orchestration tools (Airflow, Prefect, Dagster).
4. Hands-on experience with real-time data processing (Kafka, Pulsar, Flink, Spark Streaming).
5. Expertise in data warehousing solutions (Snowflake, BigQuery, Redshift, Databricks).
6. Strong understanding of database design, indexing strategies, and query optimization.
7. Experience implementing data governance, lineage, and cataloging tools.
8. Familiarity with blockchain / crypto data structures and APIs is a plus.

Leadership & Strategic Skills :

1. Experience leading and mentoring a team of data engineers.
2. Ability to design data strategies that align with business goals and regulatory requirements.
3. Strong cross-functional collaboration skills with compliance, risk, and technology teams.
4. Ability to work in a fast-paced, high-growth startup environment with a hands-on approach.

Industry & Compliance Knowledge :

1. Experience in regulated financial markets, fintech, or crypto is highly preferred.
2. Familiarity with financial data standards, KYC / AML reporting, and regulatory requirements related to data handling.

Preferred Qualifications :

1. Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.
2. Certifications in cloud data engineering (AWS / GCP / Azure), data governance, or security are a plus.
3. Experience working in a crypto exchange, trading platform, or high-frequency trading environment is an advantage.

At M2, we believe in a workplace where talent, dedication, and passion are the only factors that count, regardless of gender, background, age, and other characteristics. We embrace diversity because we know that it fuels innovation, fosters creativity, and drives success. So, if you're ready to join a team where your potential is truly valued, welcome aboard

Job Opportunity:

A seasoned data engineer with 5 years of experience in designing and maintaining data pipelines using Python. Expertise includes working with Python, SQL databases, Snowflake, Apache Airflow, and Shell scripting.

Main Responsibilities:

• Design, develop, and maintain robust data pipelines to support business operations and analytics using Python.
• Integrate APIs to fetch, process, and transform data for various use cases.
• Work with multiple SQL databases to perform data transformations, migrations, and optimizations.
• BUILD and manage data warehouses using Snowflake ensuring high performance and scalability.
• Implement and manage workflows to automate data pipeline orchestration.
• Develop and maintain Shell scripts to automate tasks and streamline processes.
• Ensure data quality, consistency, and security across all platforms.
• Collaborate with stakeholders to understand data requirements and deliver solutions.
• Monitor, troubleshoot, and optimize data pipelines for efficiency and reliability.

Required Skills and Qualifications:

• Programming: Strong proficiency in Python with experience in API integration and data processing.
• Data Warehousing: Hands-on experience with Snowflake including schema design, data modeling, and performance optimization.
• Workflow Automation: Proficiency in Apache Airflow for managing and orchestrating workflows.
• Scripting: Strong skills in Shell scripting for automation and system management.
• ETL/ELT Processes: Solid understanding of designing and maintaining ETL/ELT pipelines.
• Problem Solving: Strong analytical and problem-solving skills to handle complex data challenges.
• Communication: Excellent communication skills to work effectively with cross-functional teams.