72 Data Preprocessing jobs in Dubai

Job Title: Data Engineering Lead

We are seeking a highly skilled Data Engineering Lead to join our team. As a key member of the engineering team, you will be responsible for designing and building scalable data systems, including data lakes, ingestion pipelines, and storage solutions.

• Design and Build Scalable Data Systems: Design and build scalable data systems that can handle large volumes of structured and unstructured data.
• Ingestion Pipelines: Architect batch/stream pipelines using modern orchestration tools such as Airflow and streaming platforms like Kafka.
• Storage Solutions: Implement and tune S3 column-oriented and time-series data storage for petabyte-scale analytics.
• Tooling and Libraries: Develop internal libraries for schema management, data contracts validation, and lineage.
• Reliability and Observability: Embed monitoring, alerting, SLAs, SLOs, and CI/CD to ensure data reliability and observability.
• Collaboration: Partner with cross-functional teams to translate requirements into platform capabilities.

Required Skills and Qualifications:

• 6 years of experience building and maintaining production-grade data systems.
• Expert-level Python development skills, with knowledge of Go and C.
• Hands-on experience with modern orchestration tools and streaming platforms.
• Advanced SQL skills, including complex aggregations and query optimisation.
• Experience designing high-throughput APIs and data access libraries.
• Solid fundamentals in Linux containerisation and cloud object storage solutions.
• Strong knowledge of handling diverse data formats and storage strategies.
• English language proficiency at C1 level.

We are seeking a skilled data engineering specialist to join our team.

• This role involves designing and building a data early warning system, monitoring and optimizing pricing related thresholds, and supporting pricing algorithm changes.

• Design and develop data quality rules to monitor data inputs for anomalous data.
• Develop rules to monitor data drift and optimize thresholds.
• Set up alerts for the data quality rules.
• In-depth data investigation for any incidents.
• Propose and develop pricing algorithm changes as needed.

• Bachelor or Master's degree in Computer Science Engineering or related fields.
• 2-3 years of experience in development using Python.
• Working proficiency with Python, SQL, and/or Java.
• Experience in using statistical concepts to identify anomalies.
• Experience with cache databases (E.g. Redis, MongoDB).
• Experience with job schedulers (E.g. Airflow).
• Familiar with message queue systems (E.g. Kafka, Pub/Sub).
• Familiar with Kubernetes.
• Proficiency in English.

• Experience in using Cloud services (GCP, AWS, or Azure).
• Experience with Quality Assurance is desirable.
• Experience with Recommendation Systems is a plus.
• Knowledge of the cryptocurrency ecosystem and on-chain data is a plus.

• Not Applicable

• Full-time

• Engineering and Information Technology

**Job Title:** Data Engineering Specialist

The Role

As a Data Engineering Specialist, you will play a pivotal role in transforming complex data landscapes into actionable business insights. Your expertise will enable you to rapidly convert diverse new data sources into datasets that support our overarching business intelligence and analytics goals.

Main Responsibilities:

• Evaluate new data sources from SAP and Non-SAP systems
• Rapidly convert these data sources into datasets
• Develop and support the integration of new data streams into our Azure Data platform
• Ensure data is accessible, reliable, and structured for business insight generation

Required Skills and Qualifications:

• Expertise in data engineering and technical integration
• Ability to evaluate diverse data sources and convert them into usable datasets
• Familiarity with Azure Data platform and its features
• Excellent analytical and problem-solving skills

Benefits:

You will directly impact our ability to make data-driven decisions and maintain our competitive edge in the marketplace. Your contributions will be invaluable in driving business growth and success.

Others:

We offer a dynamic work environment and opportunities for professional growth and development.

Job Title: Data Engineering Specialist

We are seeking a highly skilled Data Engineer to join our team in a contract role with the possibility of extension.

The ideal candidate will have strong proficiency in SQL and at least one programming language (e.g., Python) for data manipulation and scripting.

They should also have experience with PySpark , ADF , Databricks , and SQL , as well as proficiency in data warehousing concepts and methodologies.

Azure Synapse and Azure Databricks knowledge is also essential, along with hand-on experience with data warehouse platforms (e.g., Snowflake, Redshift, BigQuery) and ETL tools (e.g., Informatica, Talend, Apache Spark).

The successful candidate will be able to implement data governance frameworks, ensure data quality, security, and compliance with industry standards and regulations.

They will also be responsible for developing complex SQL queries, managing relational databases, and establishing data lineage tracking within the data fabric.

Key Responsibilities:

• Design, develop, and maintain data pipelines for ingestion, transformation, and loading of data into the data warehouse.
• Develop and implement data governance frameworks to ensure data quality, security, and compliance.
• Manage relational databases and establish data lineage tracking within the data fabric.
• Collaborate with cross-functional teams, including data scientists, business analysts, and software engineers, to understand data requirements and deliver solutions that meet business objectives.

Data Engineering Specialist

A seasoned professional in data engineering with a strong background in designing and implementing end-to-end data pipelines using cloud platforms.

• Design, develop, and optimize Extract, Transform, Load (ETL) workflows for effective migration and transformation.
• Maintain data ingestion processes from diverse sources such as Salesforce, Oracle databases, PostgreSQL, and MySQL.
• Implement and maintain ETL processes and data pipelines to ensure efficient data extraction, transformation, and loading.
• Leverage Snowflake as the data warehouse solution for managing large volumes of structured and unstructured data.

This is an opportunity to work on complex data engineering projects, collaborating with cross-functional teams to deliver high-quality solutions.

Description:

We are seeking an experienced Data Engineering Expert to lead the development of our big data architecture. The successful candidate will design, implement, and maintain scalable and secure data pipelines using Spark, Hive, and Python on Cloudera.

Key Responsibilities:

• Develop real-time data workflows using Kafka.
• Design and develop APIs for data access and integration.
• Utilize Hue, Oozie, and other Cloudera tools for job orchestration and data access.
• Deploy solutions on cloud platforms such as AWS, Azure, or GCP.

Requirements:

• Over 7 years of experience in Big Data engineering.
• Hands-on experience with Cloudera Spark, Hive, Kafka, Python, Hue, and Ranger.
• Strong understanding of distributed systems and cloud data services.
• Proficient in API development and data security controls.

About this Opportunity:

This is a unique opportunity to join a rapidly growing IT services company with a presence in multiple countries, working with top clients in various sectors. Our team values teamwork, quality of life, and professional growth, offering a global community of professionals committed to your development.

What We Offer:

• A competitive salary and benefits package.
• Ongoing training and development opportunities.
• A dynamic and supportive work environment.
• The chance to work on exciting projects with cutting-edge technologies.

How to Apply:

If you are a motivated and experienced data engineer looking for a new challenge, please submit your application, including your resume and a cover letter outlining your qualifications and experience.

At General Motors we pride ourselves on designing, building and selling the world's best vehicles. We are seeking a new generation of visionaries to help launch bold engineering and business initiatives, and shape new directions for General Motors. As an intern you will gain hands-on career specific experiences to maximize your real-world potential.

Work Arrangement: This role is categorized as hybrid. This means the successful candidate is expected to report to the office at minimum three times per week or other frequency dictated by the business and two times per week remote.

Your Role:

As a Data Engineering and Analysis Intern, you'll have the opportunity to gain hands-on experience in data-driven decision-making and contribute to our global sales operations. You will collaborate with senior team members, work on data collection, analysis, and reporting, and help optimize business processes using data science techniques.

What You'll Do:

• Assist in gathering and preparing data from various sources
• Clean, transform, and analyze data to ensure quality and accuracy
• Support data analysis, hypothesis testing, and exploratory data analysis (EDA)
• Help build and validate predictive models using basic machine learning techniques
• Generate reports and create data visualizations to highlight business insights
• Collaborate with cross-functional teams (Sales, Finance, CRM, Omnichannel) to support data-driven solutions
• Contribute to business performance tracking, including dashboards and KPIs
• Participate in training and mentorship opportunities to enhance your technical skills

Your Skills & Abilities:

• Currently undergoing bachelor's or master's degree in data science, Data Engineering, Business Analytics, or a related field
• Basic knowledge of programming languages (Python preferred) and data analysis tools (e.g., SQL, Excel)
• Basic knowledge of data visualization tools (Power BI, Tableau) is a plus
• Completed coursework in AI, machine learning, deep learning, and natural language processing (NLP)
• Worked on academic projects involving generative models (like GPT-3, GANs) or reinforcement learning

What We Offer:

• Mentorship from experienced professionals in business and data
• Exposure to real-world data challenges and solutions
• Opportunity to contribute to meaningful projects that impact business outcomes

At General Motors we pride ourselves on designing, building and selling the world's best vehicles. We are seeking a new generation of visionaries to help launch bold engineering and business initiatives, and shape new directions for General Motors. As an intern you will gain hands-on career specific experiences to maximize your real-world potential.

Work Arrangement: This role is categorized as hybrid. This means the successful candidate is expected to report to the office at minimum three times per week or other frequency dictated by the business and two times per week remote.

Your Role:

As a Data Engineering and Analysis Intern, you’ll have the opportunity to gain hands-on experience in data-driven decision-making and contribute to our global sales operations. You will collaborate with senior team members, work on data collection, analysis, and reporting, and help optimize business processes using data science techniques.

What You'll Do:

• Assist in gathering and preparing data from various sources
• Clean, transform, and analyze data to ensure quality and accuracy
• Support data analysis, hypothesis testing, and exploratory data analysis (EDA)
• Help build and validate predictive models using basic machine learning techniques
• Generate reports and create data visualizations to highlight business insights
• Collaborate with cross-functional teams (Sales, Finance, CRM, Omnichannel) to support data-driven solutions
• Contribute to business performance tracking, including dashboards and KPIs
• Participate in training and mentorship opportunities to enhance your technical skills

Your Skills & Abilities:

• Currently undergoing bachelor's or master's degree in data science, Data Engineering, Business Analytics, or a related field
• Basic knowledge of programming languages (Python preferred) and data analysis tools (e.g., SQL, Excel)
• Basic knowledge of data visualization tools (Power BI, Tableau) is a plus
• Completed coursework in AI, machine learning, deep learning, and natural language processing (NLP)
• Worked on academic projects involving generative models (like GPT-3, GANs) or reinforcement learning

What We Offer:

• Mentorship from experienced professionals in business and data
• Exposure to real-world data challenges and solutions
• Opportunity to contribute to meaningful projects that impact business outcomes

As the Head of Data Engineering, you will be responsible for designing, implementing, and maintaining a robust, scalable, and compliant data architecture that supports our exchange's operations, analytics, and regulatory reporting requirements. You will lead a team of data engineers, ensuring high availability, security, and performance of our data infrastructure. You will work closely with stakeholders across technology, compliance, risk, and product teams to develop data pipelines, warehouses, and real-time analytics capabilities.

This is a strategic yet hands-on role where you will drive data engineering best practices, scalability, and automation while ensuring compliance with regulatory data requirements for a financial services entity.

Data Architecture & Strategy :

1. Define and own the data architecture strategy for the exchange, ensuring it is scalable, secure, and regulatory-compliant.
2. Design and implement data modeling, governance, and security frameworks to meet financial and regulatory requirements.
3. Architect real-time and batch processing data pipelines to handle trade data, order books, user activity, and market analytics.
4. Optimize data storage and retrieval for performance and cost efficiency, leveraging cloud-based and hybrid solutions.
5. Build and maintain ETL / ELT data pipelines for operational, analytical, and compliance-related data.
6. Ensure high data quality, reliability, and availability through robust monitoring, alerting, and data validation techniques.
7. Manage and enhance data warehouses, data lakes, and streaming platforms to support business intelligence and machine learning use cases.
8. Oversee database design and optimization for transactional and analytical workloads (e.g., Aurora, Redis, Kafka).
9. Implement data lineage, metadata management, and access control mechanisms in line with compliance requirements.

Compliance, Security & Risk Management :

1. Work closely with compliance and risk teams to ensure data retention policies, audit trails, and reporting mechanisms meet regulatory requirements (e.g., FATF, AML, GDPR, MiCA).
2. Implement encryption, anonymization, and access control policies to safeguard sensitive user and transaction data.
3. Support fraud detection and risk monitoring through data analytics and alerting frameworks.

Leadership & Team Management :

1. Lead, mentor, and grow a small team of data engineers, fostering a culture of collaboration, innovation, and accountability.
2. Drive best practices in data engineering, DevOps for data, and CI / CD automation for analytics infrastructure.
3. Collaborate with software engineers, DevOps, data analysis, and product teams to integrate data solutions into the broader exchange ecosystem.

Technical competencies and skills :

1. Proven experience in data architecture and engineering, preferably within a regulated financial or crypto environment.
2. Strong proficiency in SQL, Python, or Scala for data engineering.
3. Experience with cloud-based data platforms (AWS, GCP, or Azure) and orchestration tools (Airflow, Prefect, Dagster).
4. Hands-on experience with real-time data processing (Kafka, Pulsar, Flink, Spark Streaming).
5. Expertise in data warehousing solutions (Snowflake, BigQuery, Redshift, Databricks).
6. Strong understanding of database design, indexing strategies, and query optimization.
7. Experience implementing data governance, lineage, and cataloging tools.
8. Familiarity with blockchain / crypto data structures and APIs is a plus.

Leadership & Strategic Skills :

1. Experience leading and mentoring a team of data engineers.
2. Ability to design data strategies that align with business goals and regulatory requirements.
3. Strong cross-functional collaboration skills with compliance, risk, and technology teams.
4. Ability to work in a fast-paced, high-growth startup environment with a hands-on approach.

Industry & Compliance Knowledge :

1. Experience in regulated financial markets, fintech, or crypto is highly preferred.
2. Familiarity with financial data standards, KYC / AML reporting, and regulatory requirements related to data handling.

Preferred Qualifications :

1. Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.
2. Certifications in cloud data engineering (AWS / GCP / Azure), data governance, or security are a plus.
3. Experience working in a crypto exchange, trading platform, or high-frequency trading environment is an advantage.

At M2, we believe in a workplace where talent, dedication, and passion are the only factors that count, regardless of gender, background, age, and other characteristics. We embrace diversity because we know that it fuels innovation, fosters creativity, and drives success. So, if you're ready to join a team where your potential is truly valued, welcome aboard