190 Data Preprocessing jobs in the United Arab Emirates

Job Title: Data Engineering Lead

We are seeking a highly skilled Data Engineering Lead to join our team. As a key member of the engineering team, you will be responsible for designing and building scalable data systems, including data lakes, ingestion pipelines, and storage solutions.

• Design and Build Scalable Data Systems: Design and build scalable data systems that can handle large volumes of structured and unstructured data.
• Ingestion Pipelines: Architect batch/stream pipelines using modern orchestration tools such as Airflow and streaming platforms like Kafka.
• Storage Solutions: Implement and tune S3 column-oriented and time-series data storage for petabyte-scale analytics.
• Tooling and Libraries: Develop internal libraries for schema management, data contracts validation, and lineage.
• Reliability and Observability: Embed monitoring, alerting, SLAs, SLOs, and CI/CD to ensure data reliability and observability.
• Collaboration: Partner with cross-functional teams to translate requirements into platform capabilities.

Required Skills and Qualifications:

• 6 years of experience building and maintaining production-grade data systems.
• Expert-level Python development skills, with knowledge of Go and C.
• Hands-on experience with modern orchestration tools and streaming platforms.
• Advanced SQL skills, including complex aggregations and query optimisation.
• Experience designing high-throughput APIs and data access libraries.
• Solid fundamentals in Linux containerisation and cloud object storage solutions.
• Strong knowledge of handling diverse data formats and storage strategies.
• English language proficiency at C1 level.

Job Title: Data Engineering Specialist

About the Role:

• Create backend services that optimize data processing for IoT applications.
• Collaborate with cross-functional teams to develop and implement new features.
• Maintain and improve existing systems, ensuring high performance and quality.
• Analyze application performance, identify bottlenecks, and resolve issues promptly.
• Enhance code organization, maintainability, and automation practices.

Requirements:

• Proven experience in backend software development or similar role.
• Demonstrated expertise in programming languages such as PHP and Python.
• Strong knowledge of MySQL and database management systems.
• Familiarity with RESTful APIs and web services.
• Understanding of software development principles and best practices.
• Excellent problem-solving skills and ability to work effectively in a team.

We are seeking a skilled data engineering specialist to join our team.

• This role involves designing and building a data early warning system, monitoring and optimizing pricing related thresholds, and supporting pricing algorithm changes.

• Design and develop data quality rules to monitor data inputs for anomalous data.
• Develop rules to monitor data drift and optimize thresholds.
• Set up alerts for the data quality rules.
• In-depth data investigation for any incidents.
• Propose and develop pricing algorithm changes as needed.

• Bachelor or Master's degree in Computer Science Engineering or related fields.
• 2-3 years of experience in development using Python.
• Working proficiency with Python, SQL, and/or Java.
• Experience in using statistical concepts to identify anomalies.
• Experience with cache databases (E.g. Redis, MongoDB).
• Experience with job schedulers (E.g. Airflow).
• Familiar with message queue systems (E.g. Kafka, Pub/Sub).
• Familiar with Kubernetes.
• Proficiency in English.

• Experience in using Cloud services (GCP, AWS, or Azure).
• Experience with Quality Assurance is desirable.
• Experience with Recommendation Systems is a plus.
• Knowledge of the cryptocurrency ecosystem and on-chain data is a plus.

• Not Applicable

• Full-time

• Engineering and Information Technology

Lead – Data Engineering role with experience in data engineering and cloud transformation. This position requires strong Azure Data Engineering and Databricks experience, with enterprise-level data warehouse workload expertise and PySpark development.

• Overall 5-7 years of experience in data engineering and transformation on Cloud
• 3+ years of strong experience in Azure Data Engineering, Databricks
• Expertise in supporting/developing data warehouse workloads at enterprise level
• Experience in PySpark is required – developing and deploying workloads to run on the Spark distributed computing
• Candidate must possess at least a Graduate or bachelor's degree in Computer Science/Information Technology, Engineering (Computer/Telecommunication) or equivalent
• Cloud deployment: Preferably Microsoft Azure
• Experience in implementing the platform and application monitoring using Cloud native tools

• Not Applicable

• Full-time

• Information Technology

• IT Services and IT Consulting

WRS Health seeks a skilled and strategic Director of Data Architecture to lead our data platform initiatives on Amazon Web Services (AWS) with a strong emphasis on healthcare data, data lake strategy, and AI enablement.

This role involves architecting and managing scalable, secure, and high-performance data systems that support both traditional analytics and modern ML workloads, including embedding models and vectorized data retrieval.

• Strategic Data Platform Leadership: Develop an enterprise-wide data architecture strategy that supports interoperability, AI/ML readiness, and regulatory compliance.
• Data Lake & Storage Optimization: Design and maintain scalable, secure, and cost-effective data lakes using Amazon S3, AWS Glue, Athena, Redshift, and Lake Formation.
• AI/ML Enablement and Vector Infrastructure: Collaborate with data science teams to implement embedding models, vectorization pipelines, and real-time inference architectures.
• Healthcare Data Architecture & Interoperability: Architect systems to ingest, process, and store FHIR-compliant JSON data from EHRs, APIs, and HL7 sources.

• Leadership Opportunities: Build and lead a high-performing team of data architects and engineers.
• Cross-Functional Collaboration: Work closely with stakeholders from engineering, data science, product, and compliance teams to deliver data initiatives.
• Professional Development: Promote data literacy and foster a culture of innovation and continuous improvement.

**Job Title:** Data Engineering Specialist

The Role

As a Data Engineering Specialist, you will play a pivotal role in transforming complex data landscapes into actionable business insights. Your expertise will enable you to rapidly convert diverse new data sources into datasets that support our overarching business intelligence and analytics goals.

Main Responsibilities:

• Evaluate new data sources from SAP and Non-SAP systems
• Rapidly convert these data sources into datasets
• Develop and support the integration of new data streams into our Azure Data platform
• Ensure data is accessible, reliable, and structured for business insight generation

Required Skills and Qualifications:

• Expertise in data engineering and technical integration
• Ability to evaluate diverse data sources and convert them into usable datasets
• Familiarity with Azure Data platform and its features
• Excellent analytical and problem-solving skills

Benefits:

You will directly impact our ability to make data-driven decisions and maintain our competitive edge in the marketplace. Your contributions will be invaluable in driving business growth and success.

Others:

We offer a dynamic work environment and opportunities for professional growth and development.

Job Title: Data Engineering Specialist

We are seeking a highly skilled Data Engineer to join our team in a contract role with the possibility of extension.

The ideal candidate will have strong proficiency in SQL and at least one programming language (e.g., Python) for data manipulation and scripting.

They should also have experience with PySpark , ADF , Databricks , and SQL , as well as proficiency in data warehousing concepts and methodologies.

Azure Synapse and Azure Databricks knowledge is also essential, along with hand-on experience with data warehouse platforms (e.g., Snowflake, Redshift, BigQuery) and ETL tools (e.g., Informatica, Talend, Apache Spark).

The successful candidate will be able to implement data governance frameworks, ensure data quality, security, and compliance with industry standards and regulations.

They will also be responsible for developing complex SQL queries, managing relational databases, and establishing data lineage tracking within the data fabric.

Key Responsibilities:

• Design, develop, and maintain data pipelines for ingestion, transformation, and loading of data into the data warehouse.
• Develop and implement data governance frameworks to ensure data quality, security, and compliance.
• Manage relational databases and establish data lineage tracking within the data fabric.
• Collaborate with cross-functional teams, including data scientists, business analysts, and software engineers, to understand data requirements and deliver solutions that meet business objectives.

Data Engineering Specialist

A seasoned professional in data engineering with a strong background in designing and implementing end-to-end data pipelines using cloud platforms.

• Design, develop, and optimize Extract, Transform, Load (ETL) workflows for effective migration and transformation.
• Maintain data ingestion processes from diverse sources such as Salesforce, Oracle databases, PostgreSQL, and MySQL.
• Implement and maintain ETL processes and data pipelines to ensure efficient data extraction, transformation, and loading.
• Leverage Snowflake as the data warehouse solution for managing large volumes of structured and unstructured data.

This is an opportunity to work on complex data engineering projects, collaborating with cross-functional teams to deliver high-quality solutions.

Description:

We are seeking an experienced Data Engineering Expert to lead the development of our big data architecture. The successful candidate will design, implement, and maintain scalable and secure data pipelines using Spark, Hive, and Python on Cloudera.

Key Responsibilities:

• Develop real-time data workflows using Kafka.
• Design and develop APIs for data access and integration.
• Utilize Hue, Oozie, and other Cloudera tools for job orchestration and data access.
• Deploy solutions on cloud platforms such as AWS, Azure, or GCP.

Requirements:

• Over 7 years of experience in Big Data engineering.
• Hands-on experience with Cloudera Spark, Hive, Kafka, Python, Hue, and Ranger.
• Strong understanding of distributed systems and cloud data services.
• Proficient in API development and data security controls.

About this Opportunity:

This is a unique opportunity to join a rapidly growing IT services company with a presence in multiple countries, working with top clients in various sectors. Our team values teamwork, quality of life, and professional growth, offering a global community of professionals committed to your development.

What We Offer:

• A competitive salary and benefits package.
• Ongoing training and development opportunities.
• A dynamic and supportive work environment.
• The chance to work on exciting projects with cutting-edge technologies.

How to Apply:

If you are a motivated and experienced data engineer looking for a new challenge, please submit your application, including your resume and a cover letter outlining your qualifications and experience.