283 Big Data Hadoop jobs in the United Arab Emirates

Position Summary:

This is an excellent opportunity to utilize your exceptional data entry skills and experience in a dynamic environment. As a highly skilled detail-oriented data specialist, you will be responsible for accurately entering data into our system, ensuring timely and efficient processing.

Key Responsibilities:

• Enter data accurately and efficiently into our system
• Verify data for accuracy and completeness
• Maintain records and databases with up-to-date information
• Analyze data to identify trends and patterns
• Develop and implement data entry processes and procedures

Requirements and Skills:

• Strong computer skills and familiarity with data entry software
• Excellent attention to detail and organizational skills
• Able to work independently and as part of a team
• Good communication and problem-solving skills
• Ability to learn new software and systems quickly

This is a critical role that involves designing, building, and maintaining scalable data pipelines to enable real-time processing and analysis of streaming data. The ideal candidate will be adept at handling large volumes of data from various sources, ensuring data quality, and optimizing data flow.

• Design and develop real-time data processing systems to support business operations.
• Build and maintain scalable data pipelines to process streaming data efficiently.
• Collaborate with data scientists and analysts to accommodate analytical requirements.
• Optimize performance of data processing systems to improve speed and reliability.
• Ensure data quality and consistency across various data sources and systems.
• Monitor and troubleshoot data pipeline issues to maintain seamless operations.
• Integrate streaming data platforms with enterprise data architecture and frameworks.
• Implement data governance and security policies to protect sensitive information.
• Stay updated with emerging technologies and tools in real-time data processing.
• Contribute to the continuous improvement of data engineering practices and methodologies.
• Document all processes and pipelines to maintain a thorough engineering knowledge base.

• Bachelor's degree in Computer Science, Engineering, or a related technical field.
• Proven experience in data engineering, particularly with real-time streaming data.
• Proficiency in programming languages such as Python, Java, or Scala.
• Experience with streaming platforms like Apache Kafka, Flink, or Apache Storm.
• Strong understanding of data pipelines, ETL processes, and data warehousing solutions.
• Knowledge of cloud platforms such as AWS, Google Cloud Platform, or Azure.
• Excellent problem-solving skills and ability to work under tight deadlines.

This mid-level role is integral to businesses that rely on instantaneous data insights to drive decisions, improve operations, and enhance customer experiences. As a vital member of the technology team, you will work closely with data scientists, analysts, and software engineers to implement robust data solutions that support a variety of analytics and machine learning projects.

A competitive compensation package, opportunities for growth and development, and a collaborative work environment make this an attractive opportunity for experienced professionals looking to take their careers to the next level.

Role: Senior Lead Software Engineer

Skill: Big Data Engineer

Experience: 7 Years

Strong functional knowledge and experience in the banking domain.

Develop and optimize data pipelines using Spark, Hive, and Python on Cloudera.

Develop real-time data workflows using Kafka.

Design and develop APIs for data access and integration.

Utilize Hue, Oozie, and other Cloudera tools for job orchestration and data access.

Deploy solutions on cloud platforms such as AWS, Azure, or GCP.

Required Skills and Experience:

Over 7 years of experience in Big Data engineering.

Hands-on experience with Cloudera Spark, Hive, Kafka, Python, Hue, and Ranger.

Strong understanding of distributed systems and cloud data services.

Proficient in API development and data security controls.

About Virtusa:

Join Virtusa and gain international experience working on leading Digital Transformation programs in the Middle East.

Virtusa is a rapidly growing IT services company with a presence in the UAE, KSA, Qatar, and Oman, working with top clients in banking, finance, travel, telecom, and enterprise sectors. We have received awards from Gartner, IDC, WfMC, and others for our exceptional work.

Be part of our award-winning team that values teamwork, quality of life, and professional growth. Join a global community of 30,000 professionals committed to your development, working on exciting projects with cutting-edge technologies.

Apt Resources is seeking an experiencedBig Data Engineer for agovernment client in Abu Dhabi . You will design and implement large-scale data solutions to support AI/ML initiatives and public sector digital transformation.

Data Pipeline Development :

• Build robust data pipelines usingPython SQL/NoSQL and Airflow
• DevelopETL/ELT processes for structured/unstructured data
• Managedata lakes and optimize storage solutions

Data Infrastructure :

• Design efficientdata models for analytics
• Implementdata governance and quality frameworks
• Work withcloud-based data platforms (Azure preferred)

AI/ML Support :

• Prepare and process datasets for machine learning applications
• Collaborate with ML teams on feature engineering

• 10-12 years of hands-on big data experience
• Expertise in:
• Python andSQL/NoSQL databases
• Airflow for workflow orchestration
• ETL/ELT pipeline development
• Cloud data platforms (Azure AWS or GCP)

To be discussed

Are you an experienced data professional seeking a challenging role in a dynamic environment? We have an exciting opportunity for a skilled Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and maintaining large-scale data pipelines that meet the organization's business needs.

• Design and develop highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform
• Implement and manage data ingestion processes from various sources to the data lake or data warehouse
• Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs
• Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes
• Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline

• Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field
• 3+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform
• Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques
• Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase

• A competitive salary and benefits package
• Opportunities for career growth and professional development
• A collaborative and dynamic work environment

Location : Dubai

Who Can Apply: Candidates who are currently in Dubai
Job Type: Contract
Experience: Minimum 8+ years

Job Summary:

We are looking for an experienced Data Engineer to design, develop, and optimize data pipelines, ETL processes, and data integration solutions. The ideal candidate should have expertise in AWS cloud services, data engineering best practices, open-source tools, and data schema design. The role requires hands-on experience with large-scale data processing, real-time data streaming, and cloud-based data architectures.

Key Responsibilities:

• Develop and Maintain Data Pipelines to process structured and unstructured data efficiently.
• Implement ETL/ELT Workflows for batch and real-time data processing.
• Optimize Data Processing Workflows using distributed computing frameworks.
• Ensure Data Integrity and Quality through data validation, cleaning, and transformation techniques.
• Work with AWS Cloud Services , including S3, Redshift, Glue, Lambda, DynamoDB, and Kinesis.
• Leverage Open-Source Tools like Apache Spark, Airflow, Kafka, and Flink for data processing.
• Manage and Optimize Database Performance for both SQL and NoSQL environments.
• Collaborate with Data Scientists and Analysts to enable AI/ML model deployment and data accessibility.
• Support Data Migration Initiatives from on-premise to cloud-based data platforms.
• Ensure Compliance and Security Standards in handling sensitive and regulated data.
• Develop Data Models and Schemas for efficient storage and retrieval.

Required Skills & Qualifications:

• 8+ years of experience in data engineering, data architecture, and cloud computing.
• Strong knowledge of AWS Services such as Glue, Redshift, Athena, Lambda, and S3.
• Expertise in ETL Tools , including Talend, Apache NiFi, Informatica, dbt, and AWS Glue.
• Proficiency in Open-Source Tools such as Apache Spark, Hadoop, Airflow, Kafka, and Flink.
• Strong Programming Skills in Python, SQL, and Scala.
• Experience in Data Schema Design , normalization, and performance optimization.
• Knowledge of Real-time Data Streaming using Kafka, Kinesis, or Apache Flink.
• Experience in Data Warehouse and Data Lake Solutions .
• Hands-on experience with DevOps and CI/CD Pipelines for data engineering workflows.
• Understanding of AI and Machine Learning Data Pipelines .
• Strong analytical and problem-solving skills .

Preferred Qualifications:

• AWS Certified Data Analytics – Specialty or AWS Solutions Architect certification.
• Experience with Kubernetes, Docker, and serverless data processing.
• Exposure to MLOps and data engineering practices for AI/ML solutions.
• Experience with distributed computing and big data frameworks.

The Data Engineer will be responsible for developing semantic models on top of the Data Lake/Data Warehouse to fulfill the self-service BI foundation requirements. This includes data extraction from various data sources and integration into the central data lake/data warehouse using enterprise platforms like Informatica iPaaS.

1. Designing data warehouse data models based on business requirements.
2. Designing, developing, and testing both batch and real-time Extract, Transform and Load (ETL) processes required for data integration.
3. Ingesting both structured and unstructured data into the SMBU data lake/data warehouse system.
4. Designing and developing semantic models/self-service cubes.
5. Performing BI administration and access management to ensure access and reports are properly governed.
6. Performing unit testing and data validation to ensure business UAT is successful.
7. Performing ad-hoc data analysis and presenting results in a clear manner.
8. Assessing data quality of the source systems and proposing enhancements to achieve a satisfactory level of data accuracy.
9. Optimizing ETL processes to ensure execution time meets requirements.
10. Maintaining and architecting ETL pipelines to ensure data is loaded on time on a regular basis.

1. 5 to 8 years of overall experience.
2. Proven experience in the development of dimensional models in Azure Synapse with strong SQL knowledge.
3. Minimum of 3 years working as a Data Engineer in the Azure ecosystem specifically using Synapse, ADF & Data bricks.
4. Preferably 3 years of experience with data warehousing, ETL development, SQL Queries, Synapse, ADF, PySpark, and Informatica iPaaS for data ingestion & data modeling.

Dubai, United Arab Emirates | Posted on 07/29/2025

myZoiis changing lives for the better for those who deserve it the most. We are an excitingfintech start-up aiming to promote financial inclusion globally. Our vision isto provide a level playing field to the unbanked and the underbanked inaccessing essential financial services in an affordable, convenient, andtransparent fashion. We are looking for smart, ambitious, and purpose-drivenindividuals to join us in this journey.Please apply via the link below ifyou are interested.

You will beworking in our Data Platform team, providing data capability for internal andproduct requirements for myZoi. You will be proactive and innovative and youwill be using 100% cloud technologies based on AWS and modern Open Sourcetooling to provide a real-time data infrastructure, allowing our teams to gainunprecedented insight into our wealth of application data. You will work with aworld-class team of Data Analysts and Engineers to provide best in classsolutions.

ArchitectAWS-Centric Data Solutions:

• Designand optimize high-performance data pipelines leveraging AWS native tools.
  
• Architectmodular, AI-ready data lake with a roadmap to ensure secure ingestion,transformation, and consumption workflows.
  
• Implementscalable streaming solutions that factor in performance, scalability and cost.
  

EmbedSecurity & Compliance Across AWS Workloads

• Buildand enforce data governance protocols aligned with relevant regulatory andcompliance requirements using AWS tools.
  
• Collaboratewith cybersecurity teams to implement IAM best practices, encryptionstrategies, and secure networking.
  
• Maintaintraceability and auditability for all data flows across the AWS stack.
  

Optimizefor Observability & Cost Efficiency:

• Workwith our Cloud Architect and SRE to deploy and fine-tune monitoring dashboardsusing Datadog and AWS CloudWatch for performance, anomaly detection, andsecurity event correlation.
  
• Continuouslyevaluate storage and compute cost optimization across S3, EC2, Redshift, andGlue workloads.
  

LeadThrough Influence and Collaboration:

• Partnerwith Data Science, Cloud Architect, Security and Engineering leads to aligncloud architecture with evolving business goals and priorities to ensurefuture-readiness.
  
• Mentorjunior engineers in AWS best practices, scalable design, and secure codingstandards.
  
• Leadinnovation across key Product initiatives.
  

Innovatewith Purpose:

• Evaluateand integrate AWS-compatible orchestration tools like Airflow, Lakeformation,ECS, EKS or Managed Workflows.
  
• Contributeto middleware and third-party orchestration strategies through secure APIs andevent-driven patterns.
  
• Designdata products based on requirements that focus on key use cases such as socialimpact related.
  

• Bachelor’sor Master’s degree in Computer Science, Data Engineering, Software Engineeringor a related field.
  
• 8–10years of professional experience in data engineering, including 5+ yearsarchitecting on AWS underpinned by data governance. Mastery of AWS cloudservices (S3, Lambda, Glue, Redshift, Kinesis, Lake Formation, Crawler etc.).
  
• Deepexpertise in building scalable cloud-native solutions and managing secure datainfrastructure ensuring data governance.
  
• Strongcommand of compliance-driven architecture design and real-time monitoringstrategies.
  
• Goodunderstanding of compliance frameworks related to data privacy and informationsecurity.
  
• Excellentcommunication skills and a proven leadership in mentoring and ability to leadcross-functional initiatives.
  

• Proficiencywith agile tools (Jira).
  
• CloudInfrastructure & AWS Services: S3, Glue, Lambda, Redshift, Kinesis, IAM,CloudWatch, Lake Formation etc. Strong awareness of AWS security tools.
  
• DataOrchestration: Experience with Apache Airflow on ECS or AWS Managed Workflows.Familiarity with Step Functions and event-driven orchestration patterns.
  
• Streaming& ETL Pipelines: Expertise in Kinesis Data Streams and Kafka (AWS-hosted orcompatible). Proficiency in designing and optimizing ETL workflows using AWS.
  
• Monitoring& Observability: Awareness of or exposure to logs, alerting, monitoring,detection and tuning.
  
• Security& Governance: Awareness of or exposure to AWS KMS. In addition, buildinggovernance workflows with AWS Config and Lake Formation.
  
• DataModeling & Optimization: Extensive experience in design of AI-ready datalake with scalable ingestion and query performance.
  
• ProgrammingLanguages: Advanced coding in Python and SQL. Experience in Java and ETLprocesses is also preferred.

• Youhave strong communication skills, curiosity and are a quick learner.
  
• Youenjoy a creative fast paced agile world.
  
• Youenjoy mentoring and teaching other developers to create a world class cohesiveteam.
  
• Youenjoy making friends and having fun.
  

AtmyZoi we strive to create a both a product and a team that embraces equality,inclusion, diversity and freedom. We want people who can be themselves andbring their own brand of value to the team. Come and join us!

Do you want to love what you do at work? Do you want to make a difference, an impact, and transform people's lives? Do you want to work with a team that believes in disrupting the normal, boring, and average?

If yes, then this is the job for you. webook.com is Saudi’s #1 event ticketing and experience booking platform in terms of technology, features, agility, and revenue, serving some of the largest mega events in the Kingdom with over 2 billion in sales. webook.com is part of the Supertech Group, which also includes UXBERT Labs, one of the best digital and user experience design agencies in the GCC, along with Kafu Games, the largest esports tournament platform in MENA.

Key Responsibilities:

1. Data Integration and ETL Development: Architect and implement robust data integration pipelines to extract, transform, and load data from various sources (e.g., databases, SaaS applications, APIs, and flat files) into a centralized data platform. Design and develop complex ETL processes to ensure data quality, consistency, and reliability. Optimize data transformation workflows for performance and scalability.
2. Data Infrastructure and Platform Management: Implement and maintain data ingestion, processing, and storage solutions to support data and analytics needs. Ensure data infrastructure's reliability, security, and availability through monitoring, troubleshooting, and disaster recovery planning.
3. Data Governance and Metadata Management: Collaborate with the data governance team to establish policies, standards, and procedures. Develop and maintain metadata management systems for data lineage, provenance, and traceability. Implement data quality control measures and validation processes to ensure data integrity.

Minimum Requirements:

• 5-6 years of experience as a Data Engineer or in a related data-driven role.
• Proficient in designing and implementing data pipelines using tools like Apache Airflow, Airbyte, or cloud-based services.
• Strong experience with data infrastructure such as data lakes, data warehouses, and real-time streaming platforms (e.g., Elastic, Google BigQuery, MongoDB).
• Expertise in data modeling, data quality, and metadata management.
• Proficient in programming languages like Python or Java, and SQL.
• Familiar with cloud platforms (AWS, Google Cloud) and DevOps practices.
• Excellent problem-solving skills and ability to work collaboratively across teams.
• Strong communication skills to translate technical concepts to stakeholders.

Preferred Qualifications:

• Experience with data visualization and BI tools (e.g., Tableau, Qlik).
• Knowledge of machine learning and AI applications in data initiatives.
• Project management experience and leadership in data projects.