79 Big Data Technologies jobs in Dubai

Data Architect

We are seeking a skilled and experienced Data Architect to design and implement data solutions that support our business goals. As a key member of our team, you will play a critical role in ensuring the success of our data-driven initiatives.

• Design Scalable Data Architectures : Create robust and scalable data architectures aligned with business objectives.
• Develop Integrated Data Solutions : Design, develop, and manage integrated data solutions that meet business needs.
• Collaborate with Cross-Functional Teams : Work closely with data engineers, data analysts, and stakeholders to ensure seamless data integration and quality.
• Implement Data Governance and Best Practices : Develop and implement data governance policies and best practices for data management.
• Optimize Database Performance : Ensure efficient data storage and retrieval by optimizing database performance.
• Stay Current with Emerging Trends : Stay up-to-date with emerging trends and technologies in data architecture.
• Provide Technical Guidance : Provide technical guidance and support to cross-functional teams.

• Bachelor's or Master's Degree : Bachelor's or Master's degree in Computer Science, Information Systems, or a related field.
• 5+ Years of Experience : 5+ years of experience in data architecture or a related role.
• Strong Knowledge of Data Modeling Concepts : Strong knowledge of data modeling concepts, database design, and data management best practices.
• Experience with SQL and NoSQL Databases : Experience with SQL and NoSQL databases.
• Familiarity with ETL/ELT Processes : Familiarity with ETL/ELT processes and data integration tools.
• Excellent Problem-Solving Skills : Excellent problem-solving skills and attention to detail.
• Effective Communication and Collaboration Skills : Effective communication and collaboration skills.
• Ability to Work in a Fast-Paced Environment : Ability to work in a fast-paced environment and manage multiple priorities.

• Paid Time Off
• Performance Bonus
• Training & Development

Mid-Senior Level

Contract

IT Services and IT Consulting

Architect

We are seeking a skilled data engineering specialist to join our team.

• This role involves designing and building a data early warning system, monitoring and optimizing pricing related thresholds, and supporting pricing algorithm changes.

• Design and develop data quality rules to monitor data inputs for anomalous data.
• Develop rules to monitor data drift and optimize thresholds.
• Set up alerts for the data quality rules.
• In-depth data investigation for any incidents.
• Propose and develop pricing algorithm changes as needed.

• Bachelor or Master's degree in Computer Science Engineering or related fields.
• 2-3 years of experience in development using Python.
• Working proficiency with Python, SQL, and/or Java.
• Experience in using statistical concepts to identify anomalies.
• Experience with cache databases (E.g. Redis, MongoDB).
• Experience with job schedulers (E.g. Airflow).
• Familiar with message queue systems (E.g. Kafka, Pub/Sub).
• Familiar with Kubernetes.
• Proficiency in English.

• Experience in using Cloud services (GCP, AWS, or Azure).
• Experience with Quality Assurance is desirable.
• Experience with Recommendation Systems is a plus.
• Knowledge of the cryptocurrency ecosystem and on-chain data is a plus.

• Not Applicable

• Full-time

• Engineering and Information Technology

**Job Title:** Data Engineering Specialist

The Role

As a Data Engineering Specialist, you will play a pivotal role in transforming complex data landscapes into actionable business insights. Your expertise will enable you to rapidly convert diverse new data sources into datasets that support our overarching business intelligence and analytics goals.

Main Responsibilities:

• Evaluate new data sources from SAP and Non-SAP systems
• Rapidly convert these data sources into datasets
• Develop and support the integration of new data streams into our Azure Data platform
• Ensure data is accessible, reliable, and structured for business insight generation

Required Skills and Qualifications:

• Expertise in data engineering and technical integration
• Ability to evaluate diverse data sources and convert them into usable datasets
• Familiarity with Azure Data platform and its features
• Excellent analytical and problem-solving skills

Benefits:

You will directly impact our ability to make data-driven decisions and maintain our competitive edge in the marketplace. Your contributions will be invaluable in driving business growth and success.

Others:

We offer a dynamic work environment and opportunities for professional growth and development.

Data Engineering Specialist

A seasoned professional in data engineering with a strong background in designing and implementing end-to-end data pipelines using cloud platforms.

• Design, develop, and optimize Extract, Transform, Load (ETL) workflows for effective migration and transformation.
• Maintain data ingestion processes from diverse sources such as Salesforce, Oracle databases, PostgreSQL, and MySQL.
• Implement and maintain ETL processes and data pipelines to ensure efficient data extraction, transformation, and loading.
• Leverage Snowflake as the data warehouse solution for managing large volumes of structured and unstructured data.

This is an opportunity to work on complex data engineering projects, collaborating with cross-functional teams to deliver high-quality solutions.

Job Title: Data Engineering Lead

We are seeking a highly skilled Data Engineering Lead to join our team. As a key member of the engineering team, you will be responsible for designing and building scalable data systems, including data lakes, ingestion pipelines, and storage solutions.

• Design and Build Scalable Data Systems: Design and build scalable data systems that can handle large volumes of structured and unstructured data.
• Ingestion Pipelines: Architect batch/stream pipelines using modern orchestration tools such as Airflow and streaming platforms like Kafka.
• Storage Solutions: Implement and tune S3 column-oriented and time-series data storage for petabyte-scale analytics.
• Tooling and Libraries: Develop internal libraries for schema management, data contracts validation, and lineage.
• Reliability and Observability: Embed monitoring, alerting, SLAs, SLOs, and CI/CD to ensure data reliability and observability.
• Collaboration: Partner with cross-functional teams to translate requirements into platform capabilities.

Required Skills and Qualifications:

• 6 years of experience building and maintaining production-grade data systems.
• Expert-level Python development skills, with knowledge of Go and C.
• Hands-on experience with modern orchestration tools and streaming platforms.
• Advanced SQL skills, including complex aggregations and query optimisation.
• Experience designing high-throughput APIs and data access libraries.
• Solid fundamentals in Linux containerisation and cloud object storage solutions.
• Strong knowledge of handling diverse data formats and storage strategies.
• English language proficiency at C1 level.

Job Title: Data Engineering Specialist

We are seeking a highly skilled Data Engineer to join our team in a contract role with the possibility of extension.

The ideal candidate will have strong proficiency in SQL and at least one programming language (e.g., Python) for data manipulation and scripting.

They should also have experience with PySpark , ADF , Databricks , and SQL , as well as proficiency in data warehousing concepts and methodologies.

Azure Synapse and Azure Databricks knowledge is also essential, along with hand-on experience with data warehouse platforms (e.g., Snowflake, Redshift, BigQuery) and ETL tools (e.g., Informatica, Talend, Apache Spark).

The successful candidate will be able to implement data governance frameworks, ensure data quality, security, and compliance with industry standards and regulations.

They will also be responsible for developing complex SQL queries, managing relational databases, and establishing data lineage tracking within the data fabric.

Key Responsibilities:

• Design, develop, and maintain data pipelines for ingestion, transformation, and loading of data into the data warehouse.
• Develop and implement data governance frameworks to ensure data quality, security, and compliance.
• Manage relational databases and establish data lineage tracking within the data fabric.
• Collaborate with cross-functional teams, including data scientists, business analysts, and software engineers, to understand data requirements and deliver solutions that meet business objectives.

Description:

We are seeking an experienced Data Engineering Expert to lead the development of our big data architecture. The successful candidate will design, implement, and maintain scalable and secure data pipelines using Spark, Hive, and Python on Cloudera.

Key Responsibilities:

• Develop real-time data workflows using Kafka.
• Design and develop APIs for data access and integration.
• Utilize Hue, Oozie, and other Cloudera tools for job orchestration and data access.
• Deploy solutions on cloud platforms such as AWS, Azure, or GCP.

Requirements:

• Over 7 years of experience in Big Data engineering.
• Hands-on experience with Cloudera Spark, Hive, Kafka, Python, Hue, and Ranger.
• Strong understanding of distributed systems and cloud data services.
• Proficient in API development and data security controls.

About this Opportunity:

This is a unique opportunity to join a rapidly growing IT services company with a presence in multiple countries, working with top clients in various sectors. Our team values teamwork, quality of life, and professional growth, offering a global community of professionals committed to your development.

What We Offer:

• A competitive salary and benefits package.
• Ongoing training and development opportunities.
• A dynamic and supportive work environment.
• The chance to work on exciting projects with cutting-edge technologies.

How to Apply:

If you are a motivated and experienced data engineer looking for a new challenge, please submit your application, including your resume and a cover letter outlining your qualifications and experience.

At General Motors we pride ourselves on designing, building and selling the world's best vehicles. We are seeking a new generation of visionaries to help launch bold engineering and business initiatives, and shape new directions for General Motors. As an intern you will gain hands-on career specific experiences to maximize your real-world potential.

Work Arrangement: This role is categorized as hybrid. This means the successful candidate is expected to report to the office at minimum three times per week or other frequency dictated by the business and two times per week remote.

Your Role:

As a Data Engineering and Analysis Intern, you'll have the opportunity to gain hands-on experience in data-driven decision-making and contribute to our global sales operations. You will collaborate with senior team members, work on data collection, analysis, and reporting, and help optimize business processes using data science techniques.

What You'll Do:

• Assist in gathering and preparing data from various sources
• Clean, transform, and analyze data to ensure quality and accuracy
• Support data analysis, hypothesis testing, and exploratory data analysis (EDA)
• Help build and validate predictive models using basic machine learning techniques
• Generate reports and create data visualizations to highlight business insights
• Collaborate with cross-functional teams (Sales, Finance, CRM, Omnichannel) to support data-driven solutions
• Contribute to business performance tracking, including dashboards and KPIs
• Participate in training and mentorship opportunities to enhance your technical skills

Your Skills & Abilities:

• Currently undergoing bachelor's or master's degree in data science, Data Engineering, Business Analytics, or a related field
• Basic knowledge of programming languages (Python preferred) and data analysis tools (e.g., SQL, Excel)
• Basic knowledge of data visualization tools (Power BI, Tableau) is a plus
• Completed coursework in AI, machine learning, deep learning, and natural language processing (NLP)
• Worked on academic projects involving generative models (like GPT-3, GANs) or reinforcement learning

What We Offer:

• Mentorship from experienced professionals in business and data
• Exposure to real-world data challenges and solutions
• Opportunity to contribute to meaningful projects that impact business outcomes

Job Opportunity:

A seasoned data engineer with 5 years of experience in designing and maintaining data pipelines using Python. Expertise includes working with Python, SQL databases, Snowflake, Apache Airflow, and Shell scripting.

Main Responsibilities:

• Design, develop, and maintain robust data pipelines to support business operations and analytics using Python.
• Integrate APIs to fetch, process, and transform data for various use cases.
• Work with multiple SQL databases to perform data transformations, migrations, and optimizations.
• BUILD and manage data warehouses using Snowflake ensuring high performance and scalability.
• Implement and manage workflows to automate data pipeline orchestration.
• Develop and maintain Shell scripts to automate tasks and streamline processes.
• Ensure data quality, consistency, and security across all platforms.
• Collaborate with stakeholders to understand data requirements and deliver solutions.
• Monitor, troubleshoot, and optimize data pipelines for efficiency and reliability.

Required Skills and Qualifications:

• Programming: Strong proficiency in Python with experience in API integration and data processing.
• Data Warehousing: Hands-on experience with Snowflake including schema design, data modeling, and performance optimization.
• Workflow Automation: Proficiency in Apache Airflow for managing and orchestrating workflows.
• Scripting: Strong skills in Shell scripting for automation and system management.
• ETL/ELT Processes: Solid understanding of designing and maintaining ETL/ELT pipelines.
• Problem Solving: Strong analytical and problem-solving skills to handle complex data challenges.
• Communication: Excellent communication skills to work effectively with cross-functional teams.