100 Big Data Technologies jobs in Dubai

Job Title: Data Engineering Specialist

We are seeking a highly skilled Data Engineering Specialist to join our team. The ideal candidate will have extensive experience in designing and developing end-to-end data pipelines using Informatica PowerCenter and IDMC.

• Design, develop, and optimize ETL workflows using Informatica PowerCenter (PC) and IDMC
• Manage data ingestion processes from diverse data sources such as Salesforce, Oracle databases, PostgreSQL, and MySQL
• Implement and maintain ETL processes and data pipelines to ensure efficient data extraction, transformation, and loading
• Utilize Snowflake as the data warehouse solution for managing large volumes of structured and unstructured data
• Maintain and optimize ETL jobs for performance and reliability, ensuring timely data availability for business users

• Informatica PowerCenter and IDMC development
• Data warehousing and cloud computing
• SQL, Python, and Unix shell scripting
• Data governance and security

This role offers the opportunity to work with cutting-edge technology and be part of a dynamic team. As a Data Engineering Specialist, you will have the chance to:

• Develop and implement data pipelines that drive business insights
• Work with cross-functional teams to identify and resolve data-related issues
• Stay up-to-date with industry trends and best practices in data engineering

Our company is committed to providing a supportive and collaborative work environment. We offer:

• A competitive salary and benefits package
• Ongoing training and professional development opportunities
• A dynamic and diverse team environment

Our organization requires a highly skilled data engineer with extensive expertise in big data ecosystems, cloud-native tools, and advanced data processing techniques.

This role necessitates a strong foundation in data ingestion, transformation, and optimization using Cloudera Data Platform, along with a proven track record of implementing data engineering best practices.

• Proficiency in PySpark is essential for success in this position.
• Experience with data pipelines, workflow management, and scalable data architectures is crucial.

The ideal candidate should have hands-on experience with Cloudera Data Platform and a deep understanding of big data concepts. We are looking for a detail-oriented individual who can ensure the quality and integrity of our data assets.

As a member of our data engineering team, you will have opportunities to work on complex projects, collaborate with cross-functional teams, and contribute to the growth and development of our organization.

At General Motors we pride ourselves on designing, building and selling the world's best vehicles. We are seeking a new generation of visionaries to help launch bold engineering and business initiatives, and shape new directions for General Motors. As an intern you will gain hands-on career specific experiences to maximize your real-world potential.

Work Arrangement: This role is categorized as hybrid. This means the successful candidate is expected to report to the office at minimum three times per week or other frequency dictated by the business and two times per week remote.

Your Role:

As a Data Engineering and Analysis Intern, you’ll have the opportunity to gain hands-on experience in data-driven decision-making and contribute to our global sales operations. You will collaborate with senior team members, work on data collection, analysis, and reporting, and help optimize business processes using data science techniques.

What You'll Do:

• Assist in gathering and preparing data from various sources
• Clean, transform, and analyze data to ensure quality and accuracy
• Support data analysis, hypothesis testing, and exploratory data analysis (EDA)
• Help build and validate predictive models using basic machine learning techniques
• Generate reports and create data visualizations to highlight business insights
• Collaborate with cross-functional teams (Sales, Finance, CRM, Omnichannel) to support data-driven solutions
• Contribute to business performance tracking, including dashboards and KPIs
• Participate in training and mentorship opportunities to enhance your technical skills

Your Skills & Abilities:

• Currently undergoing bachelor's or master's degree in data science, Data Engineering, Business Analytics, or a related field
• Basic knowledge of programming languages (Python preferred) and data analysis tools (e.g., SQL, Excel)
• Basic knowledge of data visualization tools (Power BI, Tableau) is a plus
• Completed coursework in AI, machine learning, deep learning, and natural language processing (NLP)
• Worked on academic projects involving generative models (like GPT-3, GANs) or reinforcement learning

What We Offer:

• Mentorship from experienced professionals in business and data
• Exposure to real-world data challenges and solutions
• Opportunity to contribute to meaningful projects that impact business outcomes

At General Motors we pride ourselves on designing, building and selling the world's best vehicles. We are seeking a new generation of visionaries to help launch bold engineering and business initiatives, and shape new directions for General Motors. As an intern you will gain hands-on career specific experiences to maximize your real-world potential.

Work Arrangement: This role is categorized as hybrid. This means the successful candidate is expected to report to the office at minimum three times per week or other frequency dictated by the business and two times per week remote.

Your Role:

As a Data Engineering and Analysis Intern, you'll have the opportunity to gain hands-on experience in data-driven decision-making and contribute to our global sales operations. You will collaborate with senior team members, work on data collection, analysis, and reporting, and help optimize business processes using data science techniques.

What You'll Do:

• Assist in gathering and preparing data from various sources
• Clean, transform, and analyze data to ensure quality and accuracy
• Support data analysis, hypothesis testing, and exploratory data analysis (EDA)
• Help build and validate predictive models using basic machine learning techniques
• Generate reports and create data visualizations to highlight business insights
• Collaborate with cross-functional teams (Sales, Finance, CRM, Omnichannel) to support data-driven solutions
• Contribute to business performance tracking, including dashboards and KPIs
• Participate in training and mentorship opportunities to enhance your technical skills

Your Skills & Abilities:

• Currently undergoing bachelor's or master's degree in data science, Data Engineering, Business Analytics, or a related field
• Basic knowledge of programming languages (Python preferred) and data analysis tools (e.g., SQL, Excel)
• Basic knowledge of data visualization tools (Power BI, Tableau) is a plus
• Completed coursework in AI, machine learning, deep learning, and natural language processing (NLP)
• Worked on academic projects involving generative models (like GPT-3, GANs) or reinforcement learning

What We Offer:

• Mentorship from experienced professionals in business and data
• Exposure to real-world data challenges and solutions
• Opportunity to contribute to meaningful projects that impact business outcomes

As the Head of Data Engineering, you will be responsible for designing, implementing, and maintaining a robust, scalable, and compliant data architecture that supports our exchange’s operations, analytics, and regulatory reporting requirements. You will lead a team of data engineers, ensuring high availability, security, and performance of our data infrastructure. You will work closely with stakeholders across technology, compliance, risk, and product teams to develop data pipelines, warehouses, and real-time analytics capabilities.

This is a strategic yet hands-on role where you will drive data engineering best practices, scalability, and automation while ensuring compliance with regulatory data requirements for a financial services entity.

Data Architecture & Strategy :

• Define and own the data architecture strategy for the exchange, ensuring it is scalable, secure, and regulatory-compliant.
• Design and implement data modeling, governance, and security frameworks to meet financial and regulatory requirements.
• Architect real-time and batch processing data pipelines to handle trade data, order books, user activity, and market analytics.
• Optimize data storage and retrieval for performance and cost efficiency, leveraging cloud-based and hybrid solutions.
• Build and maintain ETL / ELT data pipelines for operational, analytical, and compliance-related data.
• Ensure high data quality, reliability, and availability through robust monitoring, alerting, and data validation techniques.
• Manage and enhance data warehouses, data lakes, and streaming platforms to support business intelligence and machine learning use cases.
• Oversee database design and optimization for transactional and analytical workloads (e.g., Aurora, Redis, Kafka).
• Implement data lineage, metadata management, and access control mechanisms in line with compliance requirements.

Compliance, Security & Risk Management :

• Work closely with compliance and risk teams to ensure data retention policies, audit trails, and reporting mechanisms meet regulatory requirements (e.g., FATF, AML, GDPR, MiCA).
• Implement encryption, anonymization, and access control policies to safeguard sensitive user and transaction data.
• Support fraud detection and risk monitoring through data analytics and alerting frameworks.

Leadership & Team Management :

• Lead, mentor, and grow a small team of data engineers, fostering a culture of collaboration, innovation, and accountability.
• Drive best practices in data engineering, DevOps for data, and CI / CD automation for analytics infrastructure.
• Collaborate with software engineers, DevOps, data analysis, and product teams to integrate data solutions into the broader exchange ecosystem.

Technical competencies and skills :

• Proven experience in data architecture and engineering, preferably within a regulated financial or crypto environment.
• Strong proficiency in SQL, Python, or Scala for data engineering.
• Experience with cloud-based data platforms (AWS, GCP, or Azure) and orchestration tools (Airflow, Prefect, Dagster).
• Hands-on experience with real-time data processing (Kafka, Pulsar, Flink, Spark Streaming).
• Expertise in data warehousing solutions (Snowflake, BigQuery, Redshift, Databricks).
• Strong understanding of database design, indexing strategies, and query optimization.
• Experience implementing data governance, lineage, and cataloging tools.
• Familiarity with blockchain / crypto data structures and APIs is a plus.

Leadership & Strategic Skills :

• Experience leading and mentoring a team of data engineers.
• Ability to design data strategies that align with business goals and regulatory requirements.
• Strong cross-functional collaboration skills with compliance, risk, and technology teams.
• Ability to work in a fast-paced, high-growth startup environment with a hands-on approach.

Industry & Compliance Knowledge :

• Experience in regulated financial markets, fintech, or crypto is highly preferred.
• Familiarity with financial data standards, KYC / AML reporting, and regulatory requirements related to data handling.

Preferred Qualifications :

• Bachelors or Masters degree in Computer Science, Data Engineering, or a related field.
• Certifications in cloud data engineering (AWS / GCP / Azure), data governance, or security are a plus.
• Experience working in a crypto exchange, trading platform, or high-frequency trading environment is an advantage.

At M2, we believe in a workplace where talent, dedication, and passion are the only factors that count, regardless of gender, background, age, and other characteristics. We embrace diversity because we know that it fuels innovation, fosters creativity, and drives success. So, if you're ready to join a team where your potential is truly valued, welcome aboard!

SoftServe is a global digital solutions company with headquarters in Austin, Texas, founded in 1993. Our associates are currently working on 2,000+ projects with clients across North America, EMEA, APAC, and LATAM. We are about people who create bold things, make a difference, have fun, and love their work.

Big Data & Analytics Center of Excellence is a data consulting and data engineering branch at SoftServe. Starting as a group of three enthusiasts back in 2013, hundreds of Data Engineers and Architects nowadays build Data & Analytics end-to-end solutions from strategy through technical design and PoC to full-scale implementation. We have customers in Healthcare, Finance, Manufacturing, Retail, and Energy domains.

We hold top-level partnership statuses with all the major cloud providers and collaborate with many technology partners like AWS, GCP, Microsoft, Databricks, Snowflake, Confluent, and others.

If you are

• Experienced in data engineering for 10+ years, with a strong focus on healthcare data
• Skilled in designing and implementing healthcare-compliant data architectures
• A proven leader with a track record of managing data engineering teams in regulated environments
• Proficient in working with cloud platforms such as Azure or AWS in healthcare settings
• Knowledgeable in healthcare data governance, compliance (, HIPAA), and security standards

And you want to

• Work full-time from AbuDhabi, UAE
• Drive the development and execution of data engineering strategies aligned with healthcare goals
• Lead the design of scalable, compliant data infrastructures supporting clinical and operational needs
• Ensure the highest standards of data quality, integrity, and security across systems
• Collaborate with cross-functional teams to optimize data workflows and improve healthcare outcomes

TOGETHER WE WILL

• Address different business and technology challenges, engage in impactful projects, use top-notch technologies, and drive multiple initiatives as a part of the Center of Excellence
• Support your technical and personal growth — we have a dedicated career plan for all roles in our company
• Investigate new technologies, build internal prototypes, and share knowledge with the SoftServe Data Community
• Upskill with full access to Udemy learning courses
• Pass professional certifications, encouraged and covered by the company
• Adopt best practices from experts while working in a team of top-notch engineers and architects
• Collaborate with world-leading companies and attend professional events

Job Title: Senior Data Engineering Leader

About the Role:

We are seeking a highly skilled and experienced Senior Data Engineering Leader to join our team. As a key member of our data engineering organization, you will be responsible for designing and implementing large-scale data architectures, leading cross-functional teams, and driving innovation in data engineering.

Key Responsibilities:

• Data Architecture: Design and implement scalable and efficient data architectures to support our business objectives.
• Team Leadership: Lead high-performing cross-functional teams to deliver complex data engineering projects.
• Innovation: Drive innovation in data engineering by staying up-to-date with emerging technologies and trends.
• Collaboration: Collaborate with development operations and data teams to understand requirements and deliver robust solutions.
• Security and Compliance: Ensure data engineering solutions adhere to security best practices and regulatory requirements.

Required Skills and Qualifications:

• Education: Bachelor's or Master's degree in Computer Science, Engineering, or related field.
• Experience: Minimum 8 years of experience in big data engineering or related role.
• Technical Skills:
  • Strong experience with containerization and orchestration tools like Kubernetes and OpenShift.
  • Proficiency in scripting languages such as Python, Bash, or Groovy.
  • Familiarity with infrastructure as code (IaC) tools like Terraform or Ansible.
  • Knowledge of big data technologies such as Hadoop, Spark, ETLs.

Benefits:

• Competitive Salary: We offer a competitive salary and benefits package.
• Opportunities for Growth: We provide opportunities for growth and professional development.
• Culture of Innovation: We foster a culture of innovation and experimentation.

Others:

Award Winning Company: We are an award-winning company recognized for our commitment to diversity and inclusion.

Job Title: Senior Data Engineering Expert

Job Description

• Transform complex data landscapes into actionable business insights by leveraging expertise in rapid data source conversion, dataset preparation, and business insight generation.
• Engage in early-stage design discussions and feature definition activities to predefine potential insights and shape future projects.
• Design, document, develop, implement, and test reusable and scalable data pipelines for efficiency and effectiveness.
• Bridge the gap between business needs and technical execution by converting business requirements into Technical Specification Documents (TSDs) and Functional Specification Documents (FSDs).
• Assist with regular reporting queries and support developers in utilizing tools such as Alteryx, Power BI, and Azure for data analysis and visualization.
• Collaborate with team members across multiple engineering teams to integrate proven prototypes into core intelligence products and facilitate seamless collaboration and knowledge sharing.
• Adapt to both agile and waterfall project delivery methodologies to guide internal and external teams for timely and on-spec business deliveries.

• Strong background in data engineering and data science principles.
• Experience with rapid data source conversion and dataset preparation.
• Proficiency in designing, developing, and implementing data pipelines.
• Excellent communication and collaboration skills.
• Ability to work in a fast-paced environment and adapt to changing project requirements.

• Opportunity to work with a dynamic team of engineers and scientists.
• Chance to develop and apply innovative solutions to real-world problems.
• Professional growth and development opportunities.

Join to apply for the Manager of Data Engineering and Functional Consulting role at Dicetek LLC.

Get AI-powered advice on this job and more exclusive features.

• Bachelor’s or Master’s degree in Computer Science, Information Systems, Data Analytics, or a related field.
• Strong understanding of end-to-end data warehousing concepts, including data modeling, ETL/ELT, and data integration.
• Proficient in Azure Synapse, Databricks, Azure Data Factory, Snowflake, and Power BI for modern cloud-based data solutions.
• Skilled in handling structured, semi-structured (JSON, XML), and unstructured data across Azure and AWS platforms.
• Expertise in Power Query and Power M for advanced data transformation and reporting automation.
• Solid grasp of Agile methodologies, functional requirement gathering, and stakeholder communication.
• Familiar with metadata management, data lineage, and data quality principles to ensure trusted analytics.

Experience

• 10+ years of experience in designing, implementing, and optimizing end-to-end data warehouse solutions across cloud and on-premise ecosystems.
• Orchestrated enterprise DWH solutions using Azure Synapse, Databricks, and ADF, integrating structured and semi-structured data sources while ensuring lineage, auditability, and performance at scale.
• Translated business needs into scalable dimensional models and Power BI solutions, leveraging Power Query (M) scripting and DAX for self-service analytics.
• Designed and governed hybrid data pipelines across Azure and AWS, ensuring data consistency, compliance, and traceability from ingestion to visualization.
• Collaborated with stakeholders and engineering teams to define KPIs, automate reconciliations, and embed validation rules across ETL and reporting layers.
• Led multi-cloud integration of systems into a unified analytics layer, optimizing transformations with Databricks notebooks, ADF workflows, and parameterized datasets.
• Enabled real-time insights by architecting incremental refresh in Power BI and optimizing semantic models with measures, security, and models.
• Implemented data observability frameworks to proactively detect anomalies and ensure SLA adherence across reporting layers.

• Not Applicable

• Contract

• Engineering and Information Technology

• IT Services and IT Consulting

Referrals increase your chances of interviewing at Dicetek LLC by 2x.