105 Data Pipeline jobs in the United Arab Emirates

Key Roles and Responsibilities

The team seeks a highly skilled Data Engineer to join its ranks. This individual will be instrumental in building and maintaining the next-generation data platform, focusing on real-time and batch data processing using Databricks and Azure Synapse Analytics.

Key Responsibilities Include:

• Designing, developing, and maintaining scalable and reliable data pipelines using Databricks, Azure Data Factory, and related Azure services to ingest, process, transform, and load data from diverse sources (e.g., databases, APIs, streaming platforms, cloud storage).
• Building and optimizing data models and data warehouses in Azure Synapse Analytics, ensuring performance, scalability, and data integrity for analytical workloads.
• Developing and implementing data quality checks, monitoring, and alerting systems to ensure data accuracy, completeness, and reliability across the data platform.
• Writing complex and efficient SQL queries for data extraction, transformation, and analysis in Databricks (Spark SQL), Azure Synapse (TSQL), and MySQL environments.
• Collaborating with data analysts, other engineers, Product, and other departments to understand data requirements, provide data solutions, and support data-driven initiatives.
• Optimizing data pipelines and Synapse workloads for performance and cost-efficiency.
• Implementing data security and governance best practices within the data platform.
• Documenting data pipelines, data models, and ETL processes clearly and comprehensively.
• Staying up-to-date with the latest technologies and best practices in data engineering, Databricks, Azure Synapse, and cloud data platforms.
• Troubleshooting and resolving data pipeline and data quality issues.
• Contributing to the continuous improvement of our data platform architecture and infrastructure.

• Bachelor's degree in computer science, Data Science, Engineering, or a related field.
• 4 years of experience as a Data Engineer or similar role.
• Expertise in SQL and demonstrated ability to write complex optimized queries (SQL, TSQL, Spark SQL, or similar).
• Strong hands-on experience with Databricks, including Spark programming (Python, Scala, or Java), Delta Lake, and Databricks Workflows/Jobs.
• Proven experience with Azure Synapse Analytics, including data warehousing, data modelling, and performance tuning.
• Experience with Azure Data Factory or similar ETL/Data Orchestration tools.
• Solid understanding of data warehousing concepts, data modelling principles (e.g., star schema, snowflake schema), and ETL/ELT methodologies.
• Experience working with cloud environments, particularly Microsoft Azure.
• Proficiency in at least one programming language, such as Python or Scala.
• Experience with version control systems (e.g., Git).
• Strong problem-solving and analytical skills with a data-driven approach.
• Excellent communication, collaboration, and documentation skills.

• Competitive salary and benefits package.
• Opportunity to work with an energetic and innovative company.
• A chance to contribute to the success of a well-known brand.

We are seeking a highly skilled Python Data Engineer to join our team.

• A Python Data Engineer with 5 years of experience is required to design and maintain robust, scalable data pipelines to support business operations and analytics using Python.

The successful candidate will integrate APIs using Python to fetch, process, and transform data for various use cases.

Key Responsibilities:

1. Design, develop, and maintain robust, scalable data pipelines to support business operations and analytics using Python.
2. Integrate APIs using Python to fetch, process, and transform data for various use cases.
3. Work with a variety of SQL databases including PostgreSQL, MySQL, MS SQL Server, and Oracle to perform data transformations, migrations, and optimizations.
4. Utilize Snowflake to build and manage data warehouses ensuring high performance and scalability.
5. Implement and manage workflows using Apache Airflow to automate data pipeline orchestration.
6. Develop and maintain Shell scripts to automate tasks and streamline processes.
7. Evaluate and optimize data pipelines for efficiency and reliability.

Requirements:

• Strong proficiency in Python with experience in API integration and data processing.
• Hands-on experience with Snowflake including schema design, data modeling, and performance optimization.
• Proficiency in Apache Airflow for managing and orchestrating workflows.
• Strong skills in Shell scripting for automation and system management.
• Solid understanding of designing and maintaining ETL/ELT pipelines.
• Strong analytical and problem-solving skills to handle complex data challenges.
• Excellent communication skills to work effectively with cross-functional teams.

This position offers a great opportunity for a skilled professional to work on challenging projects and contribute to the success of our organization.

Visa is a world leader in payments and technology, with over 259 billion payments transactions flowing safely between consumers, merchants, financial institutions, and government entities in more than 200 countries and territories each year. Our mission is to connect the world through the most innovative, convenient, reliable, and secure payments network, enabling individuals, businesses, and economies to thrive while driven by a common purpose – to uplift everyone, everywhere by being the best way to pay and be paid.

Make an impact with a purpose-driven industry leader. Join us today and experience Life at Visa.

Visa Consulting and Analytics (VCA) is the consulting arm of Visa, and drives tangible, impactful results for clients. Drawing on our expertise in consulting, data analytics, digital marketing and economics, VCA solves the most strategic problems for our clients.

The Data Services, Pipeline & Sales Support Manager Manager will support the Visa Consulting & Analytics (VCA) team to plan and develop a strong pipeline in data-driven projects and use cases to our clients in MENA, including issuers, merchants and governments.

The role will mainly be focused on sales pipeline development and sales support for VCA's data-driven customer projects in the region, including potential clients development, design of new service decks and proposals, organization and management of client meetings and sales follow-up tasks, client engagements for project scoping and contracting, project teams' assignments and project kick-offs, etc. This role will also support project delivery for data-driven business services to clients as needed.

Support business development initiatives by:

• Sales Pipeline Development: Investigating and suggesting potential new client lists, and keeping a strong client pipeline
• Sales Materials Design: Developing data-services decks customized for different client groups and their needs
• New Services Development: Assisting in development of new data-driven product and service offerings based on client interactions, for the practice area
• Client Engagements Planning: Contacting new clients and organizing client meetings
• Project Scoping & POCs: Engaging with the clients for new project scoping that will address their needs. Planning and execution of POC studies to assess client impact
• Proposals & Negotiation Management: Developing proposals for new client work. Following up on the new clients and projects pipeline. Overseeing and coordinating the negotiation process between client teams and Visa stakeholders to ensure mutually beneficial outcomes and alignment with contractual obligations
• Client Contracting: Preparing and executing client contracts and legal documents utilizing Visa-approved templates. This includes drafting sales and service agreements in accordance with company standards and client requirements
• Resource Onboarding: Manage the onboarding of new personnel by leveraging Visa's internal systems. Facilitate access to required tools, systems, and documentation for seamless integration into project teams
• Procurement Management: Administer the entire procurement process, ensuring compliance with Visa's policies and procedures, from requisition to purchase order and vendor management
• Data Compliance Oversight: Supervise the end-to-end data compliance process to maintain the integrity, confidentiality, and security of sensitive information handled throughout the project lifecycles
• Invoice and Client Follow-Up: Lead the development and issuance of project invoices, and proactively engage with clients for timely payment processing and resolution of billing queries
• Project Teams Coordination: Communicate closely with the project team, manager and client representative to deliver the projects on the planned timeline
• Work closely with other Visa teams; including Products, BD, Risk, Marketing, Digital Solutions, Digital Partnerships, Innovation Center etc., to ensure a seamless Consulting experience
• Contribute to the development of Visa Consulting and Analytics thought leadership materials in data services

Key Competencies:

Key elements to assess his/her candidacy will include expertise on the following topics:

• A strong understanding on payments data and possible business use cases
• Expertise in data-driven projects and the implementation of diverse use cases, particularly within government and retail sectors
• Experienced in preparing and supporting sales pitches, facilitating problem-solving sessions, and communicating insights effectively to both internal teams and senior client stakeholders
• Experience scoping data-driven projects and developing client proposals
• Experience in pre-sales and sales operations
• Effective negotiation and stakeholder management abilities, facilitating productive discussions and consensus between clients, internal teams, and external partners
• Proven ability to manage multiple client discussions simultaneously
• Demonstrates strong communication skills and the ability to navigate multi-functional environments while driving desired client outcomes
• High attention to detail, accuracy, and organizational skills, with the ability to manage multiple priorities and maintain process integrity
• Comfortable using business, procurement, and contract management systems to streamline workflows and maintain documentation

The position is based in UAE, Dubai – with the flexibility to travel for client engagements across the region.

• This is not a data scientist position.
• Bachelor's degree or equivalent
• At least 4 years of overall work experience – experience in management consulting companies is a plus
• Skills to work in a multi-functional environment and ability to extract desired outcomes
• Comfortable in interacting with and presenting to senior management of clients and Visa internal stakeholders
• Highly analytical with advanced PowerPoint skills and Excel skills
• Fluent communicator
• Results driven, flexible and both internally and externally customer focused
• Self-motivated and able to use own initiative with limited direction
• Strong teamwork and collaboration skills with the ability to work effectively with inter-disciplinary groups

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

TheData Services, Pipeline & Sales Support Manager Manager will support the Visa Consulting & Analytics (VCA) team to plan and develop a strong pipeline in data-driven projects and use cases to our clients in MENA, including issuers, merchants and governments.

The role will mainly be focused on sales pipeline development and sales support for VCA's data-driven customer projects in the region, including potential clients development, design of new service decks and proposals, organization and management of client meetings and sales follow-up tasks, client engagements for project scoping and contracting, project teams' assignments and project kick-offs, etc. This role will also support project delivery for data-driven business services to clients as needed.

• Sales Pipeline Development: Investigating and suggesting potential new client lists, and keeping a strong client pipeline
• Sales Materials Design: Developing data-services decks customized for different client groups and their needs
• New Services Development: Assisting in development of new data-driven product and service offerings based on client interactions, for the practice area
• Client Engagements Planning: Contacting new clients and organizing client meetings
• Project Scoping & POCs: Engaging with the clients for new project scoping that will address their needs. Planning and execution of POC studies to assess client impact
• Proposals & Negotiation Management: Developing proposals for new client work. Following up on the new clients and projects pipeline. Overseeing and coordinating the negotiation process between client teams and Visa stakeholders to ensure mutually beneficial outcomes and alignment with contractual obligations
• Client Contracting: Preparing and executing client contracts and legal documents utilizing Visa-approved templates. This includes drafting sales and service agreements in accordance with company standards and client requirements
• Resource Onboarding: Manage the onboarding of new personnel by leveraging Visa's internal systems. Facilitate access to required tools, systems, and documentation for seamless integration into project teams
• Procurement Management: Administer the entire procurement process, ensuring compliance with Visa's policies and procedures, from requisition to purchase order and vendor management
• Data Compliance Oversight: Supervise the end-to-end data compliance process to maintain the integrity, confidentiality, and security of sensitive information handled throughout the project lifecycles
• Invoice and Client Follow-Up: Lead the development and issuance of project invoices, and proactively engage with clients for timely payment processing and resolution of billing queries
• Project Teams Coordination: Communicate closely with the project team, manager and client representative to deliver the projects on the planned timeline
• Work closely with other Visa teams; including Products, BD, Risk, Marketing, Digital Solutions, Digital Partnerships, Innovation Center etc., to ensure a seamless Consulting experience
• Contribute to the development of Visa Consulting and Analytics thought leadership materials in data services

Key elements to assess his/her candidacy will include expertise on the following topics:

• A strong understanding on payments data and possible business use cases
• Expertise in data-driven projects and the implementation of diverse use cases, particularly within government and retail sectors
• Experienced in preparing and supporting sales pitches, facilitating problem-solving sessions, and communicating insights effectively to both internal teams and senior client stakeholders
• Experience scoping data-driven projects and developing client proposals
• Experience in pre-sales and sales operations
• Effective negotiation and stakeholder management abilities, facilitating productive discussions and consensus between clients, internal teams, and external partners
• Proven ability to manage multiple client discussions simultaneously
• Demonstrates strong communication skills and the ability to navigate multi-functional environments while driving desired client outcomes
• High attention to detail, accuracy, and organizational skills, with the ability to manage multiple priorities and maintain process integrity
• Comfortable using business, procurement, and contract management systems to streamline workflows and maintain documentation

The position is based in UAE, Dubai – with the flexibility to travel for client engagements across the region.

• This is not a data scientist position.
• Bachelor's degree or equivalent
• At least 4 years of overall work experience – experience in management consulting companies is a plus
• Skills to work in a multi-functional environment and ability to extract desired outcomes
• Comfortable in interacting with and presenting to senior management of clients and Visa internal stakeholders
• Highly analytical with advanced PowerPoint skills and Excel skills
• Results driven, flexible and both internally and externally customer focused
• Self-motivated and able to use own initiative with limited direction
• Strong teamwork and collaboration skills with the ability to work effectively with inter-disciplinary groups

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race, color, religion, sex, national origin, sexual orientation, gender identity, disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

TheData Services Pipeline & Sales Support Manager Manager will support the Visa Consulting & Analytics (VCA) team to plan and develop a strong pipeline in data-driven projects and use cases to our clients in MENA including issuers merchants and governments.

The role will mainly be focused on sales pipeline development and sales support for VCAs data-driven customer projects in the region including potential clients development design of new service decks and proposals organization and management of client meetings and sales follow-up tasks client engagements for project scoping and contracting project teams assignments and project kick-offs etc. This role will also support project delivery for data-driven business services to clients as needed.

Support business development initiatives by:

• Sales Pipeline Development: Investigating and suggesting potential new client lists and keeping a strong client pipeline
• Sales Materials Design: Developing data-services decks customized for different client groups and their needs
• New Services Development: Assisting in development of new data-driven product and service offerings based on client interactions for the practice area
• Client Engagements Planning: Contacting new clients and organizing client meetings
• Project Scoping & POCs: Engaging with the clients for new project scoping that will address their needs. Planning and execution of POC studies to assess client impact
• Proposals & Negotiation Management: Developing proposals for new client work. Following up on the new clients and projects pipeline. Overseeing and coordinating the negotiation process between client teams and Visa stakeholders to ensure mutually beneficial outcomes and alignment with contractual obligations
• Client Contracting: Preparing and executing client contracts and legal documents utilizing Visa-approved templates. This includes drafting sales and service agreements in accordance with company standards and client requirements
• Resource Onboarding: Manage the onboarding of new personnel by leveraging Visas internal systems. Facilitate access to required tools systems and documentation for seamless integration into project teams
• Procurement Management: Administer the entire procurement process ensuring compliance with Visas policies and procedures from requisition to purchase order and vendor management
• Data Compliance Oversight: Supervise the end-to-end data compliance process to maintain the integrity confidentiality and security of sensitive information handled throughout the project lifecycles
• Invoice and Client Follow-Up: Lead the development and issuance of project invoices and proactively engage with clients for timely payment processing and resolution of billing queries
• Project Teams Coordination: Communicate closely with the project team manager and client representative to deliver the projects on the planned timeline
• Work closely with other Visa teams; including Products BD Risk Marketing Digital Solutions Digital Partnerships Innovation Center etc. to ensure a seamless Consulting experience
• Contribute to the development of Visa Consulting and Analytics thought leadership materials in data services

Key Competencies:

Key elements to assess his/her candidacy will include expertise on the following topics:

• A strong understanding on payments data and possible business use cases
• Expertise in data-driven projects and the implementation of diverse use cases particularly within government and retail sectors
• Experienced in preparing and supporting sales pitches facilitating problem-solving sessions and communicating insights effectively to both internal teams and senior client stakeholders
• Experience scoping data-driven projects and developing client proposals
• Experience in pre-sales and sales operations
• Effective negotiation and stakeholder management abilities facilitating productive discussions and consensus between clients internal teams and external partners
• Proven ability to manage multiple client discussions simultaneously
• Demonstrates strong communication skills and the ability to navigate multi-functional environments while driving desired client outcomes
• High attention to detail accuracy and organizational skills with the ability to manage multiple priorities and maintain process integrity
• Comfortable using business procurement and contract management systems to streamline workflows and maintain documentation

The position is based in UAE Dubai with the flexibility to travel for client engagements across the region.

Qualifications :

• This is not a data scientist position.
• Bachelors degree or equivalent
• At least 4 years of overall work experience experience in management consulting companies is a plus
• Skills to work in a multi-functional environment and ability to extract desired outcomes
• Comfortable in interacting with and presenting to senior management of clients and Visa internal stakeholders
• Highly analytical with advanced PowerPoint skills and Excel skills
• Fluent communicator
• Results driven flexible and both internally and externally customer focused
• Self-motivated and able to use own initiative with limited direction
• Strong teamwork and collaboration skills with the ability to work effectively with inter-disciplinary groups

Additional Information :

Visa is an EEO Employer. Qualified applicants will receive consideration for employment without regard to race color religion sex national origin sexual orientation gender identity disability or protected veteran status. Visa will also consider for employment qualified applicants with criminal histories in a manner consistent with EEOC guidelines and applicable local law.

Remote Work :

No

Employment Type :

Full-time

4 weeks ago Be among the first 25 applicants

BlackStone eIT is at the forefront of data-driven decision-making and innovation. We are currently looking for a talented and motivated Data Architect to join our dynamic team. As a Data Architect at BlackStone eIT, you will play a critical role in designing and implementing data solutions that support our strategic goals and enhance our clients' business processes.

Your responsibilities will include creating robust data architectures, ensuring data quality and integrity, and optimizing data flow across various systems. You will collaborate closely with stakeholders to understand their data needs and translate them into effective technical solutions. This position requires a mix of strong technical skills and the ability to communicate effectively with both technical and non-technical team members.

Key Responsibilities

• Design, develop, and manage scalable data architectures aligned with business goals
• Create and maintain data models, data flows, and integrated data solutions
• Collaborate with data engineers and data analysts to ensure seamless data integration and quality
• Implement data governance and best practices for data management
• Optimize database performance, ensuring efficient data storage and retrieval
• Stay current with emerging trends and technologies in data architecture
• Provide technical guidance and support to cross-functional teams
  
  

• Bachelor's or Master's degree in Computer Science, Information Systems, or a related field
• 5+ years of experience in data architecture or a related role
• Strong knowledge of data modeling concepts, database design, and data management best practices
• Experience with SQL and NoSQL databases
• Familiarity with ETL/ELT processes and data integration tools
• Excellent problem-solving skills and attention to detail
• Effective communication and collaboration skills
• Ability to work in a fast-paced environment and manage multiple priorities
  
  

• Paid Time Off
• Performance Bonus
• Training & Development

• Seniority level Mid-Senior level

• Employment type Contract

• Job function Information Technology
• Industries IT Services and IT Consulting

Referrals increase your chances of interviewing at BlackStone eIT by 2x

Get notified about new Data Specialist jobs in Dubai, Dubai, United Arab Emirates.

Global Village, Dubai, United Arab Emirates 2 days ago

Dubai, Dubai, United Arab Emirates 20 hours ago

Dubai, Dubai, United Arab Emirates 23 hours ago

Dubai, Dubai, United Arab Emirates 17 hours ago

Dubai, Dubai, United Arab Emirates 17 hours ago

Dubai, Dubai, United Arab Emirates 17 hours ago

We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

As a seasoned IT professional, you will leverage your expertise to design and implement cutting-edge systems that drive business growth.

The ideal candidate will have a total of 15 years of experience in architecture design of systems and 10 years of experience in architecture design of Big Data Platforms. Key skills include:

1. Designing Big Data Platforms for both Infrastructure and Software Implementation.
2. Understanding the Design Stage of architecture.

Responsibilities:

1. Preparing detailed deployment architecture documents of the Enterprise Big Data Platform, including the redesign of existing architecture.
2. Supporting CART review of the solution architecture of the Enterprise Big Data Platform.
3. Coordinating with Big Data Platform Technical specifications related to the Enterprise Big Data Platform infrastructure software stack.

This role requires strong leadership skills, excellent communication, and a deep understanding of enterprise-level architecture and system design.

In this role, you will work closely with cross-functional teams to deliver high-quality solutions that meet business objectives and drive innovation.

We are seeking a seasoned Lead Data Architect to spearhead the design and implementation of our enterprise data architecture.

This strategic leadership role will focus on defining and evolving our data architecture to support scalable, trusted, and analytics-ready data solutions.

The successful candidate will play a pivotal role in enabling data-driven decision making across the organization by designing architecture that supports AI, machine learning, business intelligence, and operational insights.

• Design and evolve scalable data architecture across raw, curated, and semantic layers.
• Define logical and physical data models with a focus on integrity, lineage, and reusability.
• Architect data pipelines across batch, streaming, and near real-time platforms using tools such as dbt, Kafka, Oracle GoldenGate, and Airbyte.
• Design hybrid data infrastructure supporting both on-premise and cloud-based solutions.
• Embed data governance, metadata management, and lineage into architecture frameworks.
• Ensure RBAC, observability, and automation are integrated into all design principles.
• Collaborate across Data Engineering, CDO, AI, BI, and Business teams to deliver trusted and governed data products.
• Mentor engineering teams and establish best practices around architecture and delivery.

Requirements:

• Bachelor's or Master's degree in Computer Science, Information Technology, or a related field.
• Minimum 10 years of experience in enterprise data architecture, modeling, and governance.
• Strong knowledge of hybrid data platforms and batch/stream processing architectures.
• Hands-on experience with technologies including Oracle, SQL Server, SingleStore, Kafka, Hadoop, Kubernetes, and Parquet.
• Experience working with healthcare data standards such as FHIR, OMOP, Cerner, Epic, and MDR.
• Proficiency in data governance and quality tools such as OvalEdge and YData.
• Familiarity with Azure DevOps, Docker, Kubernetes, and CI/CD frameworks for DataOps and MLOps.

What We Offer:

A collaborative work environment with experienced professionals at the forefront of healthcare transformation.

Opportunities for continuous learning and professional growth.

A competitive compensation package and benefits.

A dynamic and innovative company culture.

Designing Data Pipelines and Architectures

A seasoned professional in data management is sought to spearhead the design, development, and maintenance of cutting-edge data pipelines and architectures.

The ideal candidate will possess strong technical skills in Python, SQL, Apache Airflow, PostgreSQL/MySQL, and Git/GitHub/GitLab.

• Develop efficient data storage solutions and optimize performance
• Implement robust data quality and governance practices

The successful candidate will collaborate with data scientists and analysts, automate workflows, monitor data pipelines, ensure data security and privacy, and manage data engineering infrastructure.