170 Data Scientists jobs in Dubai

Role: Senior Lead Software Engineer

Skill: Big Data Engineer

Experience: 7 Years

Strong functional knowledge and experience in the banking domain.

Develop and optimize data pipelines using Spark, Hive, and Python on Cloudera.

Develop real-time data workflows using Kafka.

Design and develop APIs for data access and integration.

Utilize Hue, Oozie, and other Cloudera tools for job orchestration and data access.

Deploy solutions on cloud platforms such as AWS, Azure, or GCP.

Required Skills and Experience:

Over 7 years of experience in Big Data engineering.

Hands-on experience with Cloudera Spark, Hive, Kafka, Python, Hue, and Ranger.

Strong understanding of distributed systems and cloud data services.

Proficient in API development and data security controls.

About Virtusa:

Join Virtusa and gain international experience working on leading Digital Transformation programs in the Middle East.

Virtusa is a rapidly growing IT services company with a presence in the UAE, KSA, Qatar, and Oman, working with top clients in banking, finance, travel, telecom, and enterprise sectors. We have received awards from Gartner, IDC, WfMC, and others for our exceptional work.

Be part of our award-winning team that values teamwork, quality of life, and professional growth. Join a global community of 30,000 professionals committed to your development, working on exciting projects with cutting-edge technologies.

We are a leading technology company with 219,000+ employees across 54 countries.

We are looking for a highly skilled and self-motivated Informatica professional to join us in advancing the technological world through innovation and creativity.

• Design and develop solutions in Informatica Big Data Management.
• Work extensively with Oozie scheduling, HQL, Hive, HDFS, and data partitioning.
• Experience working with SQL and NoSQL databases.
• Configure Linux OS and use shell scripting.
• Implement design patterns and their implementation.
• Apply Agile, DevOps, and CI/CD principles.

• Minimum 5 years of experience in Informatica BDM.
• 4+ years of development and design experience in Informatica Big Data Management.
• Extensive knowledge on Oozie scheduling, HQL, Hive, HDFS, and data partitioning.
• Experience working with SQL and NoSQL databases.
• Linux OS configuration and use, including shell scripting.

Familiarity with distributed services resiliency and monitoring in a production environment is preferred.

Apache Spark: Strong experience in batch data processing, including RDD, Data Frames, Spark SQL, pySpark, and MLlib; streaming experience is an added advantage.

Cloud & Big Data Platforms: Experience with Databricks, Oracle DRCC, Hadoop, Hive, HBase, Pig and other big data technologies

Data Caching & Performance Optimization: Experience in improving data retrieval and processing performance.

Programming Skills: Strong in Java and Scala (Scala preferred).

Database Experience: Exposure to HDFS, Data Lake, PostgreSQL/MS SQL/SQL, and similar databases.

Data Security & Compliance: Experience with PII data handling, data masking, encryption, decryption, and related security measures.

Workflow Orchestration: Hands-on experience with Apache Airflow or Nifi.

Communication: Strong verbal and written communication skills, with the ability to collaborate across teams and explain technical concepts to non-technical stakeholders.

Secondary Skills

Apache Kafka: Basic hands-on experience is sufficient.

Deployment & Release Management: Experience in application deployment and production release processes.

CI/CD Pipelines : Exposer in designing, implementing, and managing continuous integration and continuous deployment pipelines using tools such as Jenkins, ArgoCD, and Azure CI/CD.

Data Ingestion Tools: Exposer with tools such as Apache Flume, Sqoop, or similar for large-scale data ingestion.

Apply :

• Seniority level Mid-Senior level

• Employment type Contract

• Job function Engineering and Information Technology
• Industries IT Services and IT Consulting

Referrals increase your chances of interviewing at Huns Info by 2x

Get notified about new Big Data Developer jobs in Dubai, United Arab Emirates.

Dubai, Dubai, United Arab Emirates 22 hours ago

Dubai, Dubai, United Arab Emirates 8 hours ago

Dubai, Dubai, United Arab Emirates 1 year ago

We're unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.

Are you passionate about working with big data and innovative technologies?

This is an incredible opportunity to work with a prestigious financial institution, utilizing cutting-edge big data projects.

1. 7-10 years of experience as a Big Data Engineer with expertise in Hadoop (Cloudera), Apache Spark, and similar frameworks.
2. Proficient in Big Data querying tools: Pig, Hive, Impala, with strong coding skills in Python, Java, C, Linux, Ruby, PHP, or R.
3. Skill in solving complex networking, data, and software challenges.
4. Capable of planning, organizing, and delivering projects effectively, with excellent team player skills and strong interpersonal communication skills.

You will enjoy the flexibility of a fully remote role and gain exposure to leading-edge big data projects.

Total of 15 years of experience in architecture design of systems and 10 years of experience in architecture design of Big Data Platforms with the following skills:

1. Proven experience in designing Big Data Platforms for both Infrastructure and Software Implementation.
2. Good understanding of architecture Design Stage .

Responsibilities include:

1. Preparing detailed deployment architecture documents of the Enterprise Big Data Platform (redesign of existing architecture).
2. Supporting CART review of the solution architecture of the Enterprise Big Data Platform.
3. Coordinating with Big Data Platform Technical specifications related to the Enterprise Big Data Platform infrastructure software stack, etc.

Are you an experienced data professional seeking a challenging role in a dynamic environment? We have an exciting opportunity for a skilled Data Engineer to join our team. As a key member of our data engineering group, you will be responsible for designing, developing, and maintaining large-scale data pipelines that meet the organization's business needs.

• Design and develop highly scalable and optimized ETL pipelines using PySpark on the Cloudera Data Platform
• Implement and manage data ingestion processes from various sources to the data lake or data warehouse
• Use PySpark to process, cleanse, and transform large datasets into meaningful formats that support analytical needs
• Conduct performance tuning of PySpark code and Cloudera components, optimizing resource utilization and reducing runtime of ETL processes
• Implement data quality checks, monitoring, and validation routines to ensure data accuracy and reliability throughout the pipeline

• Bachelor's or Master's degree in Computer Science, Data Engineering, Information Systems, or a related field
• 3+ years of experience as a Data Engineer, with a strong focus on PySpark and the Cloudera Data Platform
• Advanced proficiency in PySpark, including working with RDDs, DataFrames, and optimization techniques
• Strong experience with Cloudera Data Platform (CDP) components, including Cloudera Manager, Hive, Impala, HDFS, and HBase

• A competitive salary and benefits package
• Opportunities for career growth and professional development
• A collaborative and dynamic work environment

Big Data Developer Job Description:

The role of a Big Data Developer involves working with large-scale data processing and analysis. This requires expertise in various big data technologies such as Apache Spark, Hadoop, Hive, HBase, Pig, and cloud-based platforms like Databricks.

Required Skills and Qualifications:

• Programming Skills: Strong experience in Java and Scala programming languages is required. Knowledge of other languages like Python is also beneficial.
• Database Experience: Familiarity with databases like HDFS, Data Lake, PostgreSQL, MS SQL, and similar databases is necessary.
• Cloud Platforms: Experience with cloud-based big data platforms like Databricks, Oracle DRCC, and other related technologies is essential.
• Data Security and Compliance: Understanding of data security measures like PII data handling, data masking, encryption, decryption, and related security protocols is crucial.
• Workflow Orchestration: Hands-on experience with workflow orchestration tools like Apache Airflow or Nifi is necessary.

Benefits:

• Opportunity to work on challenging projects involving big data processing and analysis.
• Chance to collaborate with a team of experienced professionals in the field.
• Professional growth and development opportunities.

Job Function and Industry:

• Engineering and Information Technology.
• IT Services and IT Consulting.

Employment Type:

• Contract.

Seniority Level:

• Mid-Senior level.

Job Title:

Big Data Compliance Expert

We are seeking an experienced professional with a strong understanding of data protection regulations to support our data privacy and compliance initiatives.

The ideal candidate will be responsible for configuring, managing, and optimizing the BigID platform to identify, classify, and secure personal and sensitive data across the enterprise.

• Implement, configure, and manage the BigID platform across enterprise systems.
• Work closely with data governance, privacy, and security teams to support GDPR and other data privacy regulations.
• Integrate BigID with various data sources, including structured and unstructured environments.
• Use BigID to scan, discover, and classify PII, sensitive data, and other privacy-related data types.
• Generate reports and dashboards to support compliance, auditing, and privacy impact assessments.
• Define and maintain data retention and minimization policies in line with GDPR requirements.
• Collaborate with IT and business units to remediate data privacy issues identified by BigID.
• Assist in data subject access requests (DSARs) and right-to-be-forgotten processes using BigID automation capabilities.

• Proven hands-on experience with BigID implementation, configuration, and management.
• Strong understanding of GDPR principles, requirements, and compliance practices.
• Experience in data discovery, classification, and mapping tools and processes.
• Familiarity with data privacy frameworks, governance, and security best practices.
• Ability to interpret and apply data protection regulations in a practical and technical context.

• Certifications such as CIPP/E, CIPM, or CIPT are beneficial.
• Experience with additional privacy tools is advantageous.
• Knowledge of other regulations is an advantage.

• Mid-Senior level

• Full-time

• IT Services and IT Consulting

Senior Data Flow Engineer

We are seeking an experienced Senior Data Flow Engineer to join our team. The successful candidate will be responsible for designing, implementing and optimizing complex data flows in production environments using Apache NiFi.

The ideal candidate will have a deep understanding of NiFi flow design principles, including queues, back pressure, prioritizers, and connection tuning. They will also have experience with controller services, such as SSLContextService, DBCPConnectionPool, and RecordReader/RecordWriter.

Additionally, the successful candidate will have expertise in record-based processing using Avro, JSON, CSV schemas and record processors like ConvertRecord, QueryRecord, and LookupRecord.

• Expert-level knowledge of Apache NiFi flow design principles, including queues, back pressure, prioritizers, and connection tuning.
• Experience with controller services, including SSLContextService, DBCPConnectionPool, and RecordReader/RecordWriter.
• Deep understanding of record-based processing using Avro, JSON, CSV schemas and record processors like ConvertRecord, QueryRecord, and LookupRecord.
• Ability to debug and optimize NiFi flows using Data Provenance, bulletins, and log analysis.
• Experience setting up secure NiFi clusters, configuring user authentication (LDAP, OIDC), TLS certificates, and access policies.
• Proficiency in parameter contexts, variable registry, and flow versioning using NiFi Registry.
• Understanding of Zero-Master clustering model, node coordination, and site-to-site protocol.

• Competitive salary and benefits package.
• Opportunity to work with a dynamic and growing company.
• Chance to work on challenging projects and develop new skills.
• Collaborative and supportive work environment.

We are a fast-growing technology company with a strong focus on innovation and customer satisfaction. We offer a competitive salary and benefits package, opportunities for professional growth and development, and a collaborative and supportive work environment.

We are committed to creating an inclusive and diverse work environment that values equality and opportunity for all employees, regardless of race, religion, sex, color, age, national origin, pregnancy, sexual orientation, physical disability or genetic information, military or veteran status, or any other protected classification, in accordance with federal, state, and/or local law.