Senior Data Analyst

Summary:

We are seeking a skilled Data Analyst to join our team. The ideal candidate will be responsible for data discovery, validation, rule migration, and reporting within the Databricks platform. This role involves working with SAP data to ensure data quality and accuracy across various layers.

Responsibilities:

• Identify SAP tables and fields used for existing Data Quality (DQ) rules and reports, and locate the corresponding data in Databricks.
• Validate data across all layers (bronze, silver, etc.) in Databricks for completeness, structure, and availability of required fields.
• Select and build a set of existing DQ rules in Databricks as part of the Proof of Concept (PoC).
• Execute rules in Databricks and compare results with existing DQ results, investigating and resolving differences.
• Following the PoC, migrate and build the agreed As-Is DQ rules to Databricks, including SQL logic, data mappings, rule outputs, and required transformations.
• Perform end-to-end validation of source data, DQ rules, results, and defects in Databricks to ensure the migrated process produces expected outcomes.
• Support the development of DQ outputs, KPIs, and dashboard reporting from Databricks, including validation of report results.
• Document data mappings, DQ rules, validation results, and migration processes.

Requirements:

• Experience in Data Quality / Data Validation.
• Experience working with SAP data, preferably master data.
• Experience with the Databricks platform.
• Experience in data reconciliation and comparison between different data sources.

Required Skills:

• SQL & Pyspark/Python knowledge.
• Ability to translate DQ requirements/rules into technical SQL or Python logic.
• Experience in DQ rule testing, defect analysis, and troubleshooting.

Preferred Skills:

• Good understanding of SAP tables.

#AditiConsulting
# 26-06390