
Search by job, company or skills
OSOTto Databricks Migration & Platform Ownership
. Lead the end-to-end migrationof the current OSOT solution (data lakes, data warehouse, data marts, andTalend/SQL/Python pipelines) onto the Databricks Data Intelligence Platform,defining the target lakehouse architecture, migration roadmap, sequencing, andcutover strategy with minimal disruption to production.
. Re-architect OSOT into anAPAC-specific Enterprise Data Platform on Databricks using Delta Lake, themedallion (bronze/silver/gold) architecture, Unity Catalog for governance andlineage, and Lakeflow / Delta Live Tables and Workflows for orchestration.
. Migrate and modernize ETL/ELTfrom existing tooling (Talend, SQL, Python) to Databricks-native pipelines(PySpark, Spark SQL, Databricks SQL), establishing reusable frameworks foringestion, transformation, data quality, and metadata/dependency management.
. Build data pipelines,architectures, and curated data sets from raw, loosely structured data, andstand up Databricks SQL warehouses to serve BI and self-service analytics forAPAC businesses.
. Own a modern data engineeringoperating model that follows DevOps/DataOps principles - CI/CD for notebooksand jobs (Databricks Asset Bundles, Git integration), automated testing, andinfrastructure-as-code.
AIPlatform Integration (Claude, OpenAI) on Databricks
. Define and own how Databricksworks with external AI platforms such as Anthropic Claude and OpenAI -establishing secure, governed integration patterns via Databricks Mosaic AI,Model Serving, and AI Gateway (external model endpoints) so LLMs can be consumedcentrally with rate limiting, logging, and cost controls.
. Use Unity Catalog andDatabricks governance to control which enterprise data is exposed to AI models,ensuring PII protection, access policies, and auditability before any datareaches Claude or OpenAI.
. Enable Retrieval-AugmentedGeneration (RAG) on APAC enterprise data using Databricks Vector Search andfeature/embedding pipelines, grounding Claude and OpenAI responses in trustedOSOT data for use cases such as analyst copilots, document intelligence, andnatural-language querying of the lakehouse.
. Operationalize GenAI use casesend-to-end - prompt management, evaluation, monitoring, and MLOps - usingDatabricks Mosaic AI (MLflow, Model Serving, AI Gateway, and agent/evaluationframeworks).
. Partner with Global IT andSecurity to define responsible-AI guardrails for LLM usage (data residency,prompt/response logging, content safety, and acceptable-use policies for Claudeand OpenAI).
DataModeling, Analytics & Stakeholder Delivery
. Interface with cross-functionalteams (Finance, Risk, SCM, and Product) to design and build data models on thelakehouse that provide actionable insights into key business performancemetrics and support the commercial team.
. Leverage cloud-based lakehousearchitecture and Databricks ML capabilities to deliver optimized ML and AImodels at scale.
. Work closely with leadersacross data architecture, enterprise architecture, data science/analytics, anddomain experts to build and maintain roadmaps aligned to the IT strategy.
. Act as a strategic thinker witha holistic vision, with specific focus on identifying and automating existingmanual processes - including AI-assisted automation - to drive key businessperformance.
. Participate in project workinggroups and assist in tracking project milestones and deliverables.
Critical Competencies / Critical SuccessFactors
. Bachelor's degree requiredmaster's degree (or equivalent) in computer science, engineering (allbranches), mathematics, or a related field preferred.
. 8+ years of experience withpublic and private cloud solutions, including building and maintaining a dataecosystem that includes an ERP environment.
. 8+ years of experiencedesigning and building data-intensive solutions using distributed computing.
. Hands-on experience with theDatabricks Data Intelligence Platform - Delta Lake, Unity Catalog, DatabricksSQL, Lakeflow / Delta Live Tables, Workflows, and the medallion architectureexperience leading a migration onto Databricks is strongly preferred.
. Experience integrating andoperationalizing LLM/GenAI platforms (e.g., Anthropic Claude, OpenAI) -including Databricks Mosaic AI, Model Serving, AI Gateway, Vector Search, andRAG patterns - with appropriate governance and security.
. 6+ years of programmingexperience with SQL, Python, Scala, and Java.
. Experience with distributeddata streaming frameworks such as Spark Structured Streaming, Apache Flink, andKafka.
. OLAP experience (Cubes, MSSQL)and data warehouse usage/optimization experience (Databricks SQL, Redshift,Hive, Snowflake).
. Experience building datavisualizations or analytics (e.g., Power BI, Databricks dashboards, SSRS).
. Experience building machinelearning models familiarity with MLflow and modern MLOps practices.
. Strong analytic skills andunderstanding of statistical methodologies.
. Strong knowledge of datagovernance, security, and compliance, including responsible/secure use of AIand LLMs.
. Ability to communicateeffectively with external clients and internal teams and manage expectations.
. Demonstrated ability to work asan effective team member - sharing knowledge and helping others meet teampriorities - while also working independently as an effective decision-maker.
. Good verbal, written, andpresentation skills able to engage with offshore-based development and supportteams.
. An effective troubleshooter,able to analyze problems and develop/recommend solutions, with a willingness tolearn and adapt to strategic initiatives in the pipeline.
If you're interested in this role, click apply now to forward an up-to-date copy of your CV, or call Shabnam at Hays on +65 60271964 or email [Confidential Information] for a confidential discussion.
Referrals are welcome.
Registration ID No. R1873584 | EA License number: 07C3924 | Company Registration No. 200609504D
Job ID: 153342853