Responsibilities and Requirements
- Experience in AWS services such as Lambda, CloudFormation, S3, MWAA, IAM etc.
- Experience in CI/CD with Bamboo, Bitbucket, Git workflows.
- Proficiency in Linux, Python, Bash, AWS CLI.
- Experience with Airflow/MWAA such as DAG creation, variables, connections, logs.
- Build and maintain MWAA (Airflow) orchestration workflows.
- Understanding of ML pipeline lifecycle (training evaluation deployment).
- Develop DAGs for ETL, data pipelines, ML training, and batch inference.
- Understanding of Big Data Technologies like Hadoop,spark.
