Data Engineer - TA192

• To be able to develop/design for Data Lake, System/Data Integration, App engine solutions (containing complex big data computations), ensure the product delivered on time, within budget.
• Work with Data Expert to align architecture and solution.
• Develop, optimize, and maintain scalable ETL/ELT pipelines using AWS services such as Glue, Lambda, and Kinesis.
• Build and manage data storage solutions leveraging AWS S3, Redshift, RDS, DynamoDB, and Athena.
• Design data ingestion workflows from diverse data sources ensuring data quality and integrity.
• To make sure all incidents which cannot be solved on previous lines of support are solved (Level 3 support)
• Provide Data integration standards for new IT projects within Data domain.
• Research and recommend future-proofed Data integration/processing technologies e.g., cloud, advanced data platform.
• Share coding experience with other team members.

The principal function of a Data Engineer is to make specific tasks, based on the business's specifications:

• Collaborate with Business Analysts (BAs) to design and implement data modeling solutions at Staging, Atomic, Datamart, Sematic layer to provide business intelligence or to provide data to other downstream applications.
• Design and develop solutions, including comprehensive technical documentation and operational/deployment manuals for the operation of the program by operators.
• Have deep knowledge in the data domain, with expertise in utilizing and designing/develop solutions based on batch and real-time data processing technologies such as Kafka, Flink, Kafka Connect, Debezium, Spark, Hadoop, EMR, Glue...
• Have knowledge of AWS cloud and the necessary services/open-source tools for data processing and infrastructure deployment, including EMR, Kubernetes (K8s), EKS, Docker, EventBridge, Lambda,StepFunction, Airflow, Terraform...
• Have knowledge in optimizing data processing on database systems such as Oracle, SQL Server, and big data systems on the cloud.
• Updating, repairing, modifying, developing, enhance existing development.
• Investigate and troubleshoot for issues, incidents, and problems of application as level 3 support.
• Study the technical updates and make the change or do upgrading release in Development area.
• Follow the IT processes (development, operation) and Bank processes.