Data Engineer, Sr.
This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Data Engineer, Sr. based in Ireland.
Join a high-impact engineering team focused on modernizing large-scale data infrastructure and supporting critical database migration initiatives. In this role, you will design, build, and optimize cloud-based data pipelines while ensuring the reliable movement and transformation of structured and unstructured data. You'll work with modern AWS technologies, data lakes, and warehouse architectures to deliver scalable, high-performance solutions. This position offers the opportunity to solve complex technical challenges, collaborate with cross-functional teams, and contribute to product-driven development in a fully remote environment. If you enjoy building resilient data platforms and driving cloud innovation, this is an excellent opportunity to make a meaningful impact.
Accountabilities
- Design, develop, and maintain scalable ETL pipelines to support large-scale data migration and integration projects.
- Build, configure, and optimize AWS-based data lake and warehouse solutions using services such as Glue, S3, Athena, Redshift, Lake Formation, Lambda, EventBridge, and Step Functions.
- Develop Python scripts and database procedures to automate data processing, transformation, validation, and migration workflows.
- Manage relational databases, including MySQL, SQL Server, and Aurora MySQL, with a focus on performance, replication, and reliability.
- Configure and troubleshoot AWS data engineering services, including Glue Catalog, Hudi tables, SQS, and CloudFormation resources.
- Monitor data platforms, identify performance bottlenecks, and implement improvements to ensure scalability, security, and operational excellence.
- Collaborate with product, engineering, and business teams while contributing to Agile development processes and technical planning.
Requirements
- Proven experience as a Data Engineer with strong expertise in AWS cloud data engineering and migration projects.
- Hands-on experience with AWS services including S3, Athena, Glue, Redshift, Lake Formation, Lambda, Step Functions, EventBridge, CloudFormation, and RDS.
- Strong knowledge of relational databases, including MySQL, SQL Server, and data warehouse architecture, particularly Amazon Redshift.
- Proficiency in Python for scripting, automation, ETL development, and data transformation.
- Experience working with structured and unstructured data, data cleansing, conversion, and large-scale ETL processes.
- Knowledge of Hudi table management, AWS Glue Catalog, SQS, and cloud-based data lake implementations.
- Familiarity with Agile methodologies and product-focused software development environments.
- Excellent analytical, troubleshooting, communication, and leadership skills with the ability to work independently in a remote setting.
- Experience with AWS Database Migration Service (DMS), Scala, or Aurora MySQL replication is considered an advantage.
Benefits
- Fully remote work environment with flexibility to work from anywhere.
- Competitive contract compensation based on experience.
- Opportunity to work on large-scale cloud migration and modern data platform initiatives.
- Exposure to a broad AWS ecosystem and advanced cloud-native technologies.
- Collaborative Agile environment with high levels of ownership and technical autonomy.
- Opportunity to contribute to innovative, product-driven engineering projects and expand cloud data engineering expertise.