Senior Data Engineer

Ivy Partners • Porto

Tempo inteiro Outros
Salário não especificado
Descrição da Vaga
**Our mission** Ivy Partners is a Swiss consulting firm that supports businesses in their strategic, technological, and organizational challenges. We are committed to providing our team members with a career that fosters both personal and professional growth. We support their skill development and offer real opportunities for advancement. **As a Senior Data Engineer at Ivy Partners, you will:** * Advise on data strategy, architecture, and implementation approaches within a cloud\-first environment. * Design and develop data pipelines on public cloud platforms, primarily AWS, with exposure to Azure/GCP where relevant. * Implement and optimize Lakehouse architectures, focusing primarily on Apache Iceberg (experience with Hudi and Delta is a plus). * Build, automate, and maintain ETL processes, data workflows, and orchestration pipelines using tools like Airflow, Step Functions, and AWS Workflow. * Ensure high standards of reliability, automation, maintainability, and security across all delivered solutions. * Support incident resolution, root cause analysis, and preventive measures as required. * Share knowledge and best practices with internal teams to enhance engineering capabilities. * Produce comprehensive documentation covering designs, implementations, and operational guidelines. * Participate in Agile ceremonies, technical reviews, and knowledge exchange sessions. **About You** **You are our ideal candidate if you:** * Have 3 to 5 years of experience building data pipelines on public cloud services, preferably AWS (Azure/GCP also considered). * Possess at least 1 year of experience in data solution architecture, design, development, and support. * Are experienced in database architecture, ETL, BI, Big Data, ML, and Advanced Analytics. * Have hands\-on experience with Spark ecosystems (experience with Hadoop is a plus). * Are proficient with pipeline orchestration tools such as Airflow, Step Functions, and AWS Workflow. * Have demonstrated experience with Lakehouse architectures (Apache Iceberg) and table optimizations like compaction and Z\-Order. * Have software development skills in Python, SQL, and SQL Server; familiarity with SSIS packages and Docker is beneficial. * Are experienced with AWS services including Lambda, Glue, Crawlers, Step Functions, AWS Workflow, DynamoDB, and AWS Batch. * Have experience with DevOps and Agile practices, infrastructure as code (CloudFormation, Terraform), and artifact/vulnerability management. * Understand cloud foundation services: networking, security, storage, and IAM. * Preferably have experience with container\-based applications (Docker) and Linux\-based infrastructure. * Are fluent in English (written and spoken); French is a plus. **Why Join Ivy Partners?** * **Care** \| We care about our employees by creating a supportive environment where everyone is valued and empowered with training and growth prospects. * **Trust** \| We believe in building strong and long\-lasting relationships with our clients. Working with us is about establishing a partnership based on trust, professionalism, and transparency at all times. * **Innovation** \| We empower effective digital transformation through a unique blend of innovative technology and creative thinking. * **Responsibility** \| Community is at the heart of everything we do, and we take that responsibility seriously by working hard to create a positive impact.
Precisa de estar logado para se candidatar.
Login para Candidatar