Unleash transformative data solutions — shape the future of cloud and data engineering!
Lisbon-based opportunity with hybrid work model (up to 2 remote days per week).
As a Senior Data Platform Engineer, you will be working for our client, a leading innovator in data-driven decision-making. You will play a crucial role in designing, implementing, and maintaining scalable data solutions on Google Cloud Platform (GCP). Your expertise will support strategic insights and empower organizations through robust data models, pipelines, and governance. This is your chance to drive digital transformation and take your career to new heights in a dynamic, forward-thinking environment.
Your main responsibilities:
- Develop and maintain data models using dbt, focusing on medallion architecture, including dimensional models.
- Design generic, extensible, and integration-ready data models capable of supporting numerous heterogeneous data sources.
- Implement incremental models and automated data tests (dbt tests) to ensure data reliability.
- Provide detailed documentation for data models, flows, and transformation processes.
- Handle batch and streaming data processing using Python, Apache Beam, and Google Cloud Dataflow.
- Integrate data from external APIs and perform data manipulation with pandas and pyarrow.
- Manage and operate GCP resources such as BigQuery, Cloud Run, Workflows, Cloud Scheduler, Pub/Sub, and IAM.
- Implement and maintain data governance, quality, and lineage solutions using Dataplex, Data Catalog, and industry best practices.
- Manage infrastructure as code with Terraform, ensuring organized and scalable environments across development, testing, and production stages.
- Apply software engineering best practices, including Git, CI/CD, project structuring, and YAML-based configuration for pipelines and environments.
- Monitor, troubleshoot, and optimize pipelines, jobs, and workflows, utilizing Cloud Logging and Cloud Monitoring.
- Serve data to diverse consumers, including AI, ML, BI, and business applications, ensuring high performance and reliability.
You're ideal for this role if you have:
- 4+ years of experience as a Data Engineer or in a similar senior role.
- Advanced proficiency in SQL, especially BigQuery.
- Proven experience with dbt and modern data modeling (Bronze/Silver/Gold layers).
- Deep knowledge of data processing with Python, Apache Beam, and Google Cloud Dataflow.