Thoughts on a personal journey into the modern world of data governance and DataOps.

Photo by Benjamin Davies on Unsplash

In April of 2021, I chose to leave a 10 year job at CI&T and join a nascent startup company as their first hire. In this post, I share how my personal journey mixed up with these two crazy dudes from Norway and England and their pioneering tech to solve some exciting problems in the space.


Learn how to ingest SAP HANA metadata to Google Data Catalog and extend it with your user needs


Scripts and Terraform automation to help you ensure best practices in Google Data Catalog

  • What end user should be able to discover my data assets?
  • Who should be able to classify and add tags to them?
  • And finally, be able to create templates and set standards for the data classification process?


Best practices on two approaches, with code samples!


From design decisions to step by step execution learn how to ingest Apache Atlas metadata in Google Data Catalog doing full and incremental runs.

Image created on Canva.

The Dress4Victory company


Learn how your metadata is structured on both systems.

Image created on Canva.

The one million dollar question


How to Create Data Catalog tags by inspecting all your BigQuery data with Cloud Data Loss Prevention.

Background by Kelli Tungay on Unsplash


Code samples with a practical approach on how to incrementally ingest metadata changes from an on-premise Hive server into Google Cloud Data Catalog

Background by JJ Ying on Unsplash

The Challenge


Code samples with a practical approach on how to ingest metadata from an on-premise Hive server into Google Cloud Data Catalog

Background by JJ Ying on Unsplash

The Challenge

Data Catalog


Code samples with a practical approach on how to ingest metadata from on-premise Relational Databases into Google Cloud Data Catalog

Background by JJ Ying on Unsplash

The Challenge

Marcelo Costa

software engineer & google cloud certified architect and data engineer | love to code, working with open source and writing @ alvin.ai

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store