Azure Databricks - Sync repo files automatically

Question

In Azure Databricks, I am able to add different git repos in repos filed manually through console. But not able to find any way to automatically sync new files if I committed anything in source repos. Is there any service in Azure which can do this kind of work? Thanks

Answer 1

You can update repository via Repos API, specifically via update command (even if repository is on branch already, calling it again will pull the changes). Another alternative is to use Databricks CLI that also has corresponding command to update repository.

You can setup some CI/CD pipeline in Github Action or Azure DevOps to update your Databricks repo when commit happens. Here is an example for Azure DevOps.