SQL operations you might need one dayIf you work with data you will most likely use SQL every now and then. If data is not populated in your SQL tables according to basic…May 2May 2
Run integration test between Data Factory and Databricks in Azure DevOps CI pipelineCheck out this example of how to use Powershell to run an integration test between Azure Data Factory and Databricks in your CI pipeline.May 2May 2
Published inAzure TutorialsSave Excel as CSV with custom delimiterLet’s hope that you as a Data Engineer never need this article, because that means the data you are working with is by default…Jun 8, 2023Jun 8, 2023
Building a scalable metadata-driven data ingestion frameworkIn this blog I discuss the concepts of a metadata-driven data ingestion framework, enabling scalable data processing and ingestion.Oct 18, 20221Oct 18, 20221
Published inAzure TutorialsCosmosDB (Gremlin) database design: positioning of edges in partitionsIn this blog we will focus on CosmosDB data patterns, and will look specifically at the position and usage of edges in your graph database.Jul 18, 2022Jul 18, 2022
Published inAzure TutorialsAdd/Remove Azure Resource Locks with PowershellResource locks are a powerful mechanism to protect your resources from unauthorized operations.May 30, 20221May 30, 20221
Published inAzure TutorialsUse dataset parameters to copy data to dynamically defined source and sink directories with ADFIn this blog we show how to configure dynamic source and sink directories for your Data Factory workflows.May 2, 2022May 2, 2022
Published inAzure TutorialsDo’s and Don’ts when working with CosmosDB Gremlin APISharing some of our experiences with ingesting data into Cosmos DB Gremlin API.Apr 18, 20221Apr 18, 20221
Published inAzure TutorialsConnect Azure Databricks to CosmosDB Gremlin API and run queriesHow to connect Databricks to CosmosDB (Gremlin API) and run queries on the database including a scenario for usage in an end to end flow.Apr 4, 2022Apr 4, 2022
Published inAzure TutorialsBest practices when using Databricks notebooks in an automated workflowBest practices or lessons learned when designing automated workflows with Databricks including a few examples of useful generic functions.Mar 21, 2022Mar 21, 2022