microsoft / purview-custom-types-tool-solution-accelerator

Solution accelerator for creating custom type definitions in Microsoft Purview.

page_type	languages	products
sample	javascript	microsoft-purview

Microsoft Purview Custom Types Tool

The Purview Custom Types Tool is a solution accelerator for supporting custom connectors in Microsoft Purview, which is a unified data governance service that helps you manage and govern your on-premises, multicloud, and software-as-a-service (SaaS) data.

This application supports the following actions:

• Browse existing type definitions
• Create new entity & relationship type definitions
• Create entity templates to simplify source scanning

When combined with the Purview Custom Connector Solution Accelerator, this application provides entity templates to help create source scanning functionality. The diagram below outlines the typical journey to develop a custom connector for Microsoft Purview:

Prerequisites

• Azure subscription
• Microsoft Purview
• Localhost or Windows 365:
  • Node.js
  • Azure Function Core Tools

Deployment

This application can be deployed in various ways to meet your needs. Because the creation of type definitions is often limited to a few individuals for a limited time period, this repo is built to run locally by default.

1. Local PC - default, ready for local setup below
2. Windows 365 Cloud PC - securely stream Windows from the Microsoft cloud to any device
3. Azure - using Static Web Apps and Functions

Local Setup

1. Clone or download this repository and navigate to the project's root directory
2. Follow these instructions to create an application service principal that can access the Microsoft Purview API
3. Rename /api/local.settings.json.rename to local.settings.json by removing .rename extension
4. Edit your /api/local.settings.json file:
   • Update AadTenantId with your Tenant ID for Azure Active Directory
   • Update AppClientId and AppClientSecret with your application service principal
   • Update AtlasAccountName with your Microsoft Purview account name
   • Update StorageConnectionString with your connection string for Azure Data Lake Storage
   • Update StorageContainer with your preferred container name
5. Install the application's dependencies:

   npm install

Run the App Locally

1. Start the application API from a new terminal:

   cd api
   npm start

   API now running at http://localhost:7071/api/aad/token

2. Start the application frontend from a second new terminal:

   npm start

   App now running at http://localhost:3000/. Note the package.json file has a proxy for port 7071 so you can code directly to the /api path on localhost. For more information on the local Azure Functions, see the API README.

Deploy to Azure

Please review the deployment documentation for details. Due to the two methods of API support in Azure Static Web Apps with Azure Functions, you will need to deploy a separate Azure Function for the /api/AadToken endpoint.

Please follow additional best practices for Securing Azure Functions and note the Security constraints with bringing your own functions to Azure Static Web Apps

Learn More

Microsoft Purview

• Purview Custom Connector Solution Accelerator
• Purview Machine Learning Lineage Solution Accelerator
• Tutorial using the REST APIs

Azure Static Web Apps

• Quickstart for React apps
• Authentication and authorization
• API support with Azure Functions
• Getting started with Azure Functions

Contributing

This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.

When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.

This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact [email protected] with any additional questions or comments.

Trademarks

This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.

purview-custom-types-tool-solution-accelerator's Issues

Referenced bundle Microsoft.Azure.Functions.ExtensionBundle of version 1.8.1 does not meet the required minimum version of 2.6.1. Update your extension bundle reference in host.json to reference 2.6.1 or later.

Running into the issue below after step.2 npm start

Azure Functions Core Tools
Core Tools Version:       4.0.3971 Commit hash: d0775d487c93ebd49e9c1166d5c3c01f3c76eaaf  (64-bit)
Function Runtime Version: 4.0.1.16815

[2022-03-15T13:59:57.194Z] Referenced bundle Microsoft.Azure.Functions.ExtensionBundle of version 1.8.1 does not meet the required minimum version of 2.6.1. Update your extension bundle reference in host.json to reference 2.6.1 or later.
[2022-03-15T13:59:57.200Z] A host error has occurred during startup operation 'f965be58-ce17-4295-a1dc-afe466da42e6'.
[2022-03-15T13:59:57.202Z] Microsoft.Azure.WebJobs.Script: Referenced bundle Microsoft.Azure.Functions.ExtensionBundle of version 1.8.1 does not meet the required minimum version of 2.6.1. Update your extension bundle reference in host.json to reference 2.6.1 or later. For more information see https://aka.ms/func-min-bundle-versions.
Value cannot be null. (Parameter 'provider')

Error: Unable to access application API - Deploy to Azure

I've ensured the step 7 is completed. I can confirm that these values are correct as my App starts up fine locally.

Myself who is not familiar with Azure Functions, the next step to fix and resolve this issue is unclear. Kindly assist.
Due to the two methods of [API support in Azure Static Web Apps with Azure Functions](https://docs.microsoft.com/en-us/azure/static-web-apps/apis), you need to manually create an Azure Function for the /api/AadToken endpoint. For more information, please follow the [Getting started with Azure Functions](https://docs.microsoft.com/en-us/azure/azure-functions/functions-get-started?pivots=programming-language-javascript) and learn how to [Bring your own functions to Azure Static Web Apps](https://docs.microsoft.com/en-us/azure/static-web-apps/functions-bring-your-own).

google drive files are not downloading

google drive files are not downloading

Azure-Databricks-Solution-Accelerator-Financial-Analytics-Customer-Revenue-Growth-Factor Set-Up

When I run the notebook to download data for the October 2019 - April 2020 eCommerce datasets, I receive "java.io.FileNotFoundException: File file:/databricks/driver/2019-Oct.csv does not exist"
Let me know how I can proceed please.