Azure Databricks User Connector Setup Guide

This guide describes how to configure a secure connection from Abacus.AI to data hosted on the Azure Databricks platform using the Databricks JDBC driver and Azure Entra ID.

Prerequisites

Before you begin, gather the following:

1. Databricks personal access token - In your Databricks workspace, go to User Settings and generate a personal access token.
2. JDBC connection details for your Databricks cluster - Navigate to Compute, select your cluster, open Advanced options, and then the JDBC tab to view connection info. - Note: Do not include https:// in the Database Server URL/IP. Use only the server hostname or address.
3. Azure Entra ID application (Client ID and Client Secret) - Register an app in Azure Entra ID and obtain its Client ID and Client Secret. - Microsoft quickstart: https://learn.microsoft.com/en-us/entra/identity-platform/quickstart-register-app

Set up the connector in Abacus.AI

1. Open the Abacus.AI Connected Services Dashboard: https://abacus.ai/app/profile/connected_services
2. Click Add New Connector, choose ODBC/JDBC, and select Databricks JDBC driver from the Driver dropdown.
3. Turn on the Import RBAC toggle.
4. In Azure Entra ID, create/register an application if you have not already done so.
5. Enter the app’s Client ID and Client Secret in the corresponding fields.
6. Fill in all Databricks configuration details gathered above (server hostname/address, JDBC info, token, etc.) and click Save.
7. Click Verify and wait for the connector to be validated. If an error appears, follow the message to troubleshoot and correct your configuration.

Build a ChatLLM project on your connector

Create the project

8. From the Abacus.AI logo, go to the Projects page.
9. Create a new project.
10. Select ChatLLM – Custom LLM Chat.
11. Enter a name for your project.
12. Choose Skip to project dashboard.

Train the model

13. In the left toolbar, open Model and click Train Model (top right).
14. For Structured data source, select External service from the dropdown.
15. Choose the connector you created in the steps above and add the tables to use for testing.
16. Click Train Model.
17. Once training completes, open Models and select your model.

Deploy and test

18. Click Create a new deployment.
19. Select Offline Batch + Realtime, then click Next.
20. Enter an end‑user friendly deployment name and click Deploy.
21. Return to your model (Model > your model name). When the deployment is Active, click the deployment name.
22. Under Deployment, open the Prediction Dash.
23. Click Go to Abacus.AI chat to test your bot.

End‑user sign‑in

24. Enter your prompt/question. Each user will be asked to sign in once via Azure Entra ID. Log in using your Entra ID user credentials.

Troubleshooting

• Verification fails on connector save
• Re‑check server hostname/address formatting (no https://).
• Confirm the personal access token is valid and not expired.
• Ensure JDBC details (from Compute > Advanced options > JDBC) match the active cluster.

• Validate the Azure Entra ID app’s Client ID/Secret and required permissions.

• Unable to access tables after verification

• Confirm the user or service principal has the necessary Databricks and data permissions (RBAC).
• Re‑verify the connector to surface detailed error messages.

If you need to update this document, edit /home/ubuntu/Uploads/databricks_user_connector.md and re‑publish as needed.