Databricks

Databricks combines data warehouses & data lakes into a lakehouse architecture that handles data, analytics, and AI use cases.

Use Cases the Adapter Solves

• Cloud Data Platform Asset Visibility: Discover and inventory Databricks compute clusters across workspaces and cloud providers (AWS, Azure, GCP), giving security and IT teams a unified view of data lakehouse infrastructure alongside the rest of their asset estate.
• Identity and Access Governance for Data Platforms: Correlate Databricks workspace and account-level users and groups with corporate identity sources to detect unmanaged accounts, orphaned identities, and group membership drift across the data platform.

Asset Types Fetched

This adapter fetches the following types of assets:

Devices, Users, Groups, Application Settings, SaaS Applications

Before You Begin

Required Ports

• TCP port 443 (HTTPS)

Authentication Methods

The Databricks adapter supports the following authentication methods.

APIs

Axonius uses the Databricks REST API 2.0. The endpoints called depend on the configured Databricks Level.

OAuth M2M token endpoints (when using OAuth authentication):

• POST /oidc/v1/token - Obtains an OAuth access token for workspace-level connections using the client_credentials grant.
• POST /oidc/accounts/{account_id}/v1/token - Obtains an OAuth access token for account-level connections using the client_credentials grant.

Workspace level:

• GET /api/2.0/clusters/list
• GET /api/2.0/preview/scim/v2/Users

Account level:

• GET /api/2.0/accounts/{account_id}/workspaces
• GET /api/2.0/accounts/{account_id}/scim/v2/Users
• GET /api/2.0/accounts/{account_id}/scim/v2/Groups
• GET /api/2.0/accounts/{account_id}/scim/v2/Groups/{group_id}

SQL level:

• POST /api/2.0/sql/statements
• GET /api/2.0/sql/statements/{statement_id}

Application Settings

• GET /api/2.0/accounts/{account_id}/oauth2/published-app-integrations - Fetches OAuth app integrations
• GET /api/2.0/accounts/{account_id}/oauth2/published-apps - Fetches OAuth app metadata for enrichment

Required Permissions

The credentials supplied to the adapter (Personal Access Token or OAuth service principal) must be associated with an identity that has read permission on the resources being fetched. The exact permissions required depend on the configured Databricks Level:

Workspace Level

• Workspace access (member of the workspace).
• Permission to call the Clusters API (clusters/list) - typically granted to any workspace user; cluster visibility is governed by cluster ACLs.
• Workspace admin role to read the SCIM Users API (preview/scim/v2/Users).

Account Level

• Databricks Account Admin role on the target account, which is required to:
  • List workspaces under the account (accounts/{account_id}/workspaces).
  • Read account-level users and groups via the Account SCIM API (accounts/{account_id}/scim/v2/Users, accounts/{account_id}/scim/v2/Groups).
  • When fetching Application Settings:
    • Read published OAuth app integrations (GET /api/2.0/accounts/{account_id}/oauth2/published-app-integrations)
    • Read published OAuth apps (GET /api/2.0/accounts/{account_id}/oauth2/published-apps)
• For per-workspace cluster and user fetches performed in account mode, the same workspace-level permissions listed above apply (workspace admin recommended).

SQL Level

• CAN USE permission on the configured SQL Warehouse.
• USE CATALOG on the target catalog and USE SCHEMA on the target schema.
• SELECT privilege on the table, view, or objects referenced by the configured Table/View Name or Custom SQL Statement.

📘

Note

When using OAuth M2M authentication, the service principal must be assigned to the workspace and/or account with the same role and entitlement requirements listed above.

Supported From Version

Supported from Axonius version 4.7

Connecting the Adapter in Axonius

Navigate to the Adapters page, search for Databricks, and click on the adapter tile.

Click Add Connection.

To connect the adapter in Axonius, provide the following parameters:

Required Parameters

1. Host Name or IP Address (required) - The hostname or IP address of the Databricks server. Example: mycompany.cloud.databricks.com or accounts.cloud.databricks.com for account-level.
2. Authentication Type - Select the authentication method:

3. Databricks Level - Select Account, Workspace or SQL.

Optional Parameters

1. Verify SSL - Select whether to verify the SSL certificate of the server against the CA database inside of Axonius. For more details, see SSL Trust & CA Settings.
2. HTTPS Proxy (optional) - Connect the adapter to a proxy instead of directly connecting it to the domain.
3. HTTPS Proxy User Name (optional) - The user name to use when connecting to the value supplied in Host Name or IP Address via the value supplied in HTTPS Proxy.
4. HTTPS Proxy Password (optional) - The password to use when connecting to the server using the HTTPS Proxy.

To learn more about common adapter connection parameters and buttons, see Adding a New Adapter Connection.

Advanced Settings

📘

Note

• Advanced settings can apply to either all connections of this adapter, or to a specific connection. For more detailed information, see Advanced Configuration for Adapters.
• For more general information about advanced settings, see Adapter Advanced Settings.

• Fetch SaaS applications (Account Level Only) - Fetch published OAuth app integrations as SaaS Applications. The adapter must be configured to use Account-level authentication to use this setting and you must provide a valid Databricks Account ID in the connection settings.
• Fetch Application Settings - Select this option to fetch detailed information about each OAuth application integration.
• Enable Custom Parsing - Enable this option to define how to parse specific fields from the raw data fetched. You can choose to parse the data into an already existing field, or create a new one. This adapter supports User Custom Parsing/ Device Custom Parsing. See Adapter Custom Parsing for more information.

Version Matrix

This adapter was only tested with the versions marked as supported, but may work with other versions. Contact Axonius Support if you have a version that is not listed, which is not functioning as expected.