通过Terraform部署Database时的Azure配额限制问题

h6my8fg2  于 2023-05-18  发布在  其他
关注(0)|答案(1)|浏览(106)

我正在尝试在Azure中部署一个databricks工作区并创建一个单节点集群。为此,我使用以下Terraform www.example.com文件:main.tf file:

terraform {
  required_providers {
    azurerm = {
      source  = "hashicorp/azurerm"
      version = "3.55.0"
    }
    databricks = {
      source  = "databricks/databricks"
      version = "1.0.0"
    }
  }
}

# Configure Azure provider
provider "azurerm" {
  features {}
}

# Configure Databricks provider
provider "databricks" {
  host = azurerm_databricks_workspace.databricks_workspace.workspace_url
}

# Create resource group
resource "azurerm_resource_group" "resource_group" {
  name     = var.resource_group_name
  location = var.resource_group_location
}

# Create Databricks workspace
resource "azurerm_databricks_workspace" "databricks_workspace" {
  location            = azurerm_resource_group.resource_group.location
  name                = "databricks-test-001"
  resource_group_name = azurerm_resource_group.resource_group.name
  sku                 = "standard"

  depends_on = [
    azurerm_resource_group.resource_group
  ]
}

# Create cluster
data "databricks_node_type" "smallest" {
  local_disk = true

  depends_on = [
    azurerm_databricks_workspace.databricks_workspace
  ]
}

data "databricks_spark_version" "latest_lts" {
  long_term_support = true

  depends_on = [
    azurerm_databricks_workspace.databricks_workspace
  ]
}

resource "databricks_cluster" "single_node" {
  cluster_name            = "Single Node"
  spark_version           = data.databricks_spark_version.latest_lts.id
  node_type_id            = data.databricks_node_type.smallest.id
  autotermination_minutes = 10

  spark_conf = {
    # Single-node
    "spark.databricks.cluster.profile" : "singleNode"
    "spark.master" : "local[*]"
  }

  custom_tags = {
    "ResourceClass" = "SingleNode"
  }

  depends_on = [
    azurerm_databricks_workspace.databricks_workspace
  ]
}

# Create Notebook
resource "databricks_notebook" "notebook" {
  content_base64 = base64encode("print('Welcome to Databricks-Labs notebook')")
  path           = "/Shared/Demo/demo_example_notebook"
  language       = "PYTHON"

  depends_on = [
    databricks_cluster.single_node,
    azurerm_databricks_workspace.databricks_workspace
  ]
}

当我应用terraform计划时,我得到以下错误:

  • 要求:24,(最小)新限额要求:二十四岁在https提交配额增加请求://aka. ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota。ReactView/Parameters/%7B%22subscriptionId%22:% 22a9f6a84e-aa76 - 4493-ad46 - 7335d8bc7ea5%22,%22命令%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22位置%22:%22westus%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22StandardNCADSA100v4系列%22,%22quotaRequest%22:%7B%22属性%22:%7B%22limit%22:24,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22StandardNCADSA100v4Family%22%7D%7D%7D%7D]%7D,通过指定“详细信息”部分中列出的参数来成功部署。请在https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests databricks_error_message阅读有关配额限制的更多信息:错误代码:QuotaExceeded,错误消息:操作无法完成,因为它导致超出批准的StandardNCADSA100v4系列内核配额。其他详细信息-部署模型:资源经理,位置:westus,电流限制:0、当前使用情况:0,附加要求:24、(最小)新要求限制:二十四岁在https提交配额增加请求://aka. ms/ProdportalCRP/#blade/Microsoft_Azure_Capacity/UsageAndQuota。ReactView/Parameters/%7B%22subscriptionId%22:% 22a9f6a84e-aa76 - 4493-ad46 - 7335d8bc7ea5%22,%22命令%22:%22openQuotaApprovalBlade%22,%22quotas%22:[%7B%22位置%22:%22westus%22,%22providerId%22:%22Microsoft.Compute%22,%22resourceName%22:%22StandardNCADSA100v4系列%22,%22quotaRequest%22:%7B%22属性%22:%7B%22limit%22:24,%22unit%22:%22Count%22,%22name%22:%7B%22value%22:%22StandardNCADSA100v4Family%22%7D%7D%7D%7D]%7D,通过指定“详细信息”部分中列出的参数来成功部署。请在[https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests]](https://docs.microsoft.com/en-us/azure/azure-supportability/per-vm-quota-requests%5D)阅读更多关于配额限制的信息。请访问www.example.com了解更多详情 * https://docs.databricks.com/dev-tools/api/latest/clusters.html#clusterclusterstate for more details*

我知道配额限制需要增加,我已经提交了一个请求,但被拒绝了。我也尝试在不同的Azure区域进行部署,但由于同样的原因,也没有成功。我不明白的是,为什么如果我转到创建的Databasericks工作区并尝试从那里创建一个单节点集群,它的工作完全没有问题。对我来说,这意味着配额限制可能不是真正的问题。我将感谢任何关于可能是什么问题的建议。

eimct9ow

eimct9ow1#

我已经检查了下面的代码,并尝试创建一个单节点集群。

Main.tf:

resource "azurerm_databricks_workspace" "databricks_workspace" {
  location            = data.azurerm_resource_group.example.location
  name                = "databricks-test-001"
  resource_group_name = data.azurerm_resource_group.example.name
  sku                 = "standard"

 
}

data "databricks_current_user" "me" {
  depends_on = [azurerm_databricks_workspace.databricks_workspace]
}
data "databricks_spark_version" "latest" {
  depends_on = [
    azurerm_databricks_workspace.databricks_workspace
  ]
}
data "databricks_spark_version" "latest_lts" {
 long_term_support = true
 depends_on = [
    azurerm_databricks_workspace.databricks_workspace
  ]
}
data "databricks_node_type" "smallest" {   
 local_disk = true
 depends_on = [
    azurerm_databricks_workspace.databricks_workspace
  ]
}

resource "databricks_cluster" "single_node" {
  cluster_name            = "Single Node"
  spark_version           = data.databricks_spark_version.latest_lts.id
  node_type_id            = data.databricks_node_type.smallest.id
  autotermination_minutes = 10

  spark_conf = {
    # Single-node
    "spark.databricks.cluster.profile" : "singleNode"
    "spark.master" : "local[*]"
  }

  custom_tags = {
    "ResourceClass" = "SingleNode"
  }

  depends_on = [
    azurerm_databricks_workspace.databricks_workspace
  ]
}

根据地形应用:

成功执行:
该问题主要是由于区域的Azure配额限制而发生的。正如你所说,即使在更改区域后,问题仍然存在,这个问题必须是由于Azure订阅在使用不正确的Azure creadetials登录时配置错误。
确保在使用az login登录后设置正确的订阅。

az account set --subscription "xxxx"

main.tf

provider "databricks" {
  host = azurerm_databricks_workspace.databricks_workspace.workspace_url
}


provider "azuread" {
   subscription_id = "xxx
      tenant_id              = "xxx"
}


data "azuread_client_config" "current" {
  //tenant_id = "xxxx"
}

data "azurerm_client_config" "current" {
  //tenant_id = "xxx"
 // subscription_id = "8ae0844f-xxx"
}

data azurerm_subscription "current"{
//  subscription_id = "xxx"
}

data "azurerm_storage_account" "example" {
name = "remoxxxx"
  resource_group_name = data.azurerm_resource_group.example.name
}

terraform {
  backend "azurerm" {
    resource_group_name  = "vxxx"
    storage_account_name = "remotestatekavstr233"
    container_name       = "terraform"
    key                  = "terraform.tfstate"
  }
}

如果确认订阅设置正确,则订阅可能已达到其限制。
请检查***Troubleshoot QuotaExceeded error code - Azure | Microsoft Learn***

相关问题