.net 无法使用CosmosDB V3 SDK将cosmosdb容器吞吐量从手动更改为自动缩放

up9lanfz  于 2022-12-05  发布在  .NET
关注(0)|答案(3)|浏览(139)

我正在尝试使用以下代码将手动容器的吞吐量替换为自动缩放

container.ReplaceThroughputAsync(ThroughputProperties.CreateAutoscaleThroughput(4000));

这将引发异常。错误”:[“必须提供x-ms-cosmos-migrate-offer-to-autopilot,并且offercontent不得包含autopilotSettings,以便从手动吞吐量迁移到自动缩放。”
无法在CosmosDB文档上找到任何与此相关的内容。我目前使用的是CosmosDB 3. 12 V3 .Net SDK。

bf1o4zei

bf1o4zei1#

现在不支持通过SDK将吞吐量从手动更改为自动缩放。方法ReplaceThroughputAsync只能更改吞吐量。你应在Azure门户上更改此设置。

lxkprmvk

lxkprmvk2#

Azure Cosmos DB "Replace an Offer" REST API method允许在手动和自动缩放之间切换吞吐量模式。我不会在这里重复该方法的全部文档,但要点是,根据您的方向,您必须在正文的“content”属性中提供一个特殊的值,以及一个特定的自定义HTTP请求头。

  • 手动-〉自动缩放
  • 内容:{ "offerThroughput": -1 }
  • 信头:x-ms-cosmos-migrate-offer-to-autopilot=true
  • 自动缩放-〉手动
  • 含量:{ "offerAutopilotSettings": {"maxThroughput": -1} }
  • 信头:x-ms-cosmos-migrate-offer-to-manual-throughput=true

使用REST API比SDK要复杂得多。下面的C#类总结了如何更改吞吐量方法。您需要nuget包“Microsoft.Azure.Cosmos”。

using System;
using System.Net;
using System.Net.Http;
using System.Reflection;
using System.Threading.Tasks;
using Microsoft.Azure.Cosmos;
using Newtonsoft.Json;

namespace ThroughputChangerDemo
{
    public class ThroughputChanger
    {
        private readonly HttpClient HttpClient;

        public ThroughputChanger(HttpClient httpClient)
        {
            HttpClient = httpClient;
        }

        public async Task SetThroughput(Database database, bool autoScale, int maxThroughput)
        {
            ThroughputResponse oldThroughputResponse = await database.ReadThroughputAsync(new RequestOptions { });
            var currentThroughput = oldThroughputResponse.Resource;

            ThroughputProperties newThroughput = GenerateThroughputProperties(autoScale, maxThroughput);

            if (currentThroughput.IsAutoScale() != autoScale)
            {
                await this.ChangeScalingMethodology(database.Client, currentThroughput, GenerateDatabaseLink(database));
            }

            await database.ReplaceThroughputAsync(newThroughput);
        }

        public async Task SetThroughput(Container container, bool autoScale, int maxThroughput)
        {
            ThroughputResponse oldThroughputResponse = await container.ReadThroughputAsync(new RequestOptions { });
            var currentThroughput = oldThroughputResponse.Resource;

            ThroughputProperties newThroughput = GenerateThroughputProperties(autoScale, maxThroughput);

            if (currentThroughput.IsAutoScale() != autoScale)
            {
                await this.ChangeScalingMethodology(container.Database.Client, currentThroughput, GenerateContainerLink(container));
            }

            await container.ReplaceThroughputAsync(newThroughput);
        }

        /// <summary>
        /// Toggle between Autoscale and Manual scaling methodologies for a database or container.
        /// </summary>
        /// <param name="currentThroughput"></param>
        /// <param name="scalableItemLink">The resource link for the database or container to be changed</param>
        /// <returns></returns>
        private async Task ChangeScalingMethodology(CosmosClient client, ThroughputProperties currentThroughput,
            string scalableItemLink)
        {
            bool changeToAutoScale = !currentThroughput.IsAutoScale();

            // Attempt to change between scaling schemes...
            string offerId = currentThroughput.SelfLink.Split('/')[1];
            string offerResource = $"offers/{offerId}";
            var url = $"{client.Endpoint.Scheme}://{client.Endpoint.Host}/{offerResource}";
            var restEndpointUri = new Uri(url);
            var method = HttpMethod.Put;
            var httpDate = DateTime.UtcNow.ToString("R");
            string auth = GenerateAuthToken(method, "offers", offerId,
                httpDate, extractAuthKey());
            var request = new HttpRequestMessage
            {
                RequestUri = restEndpointUri,
                Method = method,
                Headers = {
                    { HttpRequestHeader.Authorization.ToString(), auth },
                    { "x-ms-version", "2018-12-31" },
                    { "x-ms-date", httpDate },
                },
                Content = new StringContent(JsonConvert.SerializeObject(createOffer()))
            };

            if (changeToAutoScale)
            {
                request.Headers.Add("x-ms-cosmos-migrate-offer-to-autopilot", "true");
            }
            else
            {
                request.Headers.Add("x-ms-cosmos-migrate-offer-to-manual-throughput", "true");
            }

            HttpResponseMessage putResponse = await HttpClient.SendAsync(request);
            if (!putResponse.IsSuccessStatusCode)
            {
                var content = await putResponse.Content.ReadAsStringAsync();
                throw new Exception($"Error changing throughput scheme: '{putResponse.ReasonPhrase}'.\nContent: {content}");
            }

            // local function
            object createOffer()
            {
                // Read the ResourceRID using reflection because the property is protected.
                string resourceRID = currentThroughput.GetType()
                    .GetProperty("ResourceRID", BindingFlags.NonPublic | BindingFlags.Instance)
                    .GetValue(currentThroughput).ToString();
                string resourceLink = scalableItemLink;
                object content;
                if (changeToAutoScale)
                {
                    content = new
                    {
                        offerThroughput = -1
                    };
                }
                else
                {
                    content = new
                    {
                        offerAutopilotSettings = new { maxThroughput = -1 }
                    };
                }

                return new
                {
                    offerVersion = "V2",
                    offerType = "Invalid",
                    content = content,
                    resource = resourceLink,
                    offerResourceId = resourceRID,
                    id = offerId,
                    _rid = offerId,
                };
            }

            string extractAuthKey()
            {
                // Read the AccountKey using reflection because the property is protected.
                return client.GetType()
                    .GetProperty("AccountKey", BindingFlags.NonPublic | BindingFlags.Instance)
                    .GetValue(client).ToString();
            }
        }

        private string GenerateDatabaseLink(Database database) => $"dbs/{database.Id}/";

        private string GenerateContainerLink(Container container) => $"dbs/{container.Database.Id}/colls/{container.Id}/";

        private static ThroughputProperties GenerateThroughputProperties(bool autoScale, int? maxThroughput = null)
        {
            if (!autoScale)
            {
                if (!maxThroughput.HasValue || maxThroughput < 400)
                    maxThroughput = 400;
                return ThroughputProperties.CreateManualThroughput(maxThroughput.Value);
            }
            else
            {
                if (!maxThroughput.HasValue || maxThroughput < 4000)
                    maxThroughput = 4000;
                return ThroughputProperties.CreateAutoscaleThroughput(maxThroughput.Value);
            }
        }

        /// <summary>
        /// Generate the HTTP authorization header value needed to connect with Cosmos DB
        /// </summary>
        /// <param name="method">The Verb portion of the string is the HTTP verb, such as GET, POST, or PUT.</param>
        /// <param name="resourceType">The ResourceType portion of the string identifies the type of resource that the request is for, Eg. "dbs", "colls", "docs".</param>
        /// <param name="resourceLink">The ResourceLink portion of the string is the identity property of the resource that the request is directed at. ResourceLink must maintain its case for the ID of the resource. Example, for a container it looks like: "dbs/MyDatabase/colls/MyContainer".</param>
        /// <param name="date">The Date portion of the string is the UTC date and time the message was sent (in "HTTP-date" format as defined by RFC 7231 Date/Time Formats), for example, Tue, 01 Nov 1994 08:12:31 GMT. In C#, it can be obtained by using the "R" format specifier on the DateTime.UtcNow value. This same date(in same format) also needs to be passed as x-ms-date header in the request.</param>
        /// <param name="key">Cosmos DB key token (found in the Azure Portal)</param>
        /// <param name="keyType">denotes the type of token: master or resource.</param>
        /// <param name="tokenVersion">denotes the version of the token, currently 1.0.</param>
        /// <returns></returns>
        // This method borrowed from: https://learn.microsoft.com/en-us/rest/api/cosmos-db/access-control-on-cosmosdb-resources
        private string GenerateAuthToken(HttpMethod method, string resourceType, string resourceLink,
            string date, string key, string keyType = "master", string tokenVersion = "1.0")
        {
            var hmacSha256 = new System.Security.Cryptography.HMACSHA256 { Key = Convert.FromBase64String(key) };

            var verb = method?.Method ?? "";
            resourceType = resourceType ?? "";
            resourceLink = resourceLink?.ToLower() ?? ""; // Without ToLower(), we get an 'unauthorized' error.

            string payLoad = string.Format(System.Globalization.CultureInfo.InvariantCulture, "{0}\n{1}\n{2}\n{3}\n{4}\n",
                verb.ToLowerInvariant(),
                resourceType.ToLowerInvariant(),
                resourceLink,
                date.ToLowerInvariant(),
                ""
            );

            byte[] hashPayLoad = hmacSha256.ComputeHash(System.Text.Encoding.UTF8.GetBytes(payLoad));
            string signature = Convert.ToBase64String(hashPayLoad);

            return System.Web.HttpUtility.UrlEncode(String.Format(System.Globalization.CultureInfo.InvariantCulture, "type={0}&ver={1}&sig={2}",
                keyType,
                tokenVersion,
                signature));
        }
    }
}

用法:

var httpClient = new HttpClient();
var cosmosClient = new CosmosClient(EndpointUrl, PrimaryKey);
var database = cosmosClient.GetDatabase(DatabaseId);
var changer = new ThroughputChanger(httpClient);
await changer.SetThroughput(database, autoScale: true, maxThroughput: 8000);
httpClient.Dispose();
qybjjes1

qybjjes13#

注:不是此问题的答案,但与cosmos-db数据库处于自动缩放模式时以编程方式更改吞吐量有关。
使用微软Azure Cosmos;

public async Task<ThroughputResponse> UpdateThroughput(int targetLevel)
    {
        Container container = cosmosClient.GetContainer(databaseName, collectionName);
        ThroughputResponse throughput = await container.ReplaceThroughputAsync(ThroughputProperties.CreateAutoscaleThroughput(targetLevel));            
        return throughput;
    }

使用案例:我必须执行一小时的夜间工作,此工作具有高吞吐量需求(50 K RU/s),但我的正常负载不高于(10 K RU/s)。在此工作开始时,我将自动缩放增加到50 K,而在工作完成后,我将其减少10 K以最佳化成本。因为成本范围从(x的10%)到x,所以我想要维持成本最佳化的临界值。
如需进一步阅读:https://github.com/MicrosoftDocs/azure-docs/blob/master/articles/cosmos-db/how-to-provision-autoscale-throughput.md#change-the-autoscale-max-throughput-rus

相关问题