使用Powershell绕过API调用中的记录限制

lrpiutwd 于 2023-05-17 发布在 Shell

关注(0)|答案(1)|浏览(89)

我使用PowerShell从供应商站点获取位置列表，使用API调用：

# Set the initial page and limit values
$page = 1
$limit = 1000

# Initialize an array to hold all the data
$allData = @()

# Set the headers with the token key
$headers = @{
    'Content-Type' = 'application/json'
    'Authorization' = 'Bearer SomeTokenKey'
    'Accept'= 'application/json'
}

# Loop until all data is retrieved
while ($true) {
    # Set the API endpoint URL with the current page and limit values
    $uri = "https://mysite.Someapisite.com/api/locations?page=$page&limit=$limit"

    # Make the API call and retrieve the data
    $responseData = Invoke-RestMethod -Uri $uri -Method GET -Headers $headers -Body $body

    # Add the retrieved data to the allData array
    $allData += $responseData.data

    # Check if all data has been retrieved
    if ($responseData.meta.pagination.current_page -eq $responseData.meta.pagination.total_pages) {
        break
    }

    # Set the page and limit values for the next API call
    $page = $responseData.meta.pagination.current_page + 1
    $limit = $responseData.meta.pagination.per_page
}

# Export all the data to a CSV file
$allData | Export-Csv -Path 'c:\extracts\locations.csv' -NoTypeInformation -Force

供应商限制每次呼叫最多提取1000条记录。供应商API文档说明支持分页：
该站点通过限制和页面参数利用基于偏移/限制的分页，其中偏移是基于所提供的参数自动计算的。

Link: <https://mySite.someAPIsite.com/api/location?page=3&limit=10>; rel="next", 
      <https://mySite.someAPIsite.com/api/location?page=7&limit=10>; rel="last"

我的目标是提取源代码中的所有记录，但上面的脚本只返回1000条记录。
(A)难道我没有正确地实现这一点吗？
(B)使用分页是否会绕过记录限制？
有什么想法吗

powershell

来源：https://stackoverflow.com/questions/76038126/bypassing-record-limit-in-api-call-using-powershell

1条答案

按热度按时间

6ojccjat1#

我对代码进行了修改，并使其满足了我的需要：

# API endpoint
$apiUrl = 'https://someSite.siteUrl.com/api/locations'

# API credentials
$accessToken = 'InsertTokenHere'
      
# Set the limit and initial page
$limit = 1000
$page = 1

# Initialize the locations array
$locations = @()

do {
    # Construct the API request URL with pagination parameters
    $url = '{0}?limit={1}&page={2}&expand=locationHours' -f $apiUrl, $limit, $page

    try {
        # Invoke the REST API to fetch locations
        $response = Invoke-RestMethod -Uri $url -Headers @{ Authorization = "Bearer $accessToken" }

        # Add the fetched locations to the locations array
        $locations += $response.data

        # Increment the page number for the next API request
        $page++
    }
    catch {
        # Output the error message and break out of the loop
        Write-Host "API request failed with error: $_"
        break
    }
    
} until ($response.data.Count -ne $limit)

if ($locations) {
    # Output the fetched locations to a CSV file
    $locations | Export-Csv -Path "locations.csv" -NoTypeInformation

    # Output success message with record count
    $recordCount = $locations.Count
    Write-Host "Successfully fetched and saved $recordCount records to locations.csv."
} else {
    Write-Host "No Records were fetched."
}

赞(0）回复(0）举报 2023-05-17

我来回答

使用Powershell绕过API调用中的记录限制

1条答案

相关问题

热门标签

最新问答