curl 从Nexus 3.37.1下载整个存储库

cbwuti44  于 2022-11-13  发布在  其他
关注(0)|答案(3)|浏览(271)

我想知道如何从Nexus 3.37下载整个文件夹或存储库。
curl -X GET -u userid:password "https://nexus.com/abc/def/ghi/" -O
我能够使用上面提到的命令下载一个目录中的单个文件,有人知道我如何下载整个文件夹吗?
谢谢你!

vcirk6k6

vcirk6k61#

上面的脚本确实帮助我解决了这个问题。我稍微修改了一下,并把它贴在下面。这个脚本并不完美,你可能需要编辑它。我知道一个问题是这个脚本启动的第一个下载放在了错误的目录中。

sourceServer=
sourceRepo=
sourceUser=
sourcePassword=
logfile=$sourceRepo-backup.log
outputFile=$sourceRepo-artifacts.txt

# ======== GET DOWNLOAD URLs =========
url=$sourceServer"/service/rest/v1/assets?repository="$sourceRepo
contToken="initial"
while [ ! -z "$contToken" ]; do
    if [ "$contToken" != "initial" ]; then
        url=$sourceServer"/service/rest/v1/assets?continuationToken="$contToken"&repository="$sourceRepo
    fi
    echo Processing repository token: $contToken | tee -a $logfile
    response=`curl -ksSL -u "$sourceUser:$sourcePassword" -X GET --header 'Accept: application/json' "$url"`
    readarray -t artifacts < <( jq  '[.items[].downloadUrl]' <<< "$response" )
    printf "%s\n" "${artifacts[@]}" > artifacts.temp
    sed 's/\"//g' artifacts.temp > artifacts1.temp
    sed 's/,//g' artifacts1.temp > artifacts.temp
    sed 's/[][]//g' artifacts.temp > artifacts1.temp
    cat artifacts1.temp >> $outputFile
#for filter in "${filters[@]}"; do
     #   cat artifacts.temp | grep "$filter" >> $outputFile
    #done
    #cat maven-public-artifacts.txt
    contToken=( $(echo $response | sed -n 's|.*"continuationToken" : "\([^"]*\)".*|\1|p') )
done

# ======== DOWNLOAD EVERYTHING =========
    echo Downloading artifacts...
    urls=($(cat $outputFile)) > /dev/null 2>&1
    for url in "${urls[@]}"; do
        path=${url#http://*:*/*/*/}
        dir=$sourceRepo"/"${path%/*}
        mkdir -p  $dir
        cd $dir
        pwd
        curl -vks -u "$sourceUser:$sourcePassword" -D response.header -X GET "$url" -O  >> /dev/null 2>&1
        responseCode=`cat response.header | sed -n '1p' | cut -d' ' -f2`
        if [ "$responseCode" == "200" ]; then
            echo Successfully downloaded artifact: $url
        else
            echo ERROR: Failed to download artifact: $url  with error code: $responseCode
        fi
        rm response.header > /dev/null 2>&1
        cd $curFolder
    done
5kgi1eie

5kgi1eie2#

您可以使用curl通过执行以下命令来检索所有工件下载URL的列表:

filters=("\\.jar$" "\\.pom$" "\\.zip$")
sourceServer=https://myserver.com:1234
sourceRepo=myrepo
sourceUser=username
sourcePassword=password
logfile=$sourceRepo-backup.log
outputFile=$sourceRepo-artifacts.txt
# ======== GET DOWNLOAD URLs =========
url=$sourceServer"/service/rest/v1/assets?repository="$sourceRepo
contToken="initial"
while [ ! -z "$contToken" ]; do
    if [ "$contToken" != "initial" ]; then
        url=$sourceServer"/service/rest/v1/assets?continuationToken="$contToken"&repository="$sourceRepo
    fi
    echo Processing repository token: $contToken | tee -a $logfile
    response=`curl -ksSL -u "$sourceUser:$sourcePassword" -X GET --header 'Accept: application/json' "$url"`
    artifacts=( $(echo $response | sed -n 's|.*"downloadUrl" : "\([^"]*\)".*|\1|p') )
    printf "%s\n" "${artifacts[@]}" > artifacts.temp
    for filter in "${filters[@]}"; do
        cat artifacts.temp | grep "$filter" >> $outputFile
    done
    contToken=( $(echo $response | sed -n 's|.*"continuationToken" : "\([^"]*\)".*|\1|p') )
done

此时,您已经从sha1和md5文件、元数据等中过滤出了所有工件,因此您可以遍历输出文件的各行,并使用curl下载它们。

# ======== DOWNLOAD EVERYTHING =========
    echo Downloading artifacts...
    urls=($(cat $outputFile)) > /dev/null 2>&1
    for url in "${urls[@]}"; do
        path=${url#http://*:*/*/*/}
        dir=$sourceRepo"/"${path%/*}
        mkdir -p $dir
        cd $dir
        curl -vks -u "$sourceUser:$sourcePassword" -D response.header -X GET "$url" -O  >> /dev/null 2>&1
        responseCode=`cat response.header | sed -n '1p' | cut -d' ' -f2`
        if [ "$responseCode" == "200" ]; then
            echo Successfully downloaded artifact: $url  | tee -a $logfile 2>&1
        else
            echo ERROR: Failed to download artifact: $url  with error code: $responseCode  | tee -a $logfile 2>&1
        fi
        rm response.header > /dev/null 2>&1
        cd $curFolder
    done

这是下载所有内容并保留文件夹结构,以便稍后您可以使用Maven上传所有内容(假设使用Maven 2 repo)。它还将报告下载是否成功。请尝试先检查凭据,否则您将最终进行数千次错误尝试,并最终锁定您的帐户。

unhi4e5o

unhi4e5o3#

我对RESTAPI中没有“下载文件夹中的所有文件”这一事实有点感兴趣
以下是我的改动:

  • 收回简单筛选器(例如子文件夹)
  • 保留URL中的空格并转换为%20以供下载
  • 将curFolder设置为pwd的结果
  • 执行连续运行时删除输出文件(如果存在)
  • 从http更改为https
sourceServer=
sourceRepo=
sourceFolder=
sourceUser=
sourcePassword=
logfile=$sourceRepo-backup.log
outputFile=$sourceRepo-artifacts.txt
[ -e $outputFile ] && rm $outputFile

# ======== GET DOWNLOAD URLs =========
url=$sourceServer"/service/rest/v1/assets?repository="$sourceRepo
contToken="initial"
while [ ! -z "$contToken" ]; do
    if [ "$contToken" != "initial" ]; then
        url=$sourceServer"/service/rest/v1/assets?continuationToken="$contToken"&repository="$sourceRepo
    fi
    echo Processing repository token: $contToken | tee -a $logfile
    response=`curl -ksSL -u "$sourceUser:$sourcePassword" -X GET --header 'Accept: application/json' "$url"`
    readarray -t artifacts < <( jq  '[.items[].downloadUrl]' <<< "$response" )
    printf "%s\n" "${artifacts[@]}" > artifacts.temp
    sed 's/\"//g' artifacts.temp > artifacts1.temp
    sed 's/,//g' artifacts1.temp > artifacts2.temp
    sed 's/[][]//g' artifacts2.temp > artifacts3.temp
    cat artifacts3.temp | grep "$sourceFolder" >> $outputFile
    contToken=( $(echo $response | sed -n 's|.*"continuationToken" : "\([^"]*\)".*|\1|p') )
done

# ======== DOWNLOAD EVERYTHING =========
    echo Downloading artifacts...
    IFS=$'\n' read -d '' -r -a urls < $outputFile
    for url in "${urls[@]}"; do
        url="$(echo -e "${url}" | sed -e 's/^[[:space:]]*//')"
        path=${url#https://*/*/*/}
        dir="\""$sourceRepo"/"${path%/*}"\""
        curFolder=$(pwd)
        mkdir -p $dir
        cd $dir
        url="$(echo -e "${url}" | sed -e 's/\s/%20/g')"
        curl -vks -u "$sourceUser:$sourcePassword" -D response.header -X GET "$url" -O  >> /dev/null 2>&1
        responseCode=`cat response.header | sed -n '1p' | cut -d' ' -f2`
        if [ "$responseCode" == "200" ]; then
            echo Successfully downloaded artifact: $url
        else
            echo ERROR: Failed to download artifact: $url  with error code: $responseCode
        fi
        rm response.header > /dev/null 2>&1
        cd $curFolder
    done

相关问题