linux 即使设置了do_fetch[nostamp] =“1”,bitbake也不会提取文件

ao218c7q  于 2022-11-22  发布在  Linux
关注(0)|答案(2)|浏览(145)

bounty将在2天后过期。回答此问题可获得+50声望奖励。Jeanot Zubler希望吸引更多人关注此问题。

我正在尝试在我的yocto映像中安装微处理器的编译固件。此固件在启动时被发送到微处理器。此固件的编译器只在windows上运行,因此不可能在构建期间克隆git库并编译。我从GitLab CI CD管道的工件中获得此固件文件。
为了调试的目的,我想下载这个固件文件从主分支,每次我建立的形象。
我以前使用过一个自定义的do_fetch任务,并设置了do_fetch[nostamp] = "1"。这是有效的,但当curl将404错误放入文件时,我遇到了麻烦。我现在尝试切换到wgetSRC_URI,但即使nostamp仍然设置,它也会生成一个.done文件。它现在再也不会下载该文件了。
下面是我使用的配方文件:

S = "${WORKDIR}/"

PRIVATE_TOKEN = "xxxxxxxxxxx" # redacted
PROJECT_ID = "224"

VERSION = "2.00.01"
TAG = "master"
DOWNLOAD_FILE_PATH = "Production/firmware.bin"
INSTALL_NAME = "my-firmware-file.bin"
VERSION_FILE = "version.txt"

URL = "https://git.mycompany.com/api/v4/projects/${PROJECT_ID}/jobs/artifacts/${TAG}/raw/${DOWNLOAD_FILE_PATH}?job=publish_executable"

FETCHCMD_wget = "/usr/bin/env wget --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}""
BB_STRICT_CHECKSUM = "0"
SRC_URI = "${URL};downloadfilename=${INSTALL_NAME}"

do_fetch[nostamp] = "1"

do_patch(){
    echo ${VERSION} > ${VERSION_FILE}
}

do_install(){
    install -d ${D}${libdir}/test-dir/${PN}
    # Install binary from artifacts
    install -m 644 ${S}${INSTALL_NAME} ${D}${libdir}/test-dir/${PN}/${INSTALL_NAME}
    # Instal version file created from tag
    install -m 644 ${S}${VERSION_FILE} ${D}${libdir}/test-dir/${PN}/${VERSION_FILE}
}

FILES_${PN} += "${libdir}/test-dir/${PN}"

运行配方(bitbake <PN>bitbake <PN> -c fetch -f)后,我可以运行ls -la build/download | grep my-firmware-file,它显示两个条目:

-rw-rw-r--  1 root root    613924 Nov 11 11:50 my-firmware-file.bin
-rw-rw-r--  1 root root       463 Nov 14 14:26 my-firmware-file.bin.done

正如你所看到的,它生成了一个.done文件,这个文件比上次下载的实际文件要新。即使改变了TAG变量,也不会下载新文件。但是由于某种原因,它总是把正确的版本放入version.txt文件中。
我错过了什么?我可以在每次构建之前运行一个cleanall,但这似乎不是一个永久的解决方案。此外,最终目标是版本化配方并使用包版本从正确的发布版本中拉取,但目前它是从master中拉取的,因此这没有太大意义。

Update:我检查了日志文件,似乎wget不是问题所在,因为该命令从未执行过。第一次清理后,do_fetch的日志如下所示:

DEBUG: Executing python function extend_recipe_sysroot
NOTE: Direct dependencies are []
NOTE: Installed into sysroot: []
NOTE: Skipping as already exists in sysroot: []
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing python function do_fetch
DEBUG: Executing python function base_do_fetch
DEBUG: Trying PREMIRRORS
DEBUG: Trying Upstream
DEBUG: Fetching https://git.mycompany.com/api/v4/projects/224/jobs/artifacts/master/raw/Production/firmware.bin?job=publish_executable;downloadfilename=my-firmware-file.bin using command '/usr/bin/env wget -r --header "PRIVATE-TOKEN: xxxxxxxxxxxxxx" -O /home/.../build/downloads/my-firmware-file.bin.tmp -P /home/.../build/downloads 'https://git.mycompany.com/api/v4/projects/224/jobs/artifacts/master/raw/Production/firmware.bin?job=publish_executable''
DEBUG: Fetcher accessed the network with the command /usr/bin/env wget -r --header "PRIVATE-TOKEN: xxxxxxxxxxxxxx" -O /home/.../build/downloads/my-firmware-file.bin.tmp -P /home/.../build/downloads 'https://git.mycompany.com/api/v4/projects/224/jobs/artifacts/master/raw/Production/firmware.bin?job=publish_executable'
DEBUG: Running export PSEUDO_DISABLED=1; export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1005/bus"; export PATH="..."; export HOME="/home/..."; /usr/bin/env wget -r --header "PRIVATE-TOKEN: xxxxxxxxxxxxxx" -O /home/.../build/downloads/my-firmware-file.bin.tmp -P /home/.../build/downloads 'https://git.mycompany.com/api/v4/projects/224/jobs/artifacts/master/raw/Production/firmware.bin?job=publish_executable' --progress=dot -v
WARNING: combining -O with -r or -p will mean that all downloaded content
will be placed in the single file you specified.

--2022-11-18 07:01:11--  https://git.mycompany.com/api/v4/projects/224/jobs/artifacts/master/raw/Production/firmware.bin?job=publish_executable
Resolving git.mycompany.com (git.mycompany.com)... <IP>
Connecting to git.mycompany.com (git.mycompany.com)|<IP>|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 613924 (600K) [application/octet-stream]
Saving to: ‘/home/.../build/downloads/my-firmware-file.bin.tmp’

2022-11-18 07:01:11 (70.4 MB/s) - ‘/home/.../build/downloads/my-firmware-file.bin.tmp’ saved [613924/613924]

FINISHED --2022-11-18 07:01:11--
Total wall clock time: 0.5s
Downloaded: 1 files, 600K in 0.008s (70.4 MB/s)
WARNING: Missing checksum for '/home/.../build/downloads/my-firmware-file.bin', consider adding at least one to the recipe:
SRC_URI[sha256sum] = "83aa3c373228b48eea58964b4ffec7ad42226014351d18bed12cb9b5eb3d261e"
DEBUG: Python function base_do_fetch finished
DEBUG: Python function do_fetch finished

当我再次运行do_fetch时(即使在更改TAG变量时,SRC_URI也会更改),日志看起来如下所示:

DEBUG: Executing python function extend_recipe_sysroot
NOTE: Direct dependencies are []
NOTE: Installed into sysroot: []
NOTE: Skipping as already exists in sysroot: []
DEBUG: Python function extend_recipe_sysroot finished
DEBUG: Executing python function do_fetch
DEBUG: Executing python function base_do_fetch
WARNING: Missing checksum for '/home/.../build/downloads/my-firmware-file.bin', consider adding at least one to the recipe:
SRC_URI[sha256sum] = "83aa3c373228b48eea58964b4ffec7ad42226014351d18bed12cb9b5eb3d261e"
WARNING: Missing checksum for '/home/.../build/downloads/my-firmware-file.bin', consider adding at least one to the recipe:
SRC_URI[sha256sum] = "83aa3c373228b48eea58964b4ffec7ad42226014351d18bed12cb9b5eb3d261e"
DEBUG: Python function base_do_fetch finished
DEBUG: Python function do_fetch finished

注意第二个日志中重复的缺少校验和警告。我也尝试只删除./downloads/my-firmware-file.bin.done文件,但它仍然没有重新下载,日志与第二个相同。

bihw5rsg

bihw5rsg1#

尝试添加-r标志以强制wget重新下载,即使文件存在:

FETCHCMD_wget = "/usr/bin/env wget -r --header "PRIVATE-TOKEN: ${PRIVATE_TOKEN}""
htrmnn0y

htrmnn0y2#

我已经尝试了很多方法,现在我觉得这是bitbake中的一个bug。目前我实现了这个解决方案,它似乎工作正常。它在执行do_fetch之前删除下载的文件以及.done文件。

# Deletes the old, cached version of the firmware file
do_fetch_prepend(){
    import os

    fw_file = f"{d.getVar('DL_DIR')}/{d.getVar('INSTALL_NAME')}"
    done_file = f"{fw_file}.done"

    if os.path.isfile(fw_file):
        os.remove(fw_file)
    if os.path.isfile(done_file):
        os.remove(done_file)
}

我希望很快就能正确地版本我的食谱,这样我就不必依赖这样的黑客。

相关问题