Docker构建的缓存Rust依赖项

vlurs2pr  于 2022-11-12  发布在  Docker
关注(0)|答案(9)|浏览(303)

我在Rust + Actix-web中有Hello World Web项目。我有几个问题。首先是代码的每一次更改都会导致重新编译整个项目,包括下载和编译每个板条箱。我想像正常开发一样工作-这意味着缓存编译的板条箱,只重新编译我的代码库。第二个问题是它不会暴露我的应用程序。它无法通过Web浏览器访问
停靠文件:

FROM rust

WORKDIR /var/www/app

COPY . .

EXPOSE 8080

RUN cargo run

docker-compose.yml:

version: "3"
services:
  app:
    container_name: hello-world
    build: .
    ports:
      - '8080:8080'
    volumes:
      - .:/var/www/app
      - registry:/root/.cargo/registry

volumes:
  registry:
    driver: local

main.rs:

extern crate actix_web;

use actix_web::{web, App, HttpServer, Responder};

fn index() -> impl Responder {
    "Hello world"
}

fn main() -> std::io::Result<()> {
    HttpServer::new(|| App::new().service(web::resource("/").to(index)))
        .bind("0.0.0.0:8080")?
        .run()
}

Cargo.toml:

[package]
name = "hello-world"
version = "0.1.0"
authors = []
edition = "2018"

[dependencies]
actix-web = "1.0"
wkftcu5l

wkftcu5l1#

看起来您并不是唯一一个通过Docker构建过程来缓存rust依赖项的奋进。下面是一篇很棒的文章,可以帮助您:https://blog.mgattozzi.dev/caching-rust-docker-builds/
其要点是,首先需要一个dummy.rs和Cargo.toml,然后构建它来缓存依赖项,然后在以后复制应用程序源代码,以便不会在每次构建时该高速缓存无效。

停靠文件

FROM rust
WORKDIR /var/www/app
COPY dummy.rs .
COPY Cargo.toml .
RUN sed -i 's#src/main.rs#dummy.rs#' Cargo.toml
RUN cargo build --release
RUN sed -i 's#dummy.rs#src/main.rs#' Cargo.toml
COPY . .
RUN cargo build --release
CMD ["target/release/app"]

CMD应用程序名称“app”基于您在Cargo.toml中为二进制文件指定的内容。

虚拟.rs

fn main() {}

载货汽车

[package]
name = "app"
version = "0.1.0"
authors = ["..."]
[[bin]]
name = "app"
path = "src/main.rs"

[dependencies]
actix-web = "1.0.0"

源代码/main.rs

extern crate actix_web;

use actix_web::{web, App, HttpServer, Responder};

fn index() -> impl Responder {
    "Hello world"
}

fn main() -> std::io::Result<()> {
    HttpServer::new(|| App::new().service(web::resource("/").to(index)))
        .bind("0.0.0.0:8080")?
        .run()
}
9cbw7uwe

9cbw7uwe2#

使用Docker Buildkit(仍处于实验阶段),您最终可以在docker build步骤中正确地缓存构建文件夹:

停靠文件:


# syntax=docker/dockerfile:experimental

from rust
ENV HOME=/home/root
WORKDIR $HOME/app
[...]
RUN --mount=type=cache,target=/usr/local/cargo/registry \
    --mount=type=cache,target=/home/root/app/target \
    cargo build --release

然后运行:
DOCKER_BUILDKIT=1 docker build . --progress=plain
后续的Docker构建将重用缓存中的货物和目标文件夹,从而大大加快构建速度。
要清除Docker缓存装载,请执行以下操作:docker builder prune --filter type=exec.cachemount
如果您没有看到正确的缓存:如果您没有看到正确的缓存,请确保确认您的货物/注册表和目标文件夹在Docker映像中的位置。
最小工作示例:https://github.com/benmarten/sccache-docker-test/tree/no-sccache

ru9i0ody

ru9i0ody3#

您可以使用cargo-chef通过多阶段构建来利用Docker图层缓存。

FROM rust as planner
WORKDIR app

# We only pay the installation cost once,

# it will be cached from the second build onwards

RUN cargo install cargo-chef 
COPY . .
RUN cargo chef prepare  --recipe-path recipe.json

FROM rust as cacher
WORKDIR app
RUN cargo install cargo-chef
COPY --from=planner /app/recipe.json recipe.json
RUN cargo chef cook --release --recipe-path recipe.json

FROM rust as builder
WORKDIR app
COPY . .

# Copy over the cached dependencies

COPY --from=cacher /app/target target
RUN cargo build --release --bin app

FROM rust as runtime
WORKDIR app
COPY --from=builder /app/target/release/app /usr/local/bin
ENTRYPOINT ["./usr/local/bin/app"]

它不需要Buildkit,可以用于简单的项目和工作区。您可以在release announcement中找到更多的细节。

q5lcpyga

q5lcpyga4#

根据@ckaserer的回答,您可以在构建应用程序之前使用RUN echo "fn main() {}" > ./src/main.rs构建依赖项。
首先,只复制Cargo.tomlCargo.lock文件,并构建虚拟的main.rs文件:

FROM rust as rust-builder
WORKDIR /usr/src/app

# Copy Cargo files

COPY ./Cargo.toml .
COPY ./Cargo.lock .

# Create fake main.rs file in src and build

RUN mkdir ./src && echo 'fn main() { println!("Dummy!"); }' > ./src/main.rs
RUN cargo build --release

然后,您可以复制真实的的src目录并再次运行build:


# Copy source files over

RUN rm -rf ./src
COPY ./src ./src

# The last modified attribute of main.rs needs to be updated manually,

# otherwise cargo won't rebuild it.

RUN touch -a -m ./src/main.rs

RUN cargo build --release

然后,我们可以将文件复制到一个精简版的debain中。

FROM rust as rust-builder
WORKDIR /usr/src/app
COPY ./Cargo.toml .
COPY ./Cargo.lock .
RUN mkdir ./src && echo 'fn main() { println!("Dummy!"); }' > ./src/main.rs
RUN cargo build --release
RUN rm -rf ./src
COPY ./src ./src
RUN touch -a -m ./src/main.rs
RUN cargo build --release

FROM debian:buster-slim
COPY --from=rust-builder /usr/src/app/target/release/app /usr/local/bin/
WORKDIR /usr/local/bin
CMD ["app"]
qybjjes1

qybjjes15#

虽然electronix384128的答案很好,但我想通过为.cargo/git添加缓存来扩展它,这是使用git的任何依赖项所需要的,并添加一个多级Docker示例。
使用rust-musl-builder和Docker Buildkit功能,这是Docker Desktop 2.4中的默认功能。在其他版本中,您可能仍需要通过以下方式启用该功能:DOCKER_BUILDKIT=1 docker build .
rusl-musl-builder的工作目录是/home/rust/src
已尝试在--mount上设置uid/gid,但由于目标中的权限问题而无法编译rust。


# syntax=docker/dockerfile:1.2

FROM ekidd/rust-musl-builder:stable AS builder

COPY . .
RUN --mount=type=cache,target=/home/rust/.cargo/git \
    --mount=type=cache,target=/home/rust/.cargo/registry \
    --mount=type=cache,sharing=private,target=/home/rust/src/target \
    sudo chown -R rust: target /home/rust/.cargo && \
    cargo build --release && \
    # Copy executable out of the cache so it is available in the final image.
    cp target/x86_64-unknown-linux-musl/release/my-executable ./my-executable

FROM alpine
COPY --from=builder /home/rust/src/my-executable .
USER 1000
CMD ["./my-executable"]
ffscu2ro

ffscu2ro6#

我认为问题是你的volumes定义没有进行绑定挂载。我相信你当前的配置是将HOST ./registry/复制到DOCKER /root/.cargo/registry/,写入DOCKER /root/.cargo/registry/,并在容器关闭时丢弃内容。
相反,您需要在卷上指定bind类型:

version: "3"
services:
  app:
    container_name: hello-world
    build: .
    environment:
      - CARGO_HOME=/var/www/
    ports:
      - '8080:8080'
    volumes:
      - .:/var/www/app
      - type: bind
        source: ./registry
        target: /root/.cargo/registry

但是,请记住,也会创建一个/root/.cargo/.package-cache文件,但不会保存在这里。相反,您可以将source更改为./.cargo,并将目标更改为/root/.cargo
为了我自己(大多数是cli)rust项目,我喜欢使用一个drop-in replacement I've written for cargo,我已经确认它在构建之间缓存了包,大大减少了构建时间。它可以被复制到/usr/local/bin以供全局使用,或者在单个项目中作为./cargo build运行。但是请记住,这个特定的脚本假设应用程序位于容器中的/usr/src/app,所以可能需要根据您的使用情况进行调整。

kulphzqa

kulphzqa7#

这就是我所做的,它与构建脚本兼容。这是一个多阶段的构建,所以它会产生一个小的映像,但是会在第一个映像中缓存构建的依赖项。

FROM rust:1.43 AS builder

RUN apt-get update
RUN cd /tmp && USER=root cargo new --bin <projectname>
WORKDIR /tmp/<projectname>

# cache rust dependencies in docker layer

COPY Cargo.toml Cargo.lock ./
RUN touch build.rs && echo "fn main() {println!(\"cargo:rerun-if-changed=\\\"/tmp/<projectname>/build.rs\\\"\");}" >> build.rs
RUN cargo build --release

# build the real stuff and disable cache via the ADD

ADD "https://www.random.org/cgi-bin/randbyte?nbytes=10&format=h" skipcache
COPY ./build.rs ./build.rs

# force the build.rs script to run by modifying it

RUN echo " " >> build.rs
COPY ./src ./src
RUN cargo build --release

FROM rust:1.43
WORKDIR /bin
COPY --from=builder /tmp/<projectname>/target/release/server /bin/<project binary>
RUN chmod +x ./<project binary>
CMD ./<project binary>
ktca8awb

ktca8awb8#

我遇到了与您完全相同的问题,并尝试了多种方法通过缓存依赖项来缩短构建时间。

1. @卡塞勒的回答

它完成了工作,并且简单易懂地解释了它为什么工作,这是一个很好的解决方案。然而,这归结为偏好,但是如果你不以这种方式缓存依赖项,你可以遵循#2。

2.使用cargo-chef

@LukeMathWalker,创建者自己,完成了使用cargo-chef的必要步骤,但这里有一个来自github页面的 * 稍微调整的例子。

停靠文件

FROM lukemathwalker/cargo-chef:latest-rust-1.60.0 AS chef
WORKDIR /app

FROM chef as planner
COPY . .
RUN cargo chef prepare --recipe-path recipe.json

FROM chef as builder
COPY --from=planner /app/recipe.json recipe.json

# Build the dependencies (and add to docker's caching layer)

# This caches the dependency files similar to how @ckaserer's solution

# does, but is handled solely through the `cargo-chef` library.

RUN cargo chef cook --release --recipe-path recipe.json

# Build the application

COPY . .
RUN cargo build --release --bin emailer

FROM debian:buster-slim AS runtime
WORKDIR /app
COPY --from=builder /app/target/release/<Name of Rust Application> /usr/local/bin
ENTRYPOINT ["/usr/local/bin/<Name of Rust Application>"]

您应该注意到,通过上述更改,构建时间显著缩短!
边注,据我所知,this blog entry,虽然不是在dockerized构建,拥有最好的信息编译rust应用程序更快地在您的本地机器上。你可能会发现它很有帮助,所以我建议你看看它,如果你有兴趣。

kuarbcqp

kuarbcqp9#

这是对@ckaserer的回答的改进,其中包含了注解和一些个人经验。它不需要您在repo中创建一个虚拟文件,或者在构建时编辑Cargo.toml文件。

RUN echo 'fn main() { panic!("Dummy Image Called!")}' > ./src/main.rs
COPY ["Cargo.toml", "Cargo.lock",  "./"]
RUN cargo build --release
COPY src src

# need to break the cargo cache

RUN touch ./src/main.rs
RUN cargo build --release

相关问题