Skip to content

Add more options to df_engine docker build, slim down features for some packages #1540

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
albertlockett merged 9 commits into from
Dec 8, 2025
+24 −9
Merged

Add more options to df_engine docker build, slim down features for some packages #1540

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
8e43370
c
JakeDern Dec 6, 2025
20d07eb
Add more options
JakeDern Dec 6, 2025
b57ac8d
misc fixes
JakeDern Dec 6, 2025
217fe54
update dockerfile
JakeDern Dec 6, 2025
fc4b967
revert yaml change
JakeDern Dec 6, 2025
2875e26
Fix up features
JakeDern Dec 7, 2025
bb931ca
Move fs feature from workspace to otap crate
JakeDern Dec 7, 2025
de50e47
Move azure features to otap crate from workspace
JakeDern Dec 7, 2025
c6171b6
Default RUSTFLAGS in dockerfile
JakeDern Dec 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions rust/otap-dataflow/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,7 +85,7 @@ miette = { version="7.6.0", features = ["fancy"] }
mimalloc-rust = "0.2.1"
nix = { version = "0.30.1", features = ["process", "signal"] }
num_enum = "0.7"
object_store = "0.12.3"
object_store = {version = "0.12.3", default-features = false, features = ["fs"]}
once_cell = "1.20.2"
opentelemetry = "0.31.0"
opentelemetry-proto = { version = "0.31", default-features = false, features = ["gen-tonic-messages", "logs"]} #TODO - use it from submodule instead of crate(?)
Expand Down Expand Up @@ -150,10 +150,10 @@ zip = "=4.2.0"
byte-unit = "5.2.0"

# Azure Monnitor Exporter
azure_identity = "0.30.0"
azure_core = {version = "0.30.1", default-features = false, features = ["reqwest"] }
azure_identity = {version = "0.30.0", default-features = false, features = [] }
flate2 = "1.1.5"
reqwest = "0.12.24"
azure_core = "0.30.1"
reqwest = { version = "0.12.24", default-features = false, features = ["rustls-tls-native-roots"] }
time = "0.3.44"
wiremock = "0.6.5"

Expand Down
15 changes: 15 additions & 0 deletions rust/otap-dataflow/Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,12 +22,27 @@ COPY --from=otel-arrow /proto/opentelemetry-proto /build/proto/opentelemetry-pro

COPY . /build/rust/dataflow/.

# RUSTFLAGS can be used to pass argument to rustc e.g.
# --build-arg RUSTFLAGS="-C target-feature=-gfni"
ARG RUSTFLAGS
ENV RUSTFLAGS=${RUSTFLAGS}

# These are used to enable features and are passed to carg's --features argument
# e.g. --build-arg FEATURES="azure,quiver-persistence"
ARG FEATURES
ENV FEATURES=${FEATURES}

# Build dataflow engine
RUN ./cross-arch-build.sh

# The runtime image
FROM docker.io/alpine:3.22@sha256:4b7ce07002c69e8f3d704a9c5d6fd3053be500b7f1c69fc0d80990c2ad8dd412
LABEL maintainer="The OpenTelemetry Authors"

# Passing --build-arg BUILD=1 adds additional debugging features to the image.
ARG DEBUG="0"
RUN if [ "$DEBUG" = "1" ]; then apk add --no-cache gdb; fi

RUN addgroup dataflow \
&& adduser \
--ingroup dataflow \
Expand Down
2 changes: 1 addition & 1 deletion rust/otap-dataflow/crates/otap/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,7 +70,7 @@ geneva-uploader = { version = "0.3.0", optional = true }
# Azure Monitor Exporter dependencies
azure_identity = { workspace = true, optional = true }
flate2 = { workspace = true, optional = true }
reqwest = { workspace = true, optional = true }
reqwest = { workspace = true, optional = true, default-features = false, features = ["rustls-tls"] }
Copy link
Member

@lalitb lalitb Dec 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Couple of comments here -

  • No need to repeat [default-features = false] in the crate if the workspace already sets it.
  • It’s better to let the crate manage its own reqwest features instead of forcing them at the workspace level.
  • Pick one of rustls-tls or rustls-tls-native-roots. They both pull in their own CA bundles, so using both at once doesn’t help.

Copy link
Contributor Author

@JakeDern JakeDern Dec 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the review!

No need to repeat [default-features = false] in the crate if the workspace already sets it.

Good to know! Removed it in the otap crate.

It’s better to let the crate manage its own reqwest features instead of forcing them at the workspace level.

Makes perfect sense, removed at the workspace level.

Pick one of rustls-tls or rustls-tls-native-roots. They both pull in their own CA bundles, so using both at once doesn’t help.

Good catch, I meant to pick rustls-tls not rustls-tls-native-roots and forgot to update both places. Removed the workspace level feature anyways now, so we just have rustls-tls in the otap crate.

azure_core = { workspace = true, optional = true }

# OpenTelemetry proto with tonic feature enabled
Expand Down
4 changes: 2 additions & 2 deletions rust/otap-dataflow/cross-arch-build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,5 +16,5 @@ else
fi

rustup target add "${RUST_TARGET}"
cargo build --release --target="${RUST_TARGET}"
cp "target/${RUST_TARGET}/release/df_engine" .
cargo build --release --features "$FEATURES" --target="${RUST_TARGET}"
cp "target/${RUST_TARGET}/release/df_engine" .
Loading