Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Enable unused_crate_dependencies Rust lint, remove unused dependencies #6804

Merged
merged 9 commits into from
Dec 11, 2024
Merged

Enable unused_crate_dependencies Rust lint, remove unused dependencies #6804

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
ac4c87a
Disallow unused dependencies in arrow-array
findepi Nov 29, 2024
7d5e4d1
Disallow unused dependencies in arrow-schema
findepi Nov 29, 2024
dec0dc9
check all modules on gh
findepi Nov 29, 2024
c73d80e
Remove arrow-csv's unused dependencies
findepi Nov 29, 2024
58e7172
Remove arrow-ord's unused dependencies
findepi Nov 29, 2024
b5a10df
Remove arrow-row's unused dependencies
findepi Nov 29, 2024
ec2c0aa
Remove arrow's unused dependencies
findepi Nov 29, 2024
c2b973c
Remove unused dependencies in arrow-flight and in tests
findepi Nov 29, 2024
a207da9
Unify arrow workflow step names
findepi Dec 11, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
202 changes: 152 additions & 50 deletions .github/workflows/arrow.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,39 +61,39 @@ jobs:
submodules: true
- name: Setup Rust toolchain
uses: ./.github/actions/setup-builder
- name: Test arrow-buffer with all features
- name: Test arrow-buffer
run: cargo test -p arrow-buffer --all-features
- name: Test arrow-data with all features
- name: Test arrow-data
run: cargo test -p arrow-data --all-features
- name: Test arrow-schema with all features
- name: Test arrow-schema
run: cargo test -p arrow-schema --all-features
- name: Test arrow-array with all features
- name: Test arrow-array
run: cargo test -p arrow-array --all-features
- name: Test arrow-select with all features
- name: Test arrow-select
run: cargo test -p arrow-select --all-features
- name: Test arrow-cast with all features
- name: Test arrow-cast
run: cargo test -p arrow-cast --all-features
- name: Test arrow-ipc with all features
- name: Test arrow-ipc
run: cargo test -p arrow-ipc --all-features
- name: Test arrow-csv with all features
- name: Test arrow-csv
run: cargo test -p arrow-csv --all-features
- name: Test arrow-json with all features
- name: Test arrow-json
run: cargo test -p arrow-json --all-features
- name: Test arrow-avro with all features
- name: Test arrow-avro
run: cargo test -p arrow-avro --all-features
- name: Test arrow-string with all features
- name: Test arrow-string
run: cargo test -p arrow-string --all-features
- name: Test arrow-ord with all features
- name: Test arrow-ord
run: cargo test -p arrow-ord --all-features
- name: Test arrow-arith with all features
- name: Test arrow-arith
run: cargo test -p arrow-arith --all-features
- name: Test arrow-row with all features
- name: Test arrow-row
run: cargo test -p arrow-row --all-features
- name: Test arrow-integration-test with all features
- name: Test arrow-integration-test
run: cargo test -p arrow-integration-test --all-features
- name: Test arrow with default features
run: cargo test -p arrow
- name: Test arrow with all features except pyarrow
- name: Test arrow except pyarrow
run: cargo test -p arrow --features=force_validate,prettyprint,ipc_compression,ffi,chrono-tz
- name: Run examples
run: |
Expand Down Expand Up @@ -163,37 +163,139 @@ jobs:
uses: ./.github/actions/setup-builder
- name: Setup Clippy
run: rustup component add clippy
- name: Clippy arrow-buffer with all features
run: cargo clippy -p arrow-buffer --all-targets --all-features -- -D warnings
- name: Clippy arrow-data with all features
run: cargo clippy -p arrow-data --all-targets --all-features -- -D warnings
- name: Clippy arrow-schema with all features
run: cargo clippy -p arrow-schema --all-targets --all-features -- -D warnings
- name: Clippy arrow-array with all features
run: cargo clippy -p arrow-array --all-targets --all-features -- -D warnings
- name: Clippy arrow-select with all features
run: cargo clippy -p arrow-select --all-targets --all-features -- -D warnings
- name: Clippy arrow-cast with all features
run: cargo clippy -p arrow-cast --all-targets --all-features -- -D warnings
- name: Clippy arrow-ipc with all features
run: cargo clippy -p arrow-ipc --all-targets --all-features -- -D warnings
- name: Clippy arrow-csv with all features
run: cargo clippy -p arrow-csv --all-targets --all-features -- -D warnings
- name: Clippy arrow-json with all features
run: cargo clippy -p arrow-json --all-targets --all-features -- -D warnings
- name: Clippy arrow-avro with all features
run: cargo clippy -p arrow-avro --all-targets --all-features -- -D warnings
- name: Clippy arrow-string with all features
run: cargo clippy -p arrow-string --all-targets --all-features -- -D warnings
- name: Clippy arrow-ord with all features
run: cargo clippy -p arrow-ord --all-targets --all-features -- -D warnings
- name: Clippy arrow-arith with all features
run: cargo clippy -p arrow-arith --all-targets --all-features -- -D warnings
- name: Clippy arrow-row with all features
run: cargo clippy -p arrow-row --all-targets --all-features -- -D warnings
- name: Clippy arrow with all features
run: cargo clippy -p arrow --all-features --all-targets -- -D warnings
- name: Clippy arrow-integration-test with all features
run: cargo clippy -p arrow-integration-test --all-targets --all-features -- -D warnings
- name: Clippy arrow-integration-testing with all features
run: cargo clippy -p arrow-integration-testing --all-targets --all-features -- -D warnings
- name: Clippy arrow-buffer
run: |
mod=arrow-buffer
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-data
run: |
mod=arrow-data
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-schema
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nit- the names are slightly inconsistent:

Suggested change
- name: Clippy arrow-schema
- name: Clippy arrow-schema with all features

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd go opposite direction, ie remove "with all features" as now this tests various combinations: "all features", "default features", "no features"

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

run: |
mod=arrow-schema
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-array
findepi marked this conversation as resolved.
Show resolved Hide resolved
run: |
mod=arrow-array
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-select
run: |
mod=arrow-select
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-cast
run: |
mod=arrow-cast
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-ipc
run: |
mod=arrow-ipc
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-csv
run: |
mod=arrow-csv
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-json
run: |
mod=arrow-json
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-avro
run: |
mod=arrow-avro
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-string
run: |
mod=arrow-string
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-ord
run: |
mod=arrow-ord
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-arith
run: |
mod=arrow-arith
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-row
run: |
mod=arrow-row
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow
run: |
mod=arrow
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-integration-test
run: |
mod=arrow-integration-test
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
- name: Clippy arrow-integration-testing
run: |
mod=arrow-integration-testing
cargo clippy -p "$mod" --all-targets --all-features -- -D warnings
# Dependency checks excluding tests & benches.
cargo clippy -p "$mod" -- -D unused_crate_dependencies
cargo clippy -p "$mod" --all-features -- -D unused_crate_dependencies
cargo clippy -p "$mod" --no-default-features -- -D unused_crate_dependencies
2 changes: 1 addition & 1 deletion arrow-csv/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,6 @@ bench = false

[dependencies]
arrow-array = { workspace = true }
arrow-buffer = { workspace = true }
arrow-cast = { workspace = true }
arrow-schema = { workspace = true }
chrono = { workspace = true }
Expand All @@ -45,6 +44,7 @@ lazy_static = { version = "1.4", default-features = false }
regex = { version = "1.7.0", default-features = false, features = ["std", "unicode", "perf"] }

[dev-dependencies]
arrow-buffer = { workspace = true }
tempfile = "3.3"
futures = "0.3"
tokio = { version = "1.27", default-features = false, features = ["io-util"] }
Expand Down
12 changes: 7 additions & 5 deletions arrow-flight/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ base64 = { version = "0.22", default-features = false, features = ["std"] }
bytes = { version = "1", default-features = false }
futures = { version = "0.3", default-features = false, features = ["alloc"] }
once_cell = { version = "1", optional = true }
paste = { version = "1.0" }
paste = { version = "1.0" , optional = true }
prost = { version = "0.13.1", default-features = false, features = ["prost-derive"] }
# For Timestamp type
prost-types = { version = "0.13.1", default-features = false }
tokio = { version = "1.0", default-features = false, features = ["macros", "rt", "rt-multi-thread"] }
tokio = { version = "1.0", default-features = false, features = ["macros", "rt", "rt-multi-thread"], optional = true }
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💯

tonic = { version = "0.12.3", default-features = false, features = ["transport", "codegen", "prost"] }

# CLI-related dependencies
Expand All @@ -61,11 +61,10 @@ all-features = true

[features]
default = []
flight-sql-experimental = ["dep:arrow-arith", "dep:arrow-data", "dep:arrow-ord", "dep:arrow-row", "dep:arrow-select", "dep:arrow-string", "dep:once_cell"]
flight-sql-experimental = ["dep:arrow-arith", "dep:arrow-data", "dep:arrow-ord", "dep:arrow-row", "dep:arrow-select", "dep:arrow-string", "dep:once_cell", "dep:paste"]
tls = ["tonic/tls"]

# Enable CLI tools
cli = ["dep:anyhow", "arrow-array/chrono-tz", "arrow-cast/prettyprint", "dep:clap", "dep:tracing-log", "dep:tracing-subscriber", "tonic/tls-webpki-roots"]
cli = ["arrow-array/chrono-tz", "arrow-cast/prettyprint", "tonic/tls-webpki-roots", "dep:anyhow", "dep:clap", "dep:tracing-log", "dep:tracing-subscriber"]

[dev-dependencies]
arrow-cast = { workspace = true, features = ["prettyprint"] }
Expand All @@ -75,6 +74,9 @@ http-body = "1.0.0"
hyper-util = "0.1"
pin-project-lite = "0.2"
tempfile = "3.3"
tracing-log = { version = "0.2" }
tracing-subscriber = { version = "0.3.1", default-features = false, features = ["ansi", "env-filter", "fmt"] }
tokio = { version = "1.0", default-features = false, features = ["macros", "rt", "rt-multi-thread"] }
tokio-stream = { version = "0.1", features = ["net"] }
tower = { version = "0.5.0", features = ["util"] }
uuid = { version = "1.10.0", features = ["v4"] }
Expand Down
2 changes: 2 additions & 0 deletions arrow-flight/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,8 @@
//! [Flight SQL]: https://arrow.apache.org/docs/format/FlightSql.html
#![allow(rustdoc::invalid_html_tags)]
#![warn(missing_docs)]
// The unused_crate_dependencies lint does not work well for crates defining additional examples/bin targets
#![allow(unused_crate_dependencies)]

use arrow_ipc::{convert, writer, writer::EncodedData, writer::IpcWriteOptions};
use arrow_schema::{ArrowError, Schema};
Expand Down
5 changes: 2 additions & 3 deletions arrow-integration-testing/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,18 +36,17 @@ logging = ["tracing-subscriber"]
[dependencies]
arrow = { path = "../arrow", default-features = false, features = ["test_utils", "ipc", "ipc_compression", "json", "ffi"] }
arrow-flight = { path = "../arrow-flight", default-features = false }
arrow-buffer = { path = "../arrow-buffer", default-features = false }
arrow-integration-test = { path = "../arrow-integration-test", default-features = false }
async-trait = { version = "0.1.41", default-features = false }
clap = { version = "4", default-features = false, features = ["std", "derive", "help", "error-context", "usage"] }
futures = { version = "0.3", default-features = false }
prost = { version = "0.13", default-features = false }
serde = { version = "1.0", default-features = false, features = ["rc", "derive"] }
serde_json = { version = "1.0", default-features = false, features = ["std"] }
tokio = { version = "1.0", default-features = false }
tokio = { version = "1.0", default-features = false, features = [ "rt-multi-thread"] }
tonic = { version = "0.12", default-features = false }
tracing-subscriber = { version = "0.3.1", default-features = false, features = ["fmt"], optional = true }
flate2 = { version = "1", default-features = false, features = ["rust_backend"] }

[dev-dependencies]
arrow-buffer = { path = "../arrow-buffer", default-features = false }
tempfile = { version = "3", default-features = false }
3 changes: 3 additions & 0 deletions arrow-integration-testing/src/bin/arrow-file-to-stream.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
// specific language governing permissions and limitations
// under the License.

// The unused_crate_dependencies lint does not work well for crates defining additional examples/bin targets
#![allow(unused_crate_dependencies)]

use arrow::error::Result;
use arrow::ipc::reader::FileReader;
use arrow::ipc::writer::StreamWriter;
Expand Down
3 changes: 3 additions & 0 deletions arrow-integration-testing/src/bin/arrow-json-integration-test.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
// specific language governing permissions and limitations
// under the License.

// The unused_crate_dependencies lint does not work well for crates defining additional examples/bin targets
#![allow(unused_crate_dependencies)]

use arrow::error::{ArrowError, Result};
use arrow::ipc::reader::FileReader;
use arrow::ipc::writer::FileWriter;
Expand Down
3 changes: 3 additions & 0 deletions arrow-integration-testing/src/bin/arrow-stream-to-file.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
// specific language governing permissions and limitations
// under the License.

// The unused_crate_dependencies lint does not work well for crates defining additional examples/bin targets
#![allow(unused_crate_dependencies)]

use std::io;

use arrow::error::Result;
Expand Down
3 changes: 3 additions & 0 deletions arrow-integration-testing/src/bin/flight-test-integration-client.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
// specific language governing permissions and limitations
// under the License.

// The unused_crate_dependencies lint does not work well for crates defining additional examples/bin targets
#![allow(unused_crate_dependencies)]

use arrow_integration_testing::flight_client_scenarios;
use clap::Parser;
type Error = Box<dyn std::error::Error + Send + Sync + 'static>;
Expand Down
3 changes: 3 additions & 0 deletions arrow-integration-testing/src/bin/flight-test-integration-server.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,9 @@
// specific language governing permissions and limitations
// under the License.

// The unused_crate_dependencies lint does not work well for crates defining additional examples/bin targets
#![allow(unused_crate_dependencies)]

use arrow_integration_testing::flight_server_scenarios;
use clap::Parser;

Expand Down
2 changes: 2 additions & 0 deletions arrow-integration-testing/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,8 @@

//! Common code used in the integration test binaries

// The unused_crate_dependencies lint does not work well for crates defining additional examples/bin targets
#![allow(unused_crate_dependencies)]
#![warn(missing_docs)]
use serde_json::Value;

Expand Down
2 changes: 1 addition & 1 deletion arrow-ord/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ arrow-buffer = { workspace = true }
arrow-data = { workspace = true }
arrow-schema = { workspace = true }
arrow-select = { workspace = true }
half = { version = "2.1", default-features = false, features = ["num-traits"] }

[dev-dependencies]
half = { version = "2.1", default-features = false, features = ["num-traits"] }
rand = { version = "0.8", default-features = false, features = ["std", "std_rng"] }
6 changes: 0 additions & 6 deletions arrow-row/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,12 +33,6 @@ name = "arrow_row"
path = "src/lib.rs"
bench = false

[target.'cfg(target_arch = "wasm32")'.dependencies]
ahash = { version = "0.8", default-features = false, features = ["compile-time-rng"] }

[target.'cfg(not(target_arch = "wasm32"))'.dependencies]
ahash = { version = "0.8", default-features = false, features = ["runtime-rng"] }

[dependencies]
arrow-array = { workspace = true }
arrow-buffer = { workspace = true }
Expand Down
Loading
Loading