Remove duplicated dependencies

[YuhanLiin]

Calling cargo tree -d --workspace at the root of the repo reveals all of Linfa's duplicated dependencies. These are dependencies that have the same name but different versions, leading to bloat. We want to eliminate as many duplicated dependencies as possible by ensuring that multiple instances of the same crate in the dependency tree have the same version. We should also prioritize the non-dev dependencies since they affect the end users.

[bytesnake]

I investigated a bit and came up with these different dependency mismatches:

• cfg-if caused by reqwest, opened an issue here Different HTTP backend davidMcneil/mnist#7
• generic-array caused by ndarray-npy which we can bump once we moved to ndarray = 0.15
• getrandom, rand, rand-chacha, rand_core caused by using rand = 0.8 in linfa, we could downgrade to 0.7 here or wait for the ndarray bump
• hashbrown caused by an old version of hnsw. This also requires a bump in ndarray-stats/indexmap/hashbrown dependency to be on the latest version, opened a PR Bump hashbrown to 0.11.2 indexmap-rs/indexmap#193
• itertools caused by inconsistency in criterion's dependency graph and old version of ndarray-stats (can be updated once we have ndarray = 0.15)
• num-bigint, num-complex caused by ndarray-npy which can only be updated once we bump ndarray itself
• pin-project-lite caused by inconcistency in reqwest dependency graph, also depends on Different HTTP backend davidMcneil/mnist#7
• socket2 caused by using reqwest and curl
• time caused by reqwest as well, which uses an older version compared to argmin

the problems are therefore originating from three sources:

• we need the ndarray bump to be consistent to the ecosystem
• reqwest
• indexmap uses an older version of hashbrown

[YuhanLiin]

reqwest is not a big deal because it only affects dev-dependencies. Does downgrading rand in linfa break anything?

[bytesnake]

reqwest is not a big deal because it only affects dev-dependencies. Does downgrading rand in linfa break anything?

yes the ndarray bump is the main issue here. Downgrading rand requires reverting some function signatures. The dependency tree of rand is not really worth the work

[YuhanLiin]

After bumping ndarray to v0.15, approx is still duplicated since argmin v0.4.6 uses approx v0.5 but ndarray uses approx v0.4.

[YuhanLiin]

Update: Following are the main sources behind duplicated dependencies:

• approx inconsistency between ndarray and argmin
• mnist using reqwest, which has a lot of dupes (Different HTTP backend davidMcneil/mnist#7)

[oojo12]

Is using the approx-0_5 feature in ndarray a potential solution to the approx duplication?

[oojo12]

Otherwise, the approx bump is coming in v0.16.0 of ndarray

[YuhanLiin]

The approx-0_5 feature should solve the issue. Double-check to make sure.