July 2, 2024
Sylvain Kerkour writes:
Rust adoption is stagnating not because it’s missing some feature pushed by programming language theory enthusiasts, but because of a lack of focus on solving the practical problems that developers are facing every day.
… no company outside of AWS is making SDKs for Rust … it has no official HTTP library.
As a result of Rust’s lack of official packages, even its core infrastructure components need to import hundreds of third-party crates.
cargo imports over 400 crates.
crates.io has over 500 transitive dependencies.
…the offical libsignal (from the Signal messaging app) uses 500 third-party packages.
… what is really inside these packages. It has been found last month that among the 999 most popular packages on crates.io, the content of around 20% of these doesn’t even match the content of their Git repository.
…how I would do it (there may be better ways):
A stdx (for std eXtended) under the rust-lang organization containing the most-needed packages. … to make it secure: all packages in stdx can only import packages from std or stdx. No third-party imports. No supply-chain risks.
[stdx packages to include, among others]:
gzip, hex, http, json, net, rand
Read Rust has a HUGE supply chain security problem
Submitter’s note:
I find the author’s writing style immature, sensationalist, and tiresome, but they raise a number of what appear to be solid points, some of which are highlighted above.
Rust adoption is stagnating
Is it? I would like to see some evidence for that.
because of [the small standard library and potentially supply chain security issues]
Yeah I can guarantee that is not a significant reason for people to avoid Rust. If it was people wouldn’t use NPM, where the problem is even worse.
I do think it would be good to putt some more stuff in the standard library makes sense, or even just add some kind of official sanction of de facto standard library crates like
regex… But this author is an idiot.Rust adoption is stagnating
Is it? I would like to see some evidence for that.
When comparing crates.io statistics
Year Crates Yearly Crates Increase Downloads Yearly Downloads Increase 2018 21,162 - 688,268,999 - 2019 29,757 8,595 1,457,578,834 769,309,835 2020 41,539 11,782 3,079,874,235 1,622,295,401 2021 64,658 23,119 8,235,327,111 5,155,452,876 2022 86,776 22,118 17,546,769,164 9,311,442,053 2023 119,145 32,369 35,556,469,191 18,009,700,027 2024 149,970 30,825 72,083,950,414 36,527,481,223 By downloads, 2023-2024 has been Rust’s best year so far.
@FizzyOrange @ericjmorey The Regex crate is already part of the rust-lang organisation on GitHub. I don’t know what you are asking for.
I am asking for some kind of official badge or something on crates.io. Currently it just looks like any other crate. Dart has a feature like this I believe.
And
regexwas just an example. There are other crates that should be officially sanctioned but aren’t.@[email protected] It’s shown in the “owners”.
Regarding the crates that should be “officially sanctionned”, what would this mean besides a fancy badge?
It would mean a fancy badge, ideally being listed in the official docs, and probably some kind of promise about maintaining it.
It’s shown in the “owners”.
This is just way too subtle IMO.
@FizzyOrange the Rust project is not an organisation you have a contract with. The only guarantee of maintenance you get are that of the MIT and Apache licenses.
I clearly didn’t mean a legal contract. Come on.
@FizzyOrange And a lot of crates you would want to see “blessed”, are already in the Rust cookbook: https://rust-lang-nursery.github.io/rust-cookbook/
I find the author’s writing style immature, sensationalist, and tiresome, but they raise a number of what appear to be solid points, some of which are highlighted above.
I tried reading the article and gave up because life is too short for me to read a tiresome article making points that aren’t even particularly that new.
Part of this is because the article’s author pushes a lot of sensationalist content to drive traffic to their Rust book(s). I remember similar articles several times over the last year, at least one of which was a thinly disguised ad for the Black Hat Rust book. That doesn’t mean the author is wrong, necessarily, but it does get annoying after a bit.
making points that aren’t even particularly that new.
(putting my Rust historian hat on)
Even the name stdx[1][2] is not original.
It was one of multiple attempts to officially or semi-officially present a curated a list of crates. Thankfully, all these attempts failed, as the larger community pushed against them, and more relevantly, as the swarm refused to circle around any of them.
This reminds of a little-known and long-forgotten demo tool named
cargo-esr[1][2]. But it’s not the tool, but the events it was supposedly created as a response to that is worth a historical mention, namely these blog posts[1][2], and the commotion that followed them[1][2][3][4].For those who were not around back then, there was an obscure crate named
mio, created by an obscure developer named Carl Lerche, that was like the libevent/libuv equivalent for Rust.miowas so obscure I actually knew it existed before Rust even hit v1.0. Carl continued to do more obscure things liketokio, whatever that is.So, the argument was that there was absolutely no way whatsoever that one could figure out they needed to depend on
miofor a good event loop interface. It was totally an insurmountable task!That was the circus, and “no clown left behind” was the mindset, that gave birth to all these std-extending attempts.
So, let’s fast forward a bit. NTPsec didn’t actually get (re)written in go, and ended up being a trimming, hardening, and improving job on the original C impl. The security improvements were a huge success! Just the odd vulnerability here and there. You know, stuff like NULL dereferences, buffer over-reads, out-of-bounds writes, the kind of semantic errors Rust famously doesn’t protect from 🙂
To be fair, I’m not aware of any big NTP implementations written in Rust popping up around that time either. But we do finally have the now-funded
ntpd-rseffort progressing nicely.And on the crates objective metrics front, kornel of lib.rs fame, started and continues to collect A LOT of them for his service. Although, he and lib.rs are self-admittedly NOT opinion-free.
DISCLAIMER: I didn’t even visit OP’s link.
So, the argument was that there was absolutely no way whatsoever that one could figure out they needed to depend on
miofor a good event loop interface. It was totally an insurmountable task!You still see this same mindset now with people making things like blessed.rs. It’s the same idea, just not wrapped into a library. I find it hilarious when it gets shared in discussions and some people go “oh wow so helpful!”, as if we all couldn’t have found
serdeandrandon crates.io without it.
Well… it is true that it doen’t have all these crates like Url included in the rust standard library, and hence it is not official. On the other hand Url was created by Mozilla to be used in Firefox, hence it is a quite competent crate that is very well maintained. And my guess is that the http crate may have the same kind of origins… but I’m not entirely sure about that.
And even Java that includes quite a lot, still didn’t get a good Http library until very recent, until then you had to rely on some obscure library created by the unknown organization Apache… so…
As a developer you always have to think about what libraries you use, and if you trust them… that goes for pretty much any language.
Developers should think about what libraries they trust, but it seems that most of the time they’ll choose whatever is most convenient for handling the immediate problems they’re working to solve.
Funnily enough the same is true for languages that have huge standard libraries. They put anything that is convenient to solve their immediate problem in there. That is how languages like Python end up with multiple of just about everything complicated in there.
Anytime anyone mentions integrating an HTTP client into Rust’s std, all it takes is one good Python anecdote to shut that discussion right down.
Having the standard library be stable and not try to add a bunch of support for changing standards is a long-term benefit to the language. Having “de-facto standard libs” with crates like
url,http, etc ends up being better because they can evolve independently from the standard library, at the pace their respective domains evolve.Although, I suppose an argument could be made that
urlis unlikely to really evolve anymore.
It is true that having many dependencies is supply chain attack. However, this is the result of combining the following:
- The UNIX mantra of “do one thing, and do it well”
- The more things a program does, the more intuitive and useful it is.
If you want to keep the 2, you’re gonna need a lot of dependencies. To significantly decrease the amount of dependencies you’re gonna need to drop one of those, there’s no other way around it.
If you wanna know what happens when you drop the UNIX mantra look at any discussion about systemd.
If you drop the second one, everybody would have to bring their own glue. Making computers only accessible to Linux gurus that master the “|” operator and study CLI program arguments in their spare time.
I don’t know why this article focuses on rust specifically. Every language has this problem. And cargo itself has many ways to mitigate this.
- Lockfiles & caches: prevents unwanted version updates.
- Custom registries: You don’t have to use crates.io, that’s just the default. Set up your own registry with only whitelisted crates.
- Use GitHub instead of a registry: cargo has great support for this. If you want your dependencies to always match their GitHub repos, just download from GitHub and compile your own.
Java and Python projects can be based almost entirely on the standard library because there’s so much in there (or packages with minimal/no third party dependencies).
C++ it’s not uncommon for the entire code base or the majority of the code base to be internal (and maybe make use of the standard library or a library under the boost umbrella).
The “every language” is largely a “languages that became popular in the NodeJS era” issue.
Java I don’t know enough of to say. I never use dependencies in java because I can’t be bothered to learn Gradle/maven/eclipse/whatever.
Python I completely disagree. First, I believe all (non-python) libraries are distributed as binaries, no local compilation at all. Which makes the issue of the content not matching GitHub even worse.
Secondly, python is used as a glue language. Most of the time, it’s just a way to interact with bindings to a C library. There’s no big project that uses only the standard library.
Lastly, in python, whatever you want to import doesn’t always match the pip install command. In your code you might say “import MyAwesome69”, but the command to install it is “pip install awesome lib”. Any malicious actor would just need to publish a python library called “myawesome69” and it would get many people trying to install “awesomelib”. You have to know the magic words to install each library. And projects rarely tell you how to install dependencies. Requirements.txt is a joke (if you want to automatically create it, it puts every single installed library on your machine/venv, not just the ones used in your project), but you’ll be grateful if the project you want to run provides one. Also, nobody distributes python programs as executables, which means everyone who wants to run it must know the magic words, not just the developers. Moreover, not all dependencies are available through pip. The install instructions might say “install awesomelib”, but when you “pip install awesomelib” you pull a malicious library instead of installing the actual awesomelib available via “sudo apt install awesomelib”
I don’t usually use libraries in my python scripts, but that’s because I use it as a scripting language, they rarely reach 300 loc. If you want to use tl make an actual program. You’re gonna pull a lot of dependencies.
C++, like java, I can’t speak of, because installing a library was such a pain in windows without Visual Studio that I was never able to. Might explain why they don’t use many libraries.
Of the suggested packages C++ has bytes by not needing it, rand and a horribly slow regex implentation, where its faster to start PHP to parse the regex than to use the build in one. Yeah, I’m going to pass.
I keep reading about the “supply chain.” I will just leave this here. https://www.softwaremaxims.com/blog/not-a-supplier
I’ve read that. Defining a supplier as someone with whom you have a direct business relationship with seems intentionally narrow in an unhelpful way that just further muddies the waters around the issue at hand. Making something generally available to others means that you’re supplying others with that thing. While it’s true that you may have no further obligations to those that receive your software, the person receiving the software needs to evaluate their risks around using and depending on that software regardless of the existence of a business relationship with the supplier. Hence supply chain risk evaluation is always necessary. That risk evaluation, or lack thereof, can result in a security problem. These problems can propagate widely within a software ecosystem. This is true with and without the existence of direct business relationships between suppliers and recipients of software.
The whole article can be summarized by saying if you want support services related to the software written by others, negotiate a support agreement related to that software. That has nothing to do with taking a wide or narrow interpretation of the word supplier.
Exactly. A “supply-chain attack” is a very real thing in software, and it doesn’t really matter whether you consider yourself a supplier, the fact remains that something a product relies on had a security vulnerability that resulted in the product getting pwned. Nobody should be claiming that the unpaid developer maintaining that library that resulted in the vulnerability is somehow at fault in any legal sense because the license specifically states there is no warranty etc, but it is useful to point to that library as having that vulnerability to let other organizations know where the problem originated so they can either fix or replace it.
I have a better idea that can solve everyone’s problems. Businessnes make their own libraries and keep it closed for themselves. The FOSS community will use it for FOSS stuff. I’m more of a libre person that.
