Most of it is actual code - not assets. For some types of apps (see: games) assets do take up a significant portion of total size, but for most everyday apps bloat by code over-inclusion is likely a bigger problem than asset-bloat.
I wonder if it's possible to get major open-source libs to move towards more fine-grained build targets and internal dependency management, so that devs don't pull in a gigantic binary when they're only using a small slice of the functionality.
Depending on how they are made, how the linker is configured, and the phase of the moon, static linking can help by just taking the required functions.
Also I think we, as programmers, have taken the "don't reinvent the wheel" principle too far. The idea is to use 3rd party code to (1) save time writing, (2) reduce the risk of bugs, and (3) lower the maintainance burden.
But this makes sense only if the benefits outweigh the corresponding costs of integration, which also (1) takes time, (2) might be done wrong (especially because you don't understand the part you added), and (3) creates a maintainance burden. Of these, only #1 is solved when your development environment makes adding new libs quick and easy.
> I wonder if it's possible to get major open-source libs to move towards more fine-grained build targets and internal dependency management, so that devs don't pull in a gigantic binary when they're only using a small slice of the functionality.
Fwiw, this is already a trend in JavaScript land. Libraries are moving towards many small packages so you can import only the parts you need either manually or using a tool like Webpack to throw away what isn't used.
I don't think that's the right solution to the problem. Nobody wants libleftpad.so, and as someone who works on a distribution the very concept is horrific (making distribution packages for every three-line package does not make anyone happy).
What I think GP was arguing for is that you have libstring.so which you can strip down to just having leftpad or w/e with Kconfig or similar configurations (preferably at link time not build time -- otherwise you still have the same problem).
JavaScript people do not create libleftpad.so, they create libleftpad.o (metaphorically speaking), which will not clutter up your distribution, it will just clutter up some apps. I'd rather see 50 little files like libleftpad.o than 5 gigantic libraries.
But I agree with what you're saying in the second paragraph, which actually sounds just like what my GP meant by using "Webpack to throw away what isn't used". Having unused parts of larger libraries cut out would be a much cooler solution than just telling everyone to eschew kitchen sinks in favor of many tiny, bespoke items. Especially since finding the right small libraries is much harder than finding a kitchen sink that just works and has a sizable community behind it
> which will not clutter up your distribution, it will just clutter up some apps.
Not necessarily. Distributions have to build everything, and in openSUSE we have an integrated build system[1] which has a lot of very useful features (rebuild when dependencies are updated, and automated QA runs before releases). Those features require you to have all of the dependencies of a project in an RPM package. Even if you don't end up shipping that package, the package is used for tracking security issues and backports and so on. You can't pull anything from the internet during a build, you have to depend on your BuildRequires.
Now take any average JS application that has over 800 dependencies. Ruby was already bad enough with ~80 dependencies, but adding another order of magnitude is just not maintainable. One of my colleagues gave a talk at LCA about this problem[2].
I had no idea about this - thank you for bringing it up! But we seem to be discussing different ideas. I was trying to argue about apps, specifically iOS apps, like the original article was about. iOS apps aren't distributed on openSUSE, so if some devs make some kind of libleftpad.o for Objective C developers making iOS apps, I'm saying that's fine, and I doubt it would clutter up your distro because Objective C isn't exactly known for it's cross-platform adoption :)
If we're going to talk about linux packages, aren't they often written in languages with amazing optimization skills? If I write some C99 code that uses a gigantic library but I only use like 2% of it, my compiler will cut out the 98% my code doesn't use, so libleftpad sounds like an awful idea. That's one reason why packages on linux distros aren't too big.
But I'm talking about iOS apps where, as others have pointed out, the available optimizations suck[0], and as such, I think that having libleftpad.o included in everyone's iOS apps isn't a big deal (note that iOS doesn't really have a nice way to create libleftpad.so anyway AFAICT because all code for your app is supposed to be sandboxed so no one else can mess with or use it). I agree that it would be really cool to just cut out the 98% of $GIGANTIC_LIBRARY that isn't used at compile time, but since Objective C doesn't seem to have that now, I think small things would be a really nice way to give users more space on their phones without removing features.
Ah okay, I was talking about things like electron applications or desktop/server javascript projects. If you want to use $latest_js_web_framework and package it in a distribution, you're going to be in a world of pain (the same applies for Rust because their package management was based on NPM).
> I'd rather see 50 little files like libleftpad.o than 5 gigantic libraries.
And then you'll have to use software written in 50 different styles, with 50 different type and error conventions, with pieces that couple together or not at random.
> And then you'll have to use software written in 50 different styles, with 50 different type and error conventions, with pieces that couple together or not at random.
All of which matters not one hoot to folks _using_ the software.
Right, and most people could have much more free space on their phones (or, more apps). I'd much rather write some glue code than ask my users to dedicate 275 MB to install my app, not even counting cached data or anything else. There's a reason many of my friends and I have deleted apps like LinkedIn - it's not that we don't like them, it's that the the apps were too big for the little value they gave us, especially when considering the web versions give us almost all the same capabilities, and only cost a couple KB of persistent storage (cookies + history + maybe some localStorage or whatever).
Maybe you're not familiar with how it works. The idea is that you do, let's say, "import lodash" then your packaging system says "A-ha! But you're only using these 5 methods!" and only packages those.
Behind the scenes, lodash is built out of many modules, but you as a developer don't need to think about that.
Because tooling isn't perfect yet, you have the option of helping it out manually and being explicit about importing submodules. We were able to reduce our compressed JavaScript size by some 60% using a combination of automatic and manual tricks like that at my day job.
Right, but from what I've seen (though I'm not a JS developer, so I might be wrong) is that most projects are not designed like lodash. They are small, single-purpose libraries that would be much better served as part of a module of a larger library. That was the point I was trying to make with the whole "libstring" and "libleftpad" point.
I've seen people try to package JS projects inside a distribution. >800 nested dependencies is simply not sane nor sustainable. The fact that they're small is even less excusable because it means that there was an active decision to not consolidate them into a single dependency that is properly maintained.
If you want "just pull in the things I use" you can already get that by having a static library, and if you were going to be shipping a private copy of the .so file in your app then surely you would be better off with a static library -- you don't get any of the benefits of the DLL (sharing with other apps, can update the DLL without shipping a new copy of the whole app), so why pay the cost of it?
(If you link against the static library then the linker pulls in only the .o files that the app used, so assuming that the library was written in a sensibly modular way you pay only the binary size cost for more-or-less what you use. The linker can probably discard at a finer granularity than whole-object-file these days, but only-the-.o-you-use has been there for decades.)
Probably only widely used static C library with fine grained dependency tracking is glibc. The source code structure required for that is probably one of the major reasons for people to call glibc's source code unreadable and/or unmaintable.
Young minds don't like it when someone questions the wisdom of how they reinvented the wheel, even when the question comes from confusion instead of malice.
http://blog.timac.org/?p=1707
Most of it is actual code - not assets. For some types of apps (see: games) assets do take up a significant portion of total size, but for most everyday apps bloat by code over-inclusion is likely a bigger problem than asset-bloat.
I wonder if it's possible to get major open-source libs to move towards more fine-grained build targets and internal dependency management, so that devs don't pull in a gigantic binary when they're only using a small slice of the functionality.