1<html><head><title>toybox roadmap</title> 2<!--#include file="header.html" --> 3<title>Toybox Roadmap</title> 4 5<h2>Goals and use cases</h2> 6 7<p>We have several potential use cases for a new set of command line 8utilities, and are using those to determine which commands to implement 9for Toybox's 1.0 release. (Most of these have their own section in the 10<a href=status.html>status page</a>.)</p> 11 12<p>The most interesting publicly available standards are POSIX-2008 13(also known as the Single 14Unix Specification version 4) and the Linux Standard Base (version 4.1). 15The main test harness is including toybox in <a href=https://landley.net/aboriginal/about.html>Aboriginal Linux</a> and if that can 16build itself using the result to <a href=https://github.com/landley/control-images>build Linux From Scratch</a> (version 6.8). 17We also aim to replace Android's Toolbox.</p> 18 19<p>At a secondary level we'd like to meet other use cases. We've analyzed 20the commands provided by similar projects (klibc, sash, sbase, embutils, 21nash, and beastiebox), along with various vendor configurations of busybox, 22and some end user requests.</p> 23 24<p>Finally, we'd like to provide a good replacement for the Bash shell, 25which was the first program Linux ever ran and remains the standard shell 26of Linux no matter what Ubuntu says. This doesn't mean including the full 27set of Bash 4.x functionality, but does involve {various,features} <(beyond) 28posix.</p> 29 30<p>See the <a href=status.html>status page</a> for the categorized command list 31and progress towards implementing it. There's also a 32<a href=todo.html>historical todo list</a> from the project's 2011 relaunch.</p> 33 34<ul> 35<li><a href=#susv4>POSIX-2008/SUSv4</a></li> 36<li><a href=#sigh>Linux "Standard" Base</a></li> 37<li><a href=#dev_env>Development Environment</a></li> 38<li><a href=#android>Android Toolbox</a></li> 39<li><a href=#aosp>Building AOSP</a></li> 40<li><a href=#tizen>Tizen Core</a></li> 41<li><a href=#buildroot>buildroot</a></li> 42<li>Miscelaneous: <a href=#klibc>klibc</a>, <a href=#glibc>glibc</a>, 43<a href=#sash>sash</a>, <a href=#sbase>sbase</a>, 44<a href=#uclinux>uclinux</a>...</li> 45<li><a href=#packages>Other Packages</a></li> 46</ul> 47 48<hr /> 49<a name="standards"> 50<h2>Use case: standards compliance.</h2> 51 52<h3><a name=susv4 /><a href="#susv4">POSIX-2008/SUSv4</a></h3> 53<p>The best standards describe reality rather than attempting to impose a 54new one. A good standard should document, not legislate. 55Standards which document existing reality tend to be approved by 56more than one standards body, such ANSI and ISO both approving <a href=https://landley.net/c99-draft.html>C99</a>. That's why 57the IEEE POSIX committee's 2008 standard, the Single Unix Specification version 584, and the Open Group Base Specification edition 7 are all the same standard 59from three sources, but most people just call it "posix" (portable operating 60system derived from unix). It's available <a href=https://pubs.opengroup.org/onlinepubs/9699919799>online in full</a>, and may be downloaded as a tarball... 61with a caveat.</p> 62 63<p>Although previous versions of Posix have their own 64stable URLs (where you can still find 65<a href=https://pubs.opengroup.org/onlinepubs/009695399/>SUSv3</a> and 66<a href=https://pubs.opengroup.org/onlinepubs/7990989775/>SUSv2</a>), 67the 2008 release of SUSv4 was replaced by a 2013 release also claiming 68to be SUSv4, then again by a 2018 release still at the same URL. Similarly, 69the other version numbers claim not to have changed, but instead adopted some 70sort of "Windows 95" naming scheme ("The Open Group Base Specifications Issue 717, 2018 edition"). Since a moving target isn't a standard, we've stuck 72with the 2008 version and ignored whatever changes they make 73until they stop this forced-upgrade-behind-your back nonsense. 74Luckily you can still find the original content 75<a href=https://pubs.opengroup.org/onlinepubs/9699919799.2008edition/>here</a>. 76(We haven't changed the URLs in each command to the longer version yet, 77but can if conflicts arise.)</p> 78 79<h3>Why not just use posix for everything?</h3> 80 81<p>Unfortunately posix describes an incomplete subset of reality, 82lacking any mention of commands such as init or mount required to 83actually boot a system. It describes logname but not login. It provides ipcrm 84and ipcs, but not ipcmk, so you can use System V IPC resources but not create 85them. And widely used real-world commands such as tar and cpio (the basis 86of initramfs and RPM) which were present in earlier 87versions of the standard have been removed, while obsolete commands like 88cksum, compress, sccs and uucp remain with no mention of modern counterparts 89like crc32/sha1sum, gzip/xz, svn/git or scp/rsync. Meanwhile the commands 90themselves are missing dozens of features and specify silly things like ebcdic 91support in dd or that wc should use %d (not %lld) for byte counts. So 92we have to extensively filter posix to get a useful set of recommendations.</p> 93 94<p>Starting with the 95<a href="http://pubs.opengroup.org/onlinepubs/9699919799.2008edition/idx/utilities.html">full "utilities" list</a>, 96we first remove generally obsolete 97commands (compress ed ex pr uncompress uccp uustat uux), commands for the 98pre-CVS "SCCS" source control system (admin delta get prs rmdel sact sccs unget 99val what), fortran support (asa fort77), and batch processing support (batch 100qalter qdel qhold qmove qmsg qrerun qrls qselect qsig qstat qsub).</p> 101 102<p>Some commands are for a compiler toolchain (ar c99 cflow ctags cxref gencat 103iconv lex m4 make nm strings strip tsort yacc), which is outside of toybox's 104mandate and should be supplied externally. (Again, some of these may be 105revisited later, but not for toybox 1.0.)</p> 106 107<p>Some commands are part of a command shell, and can't be implemented as 108separate executables (alias bg cd command fc fg getopts hash jobs kill read 109type ulimit umask unalias wait). These may be revisited as part of a built-in 110toybox shell, but are not exported into $PATH via symlinks. (If you fork a 111child process and have it "cd" then exit, you've accomplished nothing.) 112Again, what posix provides is incomplete: a shell also needs exit, if, while, 113for, case, export, set, unset, trap, exec... (And for bash compatibility 114function, source...)</p> 115 116<p>A few other commands are judgement calls, providing command-line 117internationalization support (iconv locale localedef), System V inter-process 118communication (ipcrm ipcs), and cross-tty communication from the minicomputer 119days (talk mesg write). The "pax" utility <a href=https://slashdot.org/story/06/09/04/1335226/debian-kicks-jrg-schilling>failed</a> to replace tar, 120"mailx" is 121a command line email client, and "lp" submits files for printing to... what 122exactly? (cups?) The standard defines crontab but not crond. What is 123pathchk supposed to be portable _to_? (Linux accepts 255 byte path components 124with any char except NUL or / and no max length on the total path, and 125EXPLICITLY doesn't care if it's an invalid utf8 sequence.)</p> 126 127<p>Removing all of that leaves the following commands, which toybox should 128implement:</p> 129 130<blockquote><b> 131<span id=posix> 132at awk basename bc cal cat chgrp chmod chown cksum cmp comm cp 133csplit cut date dd df diff dirname du echo env expand expr false file find 134fold fuser getconf grep head id join kill link ln logger logname ls man 135mkdir mkfifo more mv newgrp nice nl nohup od paste patch printf ps 136pwd renice rm rmdir sed sh sleep sort split stty tabs tail tee test time 137touch tput tr true tty uname unexpand uniq unlink uudecode uuencode vi wc 138who xargs zcat 139</span> 140</b></blockquote> 141 142<h3><a name=sigh /><a href="#sigh">Linux Standard Base</a></h3> 143 144<p>One attempt to supplement POSIX towards an actual usable system was the 145Linux Standard Base. Unfortunately, the quality of this "standard" is 146fairly low.</p> 147 148<p>POSIX allowed its standards process to be compromised 149by leaving things out, thus allowing IBM mainframes and Windows NT to drive 150a truck through the holes and declare themselves compilant. But it means what 151they DID standardize tends to be respected (if sometimes obsolete).</p> 152 153<p>The Linux Standard Base's failure mode is different, they respond to 154pressure by including anything their members pay them enough to promote, 155such as allowing Red Hat to push 156RPM into the standard even though all sorts of distros (Debian, Slackware, Arch, 157Gentoo) don't use it and never will. This means anything in the LSB is 158at best a suggestion: arbitrary portions of this standard are widely 159ignored.</p> 160 161<p>The community perception seems to be that the Linux Standard Base is 162the best standard money can buy, I.E. the Linux Foundation is supported by 163financial donations from large companies and the LSB 164<a href=https://www.softwarefreedom.org/blog/2016/apr/11/lf/>represents the interests 165of those donors</a> more than technical merit. (The Linux Foundation, which 166maintains the LSB, isn't a 501c3. It's a 501c6, the 167same kind of legal entity as the Tobacco Institute and Microsoft's old 168"Don't Copy That Floppy" program.) Debian officially 169<a href=http://lwn.net/Articles/658809>washed its hands of LSB</a> when 5.0 170came out in 2015, and no longer even pretends to support it (which may affect 171Debian derivatives like Ubuntu and Knoppix). Toybox hasn't moved to 5.0 for 172similar reasons.</p> 173 174<p>That said, Posix by itself isn't enough, and this is the next most 175comprehensive standards effort for Linux so far, so we salvage what we can.</p> 176 177<p>The LSB specifies a <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/cmdbehav.html>list of command line 178utilities</a>:</p> 179 180<blockquote><b> 181ar at awk batch bc chfn chsh col cpio crontab df dmesg du echo egrep 182fgrep file fuser gettext grep groupadd groupdel groupmod groups 183gunzip gzip hostname install install_initd ipcrm ipcs killall lpr ls 184lsb_release m4 md5sum mknod mktemp more mount msgfmt newgrp od passwd 185patch pidof remove_initd renice sed sendmail seq sh shutdown su sync 186tar umount useradd userdel usermod xargs zcat 187</b></blockquote> 188 189<p>Where posix specifies one of those commands, LSB's deltas tend to be 190accomodations for broken tool versions which aren't up to date with the 191standard yet. (See <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/more.html>more</a> and <a href=http://refspecs.linuxfoundation.org/LSB_4.1.0/LSB-Core-generic/LSB-Core-generic/xargs.html>xargs</a> 192for examples.)</p> 193 194<p>Since we've already committed to using our own judgement to skip bits of 195POSIX, and LSB's "judgement" in this regard is purely bug workarounds to declare 196various legacy tool implementations "compliant", this means we're mostly 197interested in the set of LSB tools that aren't mentioned in posix.</p> 198 199<p>Of these, gettext and msgfmt are internationalization, install_initd and 200remove_initd weren't present in Ubuntu 10.04, lpr is out of scope, 201lsb_release just reports information in /etc/os-release, and sendmail's 202turned into a pile of cryptographic verification and DNS shenanigans due 203to spammers.</p> 204 205<p>This leaves:</p> 206 207<blockquote><b> 208<span id=lsb> 209chfn chsh dmesg egrep fgrep groupadd groupdel groupmod groups 210gunzip gzip hostname install killall md5sum 211mknod mktemp mount passwd pidof seq shutdown 212su sync tar umount useradd userdel usermod zcat 213</span> 214</b></blockquote> 215 216<h3><a name=rfc /><a href="#rfc">IETF RFCs</a></h3> 217 218<p>Discussion of standards wouldn't be complete without the Internet 219Engineering Task Force's "<a href=https://www.rfc-editor.org/in-notes/rfc-index.txt>Request For Comments</a>" collection.</p> 220 221<p>These are more about protocols than commands. The noise level is 222extremely high: there's thousands of RFCs, many describing a proposed idea 223that never took off, and less than 1% of the resulting documents are 224currently relevant to toybox. And the documents are numbered based on the 225order they were received, with no real attempt at coherently indexing 226the result. As with man pages they can be <a href=https://www.ietf.org/rfc/rfc0610.txt>long and complicated</a> or 227<a href=https://www.ietf.org/rfc/rfc1951.txt>terse and impenetrable</a>, 228have developed a certain amount of <a href=https://www.ietf.org/rfc/rfc8179.txt>bureaucracy</a> over the years, and often the easiest way to understand what 229they <a href=https://www.ietf.org/rfc/rfc4330.txt>document</a> is to find an <a href=https://www.ietf.org/rfc/rfc1769.txt>earlier version</a> to read first.</p> 230 231<p>That said, RFC documents can be useful (especially for networking protocols) 232and the three URL templates the recommended starting files 233for new commands (toys/example/skeleton.c or toys/example/hello.c depending on how much 234plumbing you want to start with) provide point to are posix, lsb, and 235rfc pages.</p> 236 237<hr /> 238<a name="dev_env"> 239<h2><a href="#dev_env">Use case: provide a self-hosting development environment</a></h2> 240 241<p>The following commands were enough to build the <a href=http://landley.net/aboriginal/about.html>Aboriginal Linux</a> development 242environment, boot it to a shell prompt, and build <a href=http://www.linuxfromscratch.org/lfs/view/6.8/>Linux From Scratch 6.8</a> under it.</p> 243 244<blockquote><b> 245<span id=development> 246bzcat cat cp dirname echo env patch rmdir sha1sum sleep sort sync 247true uname wc which yes zcat 248awk basename chmod chown cmp cut date dd diff 249egrep expr fdisk find grep gzip head hostname id install ln ls 250mkdir mktemp mv od readlink rm sed sh tail tar touch tr uniq 251wget whoami xargs chgrp comm gunzip less logname split 252tee test time bunzip2 chgrp chroot comm cpio dmesg 253dnsdomainname ftpd ftpget ftpput gunzip ifconfig init less 254logname losetup mdev mount mountpoint nc pgrep pkill 255pwd route split stat switch_root tac umount vi 256resize2fs tune2fs fsck.ext2 genext2fs mke2fs xzcat 257</span> 258</b></blockquote> 259 260<p>This use case includes running init scripts and other shell scripts, running 261configure, make, and install in each package, and providing basic command line 262facilities such as a text editor. (It does not include a compiler toolchain or 263C library, those are outside the scope of the toybox project, although mkroot 264has a <a href=https://landley.net/code/qcc>potentialy follow-up project</a>. 265For now we use distro toolchains, 266<a href=https://github.com/richfelker/musl-cross-make>musl-cross-make</a>, 267and the Android NDK for build testing.) 268That build system also instaled bash 2.05b as #!/bin/sh and its scripts 269required bash extensions not present in shells such as busybox ash. 270To replace that toysh needs to supply several bash extensions _and_ work 271when called under the name "bash".</p> 272 273<p>The development methodology used a <a href=http://landley.net/aboriginal/FAQ.html#debug_logging>command logging wrapper</a> 274that intercepted each command called out of the $PATH and append the 275command line to a log file, then <a href=https://github.com/landley/aboriginal/blob/master/more/report-recorded-commands.sh>analyze</a> the result to create a 276<a href=https://landley.net/notes-2008.html#23-01-2008>list of commands</a>, 277then <a href=https://github.com/landley/aboriginal/blob/master/host-tools.sh>create a directory of symlinks</a> pointing to those commands out of the 278host $PATH. Then the new implementation can replace these commands one 279at a time, checking the results and the log output to spot any behavior 280changes.</p> 281 282<h3>Stages and moving targets</h3> 283 284<p>This use case has two stages: 1) building a bootable system that can 285rebuild itself from source, and 2) a build environment capable 286of bootstrapping up to arbitrary complexity (as exemplified by building 287Linux From Scratch and Beyond Linux From Scratch under the resulting 288system). To accomplish just the first goal, the old build 289still needs the following busybox commands for which toybox does not yet 290supply adequate replacements:</p> 291 292<blockquote><b> 293awk dd diff expr fdisk ftpd gzip less route sh sha512sum tr unxz vi wget xzcat 294</b></blockquote> 295 296<p>All of those except awk, ftpd, and less have partial implementations 297in "pending".</p> 298 299<p>In 2017 Aboriginal Linux development ended, replaced by the 300<a href=https://github.com/landley/mkroot>mkroot</a> project 301designed to use an existing cross+native toolchain (such as 302<a href=https://github.com/richfelker/musl-cross-make>musl-cross-make</a> 303or the Android NDK) instead of building its own. In 2019 the still-incomplete 304mkroot was merged into toybox as the "make root" target. This is intended 305as a simpler way of providing essentially the same build environment, and doesn't 306significantly affect the rest of this analysis (although the "rebuild itself 307from source" test now includes building musl-cross-make under either mkroot 308or toybox's "make airlock" host environment).</p> 309 310<p>Building Linux From Scratch is not the same as building the 311<a href=https://source.android.com>Android Open Source Project</a>, 312but after toybox 1.0 we plan to try 313<a href=http://landley.net/aboriginal/about.html#hairball>modifying the AOSP build</a> 314to reduce dependencies. (It's fairly likely we'll have to add at least 315a read-only git utility so repo can download the build's source code, 316but that's actually <a href=https://www.youtube.com/watch?v=P7n6G2IL6eo>not 317that hard</a>. We'll probably also need our own "make" at some point after 3181.0, which is its own moving target thanks to cmake and ninja and so on.) 319The ongoing Android <a href=http://lists.landley.net/pipermail/toybox-landley.net/2018-January/009330.html>hermetic build</a> work is already advancing 320this goal.</p> 321 322<hr /> 323<h2><a name=android /><a href="#android">Use case: Replacing Android Toolbox</a></h2> 324 325<p>Android has a policy against GPL in userspace, so even though BusyBox 326predates Android by many years, they couldn't use it. Instead they grabbed 327an old version of ash (later replaced by 328<a href="https://www.mirbsd.org/mksh.htm">mksh</a>) 329and implemented their own command line utility set 330called "toolbox" (which toybox has already mostly replaced).</p> 331 332<p>Toolbox doesn't have its own repository, instead it's part of Android's 333<a href=https://android.googlesource.com/platform/system/core>system/core 334git repository</a>. Android's Native Development Kit (their standalone 335downloadable toolchain) has its own 336<a href=https://android.googlesource.com/platform/ndk/+/master/docs/Roadmap.md>roadmap</a>, and each version has 337<a href=https://developer.android.com/ndk/downloads/revision_history>release 338notes</a>.</p> 339 340<h3>Toolbox commands:</h3> 341 342<p>According to <a href=https://android.googlesource.com/platform/system/core/+/master/toolbox/Android.bp> 343system/core/toolbox/Android.bp</a> the toolbox directory builds the 344following commands:</p> 345 346<blockquote><b> 347getevent getprop modprobe setprop start 348</b></blockquote> 349 350<p>getprop/setprop/start were in toybox and moved back because they're so 351tied to non-public system interfaces. modprobe shares the implementation 352used in init. getevent is a board bringup tool built with a python script 353that pulls all the constants from the latest kernel headers.</p> 354 355<h3>Other Android /system/bin commands</h3> 356 357<p>Other than the toolbox links, the currently interesting 358binaries in /system/bin are:</p> 359 360<ul> 361<li><b>arping</b> - ARP REQUEST tool (iputils)</li> 362<li><b>blkid</b> - identify block devices (e2fsprogs)</li> 363<li><b>e2fsck</b> - fsck for ext2/ext3/ext4 (e2fsprogs)</li> 364<li><b>fsck.f2fs</b> - fsck for f2fs (f2fs-tools)</li> 365<li><b>fsck_msdos</b> - fsck for FAT (BSD)</li> 366<li><b>gzip</b> - compression/decompression tool (zlib)</li> 367<li><b>ip</b> - network routing tool (iproute2)</li> 368<li><b>iptables/ip6tables</b> - IPv4/IPv6 NAT admin (iptables)</li> 369<li><b>iw</b> - wireless device config tool (iw)</li> 370<li><b>logwrapper</b> - redirect stdio to android log (Android)</li> 371<li><b>make_ext4fs</b> - make ext4 fs (Android)</li> 372<li><b>make_f2fs</b> - make f2fs fs (f2fs-tools)</li> 373<li><b>ping/ping6</b> - ICMP ECHO_REQUEST tool (iputils)</li> 374<li><b>reboot</b> - reboot (Android)</li> 375<li><b>resize2fs</b> - resize ext2/ext3/ext4 fs (e2fsprogs)</li> 376<li><b>sh</b> - mksh (BSD)</li> 377<li><b>ss</b> - socket statistics (iproute2)</li> 378<li><b>tc</b> - traffic control (iproute2)</li> 379<li><b>tracepath/tracepath6</b> - trace network path (iputils)</li> 380<li><b>traceroute/traceroute6</b> - trace network route (iputils)</li> 381</ul> 382 383<p>The names in parentheses are the upstream source of the command.</p> 384 385<h3>Analysis</h3> 386 387<p>For reference, combining everything listed above that's still "fair game" 388for toybox, we get:</p> 389 390<blockquote><b> 391arping blkid e2fsck dd fsck.f2fs fsck_msdos gzip ip iptables 392ip6tables iw logwrapper make_ext4fs make_f2fs modpobe newfs_msdos ping ping6 393reboot resize2fs sh ss tc tracepath tracepath6 traceroute traceroute6 394</b></blockquote> 395 396<p>We may eventually implement all of that, but for toybox 1.0 we need to 397focus a bit. If Android has an acceptable external package, and the command 398isn't needed for system bootstrapping, replacing the external package is 399not a priority.</p> 400 401<p>However, several commands toybox plans to implement anyway could potentially 402replace existing Android versions, so we should take into account Android's use 403cases when doing so. This includes:</p> 404<blockquote><b> 405<span id=toolbox> 406dd getevent gzip modprobe newfs_msdos sh 407</span> 408</b></blockquote> 409 410<p>Update: <a href=https://android.googlesource.com/platform/system/core/+/master/system/core/Android.bp> 411external/toybox/Android.bp</a> has symlinks for the following toys out 412of "pending". (The toybox modprobe is also built for the device, but 413it isn't actually used and is only there for sanity checking against 414the libmodprobe-based implementation.) These should be a priority for 415cleanup:</p> 416 417<blockquote><b> 418bc dd diff expr getfattr lsof more stty tr traceroute 419</b></blockquote> 420 421<p>Android wishlist:</p> 422 423<blockquote><b> 424mtools genvfatfs mke2fs gene2fs 425</b></blockquote> 426 427<hr /> 428<h2><a name=aosp /><a href="#aosp">Use case: Building AOSP</a></h2> 429 430<p>The list of external tools used to build AOSP was 431<a href="https://android.googlesource.com/platform/build/soong/+/master/ui/build/paths/config.go">here</a>, 432but as they're switched over to toybox they disappear and reappear 433<a href="https://android.googlesource.com/platform/prebuilts/build-tools/+/refs/heads/master/path/linux-x86/">here</a>.</p> 434 435<blockquote><b> 436awk basename bash bc bzip2 cat chmod cmp comm cp cut date dd diff dirname du 437echo egrep env expr find fuser getconf getopt git grep gzip head hexdump 438hostname id jar java javap ln ls lsof m4 make md5sum mkdir mktemp mv od openssl 439paste patch pgrep pkill ps pstree pwd python python2.7 python3 readlink 440realpath rm rmdir rsync sed setsid sh sha1sum sha256sum sha512sum 441sleep sort stat tar tail tee todos touch tr true uname uniq unix2dos unzip 442wc which whoami xargs xxd xz zip zipinfo 443</b></blockquote> 444 445<p>The following are already in the tree and will be used directly:</p> 446 447<blockquote><b> 448awk bzip2 jar java javap m4 make python python2.7 python3 xz 449</b></blockquote> 450 451<p>Subtracting what's already in toybox (including the following toybox toys 452that are still in pending: <code>bc dd diff expr gzip lsof tar tr</code>), 453that leaves:</p> 454 455<blockquote><b> 456bash fuser getopt git hexdump openssl pstree rsync sh todos unzip zip zipinfo 457</b></blockquote> 458 459<p>For AOSP, zip/zipinfo/unzip are likely to be libziparchive based. The 460todos callers will use unix2dos instead if it's available. git/openssl 461seem like they should just be brought in to the tree. rsync is used to 462work around a Mac <code>cp -Rf</code> bug with broken symbolic links. That 463leaves:</p> 464 465<blockquote><b> 466bash fuser getopt hexdump pstree 467</b></blockquote> 468 469<p>(Why are fuser and pstree used during the AOSP build? They're used for 470diagnostics if something goes wrong. So it's really just bash, getopt, 471and hexdump that are actually used to build.)</p> 472 473<hr /> 474<h2><a name=tizen /><a href="#tizen">Use case: Tizen Core</a></h2> 475 476<p>The Tizen project has expressed a desire to eliminate GPLv3 software 477from its core system, and is installing toybox as 478<a href=https://wiki.tizen.org/wiki/Toybox>part of this process</a>.</p> 479 480<p>They have a fairly long list of new commands they'd like to see in toybox:</p> 481 482<blockquote><b> 483<span id=tizen> 484arch base64 users unexpand shred join csplit 485hostid nproc runcon sha224sum sha256sum sha384sum sha512sum sha3sum mkfs.vfat fsck.vfat 486dosfslabel uname stdbuf pinky diff3 sdiff zcmp zdiff zegrep zfgrep zless zmore 487</span> 488</b></blockquote> 489 490<p>In addition, they'd like to use several commands currently in pending:</p> 491 492<blockquote><b> 493<span id=tizen> 494tar diff printf wget rsync fdisk vi less tr test stty fold expr dd 495</span> 496</b></blockquote> 497 498<p>Also, tizen uses a different Linux Security Module called SMACK, so 499many of the SELinux options ala ls -Z need smack alternatives in an 500if/else setup.</p> 501 502<hr /><a name=buildroot /> 503<h2>buildroot:</h2> 504 505<p>The <a href=https://buildroot.org/downloads/manual/manual.html#requirement-mandatory>mandatory packages</a> 506section of the buildroot manual lists:</p> 507 508<blockquote><p> 509which sed make bash patch gzip bzip2 tar cpio unzip rsync file bc wget 510</p></blockquote> 511 512<p>(It also lists binutils gcc g++ perl python, and for debian it wants 513build-essential. And it wants file to be in /usr/bin because 514<a href=https://git.busybox.net/buildroot/tree/support/dependencies/dependencies.sh?h=2018.02.x#n84>libtool 515breaks otherwise</a>.)</p> 516 517<p>Buildroot does not support a cross toolchain that lives in "/usr/bin" 518with a prefix of "" (if you try, and chop out the test for a blank prefix, 519it dies trying to run "/usr/bin/-gcc"). But you can patch your way to 520making it work if you try.</p> 521 522<hr /><a name=klibc /> 523<h2>klibc:</h2> 524 525<p>Long ago some kernel developers came up with a project called 526<a href=http://en.wikipedia.org/wiki/Klibc>klibc</a>. 527After a decade of development it still has no web page or HOWTO, 528and nobody's quite sure if the license is BSD or GPL. It inexplicably 529<a href=http://www.infoworld.com/d/data-center/perl-isnt-going-anywhere-better-or-worse-211580>requires perl to build</a>, and seems like an ideal candidate for 530replacement.</p> 531 532<p>In addition to a C library even less capable than bionic (obsoleted by 533musl), klibc builds a random assortment of executables to run init scripts 534with. There's no multiplexer command, these are individual executables:</p> 535 536<blockquote><p><b> 537cat chroot cpio dd dmesg false fixdep fstype gunzip gzip halt ipconfig kill 538kinit ln losetup ls minips mkdir mkfifo mknodes 539mksyntax mount mv nfsmount nuke pivot_root poweroff readlink reboot resume 540run-init sh sha1hash sleep sync true umount uname zcat 541</b></p></blockquote> 542 543<p>To get that list, build klibc according to the instructions (I 544<a href=http://landley.net/notes-2013.html#23-01-2013>looked at</a> version 5452.0.2 and did cd klibc-*; ln -s /output/of/kernel/make/headers_install 546linux; make) then <b>echo $(for i in $(find . -type f); do file $i | grep -q 547executable && basename $i; done | grep -v '[.]g$' | sort -u)</b> to find 548executables, then eliminate the *.so files and *.shared duplicates.</p> 549 550<p>Some of those binaries are build-time tools that don't get installed, 551which removes mknodes, mksyntax, sha1hash, and fixdep from the list. 552(And sha1hash is just an unpolished sha1sum anyway.)</p> 553 554<p>The run-init command is more commonly called switch_root, nuke is just 555"rm -rf -- $@", and minips is more commonly called "ps". I'm not doing aliases 556for the oddball names.</p> 557 558<p>Yet more stale forks of dash and gzip sucked in here (see "dubious 559license terms" above), adding nothing to the other projects we've looked at. 560But we still need sh, gunzip, gzip, and zcat to replace this package.</p> 561 562<p>At the time I did the initial analysis toybox already had cat, chroot, dmesg, false, 563kill, ln, losetup, ls, mkdir, mkfifo, readlink, rm, switch_root, sleep, sync, 564true, and uname.</p> 565 566<p>The low hanging fruit is cpio, dd, ps, mv, and pivot_root.</p> 567 568<p>The "kinit" command is another gratuitous rename, it's init running as PID 1. 569The halt, poweroff, and reboot commands work with it.</p> 570 571<p>I've got mount and umount queued up already, fstype and nfsmount go with 572those. (And probably smbmount and p9mount, but this hasn't got one. Those 573are all about querying for login credentials, probably workable into the 574base mount command.)</p> 575 576<p>The ipconfig command here has a built in dhcp client, so it's ifconfig 577and dhcpcd and maybe some other stuff.</p> 578 579<p>The resume command is... weird. It finds a swap partition and reads data 580from it into a /proc file, something the kernel is capable of doing itself. 581(Even though the klibc author 582<a href=http://www.zytor.com/pipermail/klibc/2006-June/001748.html>attempted 583to remove</a> that capability from the kernel, current kernel/power/hibernate.c 584still parses "resume=" on the command line). And yet various distros seem to 585make use of klibc for this. 586Given the history of swsusp/hibernate (and 587<a href=http://lwn.net/Articles/333007>TuxOnIce</a> 588and <a href=http://lwn.net/Articles/242107>kexec jump</a>) I've lost track 589of the current state of the art here. Ah, Documentation/power/userland-swsusp.txt 590has the API docs, and <a href=http://suspend.sf.net>here's a better 591tool</a>...</p> 592 593<p>So the list of things actually in klibc are:</p> 594 595<blockquote><b> 596<span id=klibc_cmd> 597cat chroot dmesg false kill ln losetup ls mkdir mkfifo readlink rm switch_root 598sleep sync true uname 599 600cpio dd ps mv pivot_root 601mount nfsmount fstype umount 602sh gunzip gzip zcat 603kinit halt poweroff reboot 604ipconfig 605resume 606</span> 607</b></blockquote> 608 609<hr /> 610<a name=glibc /> 611<h2>glibc</h2> 612 613<p>Rather a lot of command line utilities come bundled with glibc:</p> 614 615<blockquote><b> 616catchsegv getconf getent iconv iconvconfig ldconfig ldd locale localedef 617mtrace nscd rpcent rpcinfo tzselect zdump zic 618</b></blockquote> 619 620<p>Of those, musl libc only implements ldd.</p> 621 622<p>catchsegv is a rudimentary debugger, probably out of scope for toybox.</p> 623 624<p>iconv has been <a href="#susv4">previously discussed</a>.</p> 625 626<p>iconvconfig is only relevant if iconv is user-configurable; musl uses a 627non-configurable iconv.</p> 628 629<p>getconf is a posix utility which displays several variables from 630unistd.h; it probably belongs in the development toolchain.</p> 631 632<p>getent handles retrieving entries from passwd-style databases 633(in a rather lame way) and is trivially replacable by grep.</p> 634 635<p>locale was discussed under <a href=#susv4>posix</a>. 636localedef compiles locale definitions, which musl currently does not use.</p> 637 638<p>mtrace is a perl script to use the malloc debugging that glibc has built-in; 639this is not relevant for musl, and would necessarily vary with libc. </p> 640 641<p>nscd is a name service caching daemon, which is not yet relevant for musl. 642rpcinfo and rpcent are related to rpc, which musl does not include.</p> 643 644<p>The remaining commands involve glibc's bundled timezone database, 645which seems to be derived from the <a href=http://www.iana.org/time-zones>IANA 646timezone database</a>. Unless we want to maintain our own fork of the 647standards body's database like glibc does, these are of no interest, 648but for completeness:</p> 649 650<p>tzselect outputs a TZ variable correponding to user input. 651The documentation does not indicate how to use it in a script, but it seems 652that Debian may have done so. 653zdump prints current time in each of several timezones, optionally 654outputting a great deal of extra information about each timezone. 655zic converts a description of a timezone to a file in tz format.</p> 656 657<p>None of glibc's bundled commands are currently of interest to toybox.</p> 658 659</b></blockquote> 660 661<hr /> 662<a name=sash /> 663<h2>Stand-Alone Shell</h2> 664 665<p>Wikipedia has <a href=http://en.wikipedia.org/wiki/Stand-alone_shell>a good 666summary of sash</a>, with links. The original Stand-Alone Shell project reached 667a stopping point, and then <a href=http://www.baiti.net/sash>"sash plus 668patches"</a> extended it a bit further. The result is a megabyte executable 669that provides 40 commands.</p> 670 671<p>Sash is a shell with built-in commands. It doesn't have a multiplexer 672command, meaning "sash ls -l" doesn't work (you have to go "sash -c 'ls -l'"). 673</p> 674 675<p>The list of commands can be obtained via building it and doing 676"echo help | ./sash | awk '{print $1}' | sed 's/^-//' | xargs echo", which 677gives us:</p> 678 679<blockquote><b> 680alias aliasall ar cd chattr chgrp chmod chown cmp cp chroot dd echo ed exec 681exit file find grep gunzip gzip help kill losetup losetup ln ls lsattr mkdir 682mknod more mount mv pivot_root printenv prompt pwd quit rm rmdir setenv source 683sum sync tar touch umask umount unalias where 684</b></blockquote> 685 686<p>Plus sh because it's a shell. A dozen or so commands can only sanely be 687implemented as shell builtins (alias aliasall cd exec exit prompt quit setenv 688source umask unalias), where is an alias for which, and at triage time toybox 689already has chgrp, chmod, chown, cmp, cp, chroot, echo, help, kill, losetup, 690ln, ls, mkdir, mknod, printenv, pwd, rm, rmdir, sync, and touch.</p> 691 692<p>This leaves:</p> 693 694<blockquote><b> 695<span id=sash_cmd> 696ar chattr dd ed file find grep gunzip gzip lsattr more mount mv pivot_root 697sh tar umount 698</span> 699</b></blockquote> 700 701<p>(For once, this project doesn't include a fork of gzip, instead 702it sucks in -lz from the host.)</p> 703 704<hr /> 705<a name=sbase /> 706<h2>sbase:</h2> 707 708<p>It's <a href=http://git.suckless.org/sbase>on suckless</a> in 709<a href=http://git.suckless.org/ubase>two parts</a>. As of November 2015 it's 710implemented the following (renaming "cron" to "crond" for 711consistency, and yanking "sponge", "mesg", "pagesize", "respawn", and 712"vtallow"):</p> 713 714<blockquote><p> 715<span id=sbase_cmd> 716basename cal cat chgrp chmod chown chroot cksum cmp comm cp crond cut date 717dirname du echo env expand expr false find flock fold getconf grep head 718hostname join kill link ln logger logname ls md5sum mkdir mkfifo mktemp mv 719nice nl nohup od paste printenv printf pwd readlink renice rm rmdir sed seq 720setsid sha1sum sha256sum sha512sum sleep sort split strings sync tail 721tar tee test tftp time touch tr true tty uname unexpand uniq unlink uudecode 722uuencode wc which xargs yes 723</span> 724</p></blockquote> 725 726<p>and<p> 727 728<blockquote><p> 729<span id=sbase_cmd> 730chvt clear dd df dmesg eject fallocate free id login mknod mountpoint 731passwd pidof ps stat su truncate unshare uptime watch 732who 733</span> 734</p></blockquote> 735 736<hr /> 737<a name=nash /> 738<h2>nash:</h2> 739 740<p>Red Hat's nash was part of its "mkinitrd" package, replacement for a shell 741and utilities on the boot floppy back in the 1990's (the same general idea 742as BusyBox, developed independently). Red Hat discontinued nash development 743in 2010, replacing it with dracut (which collects together existing packages, 744including busybox).</p> 745 746<p>I couldn't figure out how to beat source code out of 747<a href=http://pkgs.fedoraproject.org/git/mkinitrd>Fedora's current git</a> 748repository. The last release version that used it was Fedora Core 12 749which has <a href=http://archive.fedoraproject.org/pub/archive/fedora/linux/releases/12/Fedora/source/SRPMS/mkinitrd-6.0.93-1.fc12.src.rpm>a source rpm</a> 750that can be unwound with "rpm2cpio mkinitrd.src.rpm | cpio -i -d -H newc 751--no-absolute-filenames" and in there is a mkinitrd-6.0.93.tar.bz2 which 752has the source.</p> 753 754<p>In addition to being a bit like a command shell, the nash man page lists the 755following commands:</p> 756 757<blockquote><p> 758access echo find losetup mkdevices mkdir mknod mkdmnod mkrootdev mount 759pivot_root readlink raidautorun setquiet showlabels sleep switchroot umount 760</p></blockquote> 761 762<p>Oddly, the only occurrence of the string pivot_root in the nash source code 763is in the man page, the command isn't there. (It seems to have been removed 764when the underscoreless switchroot went in.)</p> 765 766<p>A more complete list seems to be the handlers[] array in nash.c:</p> 767 768<blockquote><p> 769access buildEnv cat cond cp daemonize dm echo exec exit find kernelopt 770loadDrivers loadpolicy mkchardevs mkblktab mkblkdevs mkdir mkdmnod mknod 771mkrootdev mount netname network null plymouth hotplug killplug losetup 772ln ls raidautorun readlink resume resolveDevice rmparts setDeviceEnv 773setquiet setuproot showelfinterp showlabels sleep stabilized status switchroot 774umount waitdev 775</p></blockquote> 776 777<p>This list is nuts: "plymouth" is an alias for "null" which is basically 778"true" (which thie above list doesn't have). Things like buildEnv and 779loadDrivers are bespoke Red Hat behavior that might as well be hardwired in 780to nash's main() without being called.</p> 781 782<p>Instead of eliminating items 783from the list with an explanation for each, I'm just going to cherry pick 784a few: the device mapper (dm, raidautorun) is probably interesting, 785hotplug (may be obsolete due to kernel changes that now load firmware 786directly), and another "resume" ala klibc.</p> 787 788<p>But mostly: I don't care about this one. And neither does Red Hat anymore.</p> 789 790<p>Verdict: ignore</p> 791 792<hr /> 793<a name=beastiebox /> 794<h2>Beastiebox</h2> 795 796<p>Back in 2008, the BSD guys vented some busybox-envy 797<a href=http://beastiebox.sourceforge.net>on sourceforge</a>. Then stopped. 798Their repository is still in CVS, hasn't been touched in years, it's a giant 799hairball of existing code sucked together. (The web page says the author 800is aware of crunchgen, but decided to do this by hand anyway. This is not 801a collection of new code, it's a katamari of existing code rolled up in a 802ball.)</p> 803 804<p>Combining the set of commands listed on the web page with the set of 805man pages in the source gives us:</P> 806 807<blockquote><p> 808[ cat chmod cp csh date df disklabel dmesg echo ex fdisk fsck fsck_ffs getty 809halt hostname ifconfig init kill less lesskey ln login ls lv mksh more mount 810mount_ffs mv pfctl ping poweroff ps reboot rm route sed sh stty sysctl tar test 811traceroute umount vi wiconfig 812</p></blockquote> 813 814<p>Apparently lv is the missing link between ed and vi, copyright 1982-1997 (do 815not want), ex is another obsolete vi mode, lesskey is "used to 816specify a set of key bindings to be used with less", and csh is a shell they 817sucked in (even though they have mksh?), [ is an alias for test. Several more bsd-isms that don't have Linux 818equivalents (even in the ubuntu "install this package" search) are 819disklabel, fsck_ffs, mount_ffs, and pfctl. And wiconfig is a 820wavelan interface network card driver utility. Subtracting all that and the 821commands toybox already implements at triage time, we get:</p> 822 823<blockquote><p> 824<span id=beastiebox_cmd> 825fdisk fsck getty halt ifconfig init kill less more mount mv ping poweroff 826ps reboot route sed sh stty sysctl tar test traceroute umount vi 827</span> 828</p></blockquote> 829 830<p>Not a hugely interesting list, but eh.</p> 831 832<p>Verdict: ignore</p> 833 834<hr /> 835<a name=BsdBox /> 836<h2>BsdBox</h2> 837 838<p>Somebody decided to do a <a href=https://wiki.freebsd.org/AdrianChadd/BsdBox>multicall binary for freebsd</a>.</p> 839 840<p>They based it on crunchgen, a tool that glues existing programs together 841into an archive and uses the name to execute the right one. It has no 842simplification or code sharing benefits whatsoever, it's basically an 843archiver that produces executables.</p> 844 845<p>That's about where I stopped reading.</p> 846 847<p>Verdict: ignore.</p> 848 849<hr /> 850<a name=slowaris /> 851<h2>OpenSolaris Busybox</h2> 852 853<p>Somebody <a href=http://hub.opensolaris.org/bin/view/Project+busybox/>wrote 854a wiki page</a> saying that Busybox for OpenSolaris would be a good idea.</p> 855 856<p>The corresponding "files" tab is an auto-generated stub. The project never 857even got as far as suggesting commands to include before Oracle discontinued 858OpenSolaris.</p> 859 860<p>Verdict: ignore.</p> 861 862<hr /> 863<a name=uclinux /> 864<h2>uClinux</h2> 865 866<p>Long ago a hardware developer named Jeff Dionne put together a 867nommu Linux distribution, which involved rewriting a lot of command line 868utilities that relied on <a href=http://nommu.org/memory-faq.txt>features 869unavailable on nommu</a> hardware.</p> 870 871<p>In 2003 Jeff moved to Japan and handed 872the project off to people who allowed it to roll to a stop. The website 873turned into a mess of 404 links, the navigation indexes stopped being 874updated over a decade ago, and the project's CVS repository suffered a 875hard drive failure for which there were no backups. The project continued 876to put out "releases" through 2014 (you have to scroll down in the "news" 877section to find them, the "HTTP download" section in the nav bar on the 878left hasn't been updated in over a decade), which were hand-updated tarball 879snapshots mostly consisting of software from the 1990's. For example the 8802014 release still contained ipfwadm, the package which predated ipchains, 881which predated iptables, which is in the process of being replaced by 882nftables.</p> 883 884<p>Nevertheless, people still try to use this because (at least until the 885launch of <a href=http://nommu.org>nommu.org</a>) the project was viewed 886as the place to discuss, develop, and learn about nommu Linux. 887The role of uclinux.org as an educational resource kept people coming 888to it long after it had collapsed as a Linux distro.</p> 889 890<p>Starting around 0.6.0 toybox began to address nommu support with the goal 891of putting uClinux out of its misery.</p> 892 893<p>An analysis of <a href=http://www.uclinux.org/pub/uClinux/dist/uClinux-dist-20140504.tar.bz2>uClinux-dist-20140504</a> found 312 package 894subdirectories under "user".</p> 895 896<h3>Taking out the trash</h3> 897 898<p>A bunch of packages (<b>inotify-tools, input-event-demon, ipsec-tools, netifd, 899keepalived, mobile-broadband-provider-info, nuttp, readline, snort, 900snort-barnyard, socat, sqlite, sysklogd, sysstat, tcl, ubus, uci, udev, 901unionfs, uqmi, usb_modeswitch, usbutils, util-linux</b>) 902are hard to evaluate because 903uclinux has directories for them, but their source isn't actually in the 904uclinux tree. In some of these the makefiles download a git repo during 905the build, so I'm assuming you can build the external package if you really 906care. (Even when I know what these packages do, I'm skipping them 907because uclinux doesn't actually contain them, and any given snapshot 908of the build system will bitrot as external web links change over time.)</p> 909 910<p>Other packages are orphaned, meaning they're not mentioned from any Kconfig 911or Makefiles outside of their directory, so uclinux can't actually build 912them: <b>mbus</b> is an orphaned i2c test program expecting to run in some sort 913of hardwired hardware context, <b>mkeccbin</b> is an orphaned "ECC annotated 914binary file" generator (meaning it's half of a flash writer), 915<b>wsc_upnp</b> is a "Ralink WPS" driver (some sort of stale wifi chip)...</p> 916 917<p>The majority of the remaining packages are probably not of interest to 918toybox due to being so obsolete or special purpose they may not actually be 919of interest to anybody anymore. (This list also includes a lot of 920special-purpose network back-end stuff that's hard for anybody but 921datacenter admins to evaluate the current relevance of.)</p> 922 923<blockquote><b><p> 924arj asterisk boottools bpalogin br2684ctl camserv can4linux cgi_generic 925cgihtml clamav clamsmtp conntrack-tools cramfs crypto-tools cxxtest 926ddns3-client de2ts-cal debug demo diald discard dnsmasq dnsmasq2 927ethattach expat-examples ez-ipupdate fakeidentd 928fconfig ferret flatfs flthdr freeradius freeswan frob-led frox fswcert 929game gettyd gnugk haserl horch 930hostap hping httptunnel ifattach ipchains 931ipfwadm ipmasqadm ipportfw ipredir ipset iso_client 932jamvm jffs-tools jpegview jquery-ui kendin-config kismet klaxon kmod 933l2tpd lcd ledcmd ledcon lha lilo lirc lissa load loattach 934lpr lrpstat lrzsz mail mbus mgetty microwin ModemManager msntp musicbox 935nooom null openswan openvpn palmbot pam_* pcmcia-cs playrt plugdaemon pop3proxy 936potrace qspitest quagga radauth 937ramimage readprofile rdate readprofile routed rrdtool rtc-ds1302 938sendip ser sethdlc setmac setserial sgutool sigs siproxd slattach 939smtpclient snmpd net-snmp snortrules speedtouch squashfs scep sslwrap stp 940stunnel tcpblast tcpdump tcpwrappers threaddemos tinylogin tinyproxy 941tpt tripwire unrar unzoo version vpnled w3cam xl2tpd zebra 942</p></b></blockquote> 943 944<p>This stuff is all over the place: arj, lha, rar, and zoo are DOS archivers, 945ethattach describes itself as just "a network tool", 946mail is a textmode smtp mailer literally described as "Some kind of mail 947proggy" in uclinux's kconfig (as opposed to clamsmtp and smtpclient and 948so on), this gettyd isn't a generic version but specifically a 949hardwired ppp dialin utility, mgetty isn't a generic version but is combined 950with "sendfax", hostap is an intersil prism driver, wlan-ng is also an 951intersil prism dirver, null is a program to intentionally dereference a 952null pointer (in case you needed one), iso_client is a 953"Demo Application for the USB Device Driver", kendin-config is 954"for configuring the Micrel Kendin KS8995M over QSPI", speedtouch configures 955a specific brand of asdl modem, portmap is part of Anfs, 956ferret, linux-igd, and miniupnp are all upnp packages, 957lanbypass "can be used to control the LAN 958bypass switches on the Advantech x86 based hardware platforms", lcd is 959"test of lcddma device driver" (an out-of-tree Coldfire driver apparently 960lost to history, the uclinux linux-2.4.x directory has a config symbol for 961it, but nothing in the code actually _uses_ it...), qspitest is another 962coldfire thing, mii-tool-fec is 963"strictly for the FEC Ethernet driver as implemented (and modified) for 964the uCdimm5272", rtc-ds1302 and rtc-m41t11 are usermode drivers for specific 965clock chips, stunnel is basically "openssl s_client -quiet -connect", 966potrace is a bitmap to vector graphic converter, radauth performs command line 967authentication against a radius server, 968clamav, klaxon, ferret, l7-protocols, and nessus are very old network security 969software (it's got a stale snapshot of nmap too), xl2tpd is a PPP over UDP 970tunnel (rfc 2661), zebra is the package quagga replaced, 971lilo is the x86-only bootloader that predated grub (and recently discontinued 972development), lissa is a "framebuffer graphics demo" from 9731998, the squashfs package here is the out of tree patches for 2.4 kernels 974and such before the filesystem was merged upstream (as opposed to the 975squashfs-new package which is a snapshot of the userspace tool from 2011), 976load is basically "dd file /dev/spi", version is basically "cat /proc/version", 977microwin is a port of the WinCE graphics API to Linux, scep is a 2003 978implementation of an IETF draft abandoned in 2010, tpt depends on 979Andrew Morton's 15 year old unmerged "timepegs" kernel patch using the pentium 980cycle counter, vpnled controls a light that reboots systems (what?), 981w3cam is a video4linux 1.0 client (v4l2 showed up during 2.5 and support for 982the old v4l1 was removed in 2.6.38 back in 2011), busybox ate tinylogin 983over a decade ago, lrpstat is a java network monitor 984from 2001, lrzsz is zmodem/ymodem/zmodem, msntp and stp implement rfc2030 985meaning it overflows in 2036 (the package was last updated in 2000), rdate 986is rfc 868 meaning it also overflows in 2036 (which is why ntp was invented 987a few decades back), reiserfsprogs development stopped abruptly after 988Hans Reiser was convicted of murdering his wife Nina (denying it on the 989stand and then leading them to the body as part of his plea bargain during 990sentencing)... 991</p> 992 993<p>Seriously, there's a lot of crap in there. It's hard to analyze most 994of it far enough to prove it _doesn't_ do anything.</p> 995 996<h3>Non-toybox programs</h3> 997 998<p>The following software may actually still do something intelligible 999(although the package versions tend to be years out of date), but 1000it's not a direction toybox has chosen to go in.</p> 1001 1002<p>There are several programming languages (<b>bash, lua, jamvm, tinytcl, 1003perl, python</b>) in there. Maybe someone somewhere wants a 2008 release of a 1004java virtual machine tested to work on nommu systems (jamvm), but it's out 1005of scope for toybox.</p> 1006 1007<p>A bunch of benchmark programs: <b>cpu, dhrystone, mathtest, nbench, netperf, 1008netpipe, and whetstone</b>.</p> 1009 1010<p>A bunch of web servers: <b>appWeb, boa, fnord (via tcpserver), goahead, httpd, 1011mini_httpd, and thttpd</b>.</p> 1012 1013<p>A bunch of shells: <b>msh</b> is a clever (I.E. obfuscated) little shell, 1014<b>nwsh</b> is "new shell" (that's what it called itself in 1999 anyway), 1015<b>sash</b> is another shell with a bunch of builtins (ls, ps, df, cp, date, reboot, 1016and shutdown, this roadmap analyzes it <a href="#sash">elsewhere</a>), 1017<b>sh</b> is a very old minix shell fork, and <b>tcsh</b> is also a shell.</p> 1018 1019<p>Also in this category, we have:</p> 1020 1021<blockquote><b><p> 1022dropbear jffs-tools jpegview kexec-tools bind ctorrent 1023iperf iproute2 ip-sentinel iptables kexec 1024nmap oggplay openssl oprofile p7zip pppd pptp play vplay 1025hdparm mp3play at clock 1026mtd-utils mysql logrotate brcfg bridge-utils flashw 1027ebtables etherwake ethtool expect gdb gdbserver hostapd 1028lm_sensors load netflash netstat-nat 1029radvd recover rootloader resolveip rp-pppoe 1030rsyslog rsyslogd samba smbmount squashfs-new squid ssh strace tip 1031uboot-envtools ulogd usbhubctrl vconfig vixie-cron watchdogd 1032wireless_tools wpa_supplicant 1033</p></b></blockquote> 1034 1035<p>An awful lot of those are borderline: play and vplay are wav file 1036audio players, there's oprofile _and_ readprofile (which just reads kernel 1037profiling data from /proc/profile), 1038radvd is a "routr advertisement daemon" (ipv6 stateless autoconf), 1039ctorrent is a bittorent client, 1040lm_sensors is hardware (heat?) monitoring, 1041resolveip is dig only less so, 1042rp-pppoe is ppp over ethernet, 1043ebtables is an ethernet version of iptables (for bridging), 1044their dropbear is from 2012, and that ssh version is from 2011 1045(which means it's about nine months too _old_ to have the heartbleed bug). 1046There's both ulogd and ulogd2 (no idea why), and pppd is version 2.4 but 1047there's a ppd-2.3 directory also.</p> 1048 1049<p>Lots of flash stuff: 1050flashw is a flash writer, load is an spi flash loader, netflash writes 1051to flash via tftp, 1052recover is also a reflash daemon intended to come up when the system can't boot, 1053rootloader seems to be another reflash daemon but without dhcp.</p> 1054 1055<h3>Already in roadmap</h3> 1056 1057<p>The following packages contain commands already in the toybox roadmap:</p> 1058 1059<blockquote><b><p> 1060agetty cal cksum cron dhcpcd dhcpcd-new dhcpd dhcp-isc dosfstools e2fsprogs 1061elvis-tiny levee fdisk fileutils ftp ftpd grep hd hwclock inetd init ntp 1062iputils login module-init-tools netcat shutils ntpdate lspci ping procps 1063proftpd rsync shadow shutils stty sysutils telnet telnetd tftp tftpd traceroute 1064unzip wget mawk net-tools 1065</p></b></blockquote> 1066 1067<p>There are some duplicates in there, levee is a tiny vi implementation 1068like elvis-tiny, ntp and ntpdate overlap, etc.</p> 1069 1070<p>Verdict: We don't really need to do a whole lot special for nommu 1071systems, just get the existing toybox roadmap working on nommu and 1072we're good. The uClinux project can rest in peace.</p> 1073 1074<hr /> 1075<h2>Requests:</h2> 1076 1077<p>The following additional commands have been requested (and often submitted) 1078by various users. I _really_ need to clean up this section.</p> 1079 1080<p>Also:</p> 1081<blockquote><b> 1082<span id=request> 1083dig freeramdisk getty halt hexdump hwclock klogd modprobe ping ping6 pivot_root 1084poweroff readahead rev sfdisk sudo syslogd taskset telnet telnetd tracepath 1085traceroute unzip usleep vconfig zip free login modinfo unshare netcat help w 1086iwconfig iwlist rdate 1087dos2unix unix2dos catv clear 1088pmap realpath setsid timeout truncate 1089mkswap swapon swapoff 1090count oneit fstype 1091acpi blkid eject pwdx 1092sulogin rfkill bootchartd 1093arp makedevs sysctl killall5 crond crontab deluser last mkpasswd watch 1094blockdev rpm2cpio arping brctl dumpleases fsck 1095tcpsvd tftpd 1096factor fallocate fsfreeze inotifyd lspci nbd-client partprobe strings 1097base64 mix 1098reset hexedit nsenter shred 1099fsync insmod ionice lsmod lsusb rmmod vmstat xxd top iotop 1100lsof ionice compress dhcp dhcpd addgroup delgroup host iconv ip 1101ipcrm ipcs netstat openvt 1102deallocvt iorenice 1103udpsvd adduser 1104microcom tunctl chrt getfattr setfattr 1105kexec 1106ascii crc32 devmem fmt i2cdetect i2cdump i2cget i2cset mcookie prlimit sntp ulimit uuidgen dhcp6 ipaddr iplink iproute iprule iptunnel cd exit toysh bash traceroute6 1107blkdiscard rtcwake 1108watchdog 1109</span> 1110</b></blockquote> 1111 1112<hr /> 1113<a name=packages /> 1114<h2>Other packages</h2> 1115 1116<p>System administrators have <a href=https://github.com/landley/toybox/issues/168#issuecomment-583725500>asked</a> what other Linux packages toybox commands 1117replace, so they can annotate alternatives in their package management system.</p> 1118 1119<p>This section uses the package definitions from Chapter 6 of 1120<a href=http://www.linuxfromscratch.org/lfs/downloads/9.0/LFS-BOOK-9.0-NOCHUNKS.html>Linux From Scratch 9.0</a>). Each package lists what we currently 1121replace, pending commands [in square brackets], and what we DON'T plan to 1122implement.</p> 1123 1124<p>Each "see also" note means the listed package also installs the listed shared 1125libraries. (While toybox contains equivalent functionality to a lot of these 1126shared libraries in its lib/ directory, it does not currently provide a shared 1127library interface.)</p> 1128 1129<h3>Packages toybox plans to provide complete-ish replacents for:</h3> 1130<ul> 1131<li><b>file</b>: file (see also: libmagic)</li> 1132<li><b>m4</b>: [m4]</li> 1133<li><b>bc</b>: [bc] [dc]</li> 1134<li><b>bison</b>: [yacc] (not: bison, see also: liby)</li> 1135<li><b>flex</b>: [lex] (not: flex flex++, see also: libfl)</li> 1136<li><b>make</b>: [make]</li> 1137<li><b>sed</b>: sed</li> 1138<li><b>grep</b>: grep egrep fgrep</li> 1139<li><b>bash</b>: bash sh (not: bashbug)</li> 1140<li><b>diffutils</b>: cmp [diff] [diff3] [sdiff]</li> 1141<li><b>gawk</b>: [awk] (not: gawk gawk-5.0.1)</li> 1142<li><b>findutils</b>: find xargs (not: locate updatedb)</li> 1143<li><b>less</b>: less (not: lessecho lesskey)</li> 1144<li><b>gzip</b>: zcat [gzip] [gunzip] [zcmp] [zdiff] [zegrep] [zfgrep] [zgrep] [zless] [zmore] 1145(not: gzexe uncompress zforce znew)</li> 1146<li><b>make</b>: [make]</li> 1147<li><b>patch</b>: patch</li> 1148<li><b>tar</b>: tar</li> 1149<li><b>procps-ng</b>: free pgrep pidof pkill ps sysctl top uptime vmstat w watch 1150[pmap] [pwdx] [slabtop] 1151(not: tload, see also libprocps)</li> 1152<li><b>sysklogd</b>: [klogd] [syslogd]</li> 1153<li><b>sysvinit</b>: [init] halt poweroff reboot killall5 [shutdown] 1154(not telinit runlevel fstab-decode bootlogd)</li> 1155<li><b>man</b>: man (but not accessdb apropos catman lexgrog mandb manpath whatis, 1156see also libman libmandb)</li> 1157<li><b>vim</b>: vi xxd (but not ex, rview, rvim, view, vim, vimdiff, vimtutor)</li> 1158<li><b>sysvinit</b>: [init] halt poweroff reboot killall5 [shutdown] 1159(not telinit runlevel fstab-decode bootlogd)</li> 1160<li><b>kmod</b>: insmod lsmod rmmod modinfo [modprobe] 1161(not: depmod kmod)</li> 1162<li><b>attr</b>: [getfattr] setfattr (not: attr, see also: libattr)</li> 1163<li><b>shadow</b>: [chfn] [chpasswd] [chsh] [groupadd] [groupdel] [groupmod] 1164[newusers] passwd [su] [useradd] [userdel] [usermod] 1165[lastlog] [login] [newgidmap] [newuidmap] 1166(not: chage expiry faillog groupmems grpck logoutd newgrp nologin pwck sg 1167vigr vipw, grpconv grpunconv pwconv pwunconv, chgpasswd gpasswd)</li> 1168<li><b>psmisc</b>: killall [fuser] [pstree] [peekfd] [prtstat] 1169(not: pslog pstree.x11)</li> 1170<li><b>inetutils</b>: dnsdomainname [ftp] hostname ifconfig ping ping6 [telnet] [tftp] [traceroute] (not: talk)</li> 1171<li><b>coreutils</b>: [ base32 base64 basename cat chgrp chmod chown chroot cksum comm cp cut date 1172dd df dirname du echo env expand factor false fmt fold groups head hostid id install 1173link ln logname ls md5sum mkdir mkfifo mknod mktemp mv nice nl nohup nproc od 1174paste printenv printf pwd readlink realpath rm rmdir seq sha1sum shred 1175sleep sort split stat sync tac tail tee test timeout touch true truncate 1176tty uname uniq unlink wc who whoami yes 1177[expr] [fold] [join] [numfmt] [runcon] [sha224sum] [sha256sum] [sha384sum] 1178[sha512sum] [stty] [b2sum] [tr] [unexpand] 1179(not: basenc chcon csplit dir dircolors pathchk 1180pinky pr ptx shuf stdbuf sum tsort users vdir, see also libstdbuf)</li> 1181<li><b>util-linux</b>: blkid blockdev cal chrt dmesg eject fallocate flock hwclock 1182ionice kill logger losetup mcookie mkswap more mount mountpoint nsenter 1183pivot_root prlimit rename renice rev setsid swapoff swapon switch_root taskset 1184umount unshare uuidgen 1185[addpart] [fdisk] [findfs] [findmnt] [fsck] [fsfreeze] [fstrim] [getopt] 1186[hexdump] [linux32] [linux64] [lsblk] [lscpu] [lsns] [setarch] 1187(not: agetty blkdiscard blkzone cfdisk chcpu chmem choom col 1188colcrt colrm column ctrlaltdel delpart fdformat fincore fsck.cramfs 1189fsck.minix ipcmk ipcrm ipcs isosize last lastb ldattach look lsipc 1190lslocks lslogins lsmem mesg mkfs mkfs.bfs mkfs.cramfs mkfs.minix namei partx 1191raw readprofile resizepart rfkill rtcwake script scriptreplay 1192setterm sfdisk sulogin swaplabel ul 1193uname26 utmpdump uuidd uuidparse wall wdctl whereis wipefs 1194i386 x86_64 zramctl)</li> 1195</ul> 1196 1197<p>Commentary: toybox init doesn't do runlevels, man and vim are just the 1198relevant commands without the piles of strange overgrowth, and if you want 1199to call a toybox binary by another name you can create a symlink to a 1200symlink. If somebody really wants to argue for "gzexe" or similar, be 1201my guest, but there's a lot of obsolete crap in shadow, coreutils, 1202util-linux...</p> 1203 1204<p>No idea why LFS is installing inetutils instead of net-tools 1205(which contains arp route ifconfig mii-tool nameif netstat and rarp that 1206toybox does or might implement, and plipconfig slattach that it probably won't.)</p> 1207 1208<h3>Packages toybox plans to provide partial replacents for:</h3> 1209 1210<p>Toybox provides replacements for some binaries from these packages, 1211but there are other useful binaries which this package provides that toybox 1212currently considers out of scope for the project:</p> 1213 1214<ul> 1215<li><b>binutils</b>: strings [ar] [nm] [readelf] [size] [objcopy] [strip] 1216(not c++filt, dwp, elfedit, gprof. The following commands belong 1217in <a href=/code/qcc>qcc</a>: addr2line as ld objdump ranlib)</li> 1218<li><b>bzip2</b>: bunzip2 bzcat [bzcmp] [bzdiff] [bzegrep] [bzfgrep] [bzgrep] [bzless] 1219[bzmore] (not: bzip2, bzip2recover, see also libbz2)</li> 1220<li><b>xz</b>: [xzcat] [lzcat] [lzcmp] [lzdiff] [lzegrep] [lzfgrep] [lzgrep] 1221[lzless] [lzmadec, lzmainfo] [lzmore] [unlzma] [unxz] [xzcat] 1222[xzcmp] [xzdec] [xzdiff] [xzegrep] [xzfgrep] [xzgrep] [xzless] [xzmore] 1223(not: compression side, see also: liblzma)</li> 1224<li><b>ncurses</b>: clear reset (not: everything else, see also: libcurses)</li> 1225<li><b>e2fsprogs</b>: chattr lsattr [e2fsck] [mkfs.ext2] [mkfs.ext3] 1226[fsck.ext2] [fsck.ext3] [e2label] [resize2fs] [tune2fs] 1227(not badblocks compile_et debugfs dumpe2fse2freefrag e2image 1228e2mmpstatus e2scrub e2scrub_all e2undo e4crypt e4defrag filefrag 1229fsck.ext4 logsave mk_cmds mkfs.ext4 mklost+found)</li> 1230</ul> 1231 1232<p>Toybox provides several decompressors but compresses to a single format 1233(deflate, ala gzip/zlib). Our e2fsprogs doesn't currently plan to support 1234ext4 or defrag. The "qcc" reference is because someday an external project to glue 1235QEMU's <a href=https://git.qemu.org/?p=qemu.git;a=blob;f=tcg/README;h=bfa2e4ed246c;hb=HEAD>Tiny Code Generator</a> 1236to Fabrice Bellard's old <a href=http://landley.net/hg/tinycc>Tiny C Compiler</a> 1237making a multicall binary that does cc/ld/as for all the targets QEMU 1238supports (then use the 1239<a href=https://github.com/JuliaComputing/llvm-cbe>LLVM C Backend</a> 1240to compile LLVM itself to C for use as a modern replacement for 1241<a href=https://en.wikipedia.org/wiki/Cfront>cfront</a> to bootstrap 1242C++ code) is under consideration 1243as a successor project to toybox. Until then things like objdump -d 1244(requiring target-specific disassembly for an unbounded number of architectures) 1245are out of scope for toybox. (This means drawing the line somewhere between 1246architecture-specific support in file and strace, and including a full 1247assembler for each architecture.)</p> 1248</span> 1249 1250<h3>Packages from LFS ch6 toybox does NOT plan to replace:</h3> 1251 1252<ul> 1253<li><b>linux-api-headers</b></li> 1254<li><b>man-pages glibc</b></li> 1255<li><b>zlib</b></li> 1256<li><b>readline</b></li> 1257<li><b>gmp</b></li> 1258<li><b>mpfr</b></li> 1259<li><b>mpc</b></li> 1260<li><b>gcc</b></li> 1261<li><b>pkg-config</b></li> 1262<li><b>ncurses</b></li> 1263<li><b>acl</b></li> 1264<li><b>libcap</b></li> 1265<li><b>psmisc</b></li> 1266<li><b>iana-etc</b></li> 1267<li><b>libtool</b></li> 1268<li><b>gdbm</b></li> 1269<li><b>gperf</b></li> 1270<li><b>expat</b></li> 1271<li><b>perl</b></li> 1272<li><b>XML::Parser</b></li> 1273<li><b>intltool</b></li> 1274<li><b>autoconf</b></li> 1275<li><b>automake</b></li> 1276<li><b>gettext</b></li> 1277<li><b>libelf</b></li> 1278<li><b>libffi</b></li> 1279<li><b>openssl</b></li> 1280<li><b>python</b></li> 1281<li><b>ninja</b></li> 1282<li><b>meson</b></li> 1283<li><b>check</b></li> 1284<li><b>groff</b></li> 1285<li><b>grub</b></li> 1286<li><b>libpipeline</b></li> 1287<li><b>texinfo</b></li> 1288</ul> 1289 1290<p>That said, we do implement our own zlib and readline replacements, and 1291presumably _could_ export them as library bindings. Plus we provide 1292our own version of a bunch of the section 1 man pages (as command help). 1293Possibly libcap and acl are interesting?</p> 1294 1295<h3>Misc</h3> 1296 1297<p>The kbd package has over a dozen commands, we only implement chvt. The 1298iproute2 package implements over a dozen commands, there's an "ip" in 1299pending but I'm not a fan (ifconfig and route and such should be extended 1300to work properly). We don't implement eudev, but toybox's maintainer 1301created busybox mdev way back when (which replaces it) and plans to do a 1302new one for toybox as soon as we work out what subset is still needed now that 1303devtmpfs is available.</p> 1304 1305<!-- #include "footer.html" --> 1306 1307