1erofs-utils 2=========== 3 4Userspace tools for EROFS filesystem, currently including: 5 6 mkfs.erofs filesystem formatter 7 erofsfuse FUSE daemon alternative 8 dump.erofs filesystem analyzer 9 fsck.erofs filesystem compatibility & consistency checker as well 10 as extractor 11 12 13EROFS filesystem overview 14------------------------- 15 16EROFS filesystem stands for Enhanced Read-Only File System. It aims to 17form a generic read-only filesystem solution for various read-only use 18cases instead of just focusing on storage space saving without 19considering any side effects of runtime performance. 20 21Typically EROFS could be considered in the following use scenarios: 22 - Firmwares in performance-sensitive systems, such as system 23 partitions of Android smartphones; 24 25 - Mountable immutable images such as container images for effective 26 metadata & data access compared with tar, cpio or other local 27 filesystems (e.g. ext4, XFS, btrfs, etc.) 28 29 - FSDAX-enabled rootfs for secure containers (Linux 5.15+); 30 31 - Live CDs which need a set of files with another high-performance 32 algorithm to optimize startup time; others files for archival 33 purposes only are not needed; 34 35 - and more. 36 37Note that all EROFS metadata is uncompressed by design, so that you 38could take EROFS as a drop-in read-only replacement of ext4, XFS, 39btrfs, etc. without any compression-based dependencies and EROFS can 40bring more effective filesystem accesses to users with reduced 41metadata. 42 43For more details of EROFS filesystem itself, please refer to: 44https://www.kernel.org/doc/html/next/filesystems/erofs.html 45 46For more details on how to build erofs-utils, see `docs/INSTALL.md`. 47 48For more details about filesystem performance, see 49`docs/PERFORMANCE.md`. 50 51 52mkfs.erofs 53---------- 54 55Two main kinds of EROFS images can be generated: (un)compressed images. 56 57 - For uncompressed images, there will be none of compresssed files in 58 these images. However, it can decide whether the tail block of a 59 file should be inlined or not properly [1]. 60 61 - For compressed images, it'll try to use the given algorithms first 62 for each regular file and see if storage space can be saved with 63 compression. If not, fallback to an uncompressed file. 64 65How to generate EROFS images (LZ4 for Linux 5.3+, LZMA for Linux 5.16+) 66~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 67 68Currently lz4(hc) and lzma are available for compression, e.g. 69 $ mkfs.erofs -zlz4hc foo.erofs.img foo/ 70 71Or leave all files uncompressed as an option: 72 $ mkfs.erofs foo.erofs.img foo/ 73 74In addition, you could specify a higher compression level to get a 75(slightly) better compression ratio than the default level, e.g. 76 $ mkfs.erofs -zlz4hc,12 foo.erofs.img foo/ 77 78Note that all compressors are still single-threaded for now, thus it 79could take more time on the multiprocessor platform. Multi-threaded 80approach is already in our TODO list. 81 82How to generate EROFS big pcluster images (Linux 5.13+) 83~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 84 85In order to get much better compression ratios (thus better sequential 86read performance for common storage devices), big pluster feature has 87been introduced since linux-5.13, which is not forward-compatible with 88old kernels. 89 90In details, -C is used to specify the maximum size of each big pcluster 91in bytes, e.g. 92 $ mkfs.erofs -zlz4hc -C65536 foo.erofs.img foo/ 93 94So in that case, pcluster size can be 64KiB at most. 95 96Note that large pcluster size can cause bad random performance, so 97please evaluate carefully in advance. Or make your own per-(sub)file 98compression strategies according to file access patterns if needed. 99 100How to generate EROFS images with multiple algorithms (Linux 5.16+) 101~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 102 103It's possible to generate an EROFS image with files in different 104algorithms due to various purposes. For example, LZMA for archival 105purposes and LZ4 for runtime purposes. 106 107In order to use alternative algorithms, just specify two or more 108compressing configurations together separated by ':' like below: 109 -zlzma:lz4hc,12:lzma,9 -C32768 110 111Although mkfs still choose the first one by default, you could try to 112write a compress-hints file like below: 113 4096 1 .*\.so$ 114 32768 2 .*\.txt$ 115 4096 sbin/.*$ 116 16384 0 .* 117 118and specify with `--compress-hints=` so that ".so" files will use 119"lz4hc,12" compression with 4k pclusters, ".txt" files will use 120"lzma,9" compression with 32k pclusters, files under "/sbin" will use 121the default "lzma" compression with 4k plusters and other files will 122use "lzma" compression with 16k pclusters. 123 124Note that the largest pcluster size should be specified with the "-C" 125option (here 32k pcluster size), otherwise all larger pclusters will be 126limited. 127 128How to generate well-compressed EROFS images 129~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 130 131Even if EROFS is not designed for such purposes in the beginning, it 132could still produce some smaller images (not always) compared to other 133approaches with better performance (see `docs/PERFORMANCE.md`). In 134order to build well-compressed EROFS images, try the following options: 135 -C1048576 (5.13+) 136 -Eztailpacking (5.16+) 137 -Efragments / -Eall-fragments ( 6.1+); 138 -Ededupe ( 6.1+). 139 140Also EROFS uses lz4hc level 9 by default, whereas some other approaches 141use lz4hc level 12 by default. So please explicitly specify 142`-zlz4hc,12 ` for comparison purposes. 143 144How to generate legacy EROFS images (Linux 4.19+) 145~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 146 147Decompression inplace and compacted indexes have been introduced in 148Linux v5.3, which are not forward-compatible with older kernels. 149 150In order to generate _legacy_ EROFS images for old kernels, 151consider adding "-E legacy-compress" to the command line, e.g. 152 153 $ mkfs.erofs -E legacy-compress -zlz4hc foo.erofs.img foo/ 154 155For Linux kernel >= 5.3, legacy EROFS images are _NOT recommended_ 156due to runtime performance loss compared with non-legacy images. 157 158Obsoleted erofs.mkfs 159~~~~~~~~~~~~~~~~~~~~ 160 161There is an original erofs.mkfs version developed by Li Guifu, 162which was replaced by the new erofs-utils implementation. 163 164git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs-utils.git -b obsoleted_mkfs 165 166PLEASE NOTE: This version is highly _NOT recommended_ now. 167 168 169erofsfuse 170--------- 171 172erofsfuse is introduced to support EROFS format for various platforms 173(including older linux kernels) and new on-disk features iteration. 174It can also be used as an unpacking tool for unprivileged users. 175 176It supports fixed-sized output decompression *without* any in-place 177I/O or in-place decompression optimization. Also like the other FUSE 178implementations, it suffers from most common performance issues (e.g. 179significant I/O overhead, double caching, etc.) 180 181Therefore, NEVER use it if performance is the top concern. 182 183How to mount an EROFS image with erofsfuse 184~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 185 186As the other FUSE implementations, it's quite easy to mount by using 187erofsfuse, e.g.: 188 $ erofsfuse foo.erofs.img foo/ 189 190Alternatively, to make it run in foreground (with debugging level 3): 191 $ erofsfuse -f --dbglevel=3 foo.erofs.img foo/ 192 193To debug erofsfuse (also automatically run in foreground): 194 $ erofsfuse -d foo.erofs.img foo/ 195 196To unmount an erofsfuse mountpoint as a non-root user: 197 $ fusermount -u foo/ 198 199 200dump.erofs and fsck.erofs 201------------------------- 202 203dump.erofs and fsck.erofs are used to analyze, check, and extract 204EROFS filesystems. Note that extended attributes and ACLs are still 205unsupported when extracting images with fsck.erofs. 206 207Note that fragment extraction with fsck.erofs could be slow now and 208it needs to be optimized later. If you are interested, contribution 209is, as always, welcome. 210 211 212Contribution 213------------ 214 215erofs-utils is a part of EROFS filesystem project, which is completely 216community-driven open source software. If you have interest in EROFS, 217feel free to send feedback and/or patches to: 218 linux-erofs mailing list <linux-erofs@lists.ozlabs.org> 219 220 221Comments 222-------- 223 224[1] According to the EROFS on-disk format, the tail blocks of files 225 could be inlined aggressively with their metadata (called 226 tail-packing) in order to minimize the extra I/Os and the storage 227 space. 228