Lines Matching +full:write +full:- +full:0 +full:- +full:bps
1 .. _cgroup-v2:
11 conventions of cgroup v2. It describes all userland-visible aspects
14 v1 is available under :ref:`Documentation/admin-guide/cgroup-v1/index.rst <cgroup-v1>`.
19 1-1. Terminology
20 1-2. What is cgroup?
22 2-1. Mounting
23 2-2. Organizing Processes and Threads
24 2-2-1. Processes
25 2-2-2. Threads
26 2-3. [Un]populated Notification
27 2-4. Controlling Controllers
28 2-4-1. Enabling and Disabling
29 2-4-2. Top-down Constraint
30 2-4-3. No Internal Process Constraint
31 2-5. Delegation
32 2-5-1. Model of Delegation
33 2-5-2. Delegation Containment
34 2-6. Guidelines
35 2-6-1. Organize Once and Control
36 2-6-2. Avoid Name Collisions
38 3-1. Weights
39 3-2. Limits
40 3-3. Protections
41 3-4. Allocations
43 4-1. Format
44 4-2. Conventions
45 4-3. Core Interface Files
47 5-1. CPU
48 5-1-1. CPU Interface Files
49 5-2. Memory
50 5-2-1. Memory Interface Files
51 5-2-2. Usage Guidelines
52 5-2-3. Memory Ownership
53 5-3. IO
54 5-3-1. IO Interface Files
55 5-3-2. Writeback
56 5-3-3. IO Latency
57 5-3-3-1. How IO Latency Throttling Works
58 5-3-3-2. IO Latency Interface Files
59 5-3-4. IO Priority
60 5-4. PID
61 5-4-1. PID Interface Files
62 5-5. Cpuset
63 5.5-1. Cpuset Interface Files
64 5-6. Device
65 5-7. RDMA
66 5-7-1. RDMA Interface Files
67 5-8. HugeTLB
68 5.8-1. HugeTLB Interface Files
69 5-9. Misc
70 5.9-1 Miscellaneous cgroup Interface Files
71 5.9-2 Migration and Ownership
72 5-10. Others
73 5-10-1. perf_event
74 5-N. Non-normative information
75 5-N-1. CPU controller root cgroup process behaviour
76 5-N-2. IO controller root cgroup process behaviour
78 6-1. Basics
79 6-2. The Root and Views
80 6-3. Migration and setns(2)
81 6-4. Interaction with Other Namespaces
83 P-1. Filesystem Support for Writeback
86 R-1. Multiple Hierarchies
87 R-2. Thread Granularity
88 R-3. Competition Between Inner Nodes and Threads
89 R-4. Other Interface Issues
90 R-5. Controller Issues and Remedies
91 R-5-1. Memory
98 -----------
107 ---------------
113 cgroup is largely composed of two parts - the core and controllers.
129 hierarchical - if a controller is enabled on a cgroup, it affects all
131 sub-hierarchy of the cgroup. When a controller is enabled on a nested
141 --------
146 # mount -t cgroup2 none $MOUNT_POINT
148 cgroup2 filesystem has the magic number 0x63677270 ("cgrp"). All
156 is no longer referenced in its current hierarchy. Because per-cgroup
163 to inter-controller dependencies, other controllers may need to be
184 ignored on non-init namespace mounts. Please refer to the
201 option is ignored on non-init namespace mounts.
209 behavior but is a mount-option to avoid regressing setups
223 controller. The pre-allocated pool does not belong to anyone.
243 The option restores v1-like behavior of pids.events:max, that is only
251 --------------------------------
257 A child cgroup can be created by creating a sub-directory::
262 structure. Each cgroup has a read-writable interface file
264 belong to the cgroup one-per-line. The PIDs are not ordered and the
270 on a single write(2) call. If a process is composed of multiple
291 format "0::$PATH"::
295 0::/test-cgroup/test-cgroup-nested
302 0::/test-cgroup/test-cgroup-nested (deleted)
328 constraint - threaded controllers can be enabled on non-leaf cgroups
352 - As the cgroup will join the parent's resource domain. The parent
355 - When the parent is an unthreaded domain, it must not have any domain
359 Topology-wise, a cgroup can be in an invalid state. Please consider
362 A (threaded domain) - B (threaded) - C (domain, just created)
377 threads in the cgroup. Except that the operations are per-thread
378 instead of per-process, "cgroup.threads" has the same format and
400 between threads in a non-leaf cgroup and its child cgroups. Each
406 - cpu
407 - cpuset
408 - perf_event
409 - pids
412 --------------------------
414 Each non-root cgroup has a "cgroup.events" file which contains
415 "populated" field indicating whether the cgroup's sub-hierarchy has
416 live processes in it. Its value is 0 if there is no live process in
419 example, to start a clean-up operation after all processes of a given
420 sub-hierarchy have exited. The populated state updates and
421 notifications are recursive. Consider the following sub-hierarchy
425 A(4) - B(0) - C(1)
426 \ D(0)
428 A, B and C's "populated" fields would be 1 while D's 0. After the one
429 process in C exits, B and C's "populated" fields would flip to "0" and
435 -----------------------
449 # echo "+cpu +memory -io" > cgroup.subtree_control
458 Consider the following sub-hierarchy. The enabled controllers are
461 A(cpu,memory) - B(memory) - C()
475 controller interface files - anything which doesn't start with
479 Top-down Constraint
482 Resources are distributed top-down and a cgroup can further distribute
484 parent. This means that all non-root "cgroup.subtree_control" files
494 Non-root cgroups can distribute domain resources to their children
509 refer to the Non-normative information section in the Controllers
522 ----------
528 user by granting write access of the directory and its "cgroup.procs",
535 shouldn't be allowed to write to them. For the first method, this is
544 delegated, the user can build sub-hierarchy under the directory,
548 happens in the delegated sub-hierarchy, nothing can escape the
552 cgroups in or nesting depth of a delegated sub-hierarchy; however,
559 A delegated sub-hierarchy is contained in the sense that processes
560 can't be moved into or out of the sub-hierarchy by the delegatee.
563 requiring the following conditions for a process with a non-root euid
567 - The writer must have write access to the "cgroup.procs" file.
569 - The writer must have write access to the "cgroup.procs" file of the
573 processes around freely in the delegated sub-hierarchy it can't pull
574 in from or push out to outside the sub-hierarchy.
580 ~~~~~~~~~~~~~ - C0 - C00
583 ~~~~~~~~~~~~~ - C1 - C10
585 Let's also say U0 wants to write the PID of a process which is
586 currently in C10 into "C00/cgroup.procs". U0 has write access to the
589 not have write access to its "cgroup.procs" files and thus the write
590 will be denied with -EACCES.
595 is not reachable, the migration is rejected with -ENOENT.
599 ----------
607 inherent trade-offs between migration and various hot paths in terms
613 resource structure once on start-up. Dynamic adjustments to resource
646 -------
652 work-conserving. Due to the dynamic nature, this model is usually
667 .. _cgroupv2-limits-distributor:
670 ------
673 Limits can be over-committed - the sum of the limits of children can
676 Limits are in the range [0, max] and defaults to "max", which is noop.
678 As limits can be over-committed, all configuration combinations are
682 "io.max" limits the maximum BPS and/or IOPS that a cgroup can consume
685 .. _cgroupv2-protections-distributor:
688 -----------
693 soft boundaries. Protections can also be over-committed in which case
697 Protections are in the range [0, max] and defaults to 0, which is
700 As protections can be over-committed, all configuration combinations
704 "memory.low" implements best-effort memory protection and is an
709 -----------
712 resource. Allocations can't be over-committed - the sum of the
716 Allocations are in the range [0, max] and defaults to 0, which is no
719 As allocations can't be over-committed, some configuration
724 "cpu.rt.max" hard-allocates realtime slices and is an example of this
732 ------
737 New-line separated values
745 (when read-only or multiple values can be written at once)
771 -----------
773 - Settings for a single feature should be contained in a single file.
775 - The root cgroup should be exempt from resource control and thus
778 - The default time unit is microseconds. If a different unit is ever
781 - A parts-per quantity should use a percentage decimal with at least
782 two digit fractional part - e.g. 13.40.
784 - If a controller implements weight based resource distribution, its
790 - If a controller implements an absolute resource guarantee and/or
799 - If a setting has a configurable default value and keyed specific
813 # cat cgroup-example-interface-file
815 8:0 300
819 # echo 125 > cgroup-example-interface-file
823 # echo "default 125" > cgroup-example-interface-file
827 # echo "8:16 170" > cgroup-example-interface-file
831 # echo "8:0 default" > cgroup-example-interface-file
832 # cat cgroup-example-interface-file
836 - For events which are not very high frequency, an interface file
843 --------------------
848 A read-write single value file which exists on non-root
854 - "domain" : A normal valid domain cgroup.
856 - "domain threaded" : A threaded domain cgroup which is
859 - "domain invalid" : A cgroup which is in an invalid state.
863 - "threaded" : A threaded cgroup which is a member of a
870 A read-write new-line separated values file which exists on
874 the cgroup one-per-line. The PIDs are not ordered and the
883 - It must have write access to the "cgroup.procs" file.
885 - It must have write access to the "cgroup.procs" file of the
888 When delegating a sub-hierarchy, write access to this file
896 A read-write new-line separated values file which exists on
900 the cgroup one-per-line. The TIDs are not ordered and the
909 - It must have write access to the "cgroup.threads" file.
911 - The cgroup that the thread is currently in must be in the
914 - It must have write access to the "cgroup.procs" file of the
917 When delegating a sub-hierarchy, write access to this file
921 A read-only space separated values file which exists on all
928 A read-write space separated values file which exists on all
935 Space separated list of controllers prefixed with '+' or '-'
937 name prefixed with '+' enables the controller and '-'
943 A read-only flat-keyed file which exists on non-root cgroups.
950 processes; otherwise, 0.
952 1 if the cgroup is frozen; otherwise, 0.
955 A read-write single value files. The default is "max".
962 A read-write single value files. The default is "max".
969 A read-only flat-keyed file with the following entries:
995 A read-only flat-keyed file which exists in non-root cgroups.
1007 frozen 0 __/ \__
1013 A read-write single value file which exists on non-root cgroups.
1014 Allowed values are "0" and "1". The default is "0".
1036 create new sub-cgroups.
1039 A write-only single value file which exists in non-root cgroups.
1051 the whole thread-group.
1054 A read-write single value file that allowed values are "0" and "1".
1057 Writing "0" to the file will disable the cgroup PSI accounting.
1058 Writing "1" to the file will re-enable the cgroup PSI accounting.
1066 This may cause non-negligible overhead for some workloads when under
1068 be used to disable PSI accounting in the non-leaf cgroups.
1071 A read-write nested-keyed file.
1079 .. _cgroup-v2-cpu:
1082 ---
1113 A read-only flat-keyed file.
1118 - usage_usec
1119 - user_usec
1120 - system_usec
1124 - nr_periods
1125 - nr_throttled
1126 - throttled_usec
1127 - nr_bursts
1128 - burst_usec
1131 A read-write single value file which exists on non-root
1134 For non idle groups (cpu.idle = 0), the weight is in the
1138 then the weight will show as a 0.
1141 A read-write single value file which exists on non-root
1142 cgroups. The default is "0".
1144 The nice value is in the range [-20, 19].
1153 A read-write two value file which exists on non-root cgroups.
1165 A read-write single value file which exists on non-root
1166 cgroups. The default is "0".
1168 The burst in the range [0, $MAX].
1171 A read-write nested-keyed file.
1177 A read-write single value file which exists on non-root cgroups.
1178 The default is "0", i.e. no utilization boosting.
1192 A read-write single value file which exists on non-root cgroups.
1203 A read-write single value file which exists on non-root cgroups.
1204 The default is 0.
1206 This is the cgroup analog of the per-task SCHED_IDLE sched policy.
1215 ------
1223 While not completely water-tight, all major memory usages by a given
1228 - Userland memory - page cache and anonymous memory.
1230 - Kernel data structures such as dentries and inodes.
1232 - TCP socket buffers.
1245 A read-only single value file which exists on non-root
1252 A read-write single value file which exists on non-root
1253 cgroups. The default is "0".
1278 A read-write single value file which exists on non-root
1279 cgroups. The default is "0".
1281 Best-effort memory protection. If the memory usage of a
1301 A read-write single value file which exists on non-root
1315 A read-write single value file which exists on non-root
1324 In default configuration regular 0-order allocations always
1329 as -ENOMEM or silently ignore in cases like disk readahead.
1332 A write-only nested-keyed file which exists for all cgroups.
1343 specified amount, -EAGAIN is returned.
1364 A read-write single value file which exists on non-root cgroups.
1369 A write of any non-empty string to this file resets it to the
1374 A read-write single value file which exists on non-root
1375 cgroups. The default value is "0".
1384 Tasks with the OOM protection (oom_score_adj set to -1000)
1392 A read-only flat-keyed file which exists on non-root cgroups.
1406 boundary is over-committed.
1426 considered as an option, e.g. for failed high-order
1442 A read-only flat-keyed file which exists on non-root cgroups.
1445 types of memory, type-specific details, and other information
1454 If the entry has no per-node counter (or not show in the
1455 memory.numa_stat). We use 'npn' (non-per-node) as the tag
1483 Amount of memory used for storing per-cpu kernel
1493 Amount of cached filesystem data that is swap-backed,
1530 Amount of memory, swap-backed and filesystem-backed,
1536 the value for the foo counter, since the foo counter is type-based, not
1537 list-based.
1548 Amount of memory used for storing in-kernel data
1626 Number of zero-filled pages swapped out with I/O skipped due to the
1677 A read-only nested-keyed file which exists on non-root cgroups.
1680 types of memory, type-specific details, and other information
1693 type N0=<bytes in node 0> N1=<bytes in node 1> ...
1702 A read-only single value file which exists on non-root
1709 A read-write single value file which exists on non-root
1714 allow userspace to implement custom out-of-memory procedures.
1725 A read-write single value file which exists on non-root cgroups.
1730 A write of any non-empty string to this file resets it to the
1735 A read-write single value file which exists on non-root
1742 A read-only flat-keyed file which exists on non-root cgroups.
1758 because of running out of swap system-wide or max
1767 A read-only single value file which exists on non-root
1774 A read-write single value file which exists on non-root
1782 A read-write single value file. The default value is "1".
1787 When this is set to 0, all swapping attempts to swapping devices
1795 0, as it still allows for pages to be written to the zswap pool.
1797 is allowed unless memory.swap.max is set to 0.
1800 A read-only nested-keyed file.
1810 Over-committing on high limit (sum of high limits > available memory)
1824 pressure - how much the workload is being impacted due to lack of
1825 memory - is necessary to determine whether a workload needs more
1839 To which cgroup the area will be charged is in-deterministic; however,
1850 --
1855 only if cfq-iosched is in use and neither scheme is available for
1856 blk-mq devices.
1863 A read-only nested-keyed file.
1872 wios Number of write IOs
1879 8:16 rbytes=1459200 wbytes=314773504 rios=192 wios=353 dbytes=0 dios=0
1880 8:0 rbytes=90430464 wbytes=299008000 rios=8950 wios=1252 dbytes=50331648 dios=3021
1883 A read-write nested-keyed file which exists only on the root
1890 line for a given device is populated on the first write for
1895 enable Weight-based control enable
1897 rpct Read latency percentile [0, 100]
1899 wpct Write latency percentile [0, 100]
1900 wlat Write latency threshold
1917 latencies is above 75ms or write 150ms, and adjust the overall
1927 devices which show wide temporary behavior changes - e.g. a
1938 A read-write nested-keyed file which exists only on the root
1945 given device is populated on the first write for the device on
1951 model The cost model in use - "linear"
1963 [r|w]bps The maximum sequential IO throughput
1977 generate device-specific coefficients.
1980 A read-write flat-keyed file which exists on non-root cgroups.
1997 8:0 50
2000 A read-write nested-keyed file which exists on non-root
2003 BPS and IOPS based IO limit. Lines are keyed by $MAJ:$MIN
2009 wbps Max write bytes per second
2011 wiops Max write IO operations per second
2014 When writing, any number of nested key-value pairs can be
2019 BPS and IOPS are measured in each IO direction and IOs are
2022 Setting read limit at 2M BPS and write at 120 IOPS for 8:16::
2030 Write IOPS limit can be removed by writing the following::
2039 A read-only nested-keyed file.
2052 write IOs.
2058 writes out dirty pages for the memory domain. Both system-wide and
2059 per-cgroup dirty memory states are examined and the more restrictive
2070 inode is assigned to a cgroup and all IO requests to write dirty pages
2082 changes over time, use cases where multiple cgroups write to a single
2097 memory controller and system-wide clean memory.
2130 your real setting, setting at 10-15% higher than the value in io.stat.
2140 - Queue depth throttling. This is the number of outstanding IO's a group is
2144 - Artificial delay induction. There are certain types of IO that cannot be
2191 no-change
2194 promote-to-rt
2195 For requests that have a non-RT I/O priority class, change it into RT.
2199 restrict-to-be
2202 of these requests to 0. Do not modify the I/O priority class of
2209 none-to-rt
2210 Deprecated. Just an alias for promote-to-rt.
2214 +----------------+---+
2215 | no-change | 0 |
2216 +----------------+---+
2217 | promote-to-rt | 1 |
2218 +----------------+---+
2219 | restrict-to-be | 2 |
2220 +----------------+---+
2222 +----------------+---+
2226 +-------------------------------+---+
2227 | IOPRIO_CLASS_NONE | 0 |
2228 +-------------------------------+---+
2229 | IOPRIO_CLASS_RT (real-time) | 1 |
2230 +-------------------------------+---+
2232 +-------------------------------+---+
2234 +-------------------------------+---+
2238 - If I/O priority class policy is promote-to-rt, change the request I/O
2241 - If I/O priority class policy is not promote-to-rt, translate the I/O priority
2247 ---
2266 A read-write single value file which exists on non-root
2272 A read-only single value file which exists on non-root cgroups.
2278 A read-only single value file which exists on non-root cgroups.
2284 A read-only flat-keyed file which exists on non-root cgroups. Unless
2302 through fork() or clone(). These will return -EAGAIN if the creation
2307 ------
2314 memory placement to reduce cross-node memory access and contention
2325 A read-write multiple values file which exists on non-root
2326 cpuset-enabled cgroups.
2333 The CPU numbers are comma-separated numbers or ranges.
2337 0-4,6,8-10
2340 setting as the nearest cgroup ancestor with a non-empty
2347 A read-only multiple values file which exists on all
2348 cpuset-enabled cgroups.
2364 A read-write multiple values file which exists on non-root
2365 cpuset-enabled cgroups.
2372 The memory node numbers are comma-separated numbers or ranges.
2376 0-1,3
2379 setting as the nearest cgroup ancestor with a non-empty
2386 Setting a non-empty value to "cpuset.mems" causes memory of
2398 A read-only multiple values file which exists on all
2399 cpuset-enabled cgroups.
2414 A read-write multiple values file which exists on non-root
2415 cpuset-enabled cgroups.
2442 exclusivity rule will be rejected with a write error.
2448 A read-only multiple values file which exists on all non-root
2449 cpuset-enabled cgroups.
2461 A read-only and root cgroup only multiple values file.
2468 A read-write single value file which exists on non-root
2469 cpuset-enabled cgroups. This flag is owned by the parent cgroup
2475 "member" Non-root member of a partition
2480 A cpuset partition is a collection of cpuset-enabled cgroups with
2487 There are two types of partitions - local and remote. A local
2503 be changed. All other non-root cgroups start out as "member".
2516 two possible states - valid or invalid. An invalid partition
2527 "member" Non-root member of a partition
2554 A valid non-root parent partition may distribute out all its CPUs
2567 by write to "cpuset.cpus.partition", cpu hotplug or other
2573 A user can pre-configure certain CPUs to an isolated state
2580 -----------------
2591 on the return value the attempt will succeed or fail with -EPERM.
2595 access type (mknod/read/write) and device (type, major and minor numbers).
2596 If the program returns 0, the attempt fails with -EPERM, otherwise it
2604 ----
2613 A readwrite nested-keyed file that exists for all the cgroups
2634 A read-only file that describes current resource usage.
2643 -------
2660 A read-only flat-keyed file which exists on non-root cgroups.
2673 use hugetlb pages are included. The per-node values are in bytes.
2676 ----
2698 A read-only flat-keyed file shown only in the root cgroup. It shows
2707 A read-only flat-keyed file shown in the all cgroups. It shows
2712 res_b 0
2715 A read-only flat-keyed file shown in all cgroups. It shows the
2724 A read-write flat-keyed file shown in the non root cgroups. Allowed
2743 A read-only flat-keyed file which exists on non-root cgroups. The
2766 ------
2777 Non-normative information
2778 -------------------------
2794 appropriately so the neutral - nice 0 - value is 100 instead of 1024).
2810 ------
2827 0::/batchjobs/container_id1
2829 The path '/batchjobs/container_id1' can be considered as system-data
2834 # ls -l /proc/self/ns/cgroup
2835 lrwxrwxrwx 1 root root 0 2014-07-15 10:37 /proc/self/ns/cgroup -> cgroup:[4026531835]
2837 0::/batchjobs/container_id1
2841 # ls -l /proc/self/ns/cgroup
2842 lrwxrwxrwx 1 root root 0 2014-07-15 10:35 /proc/self/ns/cgroup -> cgroup:[4026532183]
2844 0::/
2846 When some thread from a multi-threaded process unshares its cgroup
2858 ------------------
2869 # ~/unshare -c # unshare cgroupns in some cgroup
2871 0::/
2873 # echo 0 > sub_cgrp_1/cgroup.procs
2875 0::/sub_cgrp_1
2877 Each process gets its namespace-specific view of "/proc/$PID/cgroup"
2887 0::/sub_cgrp_1
2893 0::/batchjobs/container_id1/sub_cgrp_1
2901 0::/../container_id2/sub_cgrp_1
2908 ----------------------
2917 0::/sub_cgrp_1
2920 0::/../container_id2
2937 ---------------------------------
2940 running inside a non-init cgroup namespace::
2942 # mount -t cgroup2 none $MOUNT_POINT
2949 the view of cgroup hierarchy by namespace-private cgroupfs mount
2962 --------------------------------
2965 address_space_operations->writepage[s]() to annotate bio's using the
2982 super_block by setting SB_I_CGROUPWB in ->s_iflags. This allows for
2999 - Multiple hierarchies including named ones are not supported.
3001 - All v1 mount options are not supported.
3003 - The "tasks" file is removed and "cgroup.procs" is not sorted.
3005 - "cgroup.clone_children" is removed.
3007 - /proc/cgroups is meaningless for v2. Use "cgroup.controllers" or
3015 --------------------
3068 ------------------
3076 Generally, in-process knowledge is available only to the process
3077 itself; thus, unlike service-level organization of processes,
3084 sub-hierarchies and control resource distributions along them. This
3085 effectively raised cgroup to the status of a syscall-like API exposed
3092 and then read and/or write to it. This is not only extremely clunky
3095 that the process would actually be operating on its own sub-hierarchy.
3099 system-management pseudo filesystem. cgroup ended up with interface
3102 individual applications through the ill-defined delegation mechanism
3112 -------------------------------------------
3123 cycles and the number of internal threads fluctuated - the ratios
3139 clearly defined. There were attempts to add ad-hoc behaviors and
3153 ----------------------
3157 was how an empty cgroup was notified - a userland helper binary was
3160 to in-kernel event delivery filtering mechanism further complicating
3182 ------------------------------
3189 global reclaim prefers is opt-in, rather than opt-out. The costs for
3199 becomes self-defeating.
3201 The memory.low boundary on the other hand is a top-down allocated
3239 new limit is met - or the task writing to memory.max is killed.
3248 groups can sabotage swapping by other means - such as referencing its
3249 anonymous memory in a tight loop - and an admin can not assume full