{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Cgroups\n",
"\n",
"**cgroups** (abbreviated from control groups) is a Linux kernel feature that limits, accounts for, and isolates the resource usage (CPU, memory, disk I/O, network, etc.) of a collection of processes.\n",
"\n",
"A control group is a collection of processes that are bound by the same criteria and associated with a set of parameters or limits. These groups can be hierarchical, meaning that each group inherits limits from its parent group. The kernel provides access to multiple controllers (also called subsystems) through the cgroup interface, for example, the \"memory\" controller limits memory use, \"cpuacct\" accounts CPU usage, etc."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2016-12-08 11:42:27,154 INFO : root : Using LISA logging configuration:\n",
"2016-12-08 11:42:27,155 INFO : root : /home/vagrant/lisa/logging.conf\n"
]
}
],
"source": [
"import logging\n",
"from conf import LisaLogging\n",
"LisaLogging.setup()"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"import os\n",
"import json\n",
"import operator\n",
"\n",
"import devlib\n",
"import trappy\n",
"import bart\n",
"\n",
"from bart.sched.SchedMultiAssert import SchedMultiAssert\n",
"from wlgen import RTA, Periodic"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Target Configuration\n",
"The target configuration is used to describe and configure your test environment.\n",
"You can find more details in **examples/utils/testenv_example.ipynb**."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2016-12-08 11:42:29,845 INFO : TestEnv : Using base path: /home/vagrant/lisa\n",
"2016-12-08 11:42:29,845 INFO : TestEnv : Loading custom (inline) target configuration\n",
"2016-12-08 11:42:29,845 INFO : TestEnv : External tools using:\n",
"2016-12-08 11:42:29,846 INFO : TestEnv : ANDROID_HOME: /home/vagrant/lisa/tools/android-sdk-linux/\n",
"2016-12-08 11:42:29,846 INFO : TestEnv : CATAPULT_HOME: /home/vagrant/lisa/tools/catapult\n",
"2016-12-08 11:42:29,847 INFO : TestEnv : Loading board:\n",
"2016-12-08 11:42:29,847 INFO : TestEnv : /home/vagrant/lisa/libs/utils/platforms/pixel.json\n",
"2016-12-08 11:42:29,848 INFO : TestEnv : Devlib modules to load: [u'bl', u'cpufreq', 'cgroups']\n",
"2016-12-08 11:42:29,848 INFO : TestEnv : Connecting Android target [HT6670300102]\n",
"2016-12-08 11:42:29,848 INFO : TestEnv : Connection settings:\n",
"2016-12-08 11:42:29,849 INFO : TestEnv : {'device': 'HT6670300102'}\n",
"2016-12-08 11:42:30,008 INFO : android : ls command is set to ls -1\n",
"2016-12-08 11:42:31,253 INFO : TestEnv : Initializing target workdir:\n",
"2016-12-08 11:42:31,256 INFO : TestEnv : /data/local/tmp/devlib-target\n",
"2016-12-08 11:42:38,346 INFO : CGroups : Available controllers:\n",
"2016-12-08 11:42:39,117 INFO : CGroups : cpuset : /data/local/tmp/devlib-target/cgroups/devlib_cgh4\n",
"2016-12-08 11:42:39,840 INFO : CGroups : cpu : /data/local/tmp/devlib-target/cgroups/devlib_cgh3\n",
"2016-12-08 11:42:40,638 INFO : CGroups : cpuacct : /data/local/tmp/devlib-target/cgroups/devlib_cgh1\n",
"2016-12-08 11:42:41,416 INFO : CGroups : schedtune : /data/local/tmp/devlib-target/cgroups/devlib_cgh2\n",
"2016-12-08 11:42:42,169 INFO : CGroups : freezer : /data/local/tmp/devlib-target/cgroups/devlib_cgh0\n",
"2016-12-08 11:42:42,287 INFO : TestEnv : Topology:\n",
"2016-12-08 11:42:42,288 INFO : TestEnv : [[0, 1], [2, 3]]\n",
"2016-12-08 11:42:42,691 INFO : TestEnv : Loading default EM:\n",
"2016-12-08 11:42:42,693 INFO : TestEnv : /home/vagrant/lisa/libs/utils/platforms/pixel.json\n",
"2016-12-08 11:42:44,021 INFO : TestEnv : Enabled tracepoints:\n",
"2016-12-08 11:42:44,022 INFO : TestEnv : sched_switch\n",
"2016-12-08 11:42:44,022 INFO : TestEnv : Calibrating RTApp...\n",
"2016-12-08 11:42:44,259 INFO : RTApp : CPU0 calibration...\n",
"2016-12-08 11:42:44,328 INFO : Workload : Setup new workload rta_calib\n",
"2016-12-08 11:42:44,329 INFO : Workload : Workload duration defined by longest task\n",
"2016-12-08 11:42:44,330 INFO : Workload : Default policy: SCHED_OTHER\n",
"2016-12-08 11:42:44,330 INFO : Workload : ------------------------\n",
"2016-12-08 11:42:44,331 INFO : Workload : task [task1], sched: {'policy': 'FIFO', 'prio': 0}\n",
"2016-12-08 11:42:44,331 INFO : Workload : | calibration CPU: 0\n",
"2016-12-08 11:42:44,332 INFO : Workload : | loops count: 1\n",
"2016-12-08 11:42:44,334 INFO : Workload : + phase_000001: duration 1.000000 [s] (10 loops)\n",
"2016-12-08 11:42:44,335 INFO : Workload : | period 100000 [us], duty_cycle 50 %\n",
"2016-12-08 11:42:44,335 INFO : Workload : | run_time 50000 [us], sleep_time 50000 [us]\n",
"2016-12-08 11:42:44,466 INFO : Workload : Workload execution START:\n",
"2016-12-08 11:42:44,467 INFO : Workload : /data/local/tmp/bin/taskset 0x1 /data/local/tmp/bin/rt-app /data/local/tmp/devlib-target/rta_calib_00.json 2>&1\n",
"2016-12-08 11:42:46,114 INFO : RTApp : CPU1 calibration...\n",
"2016-12-08 11:42:46,183 INFO : Workload : Setup new workload rta_calib\n",
"2016-12-08 11:42:46,183 INFO : Workload : Workload duration defined by longest task\n",
"2016-12-08 11:42:46,184 INFO : Workload : Default policy: SCHED_OTHER\n",
"2016-12-08 11:42:46,184 INFO : Workload : ------------------------\n",
"2016-12-08 11:42:46,185 INFO : Workload : task [task1], sched: {'policy': 'FIFO', 'prio': 0}\n",
"2016-12-08 11:42:46,185 INFO : Workload : | calibration CPU: 1\n",
"2016-12-08 11:42:46,185 INFO : Workload : | loops count: 1\n",
"2016-12-08 11:42:46,186 INFO : Workload : + phase_000001: duration 1.000000 [s] (10 loops)\n",
"2016-12-08 11:42:46,186 INFO : Workload : | period 100000 [us], duty_cycle 50 %\n",
"2016-12-08 11:42:46,186 INFO : Workload : | run_time 50000 [us], sleep_time 50000 [us]\n",
"2016-12-08 11:42:46,320 INFO : Workload : Workload execution START:\n",
"2016-12-08 11:42:46,322 INFO : Workload : /data/local/tmp/bin/taskset 0x2 /data/local/tmp/bin/rt-app /data/local/tmp/devlib-target/rta_calib_00.json 2>&1\n",
"2016-12-08 11:42:48,012 INFO : RTApp : CPU2 calibration...\n",
"2016-12-08 11:42:48,084 INFO : Workload : Setup new workload rta_calib\n",
"2016-12-08 11:42:48,085 INFO : Workload : Workload duration defined by longest task\n",
"2016-12-08 11:42:48,086 INFO : Workload : Default policy: SCHED_OTHER\n",
"2016-12-08 11:42:48,086 INFO : Workload : ------------------------\n",
"2016-12-08 11:42:48,087 INFO : Workload : task [task1], sched: {'policy': 'FIFO', 'prio': 0}\n",
"2016-12-08 11:42:48,087 INFO : Workload : | calibration CPU: 2\n",
"2016-12-08 11:42:48,087 INFO : Workload : | loops count: 1\n",
"2016-12-08 11:42:48,088 INFO : Workload : + phase_000001: duration 1.000000 [s] (10 loops)\n",
"2016-12-08 11:42:48,088 INFO : Workload : | period 100000 [us], duty_cycle 50 %\n",
"2016-12-08 11:42:48,088 INFO : Workload : | run_time 50000 [us], sleep_time 50000 [us]\n",
"2016-12-08 11:42:48,220 INFO : Workload : Workload execution START:\n",
"2016-12-08 11:42:48,221 INFO : Workload : /data/local/tmp/bin/taskset 0x4 /data/local/tmp/bin/rt-app /data/local/tmp/devlib-target/rta_calib_00.json 2>&1\n",
"2016-12-08 11:42:49,900 INFO : RTApp : CPU3 calibration...\n",
"2016-12-08 11:42:49,968 INFO : Workload : Setup new workload rta_calib\n",
"2016-12-08 11:42:49,969 INFO : Workload : Workload duration defined by longest task\n",
"2016-12-08 11:42:49,969 INFO : Workload : Default policy: SCHED_OTHER\n",
"2016-12-08 11:42:49,969 INFO : Workload : ------------------------\n",
"2016-12-08 11:42:49,970 INFO : Workload : task [task1], sched: {'policy': 'FIFO', 'prio': 0}\n",
"2016-12-08 11:42:49,970 INFO : Workload : | calibration CPU: 3\n",
"2016-12-08 11:42:49,970 INFO : Workload : | loops count: 1\n",
"2016-12-08 11:42:49,971 INFO : Workload : + phase_000001: duration 1.000000 [s] (10 loops)\n",
"2016-12-08 11:42:49,971 INFO : Workload : | period 100000 [us], duty_cycle 50 %\n",
"2016-12-08 11:42:49,971 INFO : Workload : | run_time 50000 [us], sleep_time 50000 [us]\n",
"2016-12-08 11:42:50,103 INFO : Workload : Workload execution START:\n",
"2016-12-08 11:42:50,104 INFO : Workload : /data/local/tmp/bin/taskset 0x8 /data/local/tmp/bin/rt-app /data/local/tmp/devlib-target/rta_calib_00.json 2>&1\n",
"2016-12-08 11:42:51,757 INFO : RTApp : Target RT-App calibration:\n",
"2016-12-08 11:42:51,759 INFO : RTApp : {\"0\": 106, \"1\": 104, \"2\": 78, \"3\": 78}\n",
"2016-12-08 11:42:51,882 INFO : RTApp : big cores are ~36% more capable than LITTLE cores\n",
"2016-12-08 11:42:51,884 INFO : TestEnv : Using RT-App calibration values:\n",
"2016-12-08 11:42:51,886 INFO : TestEnv : {\"0\": 106, \"1\": 104, \"2\": 78, \"3\": 78}\n",
"2016-12-08 11:42:51,888 INFO : TestEnv : Set results folder to:\n",
"2016-12-08 11:42:51,889 INFO : TestEnv : /home/vagrant/lisa/results/20161208_114251\n",
"2016-12-08 11:42:51,891 INFO : TestEnv : Experiment results available also in:\n",
"2016-12-08 11:42:51,893 INFO : TestEnv : /home/vagrant/lisa/results_latest\n",
"2016-12-08 11:42:51,895 INFO : root : Connected to arm64 target\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"DONE\n"
]
}
],
"source": [
"from env import TestEnv\n",
"\n",
"my_conf = {\n",
"\n",
" # Android Pixel\n",
" \"platform\" : \"android\",\n",
" \"board\" : \"pixel\",\n",
" \n",
" \"device\" : \"HT6670300102\",\n",
" \"ANDROID_HOME\" : \"/home/vagrant/lisa/tools/android-sdk-linux/\",\n",
" \n",
" \"exclude_modules\" : [ \"hwmon\" ],\n",
"\n",
" # List of additional devlib modules to install \n",
" \"modules\" : ['cgroups', 'bl', 'cpufreq'],\n",
" \n",
" # List of additional binary tools to install\n",
" \"tools\" : ['rt-app', 'trace-cmd'],\n",
" \n",
" # FTrace events to collect\n",
" \"ftrace\" : {\n",
" \"events\" : [\n",
" \"sched_switch\"\n",
" ],\n",
" \"buffsize\" : 10240\n",
" }\n",
"}\n",
"\n",
"te = TestEnv(my_conf, force_new=True)\n",
"target = te.target\n",
"\n",
"# Report target connection\n",
"logging.info('Connected to %s target', target.abi)\n",
"print \"DONE\""
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## List available Controllers\n",
"\n",
"Details on the available controllers (or subsystems) can be found at: https://www.kernel.org/doc/Documentation/cgroup-v1/."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false,
"scrolled": true
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2016-12-08 11:42:55,652 INFO : root : CGroup - Available controllers:\n",
"2016-12-08 11:42:55,715 INFO : root : CGroup - cpuset (hierarchy id: 4) has 7 cgroups\n",
"2016-12-08 11:42:55,717 INFO : root : CGroup - cpu (hierarchy id: 3) has 2 cgroups\n",
"2016-12-08 11:42:55,718 INFO : root : CGroup - cpuacct (hierarchy id: 1) has 87 cgroups\n",
"2016-12-08 11:42:55,718 INFO : root : CGroup - schedtune (hierarchy id: 2) has 4 cgroups\n",
"2016-12-08 11:42:55,719 INFO : root : CGroup - freezer (hierarchy id: 5) has 1 cgroups\n"
]
}
],
"source": [
"logging.info('%14s - Available controllers:', 'CGroup')\n",
"ssys = target.cgroups.list_subsystems()\n",
"for (n,h,g,e) in ssys:\n",
" logging.info('%14s - %10s (hierarchy id: %d) has %d cgroups',\n",
" 'CGroup', n, h, g)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example of CPUSET controller usage\n",
"\n",
"Cpusets provide a mechanism for assigning a set of CPUs and memory nodes to a set of tasks. Cpusets constrain the CPU and memory placement of tasks to only the resources available within a task's current cpuset. They form a nested hierarchy visible in a virtual file system. These are the essential hooks, beyond what is already present, required to manage dynamic job placement on large systems.\n",
"\n",
"More information can be found in the kernel documentation: https://www.kernel.org/doc/Documentation/cgroup-v1/cpusets.txt."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Get a reference to the CPUSet controller\n",
"cpuset = target.cgroups.controller('cpuset')"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2016-12-08 11:42:58,914 INFO : root : Existing CGropups:\n",
"2016-12-08 11:42:58,915 INFO : root : /\n",
"2016-12-08 11:42:58,916 INFO : root : /system-background\n",
"2016-12-08 11:42:58,917 INFO : root : /background\n",
"2016-12-08 11:42:58,918 INFO : root : /foreground\n",
"2016-12-08 11:42:58,918 INFO : root : /foreground/boost\n",
"2016-12-08 11:42:58,920 INFO : root : /top-app\n",
"2016-12-08 11:42:58,921 INFO : root : /camera-daemon\n"
]
}
],
"source": [
"# Get the list of current configured CGroups for that controller\n",
"cgroups = cpuset.list_all()\n",
"logging.info('Existing CGropups:')\n",
"for cg in cgroups:\n",
" logging.info(' %s', cg)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2016-12-08 11:43:01,858 INFO : root : cpuset:/ cpus: 0-3\n",
"2016-12-08 11:43:02,054 INFO : root : cpuset:/system-background cpus: 0-2\n",
"2016-12-08 11:43:02,255 INFO : root : cpuset:/background cpus: 0\n",
"2016-12-08 11:43:02,450 INFO : root : cpuset:/foreground cpus: 0-2\n",
"2016-12-08 11:43:02,649 INFO : root : cpuset:/foreground/boost cpus: 0-2\n",
"2016-12-08 11:43:02,855 INFO : root : cpuset:/top-app cpus: 0-3\n",
"2016-12-08 11:43:03,053 INFO : root : cpuset:/camera-daemon cpus: 0-3\n"
]
}
],
"source": [
"# Dump the configuraiton of each controller\n",
"for cgname in cgroups:\n",
" #print cgname\n",
" cgroup = cpuset.cgroup(cgname)\n",
" attrs = cgroup.get()\n",
" #print attrs\n",
" cpus = attrs['cpus']\n",
" logging.info('%s:%-15s cpus: %s', cpuset.kind, cgroup.name, cpus)\n",
" "
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Create a LITTLE partition\n",
"cpuset_littles = cpuset.cgroup('/LITTLE')"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"LITTLE:\n",
"{\n",
" \"memory_pressure\": \"0\", \n",
" \"memory_spread_page\": \"0\", \n",
" \"notify_on_release\": \"0\", \n",
" \"sched_load_balance\": \"1\", \n",
" \"cpus\": \"\", \n",
" \"effective_mems\": \"\", \n",
" \"memory_spread_slab\": \"0\", \n",
" \"mem_hardwall\": \"0\", \n",
" \"cpu_exclusive\": \"0\", \n",
" \"mem_exclusive\": \"0\", \n",
" \"ls\": \" /data/local/tmp/devlib-target/cgroups/devlib_cgh4/LITTLE/cpuset.*\", \n",
" \"mems\": \"\", \n",
" \"memory_migrate\": \"0\", \n",
" \"sched_relax_domain_level\": \"-1\", \n",
" \"effective_cpus\": \"\"\n",
"}\n"
]
}
],
"source": [
"# Check the attributes available for this control group\n",
"print \"LITTLE:\\n\", json.dumps(cpuset_littles.get(), indent=4)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"LITTLE:\n",
"{\n",
" \"memory_pressure\": \"0\", \n",
" \"memory_spread_page\": \"0\", \n",
" \"notify_on_release\": \"0\", \n",
" \"sched_load_balance\": \"1\", \n",
" \"cpus\": \"0-1\", \n",
" \"effective_mems\": \"0\", \n",
" \"memory_spread_slab\": \"0\", \n",
" \"mem_hardwall\": \"0\", \n",
" \"cpu_exclusive\": \"0\", \n",
" \"mem_exclusive\": \"0\", \n",
" \"ls\": \" /data/local/tmp/devlib-target/cgroups/devlib_cgh4/LITTLE/cpuset.*\", \n",
" \"mems\": \"0\", \n",
" \"memory_migrate\": \"0\", \n",
" \"sched_relax_domain_level\": \"-1\", \n",
" \"effective_cpus\": \"0-1\"\n",
"}\n"
]
}
],
"source": [
"# Tune CPUs and MEMs attributes\n",
"# they must be initialize for the group to be usable\n",
"cpuset_littles.set(cpus=target.bl.littles, mems=0)\n",
"print \"LITTLE:\\n\", json.dumps(cpuset_littles.get(), indent=4)"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2016-12-08 11:43:10,335 INFO : Workload : Setup new workload simple\n",
"2016-12-08 11:43:10,337 INFO : Workload : Workload duration defined by longest task\n",
"2016-12-08 11:43:10,338 INFO : Workload : Default policy: SCHED_OTHER\n",
"2016-12-08 11:43:10,340 INFO : Workload : ------------------------\n",
"2016-12-08 11:43:10,341 INFO : Workload : task [task0], sched: using default policy\n",
"2016-12-08 11:43:10,342 INFO : Workload : | calibration CPU: 2\n",
"2016-12-08 11:43:10,343 INFO : Workload : | loops count: 1\n",
"2016-12-08 11:43:10,343 INFO : Workload : + phase_000001: duration 5.000000 [s] (50 loops)\n",
"2016-12-08 11:43:10,344 INFO : Workload : | period 100000 [us], duty_cycle 80 %\n",
"2016-12-08 11:43:10,344 INFO : Workload : | run_time 80000 [us], sleep_time 20000 [us]\n",
"2016-12-08 11:43:10,344 INFO : Workload : ------------------------\n",
"2016-12-08 11:43:10,345 INFO : Workload : task [task1], sched: using default policy\n",
"2016-12-08 11:43:10,345 INFO : Workload : | calibration CPU: 2\n",
"2016-12-08 11:43:10,345 INFO : Workload : | loops count: 1\n",
"2016-12-08 11:43:10,346 INFO : Workload : + phase_000001: duration 5.000000 [s] (50 loops)\n",
"2016-12-08 11:43:10,346 INFO : Workload : | period 100000 [us], duty_cycle 80 %\n",
"2016-12-08 11:43:10,346 INFO : Workload : | run_time 80000 [us], sleep_time 20000 [us]\n",
"2016-12-08 11:43:10,347 INFO : Workload : ------------------------\n",
"2016-12-08 11:43:10,347 INFO : Workload : task [task2], sched: using default policy\n",
"2016-12-08 11:43:10,348 INFO : Workload : | calibration CPU: 2\n",
"2016-12-08 11:43:10,348 INFO : Workload : | loops count: 1\n",
"2016-12-08 11:43:10,348 INFO : Workload : + phase_000001: duration 5.000000 [s] (50 loops)\n",
"2016-12-08 11:43:10,349 INFO : Workload : | period 100000 [us], duty_cycle 80 %\n",
"2016-12-08 11:43:10,349 INFO : Workload : | run_time 80000 [us], sleep_time 20000 [us]\n",
"2016-12-08 11:43:10,349 INFO : Workload : ------------------------\n",
"2016-12-08 11:43:10,350 INFO : Workload : task [task3], sched: using default policy\n",
"2016-12-08 11:43:10,350 INFO : Workload : | calibration CPU: 2\n",
"2016-12-08 11:43:10,350 INFO : Workload : | loops count: 1\n",
"2016-12-08 11:43:10,351 INFO : Workload : + phase_000001: duration 5.000000 [s] (50 loops)\n",
"2016-12-08 11:43:10,351 INFO : Workload : | period 100000 [us], duty_cycle 80 %\n",
"2016-12-08 11:43:10,351 INFO : Workload : | run_time 80000 [us], sleep_time 20000 [us]\n"
]
}
],
"source": [
"# Define a periodic big (80%) task\n",
"task = Periodic(\n",
" period_ms=100,\n",
" duty_cycle_pct=80,\n",
" duration_s=5).get()\n",
"\n",
"# Create one task per each CPU in the target\n",
"tasks={}\n",
"for tid in enumerate(target.core_names):\n",
" tasks['task{}'.format(tid[0])] = task\n",
"\n",
"# Configure RTA to run all these tasks\n",
"rtapp = RTA(target, 'simple', calibration=te.calibration())\n",
"rtapp.conf(kind='profile', params=tasks, run_dir=target.working_directory);"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2016-12-08 11:43:13,598 INFO : Workload : Workload execution START:\n",
"2016-12-08 11:43:13,601 INFO : Workload : /data/local/tmp/bin/shutils cgroups_run_into /LITTLE /data/local/tmp/bin/rt-app /data/local/tmp/devlib-target/simple_00.json 2>&1\n",
"2016-12-08 11:43:25,156 INFO : Workload : Pulling trace file into [/home/vagrant/lisa/results/20161208_114251/simple_00.dat]...\n"
]
}
],
"source": [
"# Test execution of all these tasks into the LITTLE cluster\n",
"trace = rtapp.run(ftrace=te.ftrace, cgroup=cpuset_littles.name, out_dir=te.res_dir)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false,
"scrolled": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"
"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Check tasks residency on little clsuter\n",
"trappy.plotter.plot_trace(trace)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"4659\": {\n",
" \"residency\": 100.0, \n",
" \"task_name\": \"rt-app\"\n",
" }, \n",
" \"4660\": {\n",
" \"residency\": 100.0, \n",
" \"task_name\": \"rt-app\"\n",
" }, \n",
" \"4661\": {\n",
" \"residency\": 100.0, \n",
" \"task_name\": \"rt-app\"\n",
" }, \n",
" \"4662\": {\n",
" \"residency\": 100.0, \n",
" \"task_name\": \"rt-app\"\n",
" }\n",
"}\n"
]
}
],
"source": [
"# Compute and visualize tasks residencies on LITTLE clusterh CPUs\n",
"s = SchedMultiAssert(trappy.FTrace(trace), te.topology, execnames=tasks.keys())\n",
"residencies = s.getResidency('cluster', target.bl.littles, percent=True)\n",
"print json.dumps(residencies, indent=4)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Assert that ALL tasks have always executed only on LITTLE cluster\n",
"s.assertResidency('cluster', target.bl.littles,\n",
" 99.9, operator.ge, percent=True, rank=len(residencies))"
]
},
{
"cell_type": "markdown",
"metadata": {
"collapsed": true
},
"source": [
"## Example of CPU controller usage\n",
"\n",
"While the CPUSET is a controller to assign CPUs and memory nodes for a set of tasks, the CPU controller is used to assign CPU bandwidth."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": true
},
"outputs": [],
"source": [
"# Get a reference to the CPU controller\n",
"cpu = target.cgroups.controller('cpu')"
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Create a big partition on that CPUS\n",
"cpu_littles = cpu.cgroup('/LITTLE')"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"LITTLE:\n",
"{\n",
" \"rt_period_us\": \"1000000\", \n",
" \"shares\": \"1024\", \n",
" \"rt_runtime_us\": \"0\"\n",
"}\n"
]
}
],
"source": [
"# Check the attributes available for this control group\n",
"print \"LITTLE:\\n\", json.dumps(cpu_littles.get(), indent=4)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"LITTLE:\n",
"{\n",
" \"rt_period_us\": \"1000000\", \n",
" \"shares\": \"512\", \n",
" \"rt_runtime_us\": \"0\"\n",
"}\n"
]
}
],
"source": [
"# Set a 1CPU equivalent bandwidth for that CGroup\n",
"# cpu_littles.set(cfs_period_us=100000, cfs_quota_us=50000)\n",
"cpu_littles.set(shares=512)\n",
"print \"LITTLE:\\n\", json.dumps(cpu_littles.get(), indent=4)"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2016-12-08 11:44:14,920 INFO : Workload : Workload execution START:\n",
"2016-12-08 11:44:14,921 INFO : Workload : /data/local/tmp/bin/shutils cgroups_run_into /LITTLE /data/local/tmp/bin/rt-app /data/local/tmp/devlib-target/simple_00.json 2>&1\n",
"2016-12-08 11:44:26,513 INFO : Workload : Pulling trace file into [.//simple_00.dat]...\n"
]
}
],
"source": [
"# Test execution of all these tasks into the LITTLE cluster\n",
"trace = rtapp.run(ftrace=te.ftrace, cgroup=cpu_littles.name)"
]
},
{
"cell_type": "code",
"execution_count": 21,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"
"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Check tasks residency on little cluster\n",
"trappy.plotter.plot_trace(trace)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example of CPUs isolation"
]
},
{
"cell_type": "code",
"execution_count": 22,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"# Isolate CPU0\n",
"\n",
"# This works by moving all user-space tasks into a cpuset\n",
"# which does not include the specified list of CPUs to be\n",
"# isolated.\n",
"sandbox, isolated = target.cgroups.isolate(cpus=[0])"
]
},
{
"cell_type": "code",
"execution_count": 23,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Sandbox:\n",
"{\n",
" \"memory_pressure\": \"0\", \n",
" \"memory_spread_page\": \"0\", \n",
" \"notify_on_release\": \"0\", \n",
" \"sched_load_balance\": \"1\", \n",
" \"cpus\": \"1-3\", \n",
" \"effective_mems\": \"0\", \n",
" \"memory_spread_slab\": \"0\", \n",
" \"mem_hardwall\": \"0\", \n",
" \"cpu_exclusive\": \"0\", \n",
" \"mem_exclusive\": \"0\", \n",
" \"ls\": \" /data/local/tmp/devlib-target/cgroups/devlib_cgh4/DEVLIB_SBOX/cpuset.*\", \n",
" \"mems\": \"0\", \n",
" \"memory_migrate\": \"0\", \n",
" \"sched_relax_domain_level\": \"-1\", \n",
" \"effective_cpus\": \"1-3\"\n",
"}\n"
]
}
],
"source": [
"# Check the attributes available for the SANDBOX group\n",
"print \"Sandbox:\\n\", json.dumps(sandbox.get(), indent=4)"
]
},
{
"cell_type": "code",
"execution_count": 24,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Isolated:\n",
"{\n",
" \"memory_pressure\": \"0\", \n",
" \"memory_spread_page\": \"0\", \n",
" \"notify_on_release\": \"0\", \n",
" \"sched_load_balance\": \"1\", \n",
" \"cpus\": \"0\", \n",
" \"effective_mems\": \"0\", \n",
" \"memory_spread_slab\": \"0\", \n",
" \"mem_hardwall\": \"0\", \n",
" \"cpu_exclusive\": \"0\", \n",
" \"mem_exclusive\": \"0\", \n",
" \"ls\": \" /data/local/tmp/devlib-target/cgroups/devlib_cgh4/DEVLIB_ISOL/cpuset.*\", \n",
" \"mems\": \"0\", \n",
" \"memory_migrate\": \"0\", \n",
" \"sched_relax_domain_level\": \"-1\", \n",
" \"effective_cpus\": \"0\"\n",
"}\n"
]
}
],
"source": [
"# Check the attributes available for the ISOLATED group\n",
"print \"Isolated:\\n\", json.dumps(isolated.get(), indent=4)"
]
},
{
"cell_type": "code",
"execution_count": 25,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"2016-12-08 11:44:50,597 INFO : Workload : Workload execution START:\n",
"2016-12-08 11:44:50,601 INFO : Workload : /data/local/tmp/bin/rt-app /data/local/tmp/devlib-target/simple_00.json 2>&1\n",
"2016-12-08 11:44:57,468 INFO : Workload : Pulling trace file into [.//simple_00.dat]...\n"
]
}
],
"source": [
"# Run some workload, which is expected to not run in the ISOLATED cpus:\n",
"trace = rtapp.run(ftrace=te.ftrace)"
]
},
{
"cell_type": "code",
"execution_count": 26,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"
"
],
"text/plain": [
""
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"# Check tasks was not running on ISOLATED CPUs\n",
"trappy.plotter.plot_trace(trace)"
]
},
{
"cell_type": "code",
"execution_count": 27,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"{\n",
" \"4968\": {\n",
" \"residency\": 0.0, \n",
" \"task_name\": \"rt-app\"\n",
" }, \n",
" \"4969\": {\n",
" \"residency\": 0.0, \n",
" \"task_name\": \"rt-app\"\n",
" }, \n",
" \"4970\": {\n",
" \"residency\": 0.0, \n",
" \"task_name\": \"rt-app\"\n",
" }, \n",
" \"4971\": {\n",
" \"residency\": 0.0, \n",
" \"task_name\": \"rt-app\"\n",
" }\n",
"}\n"
]
}
],
"source": [
"# Compute and visualize tasks residencies on ISOLATED CPUs\n",
"s = SchedMultiAssert(trappy.FTrace(trace), te.topology, execnames=tasks.keys())\n",
"residencies = s.getResidency('cpu', [0], percent=True)\n",
"print json.dumps(residencies, indent=4)"
]
},
{
"cell_type": "code",
"execution_count": 28,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"True"
]
},
"execution_count": 28,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Assert that ISOLATED CPUs was not running workload tasks\n",
"s.assertResidency('cpu', [0], 0.0, operator.eq, percent=True, rank=len(residencies))"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
},
"toc": {
"toc_cell": false,
"toc_number_sections": true,
"toc_threshold": 6,
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 0
}