\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" __comm __cpu __pid comm cpu load_avg load_sum \\\n",
"Time \n",
"0.000065 trace-cmd 0 2204 trace-cmd 0 452 21607277 \n",
"0.000081 trace-cmd 0 2204 trace-cmd 0 452 21607277 \n",
"0.000380 | \n", " | __comm | \n", "__cpu | \n", "__pid | \n", "comm | \n", "cpu | \n", "load_avg | \n", "load_sum | \n", "period_contrib | \n", "pid | \n", "util_avg | \n", "util_est | \n", "util_sum | \n", "cluster | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Time | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
| 0.000065 | \n", "trace-cmd | \n", "0 | \n", "2204 | \n", "trace-cmd | \n", "0 | \n", "452 | \n", "21607277 | \n", "125 | \n", "2204 | \n", "176 | \n", "176 | \n", "8446887 | \n", "LITTLE | \n", "
| 0.000081 | \n", "trace-cmd | \n", "0 | \n", "2204 | \n", "trace-cmd | \n", "0 | \n", "452 | \n", "21607277 | \n", "125 | \n", "2204 | \n", "176 | \n", "176 | \n", "8446887 | \n", "LITTLE | \n", "
| 0.000380 | \n", "<idle> | \n", "2 | \n", "0 | \n", "sudo | \n", "2 | \n", "0 | \n", "7 | \n", "576 | \n", "2203 | \n", "0 | \n", "941 | \n", "7 | \n", "big | \n", "
| 0.002055 | \n", "sudo | \n", "2 | \n", "2203 | \n", "sudo | \n", "2 | \n", "14 | \n", "688425 | \n", "219 | \n", "2203 | \n", "14 | \n", "941 | \n", "688425 | \n", "big | \n", "
| 0.002077 | \n", "sudo | \n", "2 | \n", "2203 | \n", "sudo | \n", "2 | \n", "14 | \n", "688425 | \n", "219 | \n", "2203 | \n", "14 | \n", "14 | \n", "688425 | \n", "big | \n", "
\n",
"\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" __comm __cpu __pid cpu capacity max_capacity \\\n",
"Time \n",
"0.002708 kschedfreq:0 4 1489 0 236 447 \n",
"0.002710 kschedfreq:0 4 1489 3 236 447 \n",
"0.002711 kschedfreq:0 4 1489 4 236 447 \n",
"0.002712 kschedfreq:0 4 1489 5 236 447 \n",
"0.410816 kschedfreq:1 2 1490 1 1024 1024 \n",
"\n",
" tip_capacity \n",
"Time \n",
"0.002708 357.6 \n",
"0.002710 357.6 \n",
"0.002711 357.6 \n",
"0.002712 357.6 \n",
"0.410816 819.2 "
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"cap_df = trace.data_frame.trace_event('cpu_capacity')\n",
"cap_df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {
"hidden": true
},
"source": [
"#### Plot the signals of interest"
]
},
{
"cell_type": "code",
"execution_count": 20,
"metadata": {
"code_folding": [],
"collapsed": false,
"hidden": true
},
"outputs": [
{
"data": {
"text/html": [
"| \n", " | __comm | \n", "__cpu | \n", "__pid | \n", "cpu | \n", "capacity | \n", "max_capacity | \n", "tip_capacity | \n", "
|---|---|---|---|---|---|---|---|
| Time | \n", "\n", " | \n", " | \n", " | \n", " | \n", " | \n", " | \n", " |
| 0.002708 | \n", "kschedfreq:0 | \n", "4 | \n", "1489 | \n", "0 | \n", "236 | \n", "447 | \n", "357.6 | \n", "
| 0.002710 | \n", "kschedfreq:0 | \n", "4 | \n", "1489 | \n", "3 | \n", "236 | \n", "447 | \n", "357.6 | \n", "
| 0.002711 | \n", "kschedfreq:0 | \n", "4 | \n", "1489 | \n", "4 | \n", "236 | \n", "447 | \n", "357.6 | \n", "
| 0.002712 | \n", "kschedfreq:0 | \n", "4 | \n", "1489 | \n", "5 | \n", "236 | \n", "447 | \n", "357.6 | \n", "
| 0.410816 | \n", "kschedfreq:1 | \n", "2 | \n", "1490 | \n", "1 | \n", "1024 | \n", "1024 | \n", "819.2 | \n", "
\n",
" \n",
" | \n",
"\n",
" \n",
" | \n",
"
| \n", " | \n", " |
\n", "We immediately capture what we are interested to evaluate!\n", "\n", "Moreover, all he produced plots are available as high resolution images, ready to be shared and/or used in other reports." ] }, { "cell_type": "code", "execution_count": 132, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[01;34m../../results/SchedTuneAnalysis/\u001b[00m\r\n", "├── \u001b[01;35mboost15_cluster_freqs.png\u001b[00m\r\n", "├── \u001b[01;35mboost15_task_util_task_ramp.png\u001b[00m\r\n", "├── energy.json\r\n", "├── output.log\r\n", "├── platform.json\r\n", "├── rt-app-task_ramp-0.log\r\n", "├── test_00.json\r\n", "├── trace_boost15.dat\r\n", "├── trace_boost15.raw.txt\r\n", "├── trace_boost15.txt\r\n", "├── trace_boost25.dat\r\n", "├── trace_boost25.raw.txt\r\n", "├── trace_boost25.txt\r\n", "├── trace.dat\r\n", "├── trace_noboost.dat\r\n", "├── trace_noboost.raw.txt\r\n", "├── trace_noboost.txt\r\n", "├── trace.raw.txt\r\n", "└── trace.txt\r\n", "\r\n", "0 directories, 19 files\r\n" ] } ], "source": [ "!tree {res_dir}" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Behavioral Analysis" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "### Is the task starting on a big core?" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "We always expect a new task to be allocated on a big core.\n", "\n", "To verify this condition we need to know what is the topology of the target.\n", "\n", "This information is **automatically collected by LISA** when the workload is executed.
\n", "Thus it can be used to write **portable tests** conditions." ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Create a SchedAssert for the specific topology" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false, "hidden": true }, "outputs": [], "source": [ "from bart.sched.SchedMultiAssert import SchedAssert\n", "\n", "# Create an object to get/assert scheduling pbehaviors\n", "sa = SchedAssert(trace_file, topology, execname='task_ramp')" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Use the SchedAssert method to investigate properties of this task" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "PASS: Task starts on big CPU: 1\n" ] } ], "source": [ "# Check on which CPU the task start its execution\n", "if sa.assertFirstCpu(platform['clusters']['big']):#, window=(4,6)):\n", " print \"PASS: Task starts on big CPU: \", sa.getFirstCpu()\n", "else:\n", " print \"FAIL: Task does NOT start on a big CPU!!!\"" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "### Is the task generating the expected load?" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "We expect 35% load in the between 2 and 4 [s] of the execution" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Identify the start of the first phase" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "The task starts execution at [s]: 1.9683\n", "Window of interest: (1.9682999999999993, 3.9682999999999993)\n" ] } ], "source": [ "# Let's find when the task starts\n", "start = sa.getStartTime()\n", "first_phase = (start, start+2)\n", "\n", "print \"The task starts execution at [s]: \", start\n", "print \"Window of interest: \", first_phase" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Use the SchedAssert module to check the task load in that period" ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "FAIL: Task duty-cycle is 18.11125% in the [2,4] execution window\n" ] } ], "source": [ "import operator\n", "\n", "# Check the task duty cycle in the second step window\n", "if sa.assertDutyCycle(10, operator.lt, window=first_phase):\n", " print \"PASS: Task duty-cycle is {}% in the [2,4] execution window\"\\\n", " .format(sa.getDutyCycle(first_phase))\n", "else:\n", " print \"FAIL: Task duty-cycle is {}% in the [2,4] execution window\"\\\n", " .format(sa.getDutyCycle(first_phase))" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "This test fails because we have not considered a scaling factor due running at a lower OPP.\n", "\n", "To write a portable test we need to account for that condition!" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Take OPP scaling into consideration" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "LITTLEs capacities range: (236, 447)\n", "LITTLE's min capacity scale: 0.527964205817\n" ] } ], "source": [ "# Get LITTLEs capacities ranges:\n", "littles = platform['clusters']['little']\n", "little_capacities = cap_df[cap_df.cpu.isin(littles)].capacity\n", "min_cap = little_capacities.min()\n", "max_cap = little_capacities.max()\n", "print \"LITTLEs capacities range: \", (min_cap, max_cap)\n", "\n", "# Get min OPP correction factor\n", "min_little_scale = 1.0 * min_cap / max_cap\n", "print \"LITTLE's min capacity scale: \", min_little_scale" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Scaled target duty-cycle: 18.9406779661\n", "1% tolerance scaled duty-cycle: 19.1300847458\n" ] } ], "source": [ "# Scale the target duty-cycle according to the min OPP\n", "target_dutycycle = 10 / min_little_scale\n", "print \"Scaled target duty-cycle: \", target_dutycycle\n", "\n", "\n", "target_dutycycle = 1.01 * target_dutycycle\n", "\n", "print \"1% tolerance scaled duty-cycle: \", target_dutycycle" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Write a more portable assertion" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "PASS: Task duty-cycle is 9.56209172258% in the [2,4] execution window\n" ] } ], "source": [ "# Add a 1% tolerance to our scaled target dutycycle\n", "if sa.assertDutyCycle(1.01 * target_dutycycle, operator.lt, window=first_phase):\n", " print \"PASS: Task duty-cycle is {}% in the [2,4] execution window\"\\\n", " .format(sa.getDutyCycle(first_phase) * min_little_scale)\n", "else:\n", " print \"FAIL: Task duty-cycle is {}% in the [2,4] execution window\"\\\n", " .format(sa.getDutyCycle(first_phase) * min_little_scale)" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "### Is the task migrated once we exceed the LITTLE CPUs capacity?" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Check that the task is switching the cluster once expected" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "PASS: Task switches to big within: (5.8682999999999996, 6.0682999999999989)\n" ] } ], "source": [ "# Consider a 100 [ms] window for the task to migrate\n", "delta = 0.1\n", "\n", "# Defined the window of interest\n", "switch_window=(start+4-delta, start+4+delta)\n", "\n", "if sa.assertSwitch(\"cluster\",\n", " platform['clusters']['little'],\n", " platform['clusters']['big'],\n", " window=switch_window):\n", " print \"PASS: Task switches to big within: \", switch_window\n", "else:\n", " print \"PASS: Task DOES NO switches to big within: \", switch_window" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Check that the task is running most of its time on the LITTLE cluster" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "PASS: Task exectuion on LITTLEs is 53.1% (less than 66% of its execution time)\n" ] } ], "source": [ "import operator\n", "\n", "if sa.assertResidency(\"cluster\", platform['clusters']['little'], 66, operator.le, percent=True):\n", " print \"PASS: Task exectuion on LITTLEs is {:.1f}% (less than 66% of its execution time)\".\\\n", " format(sa.getResidency(\"cluster\", platform['clusters']['little'], percent=True))\n", "else:\n", " print \"FAIL: Task run on LITTLE for MORE than 66% of its execution time\"" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true, "hidden": true }, "source": [ "### Check that the util estimation is properly computed and CPU capacity matches" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false, "hidden": true }, "outputs": [], "source": [ "start = 2\n", "last_phase = (start+4, start+6)\n", "\n", "analyzer_config = {\n", " \"SCALE\" : 1024,\n", " \"BOOST\" : 15,\n", "}\n", "\n", "# Verify that the margin is properly computed for each event:\n", "# margin := (scale - util) * boost\n", "margin_check_statement = \"(((SCALE - sched_boost_task:util) * BOOST) // 100) == sched_boost_task:margin\"" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false, "hidden": true }, "outputs": [], "source": [ "from bart.common.Analyzer import Analyzer\n", "\n", "# Create an Assertion Object\n", "a = Analyzer(trace.ftrace,\n", " analyzer_config,\n", " window=last_phase,\n", " filters={\"comm\": \"task_ramp\"})" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "PASS: Margin properly computed in : (6, 8)\n" ] } ], "source": [ "if a.assertStatement(margin_check_statement):\n", " print \"PASS: Margin properly computed in : \", last_phase\n", "else:\n", " print \"FAIL: Margin NOT properly computed in : \", last_phase" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Check that the CPU capacity matches the task boosted value" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false, "hidden": true }, "outputs": [], "source": [ "# Get the two dataset of interest\n", "df1 = trace.data_frame.trace_event('cpu_capacity')[['cpu', 'capacity']]\n", "df2 = trace.data_frame.trace_event('boost_task_rtapp')[['__cpu', 'boosted_util']]\n", "\n", "# Join the information from these two\n", "df3 = df2.join(df1, how='outer')\n", "df3 = df3.fillna(method='ffill')\n", "df3 = df3[df3.__cpu == df3.cpu]\n", "#df3.ix[start+4:start+6,].head()" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "data": { "text/plain": [ "19" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "len(df3[df3.boosted_util >= df3.capacity])" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "##### Do it the TRAPpy way" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "data": { "text/plain": [ "True" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Create the TRAPpy class\n", "trace.ftrace.add_parsed_event('rtapp_capacity_check', df3)\n", "# Define pivoting value\n", "trace.ftrace.rtapp_capacity_check.pivot = 'cpu'\n", "\n", "# Create an Assertion\n", "a = Analyzer(trace.ftrace,\n", " {\"CAP\" : trace.ftrace.rtapp_capacity_check},\n", " window=(start+4.1, start+6))\n", "a.assertStatement(\"CAP:capacity >= CAP:boosted_util\")" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Going further on events processing" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "### What are the relative residency on different OPPs?" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "We are not limited to the usage of pre-defined functions. We can exploit the full power of PANDAS to process the DataFrames to extract all kind of information we want." ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Use PANDAs APIs to filter and aggregate events" ] }, { "cell_type": "code", "execution_count": 40, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Residency time per OPP:\n", "Freq 450000Hz : 59.3%\n", "Freq 575000Hz : 11.7%\n", "Freq 700000Hz : 19.5%\n", "Freq 775000Hz : 8.8%\n", "Freq 850000Hz : 0.6%\n" ] } ], "source": [ "import pandas as pd\n", "\n", "# Focus on cpu_frequency events for CPU0\n", "df = trace.data_frame.trace_event('cpu_frequency')\n", "df = df[df.cpu == 0]\n", "\n", "# Compute the residency on each OPP before switching to the next one\n", "df.loc[:,'start'] = df.index\n", "df.loc[:,'delta'] = (df['start'] - df['start'].shift()).fillna(0).shift(-1)\n", "\n", "# Group by frequency and sum-up the deltas\n", "freq_residencies = df.groupby('frequency')['delta'].sum()\n", "print \"Residency time per OPP:\"\n", "df = pd.DataFrame(freq_residencies)\n", "\n", "df.head()\n", "\n", "# Compute the relative residency time\n", "tot = sum(freq_residencies)\n", "#df = df.apply(lambda delta : 100*delta/tot)\n", "for f in freq_residencies.index:\n", " print \"Freq {:10d}Hz : {:5.1f}%\".format(f, 100*freq_residencies[f]/tot)" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "#### Use MathPlot Lib to generate all kind of plot from collected data" ] }, { "cell_type": "code", "execution_count": 44, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAA8EAAAFCCAYAAADR899fAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzt3Xm8XXV57/HPNwSEAIGATCKTVqz0FkRwYjJ6ZRC4DDIY\nB3AApBUEQRBEuKYOFaulIk4VFcSKggJCpaHeq0nrhCAF5TKJVWZEgUBIICQhz/1jr8AxJDknydln\nn33W5/167dfZw9prPQsPkm+e35CqQpIkSZKkNhjX6wIkSZIkSRophmBJkiRJUmsYgiVJkiRJrWEI\nliRJkiS1hiFYkiRJktQahmBJkiRJUmsYgiVJ0gpJ8sUkpy/j84VJXjCSNUmSNJi4T7AkqS2S3AFs\nCDzVvFXA1lX1h54VNYYlWQj8RVX9rte1SJK0iJ1gSVKbFLBvVa3dPCYuHoCTjO9RbT3RtvuVJMkQ\nLElqvWbY7nuS3A7c1ry3b5IbksxM8tMkfz3g+O2T/FeSWUm+3Tw+2nz2jiQ/XsL5X9A8f06STye5\nM8kfmiHFqzefTU5yT5ITkzyQ5L4k7xhwnjWS/GOSO5I8kuQ/k6ye5Mokxy52zV8n2X8J97plU8+7\nktwJ/N/m/XcluTnJw0muSrL5gO/8U1PPo815t2neP3/RfTevT25qvifJuxa7bk/vW5KkRQzBkqS2\nyVLe3x94ObBNku2BrwJHAesB/wxckWTVJKsB3wO+DkwCvgO8kU6XeSjOBP4C2K75uSnwvwd8vhEw\nEXgecATw+STrNJ99GtgeeHVT1weAhcD5wNuevsFku+b7Vy6jjt2AvwT2akLjB4EDgecCPwa+1Zxr\nT2BX4EVVtQ5wCPBwc45adN9J9gLeD7we2Lr5ORrvW5LUcoZgSVKbBPhe092dmeTSAZ99oqoeqaon\ngXcD/1xV11bHBcCTdELYq4DxVXV2VT1VVZcA1w7p4knoBOsTm2vNBj4BTBlw2HzgI825pwGzgRcn\nGQe8Ezi+qu6vqoVVdXVVzQP+Fdg6yQubcxwGfLuqFiyjnKlV9URVzQX+prn/26pqYVPTS5tu8Dxg\nbeAlScY1xyxpDvWhwNeq6uaqehz48Ci9b0lSyxmCJUltUsD+VTWpebxxwGd3D3i+BfD+AWF5JvB8\nYBM6ncZ7FzvvnUO8/gbABOC6AeedRqf7ushDTRBd5HFgreaY1YH/ftZNdYLsxcBhTeCcAnxjkFoW\nv9+zB9T0UPP+86pqOvA54PPAA0n+OcnaSzjfJoud865Ret+SpJYzBEuS1DFwOPNdwMcHhOVJVbVW\nVV0E3E9nKO9AWwx4PodO4AMgycYDPnsQeALYZsB5162qiUOo70FgLp2hxEvydeCtdIYhP15Vvxjk\nfIvf77sXu981q+pqgKo6p6p2BLahM9T55CWc735g8wGvBz4fTfctSWo5Q7AkSc92LvA3SV6RjjWT\n7JNkLeBnwIIkxzVzhN9IZy7xIr8C/irJds3CT1MXfdB0Os8FPpNkA4AkmybZY7CCmu9+DTgrySZJ\nVkny6maOMlX1czrB9tPABct5v18CThuw4NU6SQ5pnu+Y5JVJVqXTnZ3LM1tMhWfmWF8MvCPJS5JM\nYMBw6FF835KkFjIES5K02KJWVXUdnTmsn6OzCNTtwOHNZ/PpLIT1DjrDhg8FLqUJg1X1G+AjdFZd\nvo3OIlMDz38K8Fvg6iSPAv+HTnd1ibUs5iTgRjpzkB+iM6924H/LLwD+GviX5bzf7wGfBL7d1HQj\nsGfz8UTgy3T+OdxBpzP7qQHnqeYcVwGfAX4E/Ab4IaPvviVJIlVDXcxSkiQtSZLzgHuq6owe13EY\ncFRV7dbLOkZaW+9bkrRi7ARLkrTylrbt0sgV0BmCfAydrm1rtPW+JUkrzhAsSdLKe3pYcC80e/n+\nkc7iVBf2qo6R1tb7liStHIdDS5IkSZJaw06wJEmSJKk1xve6gJGUxLa3JEmSJI1hVbXMtTpaFYIB\nHP6t4TR16lSmTp3a6zI0hvg7peHm75SGm79TGk7+Pmm4JYOvVelwaEmSJElSaxiCJUmSJEmtYQiW\nVsLkyZN7XYLGGH+nNNz8ndJw83dKw8nfJ/VCq7ZISlJtul9JkiRJapMkLowlSZIkSf1mKAs8td2K\nNjgNwZIkSZI0CjmKdelW5i8JnBMsSZIkSWoNQ7AkSZIkqTUMwZIkSZKk1jAES5IkSZJWyjve8Q7O\nOOOMQY/bcsst+eEPfzgCFS1d34bgJBsn2azXdUiSJEnSSEjS9cfK1rY8x02dOpXDDjtsha+5ovpy\ndegkhwCnAguTTAemVdX0HpclSZIkSV3WzRWjV25bpn5ZzbrvOsFJ1gOOBY4A9gTmAvsmOXSI3+/a\n335IkiRJUhtcf/31vOxlL2PixIlMmTKFuXPnPv3Z97//fV760pcyadIkdt55Z2688cZnff+qq67i\nE5/4BBdddBFrr70222+/PQDnnXce22yzDRMnTuSFL3whX/7yl4e99r4LwXRqngBUVT0MnAX8Dnhl\nkp0H/3oN8pAkSZIkLc28efM44IADePvb387MmTM55JBDuOSSS0jC9ddfzxFHHMG5557Lww8/zNFH\nH81+++3H/Pnz/+wce+21F6eddhpTpkzhscce4/rrrwdgo4024sorr2TWrFmcd955nHDCCU9/Nlz6\nLgRX1YPAd4DDkmxeVY8AFwILgP16WpwkSZIkjXFXX301CxYs4Pjjj2eVVVbhoIMO4uUvfzlVxbnn\nnsvRRx/Ny1/+cpJw+OGH85znPIerr776WeepqmcNod57773ZaqutANhtt93YY489+PGPfzys9fdF\nCE6yZ5IPJDk2yUTgouajNyXZoqpmAp8GdkqyUe8qlSRJkqSx7b777mPTTTf9s/e22GILAO68807+\n8R//kUmTJj39uOeee7jvvvuGdO5p06bxqle9ivXXX59Jkybxb//2bzz00EPDWv+oD8FJXgN8gU6n\nd1vgamAhcDmwIXBSkh2A19JZ6OuJHpUqSZIkSWPeJptswr333vtn7915550AbLbZZnzoQx9i5syZ\nTz9mz57Nm970pmedZ/H1mJ588kkOOuggPvCBD/DHP/6RmTNnsvfeew/7glujPgQDLwcuraqzqurd\nwKXAt4Df05kPvOjnUcCxVTWrZ5VKkiRJ0hi30047MX78eD772c8yf/58Lr30Uq699lqScNRRR/Gl\nL32Ja665hqpizpw5XHnllcyePftZ59l444254447ng658+bNY968eTz3uc9l3LhxTJs2jR/84AfD\nXn8/hOBfAasnWRegqk4H/gP4d2BOVZ0F7A8cUFXX9a5MSZIkSRr7Vl11VS699FLOP/981l9/fS6+\n+GIOOuggAHbYYQfOPfdcjj32WNZbbz1e9KIXccEFFyxxF55DDjkEgPXXX58dd9yRtddem89+9rMc\neuihrLfeenzrW99i//33H/b6Mxr3ckqyOfBYVc1MsiGdha++W1VfGnDMucB/VtU3luO8NfgK0Omb\n/a0kSZIkjU3Js3PJSGzn2i9ZaEn/fAa8v8x/UOO7VtUKSnIgcAowK8k1wBXA0cBlzf/o321WiH4U\nWLdnhUqSJEnSCOqXgDrajapOcJINgOnAkcAcYEfgYOBTwN3AOc3P8cAuwIFVdfNynN9OsCRJkqRR\nb2mdTnWMpU7wAuBW4IaqmpvkbuAR4P3AVOBwYGtgO+DMqrq9V4VKkiRJkvrPqOoEAyT5BjChqg5q\nXk8CDgU2rKqPruS57QRLkiRJGvXsBC/bynSCe746dJLdkxyT5ITmrWOBh5KcA1BVM4FrgZ2SrNWr\nOiVJkiRJ/a+nITjJLnRWfn4SODjJ54Btgc/T2Rbp8mZrpG2ACcBqPStWkiRJktT3ejYnOJ2lnl8B\nfK6qvpLkX4AzgAOBS4D30QnDnwNeAhxRVQ+v/JWnDng+uXlIkiRJ0ugyElsi9bsZM2YwY8aM5fpO\nT+YEJxlXVQuTvB44CTiuqn6TZHXgw8DEqjqmOXYNYJWqmj0M13VOsCRJkiSNUaNyTnCSV9GZ3zsO\nuKl57Jpkk6qaC3wEeEWSIwGq6onhCMCSJEmSJI1oCE6yF3A+MLeqFlbV/cBPgZ2AfZL8ZVU9AVwB\nzBvJ2iRJkiRJY9+IDYdOsitwEfC2qvpRkolVNav5bBfgjcCOwPXAFGByVd0yzDU4HFqSJEmSxqih\nDIceyYWx/gcwnc72R1sAZyaZDUwETqyqE5PsBmwNnFNVvx3B2iRJkiRJLdD1TnCSVwPPA+bQ2f7o\nBcD/Aj4JXA3sAewKHFpVj3a5FjvBkiRJkjRG9bwTnGR/4GPAr4Cn6Gx5NBe4tqq+2hxzH/DC5n1J\nkiRJkrqmayE4yUbA8cBbqurGJOcDGwDfAwau9jwZ2AqYADzZrXokSZIkSepmJ3gesDrwoiR3A6+h\nE4IfAX6f5JPAwXSC8luramYXa5EkSZIkqXtbJDWh9tPACcD3ga9V1T7AV4Hn05kfvDWdTvFN3apD\nkiRJkqRFurpPcFVdCrwBuBm4tXnvR8A6wGpV9cGqurmbNUiSJEmStEjXt0iqqtlJfgzsm2QW8Bxg\nC+CObl9bkiRJkqSBRmqf4H8F1gI+ADwBvLOqfj9C15YkSZIkCRiBfYL/7GLJms01Zw96cHeuP6Sb\ndZ9gSZIkSeo/Q9kneERDcK8lqTbdryRJkiS1yVBCcFcXxpIkSZIkaTQxBEuSJEmSWsMQLEmSJElq\nDUOwJEmSJKk1DMGSJEmSpNYwBEuSJEmSWsMQLEmSJElqDUOwJEmSJKk1DMGSJEmSpNYwBEuSJEmS\nWsMQLEmSJElqDUOwJEmSJKk1DMGSJEmSpNYwBEuSJEmSWsMQLEmSJElqDUOwJEmSJKk1DMGSJEmS\npNYwBEuSJEmSWsMQLEmSJElqDUOwJEmSJKk1DMGSJEmSpNYwBEuSJEmSWsMQLEmSJElqDUOwJEmS\nJKk1+jYEJ9k4yWa9rkOSJEmS1D/G97qAFZHkEOBUYGGS6cC0qpre47IkSZIkSaNcqqrXNSyXJOsB\nlwHHA3cB7wPWBH5RVRcP8t3+ulmNiH77d0CSJEnSkiWhqrKsY/qxEzwOmABUVT2c5CzgrcArk9xb\nVT9d9tcNPBpomf9+SJIkSRpj+m5OcFU9CHwHOCzJ5lX1CHAhsADYr6fFSZIkSZJGtb4YDp1kT2A7\n4HHgAmAS8F7gAeDiqrozyQbApcDBVfXAUs5TdoL15+JwaEmSJGmMGMpw6FHfCU7yGuALdDq92wJX\nAwuBy4ENgZOS7AC8ls7w7id6VKokSZIkaZQb9Z3gJCcBG1XVyc3rjwGTgSnAU8Cbgf2BecCpVXXd\nMs5lJ1iLsRMsSZIkjRVD6QT3Qwjenc5c3zOa+b8k+ThwAPDqqpqVZF1gflXNGeRchmAtxhAsSZIk\njRV9Oxw6yeZJJjUvfwW8hE7nF4Cq+hDwMzodYKrqkcECsCRJkiRJo26LpCQHAqcAs5JcA1wBHA1c\nlgTgu80K0Y8C6/asUEmSJElS3xlVw6GbFZ6nA0cCc4AdgYOBTwF3A+c0P8cDuwAHVtXNy3F+h0Nr\nMQ6HliRJksaKoQyHHm2d4KeAW4EbqmpukruBR4D3A1OBw4Gt6WyXdGZV3d6rQiVJkiRJ/WdUdIKT\nbFJV9zfPLwAmVNXBzetJwKHAhlX10ZW8jp1gLcZOsCRJkjRW9MXCWEn2BL6UZJPmrWOA2UnOBqiq\nmcC1wE5J1upRmZIkSZKkMaCnw6GT7AucDpy2qBNcVY8l+Xvg9CSXA28HtgEmAKut/FWnDng+uXlI\nkiRJkvrNjBkzmDFjxnJ9pyfDodNZ5nkD4Hbgi1V1apJNgW2BdYCLgDWAzwOr0tki6YiqumElr+tw\naC3G4dCSJEnSWDGU4dC9CsGrNwtfvQk4DriQzirQvwT2AH5WVX/bHLsGsEpVzR6G6xqCtRhDsCRJ\nkjRWjMo5wUn2AqYl2biqLgI+DZwFTKuqk4FXAtsnOR6gqp4YjgAsSZIkSdKIzgluAvBngPvoDHH+\nQ1VdluSlVXVLknFNh/gK4NGRrE2SJEmSNPaNWCc4ye7AmcC7gWl09v5d5DaAqlqY5HA6WyL9fKRq\nkyRJkiS1w4iE4CR/AbwJeG9V/SdwNjAxyRHwdPhdNcnrgPcCb6uq20aiNkmSJElSe3R9OHQzBPp9\nwAnNkOc1quqJJN8GXtAck6qan+Q3wD5V9cdu1yVJkiRJap+udoKT7Al8ks7832Ogs9BV8/FPgHcl\n2aua5Xmr6h4DsCRJkiSpW7oWgpO8ns4+v28BXgRsnWS35rNxVfVr4HTgrUnW71YdkiRJkiQt0s1O\n8CrA4VV1E7AmncWv/qr5bNG+Tf8NzAfmdbEOSZIkSZIASDMSuXsX6HR9FzZzg88H9mi6wIs+X6+q\nHu5qEc9cq6C796t+E7r974AkSZKkkZGEqsqyjhmJ1aGrWfjqKuBcYN8k45KsCjBSAViSJEmSpK6H\n4Go0L28A9gHGVdX8bl9bkiRJkqSBur5F0kBVdUmSKcDzgTtG8trPWGZnXJIkSZI0hnV9TvDTF+oM\nie7p5MtRUIIkSZIkqUtGy5xgoDMseqSuJUmSJEnSkoxYCJYkSZIkqdcMwZIkSZKk1jAES5IkSZJa\nwxAsSZIkSWoNQ7AkSZIkqTUMwZIkSZKk1jAES5IkSZJawxAsSZIkSWoNQ7AkSZIkqTUMwZIkSZKk\n1jAES5IkSZJawxAsSZIkSWoNQ7AkSZIkqTUMwZIkSZKk1jAES5IkSZJawxAsSZIkSWoNQ7AkSZIk\nqTUGDcFJrktyTJJJI1GQJEmSJEndMpRO8BRgU+DaJN9OsmeSdLkuSZIkSZKGXapqaAcm44B9gS8C\nC4GvAWdX1cPdK294Jamh3q8kSZIkqb8koaqW2bQd0pzgJNsBZwGfAi4BDgEeA360skVKkiRJkjRS\nxg92QJLrgEeBrwCnVNWTzUdXJ9m5m8VJkiRJkjScBh0OneQFVfW7EaqnqxwOLUmSJElj13ANhz4y\nyboDTjopycdWujpJkiRJkkbYUELw3lX1yKIXVTUT2Kd7JQ1Nko2TbNbrOiRJkiRJ/WPQOcHAuCSr\nV9VcgCRrAKt1t6xlS3IIcCqwMMl0YFpVTR/id7tamyStCKdqSJIkjYyhhOBvAj9M8jUgwDuBC7pa\n1TIkWQ84FjgCuAt4H7Bvkg2q6uLBz+AfNCWNNv7lnCRJ0kgZNARX1SeT/Bp4PZ0E+ZGq+veuV7Z0\n44AJndLq4SRnAW8FXpnk3qr6aQ9rkyRJkiSNYoOuDj0aJfkAsCHw2aq6K8kkOsOjqapTlvG9shMs\nafSJw6ElSZKGwbCsDp3koCS3J5mV5LHmMWv4yhxckj2TfCDJsUkmAhc1H70pyRbNYl2fBnZKstFI\n1iZJkiRJ6h9DWR36H4D9qmpiVa3dPCZ2u7BFkrwG+AKwANgWuBpYCFxOpxt8UpIdgNfSGd79xEjV\nJkmSJEnqL4MOh07y06raeYTqWdL1TwI2qqqTm9cfAyYDU4CngDcD+wPzgFOr6rplnMvh0JJGIYdD\nS5IkDYehDIceSgg+G9gY+B6doAmdRakuHZYqB5Fkd2A/4IxF+xUn+ThwAPDqqpqVZF1gflXNGeRc\nhmBJo5AhWJIkaTgMy5xgYB06Q4z3APZtHv9r5ctbuiSbN4tdAfwKeAmdzi8AVfUh4Gd0OsBU1SOD\nBWBJkiRJkoayRdI7RqCOpyU5EDgFmJXkGuAK4GjgsiQA362qB4FHgXVHsjZJkiRJUn8bynDoF9NZ\nmGrjqvqrJNvSWSjrY8NeTLIBMB04EpgD7AgcDHwKuBs4p/k5HtgFOLCqbl6O8zscWtIo5HBoSZKk\n4TBcw6HPBU7jmfnAN9JZjKobFgC3AjdU1Y3AZcBXgPfT6foeDnwd+CWw7/IEYEmSJEmShhKCJ1TV\nLxa9qE67Yn43imn2+30C+Gbz+hFgBvB9YO+qerCqflZVX6yq27tRgyRJkiRp7BpKCP5Tkr9Y9CLJ\nwcD9w1VAkt2THJPkhOatY4GHkpwDTwfja4Gdkqw1XNeVJEmSJLXPoAtj0QmlXwb+Msl9wO+Btw7H\nxZPsAlwIfBB4S5IXAhcBnweOTXI58HZgG2ACsNrKX3XqgOeTm4ckSZIkqd/MmDGDGTNmLNd3Bl0Y\n6+kDkzWBcVX12PKXttRzngisXVV/l2R14AxgDeAS4AY6YXg8nS2SjqiqG1byei6MJWkUcmEsSZKk\n4TCUhbGGsjr0h+kkxzAgQVbVR4ahwN3pLHp1XFX9pgnCHwYmVtUxzTFrAKtU1exhuJ4hWNIoZAiW\nJEkaDsO1OvSc5jEbWAjsDWy5EkVtnuQ5zfze/wBuA3ZNsklVzQU+ArwiyZEAVfXEcARgSZIkSZIG\nnRNcVZ8e+DrJp4AfrMjFkuwLfBL4KbAOcBLwL8AxnY/zk6q6NckVPLMlkyRJkiRJw2IoC2Mtbk1g\n0+X9UpLnAf8AvAe4hc6CVz8HdgY+B7wFODzJ9cAUXLFKkiRJkjTMBg3BSW4c8HIcsCGdIctDlmRT\n4DE6w59vB/5UVf+QZAHwY2Dnqjoxya7Ai4Fzquq3y3MNSZIkSZIGM5SFsbYc8HIB8EBVzR/yBZI9\ngb8DjgJOBn47cFGtJB+kE3zfU1WPD7nyFeDCWJJGJxfGkiRJGg5DWRhrKMOhZy32eu3kmXNW1cPL\nKGBPOnOA1wPeCLwPuD7JvKo6szns23T2CZ47hFokSZIkSVphQwnB/wVsDsxsXk8C7qLTUi3gBUv6\nUpLX09nnd386Q6CvAr4J/E9gRpJVgW8BuwAvo7NQ1swlnUuSJEmSpOEwlC2S/g+wb1WtX1XrA/sA\nP6iqrapqiQG4sQpweFXdRGcxrZuAvZu5vpPphOeTgWOBd1aVAViSJEmS1FVDmRP8/6rqfwz23jK+\nP66qFibZC/g68Iaq+q8kq1fV3CSTRioAOydY0ujknGBJkqThMJQ5wUPpBN+X5PQkWybZKsmHgHuX\no45Kkqq6CvgysFeSVegssoUdYEmSJEnSSBlKCH4znW2RLgMubZ6/eagXqEbz8gY6w6lTVQuWs1ZJ\nkiRJklbKoMOhnz4wWbOq5qz0BZPvACdX1R0re64VuLbjDSWNSg6HliRJWnlDGQ49lDnBOwFfAdau\nqs2SbAccXVXvWc5iUj3+U94oKEGSJEmS1CXDNSf4M8BewIMAVfUr4DXLW4zpU5IkSZLUa0MJwVTV\nXYu95XxeSZIkSVLfGT+EY+5KsjNAktWA44BbulqVJEmSJEldMJQ5wc8FPgu8HgjwA+C4qnqo++UN\nL+cES5IkSdLYNZQ5wcvsBCcZD5xdVW8Z1sokSZIkSeqBZc4Jbvby3SLJc0aoHkmSJEmSumYoc4J/\nB/wkyRXA4817VVVnda8sSZIkSZKG31I7wUm+0TzdD/h+c+xazWPt7pcmSZIkSdLwWlYneIckzwPu\nAs6hsyiWJEmSJEl9a1kh+EvAD4EXANct9lk170uSJEmS1DeGskXSl6rqb0aonq5yiyRJkiRJGruG\nskXSoCF4LDEES5IkSdLYNZQQvMwtkiRJkiRJGksMwZIkSZKk1jAES5IkSZJawxAsSZIkSWoNQ7Ak\nSZIkqTUMwZIkSZKk1jAES5IkSZJawxAsSZIkSWoNQ7AkSZIkqTUMwZIkSZKk1jAES5IkSZJawxAs\nSZIkSWoNQ7AkSZIkqTX6NgQn2TjJZr2uQ5IkSZLUP8b3uoAVkeQQ4FRgYZLpwLSqmj7E73a1Nqnb\nqqrXJUiSJEl9K/32B+ok6wGXAccDdwHvA9YEflFVFw/y3YL+ul/pz8UQLEmSJC1FEqpqmZ3PfhwO\nPQ6YAFRVPQycBfwOeGWSnXtamSRJkiRpVOu7EFxVDwLfAQ5LsnlVPQJcCCwA9utpcZIkSZKkUa0v\nhkMn2RPYDngcuACYBLwXeAC4uKruTLIBcClwcFU9sJTzOBxafc7h0JIkSdLSjInh0EleA3yBTqd3\nW+BqYCFwObAhcFKSHYDX0lno64kelSpJkiRJGuVGfSc4yUnARlV1cvP6Y8BkYArwFPBmYH9gHnBq\nVV23jHPZCVafsxMsSZIkLc1QOsH9EIJ3pzPX94xm/i9JPg4cALy6qmYlWReYX1VzBjmXIVh9zhAs\nSZIkLU3fDodOsnmSSc3LXwEvodP5BaCqPgT8jE4HmKp6ZLAALEmSJEnS+F4XsLgkBwKnALOSXANc\nARwNXJYE4LvNCtGPAuv2rFBJkiRJUt8ZVcOhmxWepwNHAnOAHYGDgU8BdwPnND/HA7sAB1bVzctx\nfodDq885HFqSJElamqEMhx5tneCngFuBG6pqbpK7gUeA9wNTgcOBrelsl3RmVd3eq0IlSZIkSf1n\nVHSCk2xSVfc3zy8AJlTVwc3rScChwIZV9dGVvI6dYPU5O8GSJEnS0vTFwlhJ9gS+lGST5q1jgNlJ\nzgaoqpnAtcBOSdbqUZmSJEmSpDGgp8Ohk+wLnA6ctqgTXFWPJfl74PQklwNvB7YBJgCrrfxVpw54\nPrl5SJIkSZL6zYwZM5gxY8Zyfacnw6HTWeZ5A+B24ItVdWqSTYFtgXWAi4A1gM8Dq9LZIumIqrph\nJa/rcGj1OYdDS5IkSUszlOHQvQrBqzcLX70JOA64kM4q0L8E9gB+VlV/2xy7BrBKVc0ehusagtXn\nDMGSJEnS0ozKOcFJ9gKmJdm4qi4CPg2cBUyrqpOBVwLbJzkeoKqeGI4ALEmSJEnSiM4JbgLwZ4D7\n6Axx/kNVXZbkpVV1S5JxTYf4CuDRkaxNkiRJkjT2jVgnOMkbgL8H3g1Mo7P37yK3AVTVwiSH09kS\n6ecjVZskSZIkqR1GJAQ3C2HtAZxYVf8JnA1MTHIEPB1+V03yOuC9wNuq6raRqE2SJEmS1B5dHw6d\nZH9gvao6oXm9WlXNS/Jt4AXNe6mq+Ul+A+xTVX/sdl2SJEmSpPbpaic4yR7A3wH3LnqvquY1T38G\nvCvJXtUsd1tV9xiAJUmSJEnd0rUtkpLsBHwP2LeqrkmyLp09gB8C5jXd4KOA3YD3VdVDXSnkz2ty\niyT1Obex6HbkAAAKXElEQVRIkiRJkpZmKFskdXM49MPAfGCTJM8FvgPMBWYDVyW5APgN8Gpg3lLP\nIkmSJEnSMOlaJxggyXbAZcDqwFTgK8A76CyS9Z6qejjJ+iPRBW7qsROsPmcnWJIkSVqaoXSCuxqC\nmyK2AV5XVZ8b8N5VwOlV9cuuXvzZtRiC1ecMwZIkSdLS9Ho4NABVdXOSWwYUdTCwIXBPt68tSZIk\nSdJAXQ/BAFVVzV7B7wROBg6pqj+MxLWfbZl/KSBJkiRJGsO6Phz66Qt1QvBk4P6qunVELvrsGsqh\npJIkSZI0No2KOcGjiSFYkiRJksauoYTgcSNVjCRJkiRJvWYIliRJkiS1hiFYkiRJktQahmBJkiRJ\nUmsYgiVJkiRJrWEIliRJkiS1hiFYkiRJktQahmBJkiRJUmsYgiVJkiRJrWEIliRJkiS1hiFYkiRJ\nktQahmBJkiRJUmsYgiVJkiRJrWEIliRJkiS1hiFYkiRJktQahmBJkiRJUmsYgiVJkiRJrWEIliRJ\nkiS1hiFYkiRJktQahmBJkiRJUmsYgiVJkiRJrWEIliRJkiS1hiFYkiRJktQaYyIEJ0mva5AkSZIk\njX59G4KTbJxkM4Cqql7XI0mSJEka/cb3uoAVkeQQ4FRgYZLpwLSqmj7E73a1NkmSJEnS6JV+a6Im\nWQ+4DDgeuAt4H7Am8IuquniQ7xb01/1KkiRJkoYqVNUyO5/9OBx6HDCBzijoh4GzgN8Br0yyMzhH\nWJIkSZK0ZH0XgqvqQeA7wGFJNq+qR4ALgQXAfs0xtnslSZIkSc/SF8Ohk+wJbAc8DlwATALeCzwA\nXFxVdybZALgUOLiqHljKeRwOLUmSJElj1hgYDp3kNcAX6HR6twWuBhYClwMbAicl2QF4LZ2Fvp7o\nUamSJEmSpFFu1HeCk5wEbFRVJzevPwZMBqYATwFvBvYH5gGnVtV1yziXnWBJkiRJGrMG7wT3Qwje\nnc5c3zOa+b8k+ThwAPDqqpqVZF1gflXNGeRchmBJkiRJGrP6dDh0ks2TTGpe/gp4CZ3OLwBV9SHg\nZ3Q6wFTVI4MFYEmSJEmSxve6gMUlORA4BZiV5BrgCuBo4LJm56PvNitEPwqs27NCJUmSJEl9Z1QN\nh25WeJ4OHAnMAXYEDgY+BdwNnNP8HA/sAhxYVTcvx/kdDi1JkiRJY9bgw6FHWyd4AXArcENVzU1y\nN/AI8H5gKnA4sDWd7ZLOrKrbe1WoJEmSJKn/jKpOMECSbwATquqg5vUk4FBgw6r66Eqe206wJEmS\nJI1ZfbAwVpLdkxyT5ITmrWOBh5KcA1BVM4FrgZ2SrNWrOiVJkiRJ/a+nw6GT7AJcCHwQeEuSFwIX\nAZ8Hjk1yOfB2YBtgArDayl916oDnk5uHJEmSJKn/zGgeQ9ez4dDpLPV8ArB2Vf1dktWBM4A1gEuA\nG+iE4fF0tkg6oqpuWMlrOhxakiRJksaswYdD9yQEJxlXVQuTvB44CTiuqn7TBOEPAxOr6pjm2DWA\nVapq9jBc1xAsSZIkSWPWKJwTnORVdOb3jgNuah67JtmkquYCHwFekeRIgKp6YjgCsCRJkiRJIxqC\nk+wFnA/MraqFVXU/8FNgJ2CfJH9ZVU8AVwDzRrI2SZIkSdLYN2LDoZPsSmfRq7dV1Y+STKyqWc1n\nuwBvBHYErgemAJOr6pZhrsHh0JIkSZI0Zg0+HHokV4f+K+DnwJ+SbAGcmWQ2MBE4sapOTLIbsDVw\nTlX9dgRrkyRJkiS1QNc7wU0HeEPgUuBv6XR79wQ+CVwN7AHsChxaVY92uRY7wZIkSZI0ZvWwE9xs\ngbQW8M/NddYAvggcC/yyqr7QHHcf8EJgbrdqkSRJkiQJuhiCq9NifizJ14EFwBuACVV1TrMV0iKT\nga2ACcCT3apHkiRJkqSudoKbIDwf2AL4GnBUkhcD85OcRmd49FHAW6tqZrdqkSRJkiQJurhFUj0z\n2fgK4A9V9UPgl8B7gLWraiGwKfCWqrqpW3VIkiRJkrTISKwO/Tjw4iRHAX8DnAm8IslbgNObMCxJ\nkiRJUtd1PQRX1X1J7gLOAI6pqn9N8jrgtwZgSZIkSdJI6voWSQBJNgM2rKrrmtfjehGAO1skSZIk\nSZLGqsG2SBqREPz0xTrbJg2cLzyinlmrS5IkSZI01iQ93Cd4SUygkiRJkqRe6trq0JIkSZIkjTaG\nYEmSJElSaxiCJUmSJEmtYQiWJEmSJLWGIViSJEmS1BqGYEmSJElSaxiCJUmSJEmtYQiWVsKMGTN6\nXYLGGH+nNNz8ndJw83dKw8nfJ/WCIVhaCf4ft4abv1Mabv5Oabj5O6Xh5O+TesEQLEmSJElqDUOw\nJEmSJKk1UlW9rmHEJGnPzUqSJElSC1VVlvV5q0KwJEmSJKndHA4tSZIkSWoNQ7AkSZIkqTVaE4KT\n7JXk1iS3Jzml1/WovyX5WpIHktzY61o0NiTZLMn0JDcl+X9Jjut1TepfSVZP8oskNyS5Ocknel2T\nxoYkqyS5Psm/9roW9b8kdyT5dfM7dU2v61H/S7Juku8muaX579+rlnhcG+YEJ1kFuA14PXAvcC3w\n5qq6paeFqW8l2RWYDVxQVX/d63rU/5JsDGxcVTckWQu4DjjA/5/SikoyoaoeTzIe+AlwUlX9pNd1\nqb8lORHYAVi7qvbrdT3qb0l+D+xQVQ/3uhaNDUm+DvxHVX2t+e/fmlX16OLHtaUT/Argt1V1R1XN\nB74N7N/jmtTHqurHwMxe16Gxo6r+UFU3NM9nA7cAz+ttVepnVfV483Q1YBXAP2RqpSR5PrA38BVg\nmSuvSsvB3yUNiyTrALtW1dcAqmrBkgIwtCcEbwrcPeD1Pc17kjTqJNkS2B74RW8rUT9LMi7JDcAD\nwPSqurnXNanv/RNwMrCw14VozCjg/yb5ZZKjel2M+t5WwJ+SnJfkv5Kcm2TCkg5sSwge+2O+JY0J\nzVDo7wLHNx1haYVU1cKqeinwfGC3JJN7XJL6WJJ9gT9W1fXYudPw2bmqtgfeABzTTDeTVtR44GXA\nF6rqZcAc4NQlHdiWEHwvsNmA15vR6QZL0qiRZFXgEuBfqup7va5HY0MzFOxKYMde16K+thOwXzOH\n81vA65Jc0OOa1Oeq6v7m55+Ay+hMYZRW1D3APVV1bfP6u3RC8bO0JQT/EnhRki2TrAa8CbiixzVJ\n0tOSBPgqcHNVfabX9ai/JXluknWb52sAuwPX97Yq9bOqOq2qNquqrYApwI+q6vBe16X+lWRCkrWb\n52sCewDuuqEVVlV/AO5OsnXz1uuBm5Z07PgRq6qHqmpBkmOBf6ezOMhXXXFVKyPJt4DXAOsnuRv4\n31V1Xo/LUn/bGXgb8Oski8LKB6vqqh7WpP61CfD1JOPo/IX3N6rqhz2uSWOLU820sjYCLuv8HTDj\ngW9W1Q96W5LGgPcC32wan/8NvHNJB7ViiyRJkiRJkqA9w6ElSZIkSTIES5IkSZLawxAsSZIkSWoN\nQ7AkSZIkqTUMwZIkSZKk1jAES5IkSZJawxAsSZIkSWoNQ7AkSZIkqTX+PwvpdkvornsqAAAAAElF\nTkSuQmCC\n", "text/plain": [ "
\n", "Advanced DataFrame usage: filtering by columns/rows, merging tables, plotting data
\n", "[notebooks/tutorial/05_TrappyUsage.ipynb](05_TrappyUsage.ipynb)\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Remote target connection and control" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Using LISA APIs to control a remote device and run custom workloads" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Configure the connection" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "# Setup a target configuration\n", "conf = {\n", " \n", " # Target is localhost\n", " \"platform\" : 'linux',\n", " \"board\" : \"juno\",\n", " \n", " # Login credentials\n", " \"host\" : \"192.168.0.1\",\n", " \"username\" : \"root\",\n", " \"password\" : \"\",\n", "\n", " # Binary tools required to run this experiment\n", " # These tools must be present in the tools/ folder for the architecture\n", " \"tools\" : ['rt-app', 'taskset', 'trace-cmd'],\n", " \n", " # Comment the following line to force rt-app calibration on your target\n", " \"rtapp-calib\" : {\n", " \"0\": 355, \"1\": 138, \"2\": 138, \"3\": 355, \"4\": 354, \"5\": 354\n", " },\n", " \n", " # FTrace events end buffer configuration\n", " \"ftrace\" : {\n", " \"events\" : [\n", " \"sched_switch\",\n", " \"sched_wakeup\",\n", " \"sched_wakeup_new\",\n", " \"sched_overutilized\",\n", " \"sched_contrib_scale_f\",\n", " \"sched_load_avg_cpu\",\n", " \"sched_load_avg_task\",\n", " \"sched_tune_config\",\n", " \"sched_tune_tasks_update\",\n", " \"sched_tune_boostgroup_update\",\n", " \"sched_tune_filter\",\n", " \"sched_boost_cpu\",\n", " \"sched_boost_task\",\n", " \"sched_energy_diff\",\n", " \"cpu_frequency\",\n", " \"cpu_capacity\",\n", " ],\n", " \"buffsize\" : 10240\n", " },\n", "\n", " # Where results are collected\n", " \"results_dir\" : \"SchedTuneAnalysis\",\n", "\n", " # Devlib module required (or not required)\n", " 'modules' : [ \"cpufreq\", \"cgroups\" ],\n", " #\"exclude_modules\" : [ \"hwmon\" ],\n", "}" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Setup the connection" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "# Support to access the remote target\n", "from env import TestEnv\n", "\n", "# Initialize a test environment using:\n", "# the provided target configuration (my_target_conf)\n", "# the provided test configuration (my_test_conf)\n", "te = TestEnv(conf)\n", "target = te.target\n", "\n", "print \"DONE\"" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Target control" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "### Run custom commands" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "# Enable Energy-Aware scheduler\n", "target.execute(\"echo ENERGY_AWARE > /sys/kernel/debug/sched_features\");\n", "target.execute(\"echo UTIL_EST > /sys/kernel/debug/sched_features\");\n", "\n", "# Check which sched_feature are enabled\n", "sched_features = target.read_value(\"/sys/kernel/debug/sched_features\");\n", "print \"sched_features:\"\n", "print sched_features" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "### Example CPUFreq configuration" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "target.cpufreq.set_all_governors('sched');\n", "\n", "# Check which governor is enabled on each CPU\n", "enabled_governors = target.cpufreq.get_all_governors()\n", "print enabled_governors" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "### Example of CGruops configuration" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "schedtune = target.cgroups.controller('schedtune')\n", "\n", "# Configure a 50% boostgroup\n", "boostgroup = schedtune.cgroup('/boosted')\n", "boostgroup.set(boost=25)\n", "\n", "# Dump the configuraiton of each groups\n", "cgroups = schedtune.list_all()\n", "for cgname in cgroups:\n", " cgroup = schedtune.cgroup(cgname)\n", " attrs = cgroup.get()\n", " boost = attrs['boost']\n", " print '{}:{:<15} boost: {}'.format(schedtune.kind, cgroup.name, boost)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Remote workloads execution" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Generate RTApp configurations" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "# RTApp configurator for generation of PERIODIC tasks\n", "from wlgen import RTA, Periodic, Ramp\n", "\n", "# Create a new RTApp workload generator using the calibration values\n", "# reported by the TestEnv module\n", "rtapp = RTA(target, 'test', calibration=te.calibration())\n", "\n", "# Ramp workload\n", "ramp = Ramp(\n", " start_pct=10,\n", " end_pct=60,\n", " delta_pct=25,\n", " time_s=2,\n", " period_ms=32\n", ")\n", "\n", "# Configure this RTApp instance to:\n", "rtapp.conf(\n", "\n", " # 1. generate a \"profile based\" set of tasks\n", " kind = 'profile',\n", " \n", " # 2. define the \"profile\" of each task\n", " params = {\n", " \n", " # 3. Composed task\n", " 'task_ramp': ramp.get(),\n", " },\n", " \n", " #loadref='big',\n", " loadref='LITTLE',\n", " run_dir=target.working_directory\n", " \n", ");" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Execution and tracing" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "def execute(te, wload, res_dir, cg='/'):\n", " \n", " logging.info('# Setup FTrace')\n", " te.ftrace.start()\n", "\n", " if te.emeter:\n", " logging.info('## Start energy sampling')\n", " te.emeter.reset()\n", "\n", " logging.info('### Start RTApp execution')\n", " wload.run(out_dir=res_dir, cgroup=cg)\n", "\n", " if te.emeter:\n", " logging.info('## Read energy consumption: %s/energy.json', res_dir)\n", " nrg_report = te.emeter.report(out_dir=res_dir)\n", " else:\n", " nrg_report = None\n", "\n", " logging.info('# Stop FTrace')\n", " te.ftrace.stop()\n", "\n", " trace_file = os.path.join(res_dir, 'trace.dat')\n", " logging.info('# Save FTrace: %s', trace_file)\n", " te.ftrace.get_trace(trace_file)\n", "\n", " logging.info('# Save platform description: %s/platform.json', res_dir)\n", " plt, plt_file = te.platform_dump(res_dir)\n", " \n", " logging.info('# Report collected data:')\n", " logging.info(' %s', res_dir)\n", " !tree {res_dir}\n", " \n", " return nrg_report, plt, plt_file, trace_file" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true, "hidden": true }, "outputs": [], "source": [ "nrg_report, plt, plt_file, trace_file = execute(te, rtapp, te.res_dir, cg=boostgroup.name)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Regression testing support" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "Writing and running regression tests using the LISA API" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Defined configurations to test and workloads" ] }, { "cell_type": "code", "execution_count": 116, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{\r\n", " /* Devlib modules to enable/disbale for all the experiments */\r\n", " \"modules\" : [ \"cpufreq\", \"cgroups\" ],\r\n", " \"exclude_modules\" : [ ],\r\n", "\r\n", " /* Binary tools required by the experiments */\r\n", " \"tools\" : [ \"rt-app\" ],\r\n", "\r\n", " /* FTrace configuration */\r\n", " \"ftrace\" : {\r\n", " \"events\" : [\r\n", " \"sched_switch\",\r\n", " \"sched_contrib_scale_f\",\r\n", " \"sched_load_avg_cpu\",\r\n", " \"sched_load_avg_task\",\r\n", " \"sched_tune_config\",\r\n", " \"sched_tune_tasks_update\",\r\n", " \"sched_tune_boostgroup_update\",\r\n", " \"sched_tune_filter\",\r\n", " \"sched_boost_cpu\",\r\n", " \"sched_boost_task\",\r\n", " \"sched_energy_diff\",\r\n", " \"cpu_frequency\",\r\n", " \"cpu_capacity\",\r\n", " ],\r\n", " \"buffsize\" : 10240,\r\n", " },\r\n", "\r\n", " /* Set of platform configurations to test */\r\n", " \"confs\" : [\r\n", " {\r\n", " \"tag\" : \"noboost\",\r\n", " \"flags\" : \"ftrace\",\r\n", " \"sched_features\" : \"ENERGY_AWARE\",\r\n", " \"cpufreq\" : { \"governor\" : \"sched\" },\r\n", " \"cgroups\" : {\r\n", " \"conf\" : {\r\n", " \"schedtune\" : {\r\n", " \"/\" : {\"boost\" : 0 },\r\n", " \"/stune\" : {\"boost\" : 0 },\r\n", " }\r\n", " },\r\n", " \"default\" : \"/\",\r\n", " }\r\n", " },\r\n", " {\r\n", " \"tag\" : \"boost15\",\r\n", " \"flags\" : \"ftrace\",\r\n", " \"sched_features\" : \"ENERGY_AWARE\",\r\n", " \"cpufreq\" : { \"governor\" : \"sched\" },\r\n", "\t \"cgroups\" : {\r\n", " \"conf\" : {\r\n", " \"schedtune\" : {\r\n", " \"/\" : {\"boost\" : 0 },\r\n", " \"/stune\" : {\"boost\" : 15 },\r\n", " }\r\n", " },\r\n", " \"default\" : \"/stune\",\r\n", " }\r\n", " },\r\n", " {\r\n", " \"tag\" : \"boost30\",\r\n", " \"flags\" : \"ftrace\",\r\n", " \"sched_features\" : \"ENERGY_AWARE\",\r\n", " \"cpufreq\" : { \"governor\" : \"sched\" },\r\n", "\t \"cgroups\" : {\r\n", " \"conf\" : {\r\n", " \"schedtune\" : {\r\n", " \"/\" : {\"boost\" : 0 },\r\n", " \"/stune\" : {\"boost\" : 30 },\r\n", " }\r\n", " },\r\n", " \"default\" : \"/stune\",\r\n", " }\r\n", " },\r\n", " {\r\n", " \"tag\" : \"boost60\",\r\n", " \"flags\" : \"ftrace\",\r\n", " \"sched_features\" : \"ENERGY_AWARE\",\r\n", " \"cpufreq\" : { \"governor\" : \"sched\" },\r\n", "\t \"cgroups\" : {\r\n", " \"conf\" : {\r\n", " \"schedtune\" : {\r\n", " \"/\" : {\"boost\" : 0 },\r\n", " \"/stune\" : {\"boost\" : 60 },\r\n", " }\r\n", " },\r\n", " \"default\" : \"/stune\",\r\n", " }\r\n", " }\r\n", "\r\n", " ],\r\n", "\r\n", " /* Set of workloads to run on each platform configuration */\r\n", " \"wloads\" : {\r\n", " \"mixprof\" : {\r\n", " \"type\": \"rt-app\",\r\n", " \"conf\" : {\r\n", " \"class\" : \"profile\",\r\n", " \"params\" : {\r\n", " \"r5_10-60\" : {\r\n", " \"kind\" : \"Ramp\",\r\n", " \"params\" : {\r\n", " \"period_ms\" : 16,\r\n", " \"start_pct\" : 5,\r\n", " \"end_pct\" : 60,\r\n", " \"delta_pct\" : 5,\r\n", " \"time_s\" : 1,\r\n", " }\r\n", " }\r\n", " }\r\n", " },\r\n", " \"loadref\" : \"LITTLE\",\r\n", " }\r\n", " },\r\n", "\r\n", " /* Number of iterations for each workload */\r\n", " \"iterations\" : 1,\r\n", "\r\n", "}\r\n", "\r\n", "// vim :set tabstop=4 shiftwidth=4 expandtab\r\n" ] } ], "source": [ "stune_smoke_test = '../../tests/stune/smoke_test_ramp.config'\n", "!cat {stune_smoke_test}" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Write Test Cases" ] }, { "cell_type": "code", "execution_count": 120, "metadata": { "collapsed": false, "hidden": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "# SPDX-License-Identifier: Apache-2.0\r\n", "#\r\n", "# Copyright (C) 2015, ARM Limited and contributors.\r\n", "#\r\n", "# Licensed under the Apache License, Version 2.0 (the \"License\"); you may\r\n", "# not use this file except in compliance with the License.\r\n", "# You may obtain a copy of the License at\r\n", "#\r\n", "# http://www.apache.org/licenses/LICENSE-2.0\r\n", "#\r\n", "# Unless required by applicable law or agreed to in writing, software\r\n", "# distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT\r\n", "# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\r\n", "# See the License for the specific language governing permissions and\r\n", "# limitations under the License.\r\n", "#\r\n", "\r\n", "import logging\r\n", "import os\r\n", "\r\n", "from test import LisaTest\r\n", "\r\n", "import trappy\r\n", "from bart.common.Analyzer import Analyzer\r\n", "\r\n", "TESTS_DIRECTORY = os.path.dirname(os.path.realpath(__file__))\r\n", "TESTS_CONF = os.path.join(TESTS_DIRECTORY, \"smoke_test_ramp.config\")\r\n", "\r\n", "class STune(LisaTest):\r\n", " \"\"\"Tests for SchedTune framework\"\"\"\r\n", "\r\n", " @classmethod\r\n", " def setUpClass(cls, *args, **kwargs):\r\n", " super(STune, cls)._init(TESTS_CONF, *args, **kwargs)\r\n", "\r\n", " def test_boosted_utilization_signal(self):\r\n", " \"\"\"The boosted utilization signal is appropriately boosted\r\n", "\r\n", " The margin should match the formula\r\n", " (sched_load_scale - util) * boost\"\"\"\r\n", "\r\n", " for tc in self.conf[\"confs\"]:\r\n", " test_id = tc[\"tag\"]\r\n", "\r\n", " wload_idx = self.conf[\"wloads\"].keys()[0]\r\n", " run_dir = os.path.join(self.te.res_dir,\r\n", " \"rtapp:{}:{}\".format(test_id, wload_idx),\r\n", " \"1\")\r\n", "\r\n", " ftrace_events = [\"sched_boost_task\"]\r\n", " ftrace = trappy.FTrace(run_dir, scope=\"custom\",\r\n", " events=ftrace_events)\r\n", "\r\n", " first_task_params = self.conf[\"wloads\"][wload_idx][\"conf\"][\"params\"]\r\n", " first_task_name = first_task_params.keys()[0]\r\n", " rta_task_name = \"task_{}\".format(first_task_name)\r\n", "\r\n", " sbt_dfr = ftrace.sched_boost_task.data_frame\r\n", " boost_task_rtapp = sbt_dfr[sbt_dfr.comm == rta_task_name]\r\n", " ftrace.add_parsed_event(\"boost_task_rtapp\", boost_task_rtapp)\r\n", "\r\n", " # Avoid the first period as the task starts with a very\r\n", " # high load and it overutilizes the CPU\r\n", " rtapp_period = first_task_params[first_task_name][\"params\"][\"period_ms\"]\r\n", " task_start = boost_task_rtapp.index[0]\r\n", " after_first_period = task_start + (rtapp_period / 1000.)\r\n", "\r\n", " boost = tc[\"cgroups\"][\"conf\"][\"schedtune\"][\"/stune\"][\"boost\"] / 100.\r\n", " analyzer_const = {\r\n", " \"SCHED_LOAD_SCALE\": 1024,\r\n", " \"BOOST\": boost,\r\n", " }\r\n", " analyzer = Analyzer(ftrace, analyzer_const,\r\n", " window=(after_first_period, None))\r\n", " statement = \"(((SCHED_LOAD_SCALE - boost_task_rtapp:util) * BOOST) // 100) == boost_task_rtapp:margin\"\r\n", " error_msg = \"task was not boosted to the expected margin: {}\".\\\r\n", " format(boost)\r\n", " self.assertTrue(analyzer.assertStatement(statement), msg=error_msg)\r\n", "\r\n", "# vim :set tabstop=4 shiftwidth=4 expandtab\r\n" ] } ], "source": [ "stune_smoke_test = '../../tests/stune/smoke_test_ramp.py'\n", "!cat {stune_smoke_test}" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Tests execution" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "The execution of a test can be triggered from a LISA shell using nosetest with the test class as a parameter. This command:\n", "\n", "```bash\n", "$ nosetests -v tests/stune/smoke_test_ramp.py\n", "```\n", "\n", "will execute all the tests described in the **smoke_test_ramp.py** module and collect all the products in a timestamp named subfolder of the results folder.\n", "Tests PASS/FAILURE is reported after the completion of each test execution." ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "## Results reporting" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Detailed results of the experiments which compares also some base configurations with each test configuration can be reported in a tablular format using this command:\n", "\n", "```bash\n", "$ lisa-report --base noboost --tests '(boost15|boost30|boost60)'\n", "```\n" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "
![](\"SchedTune_SmokeTestResults.png\"/)
"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 2",
"language": "python",
"name": "python2"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.9"
},
"toc": {
"toc_cell": false,
"toc_number_sections": true,
"toc_threshold": 6,
"toc_window_display": false
}
},
"nbformat": 4,
"nbformat_minor": 0
}