{ "cells": [ { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "# Geekbench benchmark on Android\n", "\n", "Geekbench4 is an app offering several benchmarks to run on android smartphones. The one used in this notebook is the '**CPU**' benchmark, which runs several workloads that follow the lines of what is commonly run by smartphones (AES, JPEG codec, FFT, and so on). The benchmark runs all the tests in '**Single-Core**' mode as well as in '**Multi-Core**' in order to compare the single-thread and multi-thread performances of the device.\n", "\n", "**Do note that the benchmark will attempt to upload its results, which includes some hardware information**" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2017-05-03 10:54:02,800 INFO : root : Using LISA logging configuration:\n", "2017-05-03 10:54:02,801 INFO : root : /home/vagrant/lisa/logging.conf\n" ] } ], "source": [ "from conf import LisaLogging\n", "LisaLogging.setup()" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Populating the interactive namespace from numpy and matplotlib\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "2017-05-03 10:54:03,346 WARNING : EnergyModel : Unusual max capacity (1023), overriding capacity_scale\n" ] } ], "source": [ "%pylab inline\n", "\n", "import json\n", "import os\n", "\n", "# Support to access the remote target\n", "import devlib\n", "from env import TestEnv\n", "\n", "# Import support for Android devices\n", "from android import Screen, Workload\n", "\n", "# Support for trace events analysis\n", "from trace import Trace\n", "\n", "# Suport for FTrace events parsing and visualization\n", "import trappy\n", "\n", "import pandas as pd" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Support Functions" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "This function helps us run our experiments:" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": true, "deletable": true, "editable": true }, "outputs": [], "source": [ "def experiment():\n", " \n", " # Configure governor\n", " target.cpufreq.set_all_governors('sched')\n", " \n", " # Get workload\n", " wload = Workload.getInstance(te, 'Geekbench')\n", " \n", " # Run Geekbench workload\n", " wload.run(te.res_dir, test_name='CPU', collect='ftrace')\n", " \n", " # Dump platform descriptor\n", " te.platform_dump(te.res_dir)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Test environment setup\n", "For more details on this please check out **examples/utils/testenv_example.ipynb**." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "**devlib** requires the ANDROID_HOME environment variable configured to point to your local installation of the Android SDK. If you have not this variable configured in the shell used to start the notebook server, you need to run a cell to define where your Android SDK is installed or specify the ANDROID_HOME in your target configuration." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "In case more than one Android device are conencted to the host, you must specify the ID of the device you want to target in **my_target_conf**. Run **adb devices** on your host to get the ID." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "# Setup target configuration\n", "my_conf = {\n", "\n", " # Target platform and board\n", " \"platform\" : 'android',\n", " \"board\" : 'pixel',\n", " \n", " # Device\n", " \"device\" : \"0123456789ABCDEF\",\n", " \n", " # Android home\n", " \"ANDROID_HOME\" : \"/home/vagrant/lisa/tools/android-sdk-linux/\",\n", "\n", " # Folder where all the results will be collected\n", " \"results_dir\" : datetime.datetime.now()\\\n", " .strftime(\"Geekbench_example_\" + '%Y%m%d_%H%M%S'),\n", "\n", " # Define devlib modules to load\n", " \"modules\" : [\n", " 'cpufreq' # enable CPUFreq support\n", " ],\n", "\n", " # FTrace events to collect for all the tests configuration which have\n", " # the \"ftrace\" flag enabled\n", " \"ftrace\" : {\n", " \"events\" : [\n", " \"sched_switch\",\n", " \"sched_wakeup\",\n", " \"sched_wakeup_new\",\n", " \"sched_overutilized\",\n", " \"sched_load_avg_cpu\",\n", " \"sched_load_avg_task\",\n", " \"cpu_capacity\",\n", " \"cpu_frequency\",\n", " ],\n", " \"buffsize\" : 100 * 1024,\n", " },\n", "\n", " # Tools required by the experiments\n", " \"tools\" : [ 'trace-cmd', 'taskset'],\n", "}" ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false, "deletable": true, "editable": true, "scrolled": false }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2017-05-03 10:54:03,551 INFO : TestEnv : Using base path: /home/vagrant/lisa\n", "2017-05-03 10:54:03,551 INFO : TestEnv : Loading custom (inline) target configuration\n", "2017-05-03 10:54:03,552 INFO : TestEnv : External tools using:\n", "2017-05-03 10:54:03,553 INFO : TestEnv : ANDROID_HOME: /home/vagrant/lisa/tools/\n", "2017-05-03 10:54:03,554 INFO : TestEnv : CATAPULT_HOME: /home/vagrant/lisa/tools//platform-tools/systrace/catapult\n", "2017-05-03 10:54:03,555 INFO : TestEnv : Devlib modules to load: ['bl', 'cpufreq']\n", "2017-05-03 10:54:03,556 INFO : TestEnv : Connecting Android target [DEFAULT]\n", "2017-05-03 10:54:03,557 INFO : TestEnv : Connection settings:\n", "2017-05-03 10:54:03,558 INFO : TestEnv : None\n", "2017-05-03 10:54:03,761 INFO : android : ls command is set to ls -1\n", "2017-05-03 10:54:04,637 INFO : TestEnv : Initializing target workdir:\n", "2017-05-03 10:54:04,638 INFO : TestEnv : /data/local/tmp/devlib-target\n", "2017-05-03 10:54:06,555 INFO : TestEnv : Topology:\n", "2017-05-03 10:54:06,557 INFO : TestEnv : [[0, 1], [2, 3]]\n", "2017-05-03 10:54:06,748 INFO : TestEnv : Loading default EM:\n", "2017-05-03 10:54:06,749 INFO : TestEnv : /home/vagrant/lisa/libs/utils/platforms/pixel.json\n", "2017-05-03 10:54:07,301 INFO : TestEnv : Enabled tracepoints:\n", "2017-05-03 10:54:07,302 INFO : TestEnv : sched_switch\n", "2017-05-03 10:54:07,303 INFO : TestEnv : sched_wakeup\n", "2017-05-03 10:54:07,304 INFO : TestEnv : sched_wakeup_new\n", "2017-05-03 10:54:07,305 INFO : TestEnv : sched_overutilized\n", "2017-05-03 10:54:07,306 INFO : TestEnv : sched_load_avg_cpu\n", "2017-05-03 10:54:07,307 INFO : TestEnv : sched_load_avg_task\n", "2017-05-03 10:54:07,308 INFO : TestEnv : cpu_capacity\n", "2017-05-03 10:54:07,309 INFO : TestEnv : cpu_frequency\n", "2017-05-03 10:54:07,310 INFO : TestEnv : Set results folder to:\n", "2017-05-03 10:54:07,311 INFO : TestEnv : /home/vagrant/lisa/results/Geekbench_example_20170503_105403\n", "2017-05-03 10:54:07,312 INFO : TestEnv : Experiment results available also in:\n", "2017-05-03 10:54:07,313 INFO : TestEnv : /home/vagrant/lisa/results_latest\n" ] } ], "source": [ "# Initialize a test environment using:\n", "te = TestEnv(my_conf, wipe=False)\n", "target = te.target" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Workloads execution\n", "\n", "This is done using the **experiment** helper function defined above which is configured to run a **Geekbench - CPU** experiment." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stderr", "output_type": "stream", "text": [ "2017-05-03 10:54:07,990 INFO : Workload : Supported workloads available on target:\n", "2017-05-03 10:54:07,992 INFO : Workload : gmaps, youtube, jankbench, geekbench\n", "2017-05-03 10:54:10,644 INFO : Screen : Force manual orientation\n", "2017-05-03 10:54:10,645 INFO : Screen : Set orientation: PORTRAIT\n", "2017-05-03 10:54:12,496 INFO : Screen : Set brightness: 0%\n", "2017-05-03 10:54:14,950 INFO : Geekbench : adb -s HT67M0300128 logcat ActivityManager:* System.out:I *:S GEEKBENCH_RESULT:*\n", "2017-05-03 10:54:17,071 INFO : Geekbench : FTrace START\n", "2017-05-03 10:58:23,429 INFO : Geekbench : FTrace STOP\n", "2017-05-03 10:58:33,944 INFO : Screen : Set orientation: AUTO\n", "2017-05-03 10:58:35,526 INFO : Screen : Set brightness: AUTO\n" ] } ], "source": [ "# Initialize Workloads for this test environment\n", "results = experiment()" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Results analysis" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true, "deletable": true, "editable": true }, "source": [ "Geekbench4 uses a baseline score of 4000, which is the benchmark score of an Intel Core i7-6600U. Higher scores are better, with double the score indicating double the performance. You can have a look at the results for several android phones here https://browser.primatelabs.com/android-benchmarks" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "class Geekbench(object):\n", " \"\"\"\n", " Geekbench json results parsing class\n", " \"\"\"\n", " def __init__(self, filepath):\n", " with open(filepath) as fd:\n", " self.__json = json.loads(fd.read())\n", " \n", " self.benchmarks = {}\n", " for section in self.__json[\"sections\"]:\n", " self.benchmarks[section[\"name\"]] = section\n", " for workload in section[\"workloads\"]:\n", " self.benchmarks[section[\"name\"]][workload[\"name\"]] = workload \n", " \n", " def name(self):\n", " \"\"\"Get a human-readable name for the geekbench run\n", " \"\"\"\n", " gov = \"\"\n", " build = \"\"\n", " for metric in self.__json[\"metrics\"]:\n", " if metric[\"name\"] == \"Governor\":\n", " gov = metric[\"value\"]\n", " elif metric[\"name\"] == \"Build\":\n", " build = metric[\"value\"]\n", "\n", " return \"[build]=\\\"{}\\\" [governor]=\\\"{}\\\"\".format(build, gov)\n", " \n", " def benchmarks_names(self):\n", " \"\"\"Get a list of benchmarks (e.g. Single-Core, Multi-Core) found in the run results \n", " \"\"\"\n", " return [section[\"name\"] for section in self.__json[\"sections\"]]\n", " \n", " def workloads_names(self):\n", " \"\"\"Get a list of unique workloads (e.g. EAS, Dijkstra) found in the run results\n", " \"\"\"\n", " return [workload[\"name\"] for workload in self.benchmarks.values()[0][\"workloads\"]]\n", " \n", " def global_scores(self):\n", " \"\"\"Get the overall scores of each benchmark\n", " \"\"\"\n", " data = {}\n", " for benchmark in self.benchmarks_names():\n", " data[benchmark] = self.benchmarks[benchmark][\"score\"]\n", " return data\n", " \n", " def detailed_scores(self):\n", " \"\"\"Get the detailed workload scores of each benchmark\n", " \"\"\"\n", " benchmark_fields = [\"score\", \"runtime_mean\", \"rate_string\"]\n", " benches = {}\n", " benchmarks = self.benchmarks_names()\n", " workloads = self.workloads_names() \n", " \n", " for benchmark in benchmarks:\n", " data = {}\n", " for workload in workloads:\n", " data[workload] = {}\n", " for field in benchmark_fields:\n", " data[workload][field] = self.benchmarks[benchmark][workload][field] \n", " benches[benchmark] = data\n", " \n", " return benches" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "def display_bench_results(geekbench, detailed=False):\n", " print \"===== Global results =====\"\n", " \n", " scores = geekbench.global_scores()\n", " \n", " # Build dataframe for display\n", " row = []\n", " for bench_type, score in scores.iteritems():\n", " row.append(score)\n", " \n", " df = pd.DataFrame(data=row, index=scores.keys(), columns=[\"Global score\"])\n", " display(df)\n", " \n", " if not detailed:\n", " return\n", " \n", " print \"===== Detailed results =====\"\n", " \n", " scores = geekbench.detailed_scores()\n", " \n", " for benchmark, results in geekbench.detailed_scores().iteritems():\n", " print \"----- {} benchmark -----\".format(benchmark)\n", " # Build dataframe for display\n", " data = []\n", " idx = []\n", " columns = results.values()[0].keys()\n", " for workload, fields in results.iteritems():\n", " data.append(tuple(fields.values()))\n", " idx.append(workload)\n", " display (pd.DataFrame(data=data, index=idx, columns=columns))" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false, "deletable": true, "editable": true, "scrolled": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Analysing geekbench [build]=\"sailfishf-userdebug 7.1.1 NMF26P 3525730 dev-keys\" [governor]=\"sched\"\n", "===== Global results =====\n" ] }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
Global score
Single-Core1594
Multi-Core4170
\n", "
" ], "text/plain": [ " Global score\n", "Single-Core 1594\n", "Multi-Core 4170" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "===== Detailed results =====\n", "----- Single-Core benchmark -----\n" ] }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
scoreruntime_meanrate_string
AES6490.261358501.0 MB/sec
HDR30500.39478211.1 Mpixels/sec
Rigid Body Physics21470.2392386285.5 FPS
HTML5 Parse15650.1736687.11 MB/sec
Lua12200.2544861.25 MB/sec
Camera26050.1483247.22 images/sec
Histogram Equalization14250.24572944.6 Mpixels/sec
SQLite11890.61201233.0 Krows/sec
Face Detection21060.123343615.2 Ksubwindows/sec
Memory Copy26910.2693747.46 GB/sec
Memory Latency9171.001564471.8 ns
Canny21910.16768530.4 Mpixels/sec
PDF Rendering21370.45217456.8 Mpixels/sec
Gaussian Blur23130.12664940.5 Mpixels/sec
Speech Recognition12631.12676010.8 Words/sec
LLVM18171.057730124.9 functions/sec
Ray Tracing18180.339196265.5 Kpixels/sec
JPEG26050.24890221.0 Mpixels/sec
SGEMM5510.79032911.7 Gflops
LZMA12110.4951951.89 MB/sec
SFFT11300.1692162.82 Gflops
Memory Bandwidth24200.10046012.9 GB/sec
N-Body Physics14450.2564961.08 Mpairs/sec
Dijkstra20630.3747751.40 MTE/sec
HTML5 DOM6130.913885556.2 KElements/sec
\n", "
" ], "text/plain": [ " score runtime_mean rate_string\n", "AES 649 0.261358 501.0 MB/sec\n", "HDR 3050 0.394782 11.1 Mpixels/sec\n", "Rigid Body Physics 2147 0.239238 6285.5 FPS\n", "HTML5 Parse 1565 0.173668 7.11 MB/sec\n", "Lua 1220 0.254486 1.25 MB/sec\n", "Camera 2605 0.148324 7.22 images/sec\n", "Histogram Equalization 1425 0.245729 44.6 Mpixels/sec\n", "SQLite 1189 0.612012 33.0 Krows/sec\n", "Face Detection 2106 0.123343 615.2 Ksubwindows/sec\n", "Memory Copy 2691 0.269374 7.46 GB/sec\n", "Memory Latency 917 1.001564 471.8 ns\n", "Canny 2191 0.167685 30.4 Mpixels/sec\n", "PDF Rendering 2137 0.452174 56.8 Mpixels/sec\n", "Gaussian Blur 2313 0.126649 40.5 Mpixels/sec\n", "Speech Recognition 1263 1.126760 10.8 Words/sec\n", "LLVM 1817 1.057730 124.9 functions/sec\n", "Ray Tracing 1818 0.339196 265.5 Kpixels/sec\n", "JPEG 2605 0.248902 21.0 Mpixels/sec\n", "SGEMM 551 0.790329 11.7 Gflops\n", "LZMA 1211 0.495195 1.89 MB/sec\n", "SFFT 1130 0.169216 2.82 Gflops\n", "Memory Bandwidth 2420 0.100460 12.9 GB/sec\n", "N-Body Physics 1445 0.256496 1.08 Mpairs/sec\n", "Dijkstra 2063 0.374775 1.40 MTE/sec\n", "HTML5 DOM 613 0.913885 556.2 KElements/sec" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "----- Multi-Core benchmark -----\n" ] }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
scoreruntime_meanrate_string
AES20860.3365751.57 GB/sec
HDR84350.58892630.6 Mpixels/sec
Rigid Body Physics59760.36393717496.3 FPS
HTML5 Parse47470.26998921.6 MB/sec
Lua34750.3857673.57 MB/sec
Camera72720.20374320.2 images/sec
Histogram Equalization37850.368308118.3 Mpixels/sec
SQLite32871.16635091.1 Krows/sec
Face Detection57020.1880871.67 Msubwindows/sec
Memory Copy39740.37865111.0 GB/sec
Memory Latency18650.705881232.1 ns
Canny66480.26539092.2 Mpixels/sec
PDF Rendering59100.663689157.0 Mpixels/sec
Gaussian Blur61750.200005108.2 Mpixels/sec
Speech Recognition37551.52038432.1 Words/sec
LLVM67851.141085466.6 functions/sec
Ray Tracing50020.498715730.4 Kpixels/sec
JPEG71920.36303357.9 Mpixels/sec
SGEMM16971.05781435.9 Gflops
LZMA39770.5872796.21 MB/sec
SFFT34430.2506608.58 Gflops
Memory Bandwidth33750.14245118.0 GB/sec
N-Body Physics38910.3986172.91 Mpairs/sec
Dijkstra51680.6018403.50 MTE/sec
HTML5 DOM21581.0310611.96 MElements/sec
\n", "
" ], "text/plain": [ " score runtime_mean rate_string\n", "AES 2086 0.336575 1.57 GB/sec\n", "HDR 8435 0.588926 30.6 Mpixels/sec\n", "Rigid Body Physics 5976 0.363937 17496.3 FPS\n", "HTML5 Parse 4747 0.269989 21.6 MB/sec\n", "Lua 3475 0.385767 3.57 MB/sec\n", "Camera 7272 0.203743 20.2 images/sec\n", "Histogram Equalization 3785 0.368308 118.3 Mpixels/sec\n", "SQLite 3287 1.166350 91.1 Krows/sec\n", "Face Detection 5702 0.188087 1.67 Msubwindows/sec\n", "Memory Copy 3974 0.378651 11.0 GB/sec\n", "Memory Latency 1865 0.705881 232.1 ns\n", "Canny 6648 0.265390 92.2 Mpixels/sec\n", "PDF Rendering 5910 0.663689 157.0 Mpixels/sec\n", "Gaussian Blur 6175 0.200005 108.2 Mpixels/sec\n", "Speech Recognition 3755 1.520384 32.1 Words/sec\n", "LLVM 6785 1.141085 466.6 functions/sec\n", "Ray Tracing 5002 0.498715 730.4 Kpixels/sec\n", "JPEG 7192 0.363033 57.9 Mpixels/sec\n", "SGEMM 1697 1.057814 35.9 Gflops\n", "LZMA 3977 0.587279 6.21 MB/sec\n", "SFFT 3443 0.250660 8.58 Gflops\n", "Memory Bandwidth 3375 0.142451 18.0 GB/sec\n", "N-Body Physics 3891 0.398617 2.91 Mpairs/sec\n", "Dijkstra 5168 0.601840 3.50 MTE/sec\n", "HTML5 DOM 2158 1.031061 1.96 MElements/sec" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "for f in os.listdir(te.res_dir):\n", " if f.endswith(\".gb4\"):\n", " geekbench = Geekbench(te.res_dir + \"/\" + f)\n", " \n", " print \"Analysing geekbench {}\".format(geekbench.name())\n", " display_bench_results(geekbench, True)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Analysing several runs" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "It can be interesting to compare Geekbench results with different parameters (kernel, drivers) and even different devices to gauge the impact of these parameters. As Geekbench results can vary a bit from one run to another, having a set of repeated runs is preferable.\n", "\n", "The following section will grab the results of all the **Geekbench\\_exemple\\_\\*** results found in the LISA results directory" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "import glob\n", "\n", "def fetch_results():\n", " results_path = os.path.join(te.LISA_HOME, \"results\")\n", " \n", " results_dirs = [results_path + \"/\" + d for d in os.listdir(results_path) if d.startswith(\"Geekbench_example_\")]\n", " \n", " res = []\n", " \n", " for d in results_dirs:\n", " bench_file = glob.glob(\"{}/*.gb4\".format(d))[0]\n", " res.append(Geekbench(bench_file))\n", " \n", " return res\n", "\n", "def compare_runs():\n", " geekbenches = fetch_results()\n", " \n", " # Pick one run to build a baseline template\n", " benchmarks = geekbenches[0].benchmarks_names()\n", " workloads = geekbenches[0].workloads_names()\n", " \n", " stats = [\"avg\", \"min\", \"max\"]\n", " count = len(geekbenches)\n", " \n", " print \"Parsing {} runs\".format(count)\n", "\n", " \n", " # Initialize stats\n", " results = {benchmark : \n", " {\"min\" : sys.maxint, \"max\" : 0, \"avg\" : 0} \n", " for benchmark in benchmarks}\n", " \n", " # Get all the data\n", " for benchmark in results.iterkeys():\n", " for bench in geekbenches:\n", " score = bench.global_scores()[benchmark]\n", " \n", " if score > results[benchmark][\"max\"]:\n", " results[benchmark][\"max\"] = score\n", " \n", " if score < results[benchmark][\"min\"]:\n", " results[benchmark][\"min\"] = score\n", " \n", " results[benchmark][\"avg\"] += score\n", " \n", " results[benchmark][\"avg\"] /= count\n", " \n", " # Convert data to Dataframe\n", " data = []\n", "\n", " for benchmark in results.iterkeys():\n", " row = []\n", " for stat in stats:\n", " row.append(results[benchmark][stat])\n", " data.append(tuple(row))\n", " \n", " df = pd.DataFrame(data, index=results.iterkeys(), columns=stats)\n", " \n", " return df" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Parsing 2 runs\n" ] }, { "data": { "text/html": [ "
\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
avgminmax
Single-Core160215941610
Multi-Core417641704182
\n", "
" ], "text/plain": [ " avg min max\n", "Single-Core 1602 1594 1610\n", "Multi-Core 4176 4170 4182" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "display(compare_runs())" ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.6" }, "toc": { "toc_cell": false, "toc_number_sections": true, "toc_threshold": 6, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 0 }