1page.title=Instrumentation Testing 2pdk.version=1.0 3doc.type=guide 4@jd:body 5 6<div id="qv-wrapper"> 7<div id="qv"> 8<h2>In this document</h2> 9<a name="toc"/> 10<ul> 11<li><a href="#androidInstrumentationTestingFramework">Instrumentation Framework</a></li> 12<li><a href="#androidInstrumentationFrameworkPlatform">Platform Test Suites</a></li> 13<li><a href="#androidInstrumentationFrameworkWritingRunning">Running Tests</a></li> 14<li><a href="#androidInstrumentationTestingCreating">Writing Tests</a></li> 15<li><a href="#troubleshooting">Troubleshooting</a></li> 16</ul> 17</div> 18</div> 19 20<p>This document describes how to use the Instrumentation Framework to write test cases. Instrumentation testing allows you to verify a particular feature or behavior with an automated JUnit TestCase. You can launch activities and providers within an application, send key events, and make assertions about various UI elements. </p> 21<p>You should have a working knowledge of the following:</p> 22<ul> 23 <li> Android Application Framework</li> 24 <li> Using <code>adb</code>, <code>am</code> and various logging functionality </li> 25 <li> A brief understanding of the application of interest, that is, the names of the classes which handle the intents etc. </li> 26 <li> JUnit testing.</li> 27</ul> 28<p> Each Android application runs in its own process. Instrumentation kills the application process and restarts the process with Instrumentation. Instrumentation gives a handle to the application context used to poke around the application to validate test assertions, allowing you to write test cases to test applications at a much lower level than UI screen shot tests. Note that Instrumentation cannot catch UI bugs. </p> 29 30 31<a name="androidInstrumentationTestingFramework"></a><h3>Instrumentation Framework</h3> 32 33 34 35<a name="androidInstrumentationTestingClasses"></a><h4>Classes</h4> 36 37<p> The following classes help glue together <code>Instrumentation</code> with JUnit testing. </p> 38<table> 39 <tr> 40 <th scope="col">Class</th> 41 <th scope="col">Description</th></tr> 42 <tr> 43 <td valign="top"><code>InstrumentationTestCase</code></td> 44 <td valign="top"> 45 <p>This extends the standard JUnit <code>TestCase</code> and offers access to an <code>Instrumentation</code> class. Write tests inside your instrumentation class any way you see fit. For example, your test might launch activities and send key events. For this to work properly, the instrumentation needs to be injected into the test case.</p> </td> 46 </tr> 47 <tr> 48 <td valign="top"><code>InstrumentationTestRunner</code></td> 49 <td valign="top">The instrumentation test runner is an instrumentation that runs instrumentation test cases and injects itself into each test case. Instrumentation test cases need to be grouped together with an instrumentation test runner with the appropriate target package.</td> 50 </tr> 51 <tr> 52 <td valign="top"><code>InstrumentationTestSuite</code></td> 53 <td valign="top">The instrumentation test suite is a simple extension of the standard JUnit <code>TestSuite</code> that keeps a member <code>Instrumentation</code> variable on hand to inject into each <code>TestCase</code> before running them. It is used by <code>InstrumentationTestRunner</code>.</td> 54 </tr> 55</table> 56<p> Three additional base classes extend <code>InstrumentationTestCase</code> to allow you to test <code>Activity</code> and <code>Provider</code> classes:</p> 57<table> 58 <tr> 59 <th scope="col">Class</th> 60 <th scope="col">Description</th> 61 </tr> 62 <tr> 63 <td valign="top"><code>ActivityTestCase</code></td> 64 <td valign="top"><p>This class can be used to write tests for a specific activity. An activity is launched in its <code>setUp()</code> method and finished with <code>tearDown</code>. If you write a test case that extends <code>ActivityTestCase</code>, you can write tests that access the activity using <code>getActivity()</code> and assume it has been set up properly.</p></td> 65 </tr> 66 <tr> 67 <td valign="top"><code>ServiceTestCase</code></td> 68 <td valign="top">This test case provides a framework in which you can test Service classes in a controlled environment. It provides basic support for the lifecycle of a Service, and hooks by which you can inject various dependencies and control the environment in which your Service is tested.</td> 69 </tr> 70 <tr> 71 <td valign="top"><code>SingleLaunchActivityTestCase</code></td> 72 <td valign="top">This class is similar to <code>ActivityTestCase</code> except that the activity is launched once per class instead of every time the test case calls setup. </td> 73 </tr> 74 <tr> 75 <td valign="top"><code>ProviderTestCase</code></td> 76 <td valign="top">This class is similar to <code>ActivityTestCase</code> except that it will setup, tear down, and provide access to the <code>Provider</code> of your choice.</td> 77 </tr> 78</table> 79 80 81<a name="androidInstrumentationFrameworkamCommand"></a><h4>Understanding the am Command</h4> 82 83<p>The am command is a command-line interface to the ActivityManager (see <a href="http://code.google.com/android/reference/android/app/ActivityManager.html">http://code.google.com/android/reference/android/app/ActivityManager.html</a> for details). <code>am</code> is used to start and instrument activities using the adb shell command, as shown in the snippet below:</p> 84<pre class="prettify"> 85> adb shell am 86usage: am [start|instrument] 87 am start [-a <ACTION>] [-d <DATA_URI>] [-t <MIME_TYPE>] 88 [-c <CATEGORY> [-c <CATEGORY>] ...] 89 [-e <EXTRA_KEY> <EXTRA_VALUE> [-e <EXTRA_KEY> <EXTRA_VALUE> ...] 90 [-n <COMPONENT>] [-D] [<URI>] 91 am instrument [-e <ARG_NAME> <ARG_VALUE>] [-p <PROF_FILE>] 92 [-w] <COMPONENT> 93For example, to start the Contacts application you can use 94> adb shell am start -n com.google.android.contacts/.ContactsActivity 95</pre> 96 97 98<a name="androidInstrumentationFrameworkPlatform"></a><h3>Platform Test Suites</h3> 99 100<p>This section provides an overview for various unit and functional test cases that can be executed through the instrumentation framework.</p> 101 102 103<a name="androidTestingPlatformFramework"></a><h4>Framework Tests</h4> 104 105<p>Framework test cases test the Android application framework or specific Android application functionality that requires an Android runtime context. These tests can be found in <code>//device/tests</code> and <code>//device/apps/AndroidTests</code>.</p> 106 107 108<a name="androidTestingPlatformCoreLibrary"></a><h4>Core Library</h4> 109 110<p>Core library test cases test the Android library functionality that does not require an Android runtime context. These tests are split into Android library (android.* package space) tests at <code>//device/java/tests</code> and Java library (java.*, javax.*, etc. packages) tests at <code>//device/dalvik/libcore/.../tests</code>.</p> 111 112 113<a name="androidInstrumentationFrameworkWritingRunning"></a><h3>Running Tests</h3> 114 115<p>Each instrumentation test case is similar to an Android application with the distinction that it starts another application. For example, have a look in the <code>tests/Contacts</code> directory. </p> 116<ul> 117 <li> There should be a Makefile and an Android Manifest file. </li> 118 <li> Tests are located in <code>tests/Contacts/src/com/google/android/contactstests</code>. </li> 119 <li> The Instrumentation Test Runner is located at <code>tests/Contacts/src/com/google/android/contactstests/functional/ContactsInstrumentationTestRunner.java</code>.</li> 120</ul> 121<p>Suppose you have a makefile with <code>Contactstests</code> as the target. </p> 122<ul> 123 <li> <code>make Contactstests</code>: Compiles the test cases. </li> 124 <li> <code>adb install Contactstests.apk</code>: Installs the apk on the device. </li> 125 <li> Use the adb shell <code>am</code> command to run them. </li> 126</ul> 127<p> To run your tests, use the <code>am instrument</code> command with your <code>InstrumentationTestRunner</code> as its argument. Results are printed as a result of the instrumentation. For example, the following snippet displays the output after running the framework tests with one test failing (note the unusual syntax caused by how instrumentations are run via <code>am</code>):</p> 128<pre class="prettify"> 129$ adb shell am instrument -w com.google.android.frameworktest/.tests.FrameworkInstrumentationTestRunner 130INSTRUMENTATION_RESULT: test results:=.......F....... 131Time: 6.837 132There was 1 failure: 1331) testSetUpConditions(com.google.android.frameworktest.tests.focus.RequestFocusTest)junit.framework.AssertionFailedError: requestFocus() should work from onCreate. 134 at com.google.android.frameworktest.tests.focus.RequestFocusTest.testSetUpConditions(RequestFocusTest.java:66) 135 at java.lang.reflect.Method.invokeNative(Native Method) 136 at android.test.InstrumentationTestSuite.runTest(InstrumentationTestSuite.java:73) 137 at android.test.InstrumentationTestSuite.runTest(InstrumentationTestSuite.java:73) 138 at android.test.InstrumentationTestRunner.onStart(InstrumentationTestRunner.java:151) 139 at android.app.Instrumentation$InstrumentationThread.run(Instrumentation.java:1088) 140 141FAILURES!!! 142Tests run: 14, Failures: 1, Errors: 0 143 144<RETURN> to continue 145 146INSTRUMENTATION_CODE: -1 147$ 148</pre> 149 150 151<a name="androidInstrumentationTestingRunningAll"></a><h4>All Tests with Default TestRunner behavior</h4> 152 153<p>If no class or package is passed in to run, InstrumentationTestRunner will automatically find and run all tests under the package of the test application (as defined by the <code>android:targetPackage</code> attribute of the instrumentation defined in its manifest file). 154</p> 155<pre> 156$ adb shell am instrument -w \ 157 com.android.samples.tests/android.test.InstrumentationTestRunner 158 159INSTRUMENTATION_RESULT: Test results for InstrumentationTestRunner=.......... 160Time: 2.317 161 162OK (10 tests) 163 164 165INSTRUMENTATION_CODE: -1 166</pre> 167 168 169<a name="androidTestingTestSinglePakcage"></a><h4>Running all Tests Under Single Package</h4> 170 171<p>If you have many tests under one package, use the <code>-e package <packagename></code> option to run all tests under that package without having to manually create a test suite.</p> 172<pre> 173$ adb shell am instrument -w \ 174 -e package com.android.samples.view \ 175 com.android.samples.tests/android.test.InstrumentationTestRunner 176INSTRUMENTATION_RESULT: Test results for InstrumentationTestRunner=........ 177Time: 1.587 178 179OK (8 tests) 180</pre> 181 182 183<a name="androidTestingSingleTestSuite"></a><h4>Running a Single Test Suite</h4> 184 185<p>If you prefer to explicitly state which tests comprise all of your tests, you can define a test suite and run that directly. By convention, all test packages in your system should have at least one suite called <code>AllTests</code> (see <code>AllTests.java</code>). To run all of the tests using the <code>AllTests</code> suite for the api demos test app:</p> 186 187<pre> 188$ adb shell am instrument -w \ 189 -e class com.android.samples.AllTests \ 190 com.android.samples.tests/android.test.InstrumentationTestRunner 191 192INSTRUMENTATION_RESULT: Test results for AllTests=.......... 193Time: 2.286 194 195OK (10 tests) 196 197 198INSTRUMENTATION_CODE: -1 199</pre> 200 201 202<a name="androidInstrumentationTestingRunningSingleTestCase"></a><h4>A Single Test Case</h4> 203 204<pre> 205$ adb shell am instrument -w \ 206 -e class com.android.samples.view.Focus2ActivityTest \ 207 com.android.samples.tests/android.test.InstrumentationTestRunner 208 209INSTRUMENTATION_RESULT: Test results for Focus2ActivityTest=.... 210Time: 1.359 211 212OK (4 tests) 213 214 215INSTRUMENTATION_CODE: -1 216</pre> 217 218 219<a name="androidInstrumentationTestingRunningSingleTest"></a><h4>A Single Test</h4> 220 221<pre> 222$ adb shell am instrument -w \ 223 -e class com.android.samples.view.Focus2ActivityTest#testGoingLeftFromRightButtonGoesToCenter \ 224 com.android.samples.tests/android.test.InstrumentationTestRunner 225 226INSTRUMENTATION_RESULT: Test results for Focus2ActivityTest=. 227Time: 0.51 228 229OK (1 test) 230 231 232INSTRUMENTATION_CODE: -1 233</pre> 234 235 236<a name="androidTestingDebugging"></a><h4>Attaching a debugger to your test</h4> 237 238<p>In order to debug your test code, instruct the controller to stop and wait for the debugger by adding <code>-e debug true</code> to your 239command line. This causes the test runner to stop and wait for the debugger just before calling your <code>setUp()</code> method. For example,</p> 240 241<pre> 242$ adb shell am instrument -w \ 243 -e debug true \ 244 com.android.samples.tests/android.test.InstrumentationTestRunner 245</pre> 246 247 248<a name="androidInstrumentationTestingCreating"></a><h3>Writing Tests</h3> 249 250<p>When writing tests, refer to the ApiDemos tests as models (located at <code>//device/samples/ApiDemos</code>). This section provides an overview of the test structure with ApiDemos.</p> 251 252 253<a name="androidTestingLocationFiles"></a><h4>Location of Files</h4> 254 255<p>Test packages should use the following structure and include <code>Android.mk</code>, <code>AndroidManifest.xml</code>, <code>AllTests.java</code>, and a src directory that mirrors the src directory of the tested application.</p> 256<p>Files are located within a <code>tests</code> directory found in the root directory:</p> 257<pre> 258$ find samples/ApiDemos/tests 259samples/ApiDemos/tests 260samples/ApiDemos/tests/Android.mk 261samples/ApiDemos/tests/AndroidManifest.xml 262samples/ApiDemos/tests/src 263samples/ApiDemos/tests/src/com 264samples/ApiDemos/tests/src/com/google 265samples/ApiDemos/tests/src/com/google/android 266samples/ApiDemos/tests/src/com/google/android/samples 267samples/ApiDemos/tests/src/com/google/android/samples/AllTests.java 268samples/ApiDemos/tests/src/com/google/android/samples/ApiDemosTest.java 269samples/ApiDemos/tests/src/com/google/android/samples/os 270samples/ApiDemos/tests/src/com/google/android/samples/os/MorseCodeConverterTest.java 271samples/ApiDemos/tests/src/com/google/android/samples/view 272samples/ApiDemos/tests/src/com/google/android/samples/view/Focus2ActivityTest.java 273samples/ApiDemos/tests/src/com/google/android/samples/view/Focus2AndroidTest.java 274</pre> 275 276 277<a name="androidTestingContentMakefile"></a><h4>Contents of makefile</h4> 278 279<p>The contents of the makefile are similar to a normal application with the addition of a <code>LOCAL_INSTRUMENTATION_FOR</code> declaration.<p /> 280<pre> 281# Add appropriate copyright banner here 282LOCAL_PATH:= $(call my-dir) 283include $(CLEAR_VARS) 284 285# We only want this apk build for tests. 286LOCAL_MODULE_TAGS := tests 287 288# Include all test java files. 289LOCAL_SRC_FILES := $(call all-java-files-under, src) 290 291# Notice that we don't have to include the src files of ApiDemos because, by 292# running the tests using an instrumentation targeting ApiDemos, we 293# automatically get all of its classes loaded into our environment. 294 295LOCAL_PACKAGE_NAME := ApiDemosTests 296 297LOCAL_INSTRUMENTATION_FOR := ApiDemos 298 299include $(BUILD_PACKAGE) 300</pre> 301 302 303<a name="androidTestingContentManifest"></a><h4>Content of Manifest</h4> 304 305<p>Use the following example to create an <code>AndroidManifest.xml</code> file that declares the instrumentation. Specify that the framework supplied Instrumentation TestRunner targest the package of your application, allowing the tests that are run with the instrumentation to get access to all of the classes of your application without having to build the source into the test app. The name of the test application is typically the same as your target application with <code>.tests</code> appended. </p> 306<pre> 307# Add appropriate copyright banner here 308<manifest xmlns:android="http://schemas.android.com/apk/res/android" 309 package="com.android.samples.tests"> 310 311 <uses-permission android:name="android.permission.RUN_INSTRUMENTATION" /> 312 313 <!-- 314 This declares that this app uses the instrumentation test runner targeting 315 the package of com.android.samples. To run the tests use the command: 316 "adb shell am instrument -w com.android.samples.tests/android.test.InstrumentationTestRunner" 317 --> 318 <instrumentation android:name="android.test.InstrumentationTestRunner" 319 android:targetPackage="com.android.samples" 320 android:label="Tests for Api Demos."/> 321 322</manifest> 323</pre> 324<p> </p> 325<p>The following snippet will prefix the <code>/android.test.InstrumentationTestRunner</code> when running tests from the command line:</p> 326<pre> 327$ adb shell am instrument -w \ 328 com.android.samples.tests/android.test.InstrumentationTestRunner 329</pre> 330 331 332<a name="androidInstrumentationTestingCreatingTestRunner"></a><h4>New Instrumentation TestRunner</h4> 333 334<p>Create a class that derives from this class. You must override two abstract methods; one that returns the class loader of the target package, and another that defines all of the tests within the package. For example, the snippet below displays the test runner for the framework tests.</p> 335<pre class="prettify"> 336public class FrameworkInstrumentationTestRunner extends InstrumentationTestRunner { 337 338 @Override 339 public TestSuite getAllTests() { 340 InstrumentationTestSuite suite = new InstrumentationTestSuite(this); 341 342 suite.addTestSuite(FocusAfterRemovalTest.class); 343 suite.addTestSuite(RequestFocusTest.class); 344 suite.addTestSuite(RequestRectangleVisibleTest.class); 345 return suite; 346 } 347 348 @Override 349 public ClassLoader getLoader() { 350 return FrameworkInstrumentationTestRunner.class.getClassLoader(); 351 } 352} 353</pre> 354<p> Next, in an appropriate <code>AndroidManifest.xml</code>, define the instrumentation for the derived class with the appropriate <code>android:targetPackage</code> set. For example, the snippet below defines the instrumentation runner for the framework tests.</p> 355<pre class="prettify"> 356<uses-permission android:name="android.permission.RUN_INSTRUMENTATION" /> 357 358<instrumentation android:name="android.tests.FrameworkInstrumentationTestRunner" 359 android:targetPackage="com.google.android.frameworktest" 360 android:label="framework instrumentation test runner" /> 361</pre> 362 363 364<a name="androidInstrumentationTestingCreatingTestCase"></a><h4>New InstrumentationTestCase</h4> 365 366<p> To create a new test case, write a class that extends <code>InstrumentationTestCase</code> in the same application as your test runner. The following snippet illustrates an example <code>ActivityTestCase</code> that tests an activity named <code>MyActivity</code>.</p> 367<pre class="prettify"> 368public class ButtonPressTest extends ActivityTestCase<MyActivity> { 369 370 Button mLeftButton; 371 372 public ButtonPressTest() { 373 super("com.example", MyActivity.class); 374 } 375 376 @Override 377 public void setUp() throws Exception { 378 super.setUp(); 379 mLeftButton = (Button) getActivity().findViewById(R.id.leftButton); 380 } 381 382 public void testFocusMovesToRight() throws Exception { 383 assertTrue(mLeftButton.hasFocus()); 384 getInstrumentation().sendCharacterSync(KeyEvent.KEYCODE_DPAD_RIGHT); 385 386 Button rightButton = (Button) getActivity().findViewById(R.id.rightButton); 387 assertTrue(rightButton.hasFocus()); 388 } 389 390 // could have several more tests... 391} 392</pre> 393 394 395<a name="androidInstrumentationFrameworkTestCase"></a><h4>Exploring a Test Case</h4> 396 397<p> The test case described in this section adds and tests a new Contact. Note that you can send intents, register intent receivers, etc. </p> 398<p><code>Instrumentation.java</code> has helper functions that send key events and strings, for example: </p> 399<ul> 400 <li><code>getInstrumentation()</code>: Returns the handle to the instrumentation </li> 401 <li><code>sendCharacterSync</code>: Sends a character. </li> 402 <li><code>sendStringSync</code>: Sends a string to an input box. </li> 403 <li><code>sendKeyDownUpSync</code>: Sends a specific keyevent. </li> 404 <li><code>sendTrackballEventSync</code>: Sends a trackball event.</li> 405</ul> 406<p> You can find the test case below at <code>device/tests/Contacts.</code></p> 407<pre class="prettify"> 408private void addNewContact(String name, int star, int phoneType, String number, String label, 409 String email, int emailType){ 410 ContentValues values = new ContentValues(); 411 Uri phoneUri = null; 412 Uri emailUri = null; 413 414 values.put(Contacts.People.NAME, name); 415 values.put(Contacts.People.STARRED, star); 416 417 //Add Phone Numbers 418 Uri uri = mActivity.getContentResolver().insert(Contacts.People.CONTENT_URI, values); 419 phoneUri = Uri.withAppendedPath(uri, Contacts.People.Phones.CONTENT_DIRECTORY); 420 421 values.clear(); 422 values.put(Contacts.Phones.TYPE, phoneType); 423 values.put(Contacts.Phones.NUMBER, number); 424 values.put(Contacts.Phones.LABEL, label); 425 mActivity.getContentResolver().insert(phoneUri, values); 426 427 //Add Email 428 emailUri = Uri.withAppendedPath(uri, ContactMethods.CONTENT_DIRECTORY); 429 430 values.clear(); 431 values.put(ContactMethods.KIND, Contacts.KIND_EMAIL); 432 values.put(ContactMethods.DATA, email); 433 values.put(ContactMethods.LABEL, ""); 434 values.put(ContactMethods.TYPE, emailType); 435 mActivity.getContentResolver().insert(emailUri, values); 436} 437 438 439 public void testAddSaveSingleContact(){ 440 int previousCount = mActivity.getListView().getCount(); 441 String message; 442 443 addNewContact(INPUT_NAME_1 + "1", "5435754532", "1" + INPUT_EMAIL_1, CONFIRM_OPTION); 444 445 message = "Added 1 to initial length=" + previousCount + ", but resulted with a count=" + 446 mActivity.getListView().getCount(); 447 assertEquals(message, ++previousCount, mActivity.getListView().getCount()); 448 449 // Check Content; Name; Num; Starred 450 assertEquals(INPUT_NAME_1 + "1", getTextFromView(0, android.R.id.text1)); 451 assertEquals("5435754532", getTextFromView(0, android.R.id.text2)); 452 453 //Check email is saved 454 //cursor = returnEmailCursorAtId("1"); 455 Uri uri = Uri.parse("content://contacts/people/1"); 456 uri = Uri.withAppendedPath(uri, ContactMethods.CONTENT_DIRECTORY); 457 Cursor cursor = mActivity.getContentResolver().query(uri, CONTACTS_COLUMNS, null, null, null); 458 assertTrue("returnEmailCursorAtId: Moving cursor to first row has failed", cursor.first()); 459 460 int dataIndex = cursor.getColumnIndexOrThrow("data"); 461 assertEquals("1" + INPUT_EMAIL_1, cursor.getString(dataIndex)); 462 cursor.deactivate(); 463} 464 </pre> 465 466 467<a name="androidTestingKindsofTests"></a><h4>Deciding Kinds of Tests to Write</h4> 468 469<p>Once you are bootstrapped with your test application, you can start writing tests. There are three of types of tests you may wish to write:</p> 470<p><ul> 471<li> <strong>TestCase</strong>: The standard junit test case. 472</li> 473<li> <strong>AndroidTestCase</strong>: A test case with access to a Context object that is injected for you by the instrumentation test runner. 474</li> 475<li> <strong>InstrumentationTestCase</strong>: A test case with access to an Instrumentation, which can be used to launch activities, content providers, send key events, etc. 476</li> 477</ul> 478</p> 479<p>The API Demos test suite includes examples of all three styles and can be used as a guideline for writing each type of test.</p> 480<p>There are two utility classes available for the most common uses of InstrumentationTestCase: ActivityTestCase and ProviderTestCase. See their javadoc for more information. 481</p> 482 483 484<a name="troubleshooting"></a><h3>Troubleshooting</h3> 485 486<p>If you run your test cases and nothing appears to happen, have a look at <code>adb logcat</code>. The following is a common problem:</p> 487<pre class="prettify"> 488I/dalvikvm( 688): threadid=11: attached from native, name=Binder Thread #1 489I/dalvikvm( 688): threadid=13: attached from native, name=Binder Thread #2 490W/ActivityManager( 469): Unable to find instrumentation info for: ComponentInfo{com.google.android.browser_instrumentation/com.google.android.browser_instrumentation.BrowserWebkitLayoutInstrumentation} 491D/AndroidRuntime( 688): Shutting down VM 492E/AndroidRuntime( 688): ERROR: thread attach failed 493</pre> 494<p>It's possible that the instrumentation apk isn't installed on your device or that the package name is incorrect in the Manifest file. </p> 495