1page.title=Testing from Other IDEs 2parent.title=Testing 3parent.link=index.html 4@jd:body 5 6<div id="qv-wrapper"> 7 <div id="qv"> 8 <h2>In this document</h2> 9 <ol> 10 <li> 11 <a href="#CreateTestProjectCommand">Working with Test Projects</a> 12 <ol> 13 <li> 14 <a href="#CreateTestProject">Creating a test project</a> 15 </li> 16 <li> 17 <a href="#UpdateTestProject">Updating a test project</a> 18 </li> 19 </ol> 20 </li> 21 <li> 22 <a href="#CreateTestApp">Creating a Test Package</a> 23 </li> 24 <li> 25 <a href="#RunTestsCommand">Running Tests</a> 26 <ol> 27 <li> 28 <a href="#RunTestsAnt">Quick build and run with Ant</a> 29 </li> 30 <li> 31 <a href="#RunTestsDevice">Running tests on a device or emulator</a> 32 </li> 33 </ol> 34 </li> 35 <li> 36 <a href="#AMSyntax">Using the Instrument Command</a> 37 <ol> 38 <li> 39 <a href="#AMOptionsSyntax">Instrument options</a> 40 </li> 41 <li> 42 <a href="#RunTestExamples">Instrument examples</a> 43 </li> 44 </ol> 45 </li> 46 </ol> 47 <h2>See Also</h2> 48 <ol> 49 <li> 50 <a href="{@docRoot}guide/topics/testing/testing_android.html"> 51 Testing Fundamentals</a> 52 </li> 53 <li> 54 <a href="{@docRoot}guide/developing/tools/adb.html">Android Debug Bridge</a> 55 </li> 56 </ol> 57 </div> 58</div> 59<p> 60 This document describes how to create and run tests directly from the command line. 61 You can use the techniques described here if you are developing in an IDE other than Eclipse 62 or if you prefer to work from the command line. This document assumes that you already know how 63 to create a Android application in your programming environment. Before you start this 64 document, you should read the topic 65 <a href="{@docRoot}guide/topics/testing/testing_android.html">Testing Fundamentals</a>, 66 which provides an overview of Android testing. 67</p> 68<p> 69 If you are developing in Eclipse with ADT, you can set up and run your tests 70 directly in Eclipse. For more information, please read 71 <a href="{@docRoot}guide/developing/testing/testing_eclipse.html"> 72 Testing from Eclipse with ADT</a>. 73</p> 74<h2 id="CreateTestProjectCommand">Working with Test Projects</h2> 75<p> 76 You use the <code>android</code> tool to create test projects. 77 You also use <code>android</code> to convert existing test code into an Android test project, 78 or to add the <code>run-tests</code> Ant target to an existing Android test project. 79 These operations are described in more detail in the section <a href="#UpdateTestProject"> 80 Updating a test project</a>. The <code>run-tests</code> target is described in 81 <a href="#RunTestsAnt">Quick build and run with Ant</a>. 82</p> 83<h3 id="CreateTestProject">Creating a test project</h3> 84<p> 85 To create a test project with the <code>android</code> tool, enter: 86</p> 87<pre> 88android create test-project -m <main_path> -n <project_name> -p <test_path> 89</pre> 90<p> 91 You must supply all the flags. The following table explains them in detail: 92</p> 93<table> 94 <tr> 95 <th>Flag</th> 96 <th>Value</th> 97 <th>Description</th> 98 </tr> 99 <tr> 100 <td><code>-m, --main</code></td> 101 <td> 102 Path to the project of the application under test, relative to the test package 103 directory. 104 </td> 105 <td> 106 For example, if the application under test is in <code>source/HelloAndroid</code>, and 107 you want to create the test project in <code>source/HelloAndroidTest</code>, then the 108 value of <code>--main</code> should be <code>../HelloAndroid</code>. 109 <p> 110 To learn more about choosing the location of test projects, please read 111 <a href="{@docRoot}guide/topics/testing/testing_android.html#TestProjects"> 112 Testing Fundamentals</a>. 113 </p> 114 </td> 115 </tr> 116 <tr> 117 <td><code>-n, --name</code></td> 118 <td>Name that you want to give the test project.</td> 119 <td> </td> 120 </tr> 121 <tr> 122 <td><code>-p, --path</code></td> 123 <td>Directory in which you want to create the new test project.</td> 124 <td> 125 The <code>android</code> tool creates the test project files and directory structure 126 in this directory. If the directory does not exist, <code>android</code> creates it. 127 </td> 128 </tr> 129</table> 130<p> 131 If the operation is successful, <code>android</code> lists to STDOUT the names of the files 132 and directories it has created. 133</p> 134<p> 135 This creates a new test project with the appropriate directories and build files. The directory 136 structure and build file contents are identical to those in a regular Android application 137 project. They are described in detail in the topic 138 <a href="{@docRoot}guide/developing/projects/index.html">Managing Projects</a>. 139</p> 140<p> 141 The operation also creates an <code>AndroidManifest.xml</code> file with instrumentation 142 information. When you run the test, Android uses this information to load the application you 143 are testing and control it with instrumentation. 144</p> 145<p> 146 For example, suppose you create the <a href="{@docRoot}resources/tutorials/hello-world.html"> 147 Hello, World</a> tutorial application in the directory <code>~/source/HelloAndroid</code>. 148 In the tutorial, this application uses the package name <code>com.example.helloandroid</code> 149 and the activity name <code>HelloAndroid</code>. You can to create the test for this in 150 <code>~/source/HelloAndroidTest</code>. To do so, you enter: 151</p> 152<pre> 153$ cd ~/source 154$ android create test-project -m ../HelloAndroid -n HelloAndroidTest -p HelloAndroidTest 155</pre> 156<p> 157 This creates a directory called <code>~/src/HelloAndroidTest</code>. In the new directory you 158 see the file <code>AndroidManifest.xml</code>. This file contains the following 159 instrumentation-related elements and attributes: 160</p> 161<ul> 162 <li> 163 <code><application></code>: to contain the 164 <code><uses-library></code> element. 165 </li> 166 <li> 167 <code><uses-library android:name="android.test.runner"</code>: 168 specifies this testing application uses the <code>android.test.runner</code> library. 169 </li> 170 <li> 171 <code><instrumentation></code>: contains attributes that control Android 172 instrumentation. The attributes are: 173 <ul> 174 <li> 175 <code>android:name="android.test.InstrumentationTestRunner"</code>: 176 {@link android.test.InstrumentationTestRunner} runs test cases. It extends both 177 JUnit test case runner classes and Android instrumentation classes. 178 </li> 179 <li> 180 <code>android:targetPackage="com.example.helloandroid"</code>: specifies 181 that the tests in HelloAndroidTest should be run against the application with the 182 <em>Android</em> package name <code>com.example.helloandroid</code>. This is the 183 package name of the <a 184 href="{@docRoot}resources/tutorials/hello-world.html">Hello, World</a> 185 tutorial application. 186 </li> 187 <li> 188 <code>android:label="Tests for .HelloAndroid"</code>: specifies a 189 user-readable label for the instrumentation class. By default, 190 the <code>android</code> tool gives it the value "Tests for " plus 191 the name of the main Activity of the application under test. 192 </li> 193 </ul> 194 </li> 195</ul> 196<h3 id="UpdateTestProject">Updating a test project</h3> 197<p> 198 You use the <code>android</code> tool when you need to change the path to the 199 project of the application under test. If you are changing an existing test project created in 200 Eclipse with ADT so that you can also build and run it from the command line, you must use the 201 "create" operation. See the section <a href="#CreateTestProject">Creating a test project</a>. 202</p> 203<p class="note"> 204 <strong>Note:</strong> If you change the Android package name of the application under test, 205 you must <em>manually</em> change the value of the <code><android:targetPackage></code> 206 attribute within the <code>AndroidManifest.xml</code> file of the test package. 207 Running <code>android update test-project</code> does not do this. 208</p> 209<p> 210 To update a test project with the <code>android</code> tool, enter: 211</p> 212<pre>android update-test-project -m <main_path> -p <test_path></pre> 213 214<table> 215 <tr> 216 <th>Flag</th> 217 <th>Value</th> 218 <th>Description</th> 219 </tr> 220 <tr> 221 <td><code>-m, --main</code></td> 222 <td>The path to the project of the application under test, relative to the test project</td> 223 <td> 224 For example, if the application under test is in <code>source/HelloAndroid</code>, and 225 the test project is in <code>source/HelloAndroidTest</code>, then the value for 226 <code>--main</code> is <code>../HelloAndroid</code>. 227 </td> 228 </tr> 229 <tr> 230 <td><code>-p, --path</code></td> 231 <td>The of the test project.</td> 232 <td> 233 For example, if the test project is in <code>source/HelloAndroidTest</code>, then the 234 value for <code>--path</code> is <code>HelloAndroidTest</code>. 235 </td> 236 </tr> 237</table> 238<p> 239 If the operation is successful, <code>android</code> lists to STDOUT the names of the files 240 and directories it has created. 241</p> 242<h2 id="CreateTestApp">Creating a Test Package</h2> 243<p> 244 Once you have created a test project, you populate it with a test package. 245 The application does not require an {@link android.app.Activity Activity}, 246 although you can define one if you wish. Although your test package can 247 combine Activities, Android test class extensions, JUnit extensions, or 248 ordinary classes, you should extend one of the Android test classes or JUnit classes, 249 because these provide the best testing features. 250</p> 251<p> 252 If you run your tests with {@link android.test.InstrumentationTestRunner} 253 (or a related test runner), then it will run all the methods in each class. You can modify 254 this behavior by using the {@link junit.framework.TestSuite TestSuite} class. 255</p> 256 257<p> 258 To create a test package, start with one of Android's test classes in the Java package 259 {@link android.test android.test}. These extend the JUnit 260 {@link junit.framework.TestCase TestCase} class. With a few exceptions, the Android test 261 classes also provide instrumentation for testing. 262</p> 263<p> 264 For test classes that extend {@link junit.framework.TestCase TestCase}, you probably want to 265 override the <code>setUp()</code> and <code>tearDown()</code> methods: 266</p> 267<ul> 268 <li> 269 <code>setUp()</code>: This method is invoked before any of the test methods in the class. 270 Use it to set up the environment for the test. You can use <code>setUp()</code> 271 to instantiate a new <code>Intent</code> object with the action <code>ACTION_MAIN</code>. 272 You can then use this intent to start the Activity under test. 273 <p class="note"> 274 <strong>Note:</strong> If you override this method, call 275 <code>super.setUp()</code> as the first statement in your code. 276 </p> 277 </li> 278 <li> 279 <code>tearDown()</code>: This method is invoked after all the test methods in the class. Use 280 it to do garbage collection and re-setting before moving on to the next set of tests. 281 <p class="note"><strong>Note:</strong> If you override this method, you must call 282 <code>super.tearDown()</code> as the <em>last</em> statement in your code.</p> 283 </li> 284</ul> 285<p> 286 Another useful convention is to add the method <code>testPreConditions()</code> to your test 287 class. Use this method to test that the application under test is initialized correctly. If this 288 test fails, you know that that the initial conditions were in error. When this happens, further 289 test results are suspect, regardless of whether or not the tests succeeded. 290</p> 291<p> 292 To learn more about creating test packages, see the topic <a 293 href="{@docRoot}guide/topics/testing/testing_android.html">Testing Fundamentals</a>, 294 which provides an overview of Android testing. If you prefer to follow a tutorial, 295 try the <a href="{@docRoot}resources/tutorials/testing/activity_test.html">Activity Testing</a> 296 tutorial, which leads you through the creation of tests for an actual Android application. 297</p> 298<h2 id="RunTestsCommand">Running Tests</h2> 299<p> 300 You run tests from the command line, either with Ant or with an 301 <a href="{@docRoot}guide/developing/tools/adb.html"> 302 Android Debug Bridge (adb)</a> shell. 303</p> 304<h3 id="RunTestsAnt">Quick build and run with Ant</h3> 305<p> 306 You can use Ant to run all the tests in your test project, using the target 307 <code>run-tests</code>, which is created automatically when you create a test project with 308 the <code>android</code> tool. 309</p> 310<p> 311 This target re-builds your main project and test project if necessary, installs the test 312 application to the current AVD or device, and then runs all the test classes in the test 313 application. The results are directed to <code>STDOUT</code>. 314</p> 315<p> 316 You can update an existing test project to use this feature. To do this, use the 317 <code>android</code> tool with the <code>update test-project</code> option. This is described 318 in the section <a href="#UpdateTestProject">Updating a test project</a>. 319</p> 320<h3 id="RunTestsDevice">Running tests on a device or emulator</h3> 321<p> 322 When you run tests from the command line with 323 <a href="{@docRoot}guide/developing/tools/adb.html"> 324 Android Debug Bridge (adb)</a>, you get more options for choosing the tests 325 to run than with any other method. You can select individual test methods, filter tests 326 according to their annotation, or specify testing options. Since the test run is controlled 327 entirely from a command line, you can customize your testing with shell scripts in various ways. 328</p> 329<p> 330 To run a test from the command line, you run <code>adb shell</code> to start a command-line 331 shell on your device or emulator, and then in the shell run the <code>am instrument</code> 332 command. You control <code>am</code> and your tests with command-line flags. 333</p> 334<p> 335 As a shortcut, you can start an <code>adb</code> shell, call <code>am instrument</code>, and 336 specify command-line flags all on one input line. The shell opens on the device or emulator, 337 runs your tests, produces output, and then returns to the command line on your computer. 338</p> 339<p> 340 To run a test with <code>am instrument</code>: 341</p> 342<ol> 343 <li> 344 If necessary, rebuild your main application and test package. 345 </li> 346 <li> 347 Install your test package and main application Android package files 348 (<code>.apk</code> files) to your current Android device or emulator</li> 349 <li> 350 At the command line, enter: 351<pre> 352$ adb shell am instrument -w <test_package_name>/<runner_class> 353</pre> 354 <p> 355 where <code><test_package_name></code> is the Android package name of your test 356 application, and <code><runner_class></code> is the name of the Android test 357 runner class you are using. The Android package name is the value of the 358 <code>package</code> attribute of the <code>manifest</code> element in the manifest file 359 (<code>AndroidManifest.xml</code>) of your test package. The Android test runner 360 class is usually {@link android.test.InstrumentationTestRunner}. 361 </p> 362 <p> 363 Your test results appear in <code>STDOUT</code>. 364 </p> 365 </li> 366</ol> 367<p> 368 This operation starts an <code>adb</code> shell, then runs <code>am instrument</code> 369 with the specified parameters. This particular form of the command will run all of the tests 370 in your test package. You can control this behavior with flags that you pass to 371 <code>am instrument</code>. These flags are described in the next section. 372</p> 373<h2 id="AMSyntax">Using the am instrument Command</h2> 374<p> 375 The general syntax of the <code>am instrument</code> command is: 376</p> 377<pre> 378 am instrument [flags] <test_package>/<runner_class> 379</pre> 380<p> 381 The main input parameters to <code>am instrument</code> are described in the following table: 382</p> 383<table> 384 <tr> 385 <th> 386 Parameter 387 </th> 388 <th> 389 Value 390 </th> 391 <th> 392 Description 393 </th> 394 </tr> 395 <tr> 396 <td> 397 <code><test_package></code> 398 </td> 399 <td> 400 The Android package name of the test package. 401 </td> 402 <td> 403 The value of the <code>package</code> attribute of the <code>manifest</code> 404 element in the test package's manifest file. 405 </td> 406 </tr> 407 <tr> 408 <td> 409 <code><runner_class></code> 410 </td> 411 <td> 412 The class name of the instrumented test runner you are using. 413 </td> 414 <td> 415 This is usually {@link android.test.InstrumentationTestRunner}. 416 </td> 417 </tr> 418</table> 419<p> 420 The flags for <code>am instrument</code> are described in the following table: 421</p> 422<table> 423 <tr> 424 <th> 425 Flag 426 </th> 427 <th> 428 Value 429 </th> 430 <th> 431 Description 432 </th> 433 </tr> 434 <tr> 435 <td> 436 <code>-w</code> 437 </td> 438 <td> 439 (none) 440 </td> 441 <td> 442 Forces <code>am instrument</code> to wait until the instrumentation terminates 443 before terminating itself. The net effect is to keep the shell open until the tests 444 have finished. This flag is not required, but if you do not use it, you will not 445 see the results of your tests. 446 </td> 447 </tr> 448 <tr> 449 <td> 450 <code>-r</code> 451 </td> 452 <td> 453 (none) 454 </td> 455 <td> 456 Outputs results in raw format. Use this flag when you want to collect 457 performance measurements, so that they are not formatted as test results. This flag is 458 designed for use with the flag <code>-e perf true</code> (documented in the section 459 <a href="#AMOptionsSyntax">Instrument options</a>). 460 </td> 461 </tr> 462 <tr> 463 <td> 464 <code>-e</code> 465 </td> 466 <td> 467 <test_options> 468 </td> 469 <td> 470 Provides testing options as key-value pairs. The 471 <code>am instrument</code> tool passes these to the specified instrumentation class 472 via its <code>onCreate()</code> method. You can specify multiple occurrences of 473 <code>-e <test_options></code>. The keys and values are described in the 474 section <a href="#AMOptionsSyntax">am instrument options</a>. 475 <p> 476 The only instrumentation class that uses these key-value pairs is 477 {@link android.test.InstrumentationTestRunner} (or a subclass). Using them with 478 any other class has no effect. 479 </p> 480 </td> 481 </tr> 482</table> 483 484<h3 id="AMOptionsSyntax">am instrument options</h3> 485<p> 486 The <code>am instrument</code> tool passes testing options to 487 <code>InstrumentationTestRunner</code> or a subclass in the form of key-value pairs, 488 using the <code>-e</code> flag, with this syntax: 489</p> 490<pre> 491 -e <key> <value> 492</pre> 493<p> 494 Some keys accept multiple values. You specify multiple values in a comma-separated list. 495 For example, this invocation of <code>InstrumentationTestRunner</code> provides multiple 496 values for the <code>package</code> key: 497</p> 498<pre> 499$ adb shell am instrument -w -e package com.android.test.package1,com.android.test.package2 \ 500> com.android.test/android.test.InstrumentationTestRunner 501</pre> 502<p> 503 The following table describes the key-value pairs and their result. Please review the 504 <strong>Usage Notes</strong> following the table. 505</p> 506<table> 507 <tr> 508 <th>Key</th> 509 <th>Value</th> 510 <th>Description</th> 511 </tr> 512 <tr> 513 <td> 514 <code>package</code> 515 </td> 516 <td> 517 <Java_package_name> 518 </td> 519 <td> 520 The fully-qualified <em>Java</em> package name for one of the packages in the test 521 application. Any test case class that uses this package name is executed. Notice that 522 this is not an <em>Android</em> package name; a test package has a single 523 Android package name but may have several Java packages within it. 524 </td> 525 </tr> 526 <tr> 527 <td rowspan="2"><code>class</code></td> 528 <td><class_name></td> 529 <td> 530 The fully-qualified Java class name for one of the test case classes. Only this test 531 case class is executed. 532 </td> 533 </tr> 534 <tr> 535 <td><class_name><strong>#</strong>method name</td> 536 <td> 537 A fully-qualified test case class name, and one of its methods. Only this method is 538 executed. Note the hash mark (#) between the class name and the method name. 539 </td> 540 </tr> 541 <tr> 542 <td><code>func</code></td> 543 <td><code>true</code></td> 544 <td> 545 Runs all test classes that extend {@link android.test.InstrumentationTestCase}. 546 </td> 547 </tr> 548 <tr> 549 <td><code>unit</code></td> 550 <td><code>true</code></td> 551 <td> 552 Runs all test classes that do <em>not</em> extend either 553 {@link android.test.InstrumentationTestCase} or 554 {@link android.test.PerformanceTestCase}. 555 </td> 556 </tr> 557 <tr> 558 <td><code>size</code></td> 559 <td> 560 [<code>small</code> | <code>medium</code> | <code>large</code>] 561 </td> 562 <td> 563 Runs a test method annotated by size. The annotations are <code>@SmallTest</code>, 564 <code>@MediumTest</code>, and <code>@LargeTest</code>. 565 </td> 566 </tr> 567 <tr> 568 <td><code>perf</code></td> 569 <td><code>true</code></td> 570 <td> 571 Runs all test classes that implement {@link android.test.PerformanceTestCase}. 572 When you use this option, also specify the <code>-r</code> flag for 573 <code>am instrument</code>, so that the output is kept in raw format and not 574 re-formatted as test results. 575 </td> 576 </tr> 577 <tr> 578 <td><code>debug</code></td> 579 <td><code>true</code></td> 580 <td> 581 Runs tests in debug mode. 582 </td> 583 </tr> 584 <tr> 585 <td><code>log</code></td> 586 <td><code>true</code></td> 587 <td> 588 Loads and logs all specified tests, but does not run them. The test 589 information appears in <code>STDOUT</code>. Use this to verify combinations of other 590 filters and test specifications. 591 </td> 592 </tr> 593 <tr> 594 <td><code>emma</code></td> 595 <td><code>true</code></td> 596 <td> 597 Runs an EMMA code coverage analysis and writes the output to 598 <code>/data//coverage.ec</code> on the device. To override the file location, use the 599 <code>coverageFile</code> key that is described in the following entry. 600 <p class="note"> 601 <strong>Note:</strong> This option requires an EMMA-instrumented build of the test 602 application, which you can generate with the <code>coverage</code> target. 603 </p> 604 </td> 605 </tr> 606 <tr> 607 <td><code>coverageFile</code></td> 608 <td><code><filename></code></td> 609 <td> 610 Overrides the default location of the EMMA coverage file on the device. Specify this 611 value as a path and filename in UNIX format. The default filename is described in the 612 entry for the <code>emma</code> key. 613 </td> 614 </tr> 615</table> 616<strong><code>-e</code> Flag Usage Notes</strong> 617<ul> 618 <li> 619 <code>am instrument</code> invokes 620 {@link android.test.InstrumentationTestRunner#onCreate(Bundle)} 621 with a {@link android.os.Bundle} containing the key-value pairs. 622 </li> 623 <li> 624 The <code>package</code> key takes precedence over the <code>class</code> key. If you 625 specifiy a package, and then separately specify a class within that package, Android 626 will run all the tests in the package and ignore the <code>class</code> key. 627 </li> 628 <li> 629 The <code>func</code> key and <code>unit</code> key are mutually exclusive. 630 </li> 631</ul> 632<h3 id="RunTestExamples">Usage examples</h3> 633<p> 634The following sections provide examples of using <code>am instrument</code> to run tests. 635They are based on the following structure:</p> 636<ul> 637 <li> 638 The test package has the Android package name <code>com.android.demo.app.tests</code> 639 </li> 640 <li> 641 There are three test classes: 642 <ul> 643 <li> 644 <code>UnitTests</code>, which contains the methods 645 <code>testPermissions</code> and <code>testSaveState</code>. 646 </li> 647 <li> 648 <code>FunctionTests</code>, which contains the methods 649 <code>testCamera</code>, <code>testXVGA</code>, and <code>testHardKeyboard</code>. 650 </li> 651 <li> 652 <code>IntegrationTests</code>, 653 which contains the method <code>testActivityProvider</code>. 654 </li> 655 </ul> 656 </li> 657 <li> 658 The test runner is {@link android.test.InstrumentationTestRunner}. 659 </li> 660</ul> 661<h4>Running the entire test package</h4> 662<p> 663 To run all of the test classes in the test package, enter: 664</p> 665<pre> 666$ adb shell am instrument -w com.android.demo.app.tests/android.test.InstrumentationTestRunner 667</pre> 668<h4>Running all tests in a test case class</h4> 669<p> 670 To run all of the tests in the class <code>UnitTests</code>, enter: 671</p> 672<pre> 673$ adb shell am instrument -w \ 674> -e class com.android.demo.app.tests.UnitTests \ 675> com.android.demo.app.tests/android.test.InstrumentationTestRunner 676</pre> 677<p> 678 <code>am instrument</code> gets the value of the <code>-e</code> flag, detects the 679 <code>class</code> keyword, and runs all the methods in the <code>UnitTests</code> class. 680</p> 681<h4>Selecting a subset of tests</h4> 682<p> 683 To run all of the tests in <code>UnitTests</code>, and the <code>testCamera</code> method in 684 <code>FunctionTests</code>, enter: 685</p> 686<pre> 687$ adb shell am instrument -w \ 688> -e class com.android.demo.app.tests.UnitTests,com.android.demo.app.tests.FunctionTests#testCamera \ 689> com.android.demo.app.tests/android.test.InstrumentationTestRunner 690</pre> 691<p> 692 You can find more examples of the command in the documentation for 693 {@link android.test.InstrumentationTestRunner}. 694</p> 695