1 [The "BSD license"] 2 Copyright (c) 2010 Terence Parr 3 Maven Plugin - Copyright (c) 2009 Jim Idle 4 5 All rights reserved. 6 7 Redistribution and use in source and binary forms, with or without 8 modification, are permitted provided that the following conditions 9 are met: 10 1. Redistributions of source code must retain the above copyright 11 notice, this list of conditions and the following disclaimer. 12 2. Redistributions in binary form must reproduce the above copyright 13 notice, this list of conditions and the following disclaimer in the 14 documentation and/or other materials provided with the distribution. 15 3. The name of the author may not be used to endorse or promote products 16 derived from this software without specific prior written permission. 17 18 THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR 19 IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES 20 OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. 21 IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, 22 INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT 23 NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 24 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY 25 THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT 26 (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF 27 THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. 28 29============================================================================ 30 31This file contains the build instructions for the ANTLR toolset as 32of version 3.1.3 and beyond. 33 34The ANTLR toolset must be built using the Maven build system as 35this build system updates the version numbers and controls the 36whole build process. However, if you just want the latest build 37and do not care to learn anything about Maven, then visit the 'target' 38directories (for jars) under the depot mirror root here: 39 40 http://antlr.org/depot 41 42If you are looking for the latest released version of ANTLR, then 43visit the downloads page on the main antlr.org website. 44 45These instructions are mainly for the ANTLR development team, 46though you are free to build ANTLR yourself of course. 47 48Source code Structure 49----------------------- 50 51The main development branch of ANTLR is stored within the Perforce SCM at: 52 53 //depot/code/antlr/main/... 54 55release branches are stored in Perforce like so: 56 57 //depot/code/antlr/release-3.1.3/... 58 59In this top level directory, you will find a master build file for 60Maven called pom.xml and you will also note that there are a number of 61subdirectories: 62 63 tool - The ANTLR tool itself 64 runtime/Java - The ANTLR Java runtime 65 runtime/X - The runtime for language target X 66 gunit - The grammar test tool 67 antlr3-maven-plugin - The plugin tool for Maven allowing Maven 68 projects to process ANTLR grammars. 69 70Each of these sub-directories also contains a file pom.xml that 71controls the build of each sub-component (or module in Maven 72parlance). 73 74Build Parameters 75----------------- 76 77Alongside each pom.xml (other than for the antlr3-maven-plugin), you 78will see that there is a file called antlr.config. This file is called 79a filter and should contain a set of key/value pairs in the same 80manner as Java properties files: 81 82antlr.something="Some config thang!" 83 84When the build of any component happens, any values in the 85antlr.config for the master build file and any values in the 86antlr.config file for each component are made available to the 87build. This is mainly used by the resource processor, which will 88filter any file it finds under: src/main/resources/** and replace any 89references such as ${antlr.something} with the actual value at the 90time of the build. 91 92Building 93-------- 94 95Building ANTLR is trivial, assuming that you have loaded Maven version 963.0.3 or better on to your build system and installed it as explained 97here: 98 99http://maven.apache.org/download.html 100 101Note that the ANTLR toolset will ONLY build with version 3.0.3 of Maven 102as of release 3.4. 103 104If you are unfamiliar with Maven (and even if you are), the best 105resource for learning about it is The Definitive Guide: 106 107http://www.sonatype.com/books/maven-book/reference/public-book.html 108 109The instructions here assume that Maven is installed and working correctly. 110 111If this is the first time you have built the ANTLR toolset, you will 112possibly need to install the master pom in your local repository 113(however the build may be able to locate this in the ANTLR snapshot or 114release repository). If you try to build sub-modules on their own (as 115in run the mvn command in the sub directory for that tool, such as 116runtime/Java), and you receive a message that maven cannot find the 117master pom, then execute this in the main (or release) directory: 118 119mvn -N install 120 121This command will install the master build pom in your local maven 122repository (it's ~/.m2 on UNIX) and individual builds of sub-modules 123will now work correctly. 124 125To build then, simply cd into the master build directory 126(e.g. $P4ROOT//code/antlr/main) and type: 127 128mvn -Dmaven.test.skip=true 129 130Assuming that everything is correctly installed and synchronized, then 131ANTLR will build and skip any unit tests in the modules (the ANTLR 132tool tests can take a long time). 133 134This command will build each of the tools in the correct order and 135will create the jar artifacts of all the components in your local 136development Maven repository (which takes precedence over remote 137repositories by default). At the end of the build you should see: 138 139[INFO] ------------------------------------------------------------------------ 140[INFO] Reactor Summary: 141[INFO] ------------------------------------------------------------------------ 142[INFO] ANTLR Master build control POM ........................ SUCCESS [1.373s] 143[INFO] Antlr 3 Runtime ....................................... SUCCESS [0.879s] 144[INFO] ANTLR Grammar Tool .................................... SUCCESS [5.431s] 145[INFO] Maven plugin for ANTLR V3 ............................. SUCCESS [1.277s] 146[INFO] ANTLR gUnit ........................................... SUCCESS [1.566s] 147[INFO] Maven plugin for gUnit ANTLR V3 ....................... SUCCESS [0.079s] 148[INFO] ------------------------------------------------------------------------ 149[INFO] ------------------------------------------------------------------------ 150[INFO] BUILD SUCCESSFUL 151[INFO] ------------------------------------------------------------------------ 152[INFO] Total time: 11 seconds 153 154However, unless you are using Maven exclusively in your projects, you 155will most likely want to build the ANTLR Uber Jar, which is an 156executable jar containing all the components that ANTLR needs to build 157and run parsers (note that at runtime, you need only the runtime 158components you use, such as the Java runtime and say stringtemplate). 159 160Because the Uber jar is not something we want to deploy to Maven 161repositories it is built with a special invocation of Maven: 162 163mvn -Dmaven.test.skip=true package assembly:assembly 164 165Note that Maven will appear to build everything twice, which is a 166quirk of how it calculates the dependencies and makes sure it has 167everything packaged up so it can build the uber-jar assembly. 168 169Somewhere in the build output (towards the end), you will find a line 170like this: 171 172[INFO] Building jar: /home/jimi/antlrsrc/code/antlr/main/target/antlr-master-3.4-SNAPSHOT-completejar.jar 173 174This is the executable jar that you need and you can either copy it 175somewhere or, like me, you can create this script (assuming UNIX) 176somewhere in your PATH: 177 178#! /bin/bash 179java -jar ~/antlrsrc/code/antlr/main/target/antlr-master-3.4-SNAPSHOT-completejar.jar $* 180 181Version Numbering 182------------------- 183 184The first and Golden rule is that any pom files stored under the main 185branch of the toolset should never be modified to contain a release 186version number. They should always contain a.b.c-SNAPSHOT 187(e.g. 3.1.3-SNAPSHOT). Only release branches should have their pom 188version numbers set to a release version. You can release as many 189SNAPSHOTS as you like, but only one release version. However, release 190versions may be updated with a patch level: 3.1.3-1, 3.1.3-2 and so 191on. 192 193Fortunately, Maven helps us with the version numbering in a number of 194ways. Firstly, the pom.xml files for the various modules do not 195specify a version of the artifacts themselves. They pick up their 196version number from the master build pom. However, there is a catch, 197because they need to know what version of the parent pom they inherit 198from and so they DO mention the version number. However, this does 199prevent accidentally releasing different versions of sub-modules than 200the master pom describes. 201 202Fortunately once again, Maven has a neat way of helping us change the 203version. All you need do is check out all the pom.xml files from 204perforce, then modify the <version>a.b.c-SNAPSHOT</version> in the 205master pom. When the version number is correct in the master pom, you 206make sure your working directory is the location of the master pom and 207type: 208 209mvn versions:update-child-modules 210 211This command will then update the child pom.xml files to reflect the 212version number defined in the master pom.xml. 213 214There is unfortunately one last catch here though and that is that the 215antlr3-maven-plugin and the gunit-maven-plugin are not able to use the 216parent pom. The reason for this is subtle but makes sense as doing so 217would create a circular dependency between the ANTLR tool (which uses 218the plugin to build its own grammar files), and the plugins (which 219uses the tool to build grammar files and gunit to test). 220 221This catch-22 situation means that the pom.xml file in the 222antlr3-maven-plugin directory and the one in the gunit-maven-plugin 223directory MUST be updated manually (or we must write a script to do 224this). 225 226Finally, we need to remember that because the tool is dependent on the 227antlr3-maven-plugin and the plugin is itself dependent on the 228tool, that we must manually update the versions of each that they 229reference. So, when we bump the version of the toolset to say 2303.1.4-SNAPSHOT, we need to change the antlr3-maven-plugin pom.xml and 231the gunit-maven-plugin pom.xml to reference that version of the antlr 232tool. The tool itself is always built with the prior released version 233of the plugin, so when we release we must change the main branch of 234the plugin to use the newly released version of the plugin. This is 235covered in the release checklist. 236 237Deploying 238---------- 239 240Deploying the tools at the current version is relatively easy, but to 241deploy to the ANTLR repositories (snapshot or release) you must have 242been granted access to the Sonatype OSS repositories' ANTLR login. 243Few people will have this access of course. 244 245Next, because we do not publish access information for antlr.org, you 246will need to configure the repository server names locally. You do 247this by creating (or adding to) the file: 248 249~/.m2/settings.xml 250 251Which should look like this: 252 253<?xml version="1.0" encoding="UTF-8"?> 254<settings> 255 <servers> 256 <server> 257 <id>sonatype-nexus-snapshots</id> 258 <username>xxxxxxx</username> 259 <password>xxxxxxx</password> 260 </server> 261 <server> 262 <id>sonatype-nexus-staging</id> 263 <username>xxxxxxx</username> 264 <password>xxxxxxx</password> 265 </server> 266 </servers> 267</settings> 268 269When this configuration is in place, you will be able to deploy the components, 270either individually or from the master directory: 271 272mvn -Dmaven.test.skip=true -Ddeplot deploy 273 274You will then see lots of information about checking existing version 275information and so on, and the components will be deployed once you 276supply the ANTLR public key passphrase to sign the jars. 277 278Note that so long as the artifacts are versioned with a.b.c-SNAPSHOT 279then deployment will always be to the development snapshot 280directory. When the artifacts are versioned with a release version 281then deployment will be to the release stahinh repository, which 282will then be mirrored around the world if closed and release. 283The sonatype documentation should be consulted. 284 285Release Checklist 286------------------ 287 288Here is the procedure to use to make a release of ANTLR. Note that we 289should really use the mvn release:release command, but the perforce 290plugin for Maven is not commercial quality and I want to rewrite it. 291 292For this checklist, let's assume that the current development version 293of ANTLR is 3.1.3-SNAPSHOT. This means that it will probably (but not 294necessarily) become release version 3.1.3 and that the development 295version will bump to 3.1.4-SNAPSHOT. 296 2970) Run a build of the main branch and check that it is builds and 298 passes as many tests as you want it to. 299 3001) First make a branch from main into the target release 301 directory. Then submit this to perforce. You could change versions 302 numbers before submitting, but doing that in separate stages will 303 keep things sane; 304 305--- Use main development branch from here --- 306 3072) Before we deploy the release, we want to update the versions of the 308 development branch, so we don't deploy what is now the new release 309 as an older snapshot (this is not super important, but procedure is 310 good right?). 311 312 Check out all the pom.xml files (and if you are using any 313 antlr.config parameters that must change, then do that too). 314 3153) Edit the master pom.xml in the main directory and change the version from 316 3.1.3-SNAPSHOT to 3.1.4-SNAPSHOT. 317 3184) Edit the pom.xml file for antlr3-maven-plugin under the main 319 directory and change the version from 3.1.3-SNAPSHOT to 320 3.1.4-SNAPSHOT. Do the same for the pom.xml in the 321 gunit-maven-plugin directory. 322 323 Update the pom.xml for the archetype manually too. 324 3255) Now (from the main directory), run the command: 326 327 mvn versions:update-child-modules 328 329 You should see: 330 331 [INFO] [versions:update-child-modules] 332 [INFO] Module: gunit 333 [INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT 334 [INFO] Module: runtime/Java 335 [INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT 336 [INFO] Module: tool 337 [INFO] Parent is org.antlr:antlr-master:3.1.4-SNAPSHOT 338 3396) Run a build of the main branch: 340 341 mvn -Dmaven.test.skip=true 342 343 All should be good. 344 3457) Submit the pom changes of the main branch to perforce. 346 3478) Deploy the new snapshot as a placeholder for the next release. It 348 will go to the snapshot repository of course: 349 350 mvn -N deploy 351 mvn -Dmaven.test.skip=true deploy 352 3539) You are now finished with the main development branch and should change 354 working directories to the release branch you made earlier. 355 356--- Use release branch from here --- 357 35810) Check out all the pom.xml files in the release branch (and if you are 359 using any antlr.config parameters that must change, then do that too). 360 36111) Edit the master pom.xml in the release-3.1.3 directory and change 362 the version from 3.1.3-SNAPSHOT to 3.1.3. 363 36412) Edit the pom.xml file for antlr3-maven-plugin under the 365 release-3.1.3 directory and change the version from 3.1.3-SNAPSHOT 366 to 3.1.3. Also change the version of the tool that the this 367 pom.xml references from 3.1.3-SNAPSHOT to 3.1.3 as we are now 368 releasing the plugin of course and it needs to reference the 369 version we are about to release. You will find this reference in 370 the dependencies section of the antlr3-maven-plugin pom.xml. Also 371 change the version references in the pom for gunit-maven-plugin. 372 37313) Now (from the release-3.1.3 directory), run the command: 374 375 mvn versions:update-child-modules 376 377 You should see: 378 379 [INFO] [versions:update-child-modules] 380 [INFO] Module: gunit 381 [INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT, 382 now org.antlr:antlr-master:3.1.3 383 [INFO] Module: runtime/Java 384 [INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT, 385 now org.antlr:antlr-master:3.1.3 386 [INFO] Module: tool 387 [INFO] Parent was org.antlr:antlr-master:3.1.3-SNAPSHOT, 388 now org.antlr:antlr-master:3.1.3 389 39014) Run a build of the release-3.1.3 branch: 391 392 mvn # Note I am letting unit tests run here! 393 394 All should be good, or as good as it gets ;-) 395 39615) Submit the pom changes of the release-3.1.3 branch to perforce. 397 39816) Deploy the new release (this is it guys, make sure you are happy): 399 400 mvn -N deploy 401 mvn -Dmaven.test.skip=true deploy 402 403 Note that we must skip the tests as Maven will not let you 404 deploy releases that fail any junit tests. 405 40617) The final step is that we must update the main branch pom.xml for 407 the tool to reference the newly release version of the 408 antlr3-maven-plugin. This is because each release of ANTLR is 409 built with the prior release of ANTLR, and we have just released 410 a new version. Edit the pom.xml for the tool (main/tool/pom.xml) 411 under the main (that's the MAIN branch, not the release branch) 412 and find the dependency reference to the antlr plugin. If you 413 just released say 3.1.3, then the tool should now reference 414 version 3.1.3 of the plugin. Having done this, you should 415 probably rebuild the main branch and let it run the junit 416 tests. Later, I will automate this dependency update as mvn can 417 do this for us. 418 41918) Having deployed the release to maven, you will want to create the 420 uber jar for the new release, to make it downloadable from the 421 antlr.org website. This is a repeat of the earlier described step 422 to build the uber jar: 423 424 mvn -Dmaven.test.skip=true package assembly:assembly 425 426 MAven will produce the uber jar in the target directory: 427 428 antlr-master-3.1.3-completejar.jar 429 430 And this is the complete jar that can be downloaded from the web site. You 431 may wish to produce an md5 checksum to go with the jar: 432 433 md5sum target/antlr-master-3.1.3-completejar.jar 434 xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx target/antlr-master-3.1.4-SNAPSHOT-completejar.jar 435 436 The command you just ran will also produce a second jar: 437 438 antlr-master-3.1.3-src.jar 439 440 This is the source code for everythign you just deployed and can 441 be unjarred and built from scratch using the very procedures 442 described here, which means you will now be reading this 443 BUILD.txt file for ever. 444 44519) Reward anyone around you with good beer. 446 447 448Miscellany 449----------- 450 451It was a little tricky to get all the interdependencies correct 452because ANTLR builds itself using itself and the maven plugin 453references the ANTLR Tool as well. Hence the maven tool is not a child 454project of the master pom.xml file, even though it is built by it. 455 456An observant person will not that when the assembly:assembly phase is 457run, that it invokes the build of the ANTLR tool using the version of 458the Maven plugin that it has just built, and this results in the 459plugin using the version of ANTLR tool that it has just built. This is 460safe because everything will already be up to date and so we package 461up the version of the tool that we expect, but the Maven plugin we 462deploy will use the correct version of ANTLR, even though there is 463technically a circular dependency. 464 465The master pom.xml does give us a way to cause the build of the ANTLR 466tool to use itself to build itself. This is because in 467dependencyManagement in the master pom.xml, we can reference the 468current version of the Tool and the Maven plugin, even though in the 469pom.xml for the tool itself refers to the previous version of the 470plugin. 471 472What happens is that if we first cd into the tool and maven 473directories and build ANTLR, it will build itself with the prior 474version and this will deploy locally (.m2). We can then clean build 475from the master pom and when ANTLR asks for the prior version of the 476tool, the master pom.xml will override it and build with the interim 477versions we just built manually. 478 479However, strictly speaking, we need a third build where we rebuild the 480tool again with the version of the tool that was built with itself and 481not deploy the version that was built by the version of itself that 482was built by a prior version of itself. I decided that this was not 483particularly useful and complicates things too much. Building with a 484prior version of the tool is fine and if there was ever a need to, we 485could release twice in quick succession. 486 487I have occasionally seen the MAven reactor screw up (or perhaps it is 488the ANTLR tool) when building. If this happens you will see an ANTLR 489Panic - cannot find en.stg message. If this happens to you, then just 490rerun the build and it will eventually work. 491 492Jim Idle - March 2009 493 494