Ant: 2nd article

My 2nd (German) article about Ant is online. Here I write about subprojects and sandbox experiments like Ivy, props, autoconf, parallel executor, …

Ant Article online

After a long time of not-writing I have finished my (German) article about Ant 1.8.0. It is now published on JaxCenter: Der ANTwickler – Das neue Majorrelease

Ant: Use optional dependencies in JavaDoc

If you create your Javadocs you sometimes get unresolved reference warnings: package xy does not exist or cannot find symbol

If you define the path with these dependencies outside of the target you could reference that path using the classpathref attribute.

But if the path definition is done in an other target and you don’t want to have a dependency on that (e.g. many dependencies are downloaded there, e.g. using Ivy), using the classpathref would throw a BuildException because the path is not defined: Reference my-path.reference not found.

You could do a little hack: use a classpath attribute and use a special PropertyEvaluator for using that path: classpathref=“${toString:my-path.reference}“. If your defining target is executed before, the toString: resolves that into a usable path-string. If the target is not executed before, the resulting string toString:my-path.reference is just ignored by the javadoc task.

Werbung

Let Ant tasks choose the attribute values by themself

For a long time an idea travelled in my head. But now it arrived.

In my several Ant build files I have constructs like

<javac source=”${javac.source}” debug=”${javac.debug}” target=”${javac.target}” …

and the according properties defined in an external file. Now have a compile run for the source code and the test code and you have doubled this amount of configuration. And I thought that just writing a <javac> and starting with an -autoconf option would be easier.

The idea is: apply the properties directly before the task execution.

So I could implement a method call in oata.Task.perform() direclty before calling the execute() method.
Hhm …. I don’t want to change the Ant core that deeply because so many external tasks exist and I don’t want to (maybe) break their build.

Another idea is using an AOP framework like AspectJ for jumping in: before <? extends Task>.execute() : applyAttributeValues()
(I am not familiar with AspectJ but you get the idea.)
But then I would depend on the AOP library. That’s nothing for the Core. And Ant options should not depend on any further libraries.
I could implement it as a task:

  <project><autoconf/><javac/></project>    

Better. But I have to learn AspectJ … so not for now…

On the Hudson dev-mailinglist I heard from the Hudson Clover Plugin. It gatheres code coverage from Ant jobs WITHOUT configuring the job itself.-It adds a BuildListener which stores the srcdir and destdir values from <javac> tasks.
Nice idea …. use the Listeners taskStarted(event) and taskFinished(event) methods for doing AOP-stuff.

This results in the <autoconf> task, currently in the sandbox and feedback is welcome.

The goal:

• apply attribute values from properties
• do not overwrite user specified values
• switch on/off the behaviour
• support name prefix for using different values for different targets

While the last two points are easily to implement (add/remove the listener, use a prefix for property search) the first two are difficult.

apply attribute values from properties / which attributes are supported by a given task?

Why is this difficult? Just ask mytask.getClass() for declared setters ….

The problem is that you don’t get the task object. Due lazy instantiation/configuration the only thing you get from the BuildEvent is an UnknownElement. And therefore you cannot just ask getClass() for the class object you need.

I saw three different strategies according to that value:

• if it is a <macrodef> I could ask it directly for its <attribute>s
• if it is a normal task I ask the class object
• if it is a <presetdef> I ask the class object from preset.getTypeClass()

do not overwrite user specified values

I could ask an object for its values, but which are set by the user and which are just implementation defaults? From the Java perspective you cannot distinguish between them…

My strategy is using a “clean” object (I called it template object) and compare its values with the values from the given object.

ApacheCon EU 2009: what is interesting for me …

ApacheCon EU 2009 will be from 23 to 27 March 2009 in Amsterdam. Since the scheduler is open I could have a look and there are some interesting talks:

• AntUnit Your Functional Test: Of course I know AntUnit, we use it in Ant … But it would be interesting so see other scenarios.
• Apache Continuum – Build, Test, and Release : I have to set up a CI-Server at work. My favorite is Hudson but I only heard from Continuum. So having a look widens your horizon.
• Pragmatic Models: building systems with domain specific languages : just because I am a fan of DSLs and MDSD …
• Eclipse IAM, Maven integration for Eclipse : IvyDE is in alpha phase and how feels the Maven pendant?

And of course, Hackathon, meeting people … and I should create a PGP key for the key-signing-event …

But sadly I dont know if I would be there 😦

Let JavaDoc produce tables

JavaDoc is good for documenting Java code. But sometimes it is better to have tables than plain text. Of course you could code the HTML code for tables manually, but it is more comfortable if JavaDoc would support that ‚requirement‘ directly – which it does not. But you can extend JavaDoc by writing a Taglet.

We want to achieve that your JavaDoc comment

/**
@my.tag
    #column1 Text for column 1
    #column2 Text for column 2
*/

produces the required HTML.

JavaDoc uses Doclets as extension point. Writing a doclet is possible but too much overhead because its responsibility is to do the whole generation, but we only want to add a new feature. By default the StandardDoclet is used. And that provides an additional extension point: the taglet. With that you can write new JavaDoc tags (like the @my.tag) and specify the generation behaviour. Using Taglets consist on these steps:

• writing an implementation of com.sun.tools.doclets.Taglet
• add the taglet to the generation process, e.g. by Ant’s <javadoc><taglet name=classname path=pathOfJar>.

When you implement the Taglet interface you have to implement these methods

public String getName() {
  return NAME;
}
public boolean inField() {
  return false;
}
public boolean inConstructor() {
  return false;
}
public boolean inMethod() {
  return true;
}
public boolean inOverview() {
  return false;
}
public boolean inPackage() {
  return false;
}
public boolean inType() {
  return false;
}
public boolean isInlineTag() {
  return false;
}

public static void register(Map registeredTaglets);
public String toString(Tag tag);
public String toString(Tag[] tags);

getName() returns the name of the taglet which should be the name used after the @ sign. I used a constant NAME=“my.tag“.

The inXXX and isXXX methods specify the kind of the taglet. Because I want to have the tables only in method documentation this taglet is not valid for fields and constructors, types (means the class itself) or package, but for methods. Its content shouldnt be on the overview page (only on the detail page). This taglet shouldnt define an inline tag (like in ‚text text {@link target} text text‘).

The next three methods are the more interesting …

The register() method is a static method (and therefore not defined by the interface) and responsible for registering the taglet by the StandardDoclet. If you specify the -taglet option or use the <taglet> instruction in Ant that method is invoked. The StandardDoclet passes a Map with all registered taglets. So do these steps

1. remove the (old) registration if present
2. add the new registration

public static void register(Map registeredTaglets) {
  MyTaglet tag = new MyTaglet();
  Taglet t = (Taglet) registeredTaglets.get(tag.getName());
  if (t != null) {
    registeredTaglets.remove(tag.getName());
  }
  registeredTaglets.put(tag.getName(), tag);
}

The two toString() methods are responsible for the conversion from the taglet text to the HTML code. The first gets one taglet comment, the second gets multiple comments. So the easiest implementation for the ‚one‘ method is delegating to the ‚multiple‘ method.

public String toString(Tag tag) {
  return toString(new Tag[]{tag});
}

So the last thing to do is implementing the conversion of the tag-array:

1. print the tag name as header
2. indent the output as all other (standard) tags do
3. print the table header
4. for each tag print the row
5. close opened HTML tags

public String toString(Tag[] tags) {
  // Quick exit
  if (tags == null || tags.length == 0) {
    return null;
  }
  // HTML collector
  StringBuffer sb = new StringBuffer();
  // 1.Header
  sb.append("<dt><b>").append(getName()).append("</b></dt>").append(BR);
  // 2.Indent
  sb.append("<dd>").append(BR);
  // 3.table header
  sb.append("
<table border='1'>").append(BR);
  sb.append("
<tr>").append(BR);
  sb.append("
<th>column 1</th>
").append(BR);
  sb.append("
<th>column 2</th>
").append(BR);
  sb.append("</tr>
").append(BR);
  // 4. for each tag make a row
  String[] stopwords = new String[]{"column1", "column2"};
  for (int i = 0; i < tags.length; i++) {
    appendTagAsHtml(sb, new TagInfo(tags&#91;i&#93;.text(), stopwords));
  }
  // 5.close all opened tags
  sb.append("</table>
").append(BR);
  sb.append("</dd>").append(BR);
  // return the HTML coding
  return sb.toString();
}

In this code I use two helpers: the method appendTagAsHtml() and the class TagInfo.

The TagInfo class constructor takes the text for the tag (all after the @tagname until the next @tag or end of javadoc) and the supported stopwords. It creates a slot for each stopword and one for the text before the first stopword. Then it parses the text and stores the words in the current slot. It starts with the before-slot and goes to the next slot if a stopword occurs.

public class TagInfo {
  HashMap<String,List<String>> map = new HashMap();

  TagInfo(String tagText, String[] stopwords) {
    map.put("#__text__", new ArrayList());
    for (int i = 0; i < stopwords.length; i++) {
      map.put("#" + stopwords&#91;i&#93;, new ArrayList());
    }
    String currentSlot = "#__text__";
    StringTokenizer tokenizer = new StringTokenizer(tagText);
    while (tokenizer.hasMoreTokens()) {
      String token = tokenizer.nextToken();
      if (map.containsKey(token)) {
        currentSlot = token;
      } else {
        ((List)map.get(currentSlot)).add(token);
      }
    }
  }

  public List get(String stopword) {
    return (List)map.get("#" + stopword);
  }
}

&#91;/sourcecode&#93;

The appendInfoAlsHtml() method converts the TagInfo to HTML. It uses another helper method for converting lists to strings.

&#91;sourcecode language='java'&#93;

private static void appendInfoAlsHtml2(StringBuffer sb, PruefungsInfo info) {
  sb.append("
<tr valign='top'>").append(BR);
  sb.append("
<td>")
    .append(list2stringbuffer(info.get("column1"), " "))
    .append("</td>
")
    .append(BR);
  sb.append("
<td>")
    .append(list2stringbuffer(info.get("column2"), " "))
    .append("</td>
")
    .append(BR);
    .append("</td>
")
    .append(BR);
    .append(BR);
  sb.append("</tr>
").append(BR);
}

protected static StringBuffer list2stringbuffer(List list, String separator) {
  StringBuffer rv = new StringBuffer();
  for (int i=0; i
	<list.size()-1; i++) {
    rv.append(list.get(i));
    rv.append(separator);
  }
  if (list.size() > 0) {
    // 'get' is 0-based but 'size' is 1-based
    rv.append(list.get(list.size()-1));
  }
  return rv;
}

Finally the use of the taglet in Ant is this:

<target name="javadoc">
  <mkdir dir="${javadoc.dir}"/>
  <javadoc sourcepath="${java.src}"
           destdir="${javadoc.dir}"
           author="${javadoc.author}"
           version="${javadoc.version}"
           use="${javadoc.use}"
           access="${javadoc.access}"
           excludepackagenames="${javadoc.excludes}"
           windowtitle="${ant.project.name} - API">
    <doctitle><!&#91;CDATA&#91;
<h1>${ant.project.name} - API</h1>
&#93;&#93;></doctitle>
    <bottom><!&#91;CDATA&#91;<i>Copyright &#169; 2002-${tstamp.jahr} ACME All Rights Reserved.</i>&#93;&#93;></bottom>
    <taglet name="pack.age.of.MyTaglet" path="${taglet.jar}"/>
  </javadoc>
</target>

p.s.

Sorry for unintentional linebreaks in the coding. Maybe my WordPress knowledge is not big enough …

Ant + JavaFront: start Ant tasks from the command line (2)

In my last blog I wrote about a new AntLib in Ants sandbox: javafront with which you have the ability to start tasks directly from the command line – without any buildfile. With that I was able to do the pom2ivy conversion Torsten Curdt asked for:

task.bat -lib path-to-ivy.jar -xmlns:jar antlib:org.apache.ivy.ant ivy:convertpom pomFile my.pom ivyFile ivy.xml

Ant + JavaFront: start Ant tasks from the command line

Reading the blog Converting maven pom to ivy I asked myself, why it should not be possible to start Ant tasks from the command line.

Stefan on the other side was bored always to here that Ant relies on XML and he presented a way for writing Buildfiles in plain (annotated) Java using JavaFront, a library in Ants sandbox.

Ant provides a way to override the class to start by providing an implementation of AntStart and specifying that class by -main option. So I wrote a new start class which parses the command line parameters and executes that task. The first parameter is the taskname and following parameters are attributes with their values. (There is no support for nested elements yet.)

public class TaskExec implements AntMain {

public void startAnt(String[] args, Properties additionalUserProperties, ClassLoader coreLoader) {
Map attributes = new Hashtable();

// Analyzing the command line arguments
String taskname = args[0];
for(int i=1; iant -lib build\classes -main org.apache.ant.javafront.TaskExec echo message „Hello World“

Other examples are in the launcher script.

Ant: catching all marked lines in text files for creating a changes document

MartianSoftware has released (a long time ago) two Ant task: <snip> and <rundoc>. While I am not sure about the benefits of <rundoc> (as I dont know docbook really well) <snip> seems to be a nice task. Extracting parts of a file with Ant on board utilities is not that easy.

Using the snip task extracting is a few-liner. Just think we have a class

/**
 * This is normal JavaDoc comment.
 * @@snip:changes@@
 * This is a comment which should be in the changes.txt!
 * @@endSnip@@
 * @author Jan Materne
 */
public class MyProgram {
}

Ant I want to generate the document containing this line
This is a comment which should be in the changes.txt!

<project>
<property name="build.dir" value="build"/>

    <target name="clean">
        <delete dir="${build.dir}"/>
    </target>

    <target name="snip">
        <!-- Getting snip task -->
        <mkdir dir="${build.dir}"/>
        <get src="http://www.martiansoftware.com/lab/snip/snip-0.11.jar" dest="${build.dir}/snip.jar"/>
        <taskdef name="snip" classname="com.martiansoftware.snip.Snip" classpath="${build.dir}/snip.jar"/>
        <!-- Collecting the information -->
        <snip>
            <fileset dir="src" includes="**/*.java" />
        </snip>
        <!-- Use these informaion -->
        <echo file="${build.dir}/changes.txt">
        Changes
        -----------------------------------------------------------------
        ${snip.changes}
        </echo>
        <!-- For easier test: output the file -->
        <concat><fileset file="${build.dir}/changes.txt"/></concat>
    </target>

</project>

I can see, that the information is gathered:

..>ant snip
Buildfile: build.xml
snip:
      [get] Getting: http://www.martiansoftware.com/lab/snip/snip-0.11.jar
      [get] To: ...\build\snip.jar
   [concat]
   [concat]         Changes
   [concat]         -----------------------------------------------------------------
   [concat]
   [concat]  * This is a comment which should be in the changes.txt!
   [concat]

BUILD SUCCESSFUL

Fine, let’s try having more information with a second class:

/**
 * This is normal JavaDoc comment.
 * @@snip:changes@@
 * This is another comment which should be in the changes.txt!
 * @@endSnip@@
 *
 * @author Jan Materne
 *
 * @@snip:changes@@
 * Maybe another line in the comments.
 * @@endSnip@@
 */
public class Class2 {
}

And I want to get:

...>ant snip
Buildfile: build.xml

snip:
[get] Getting: http://www.martiansoftware.com/lab/snip/snip-0.11.jar
[get] To: ...\build\snip.jar
[concat]
[concat]         Changes
[concat]         -----------------------------------------------------------------
[concat]
[concat]  * This is a comment which should be in the changes.txt!
[concat]  * This is another comment which should be in the changes.txt!
[concat]  * Maybe another line in the comments.
[concat]

BUILD SUCCESSFUL

But the result doesnt change. Thats because <snip> sets properties directly when getting the @@endSnip@@ mark. And – as Ant properties are immutable – once set a property cant be extended.

It would be better to extend <snip> for first collecting all values and then storing the values into Ant properties.

I thought I could start collecting these lines with Ant built-in facilities could be done like

    <target name="ant" depends="clean">
        <mkdir dir="${build.dir}"/>
        <!-- Collecting and using the information -->
        <concat destfile="${build.dir}/changes.txt">
            <header>
                Changes
                -----------------------------------------------------------------
            </header>
            <fileset dir="src" includes="**/*.java" />
            <filterchain>
                <!-- Something special here -->
            </filterchain>
        </concat>
        <!-- For easier test: output the file -->
        <concat><fileset file="${build.dir}/changes.txt"/></concat>
    </target>

But I couldnt get one file after the next in that filterchain. By default I get one line after the next and then I have troubles for storing the current state (in @@snip-@@endSnip section or out). When using a <tokenfilter><filetokenizer/> I’ll get not only one whole file – I’ll get one token containing ALL catched files including the specified headers. And I dont want to run THAT on a larger codebase …

Ok, it is difficult to gather all content BETWEEN two special markers. But it is very easy to catch all lines with a PREPENDING marker:

    <target name="ant">
        <mkdir dir="${build.dir}"/>
        <!-- Collecting and using the information -->
        <echo file="${build.dir}/changes.txt">
        Changes
        -----------------------------------------------------------------
        </echo>
        <concat destfile="${build.dir}/changes.txt" append="true">
            <fileset dir="src" includes="**/*.java" />
            <filterchain>
	<linecontainsregexp>
                    <regexp pattern="^&#91; \*&#93;*CHANGES:"/>
                </linecontainsregexp>
                <replaceregex pattern="^&#91; \*&#93;*CHANGES: *" replace=""/>
            </filterchain>
        </concat>
        <!-- For easier test: output the file -->
        <concat><fileset file="${build.dir}/changes.txt"/></concat>
    </target>

/**
 * This is normal JavaDoc comment.
 * CHANGES: This is a comment which should be in the changes.txt!
 * @author Jan Materne
 */
public class MyProgram {
}

Links for 2008-10-16

According to this German article (relies on an article by ComputerWorld Australia)Australia plans to filter all internet traffic in AUS. There are two blacklists: one for illegal content (which should be blocked) and one for content for adults.

Trustin Lee has released Ant2IDE. Basically this is a BuildListener that collects (almost) all data required for generating Eclipse settings.