Hudson: how to get a list of all failed jobs?

Posted 3. Juli 2009 by janmaterne
Categories: Hudson, en

If you have a lot of job configured on your Hudson installation, you maybe want to have a list of job which last run fails. Sadly there is no view to that, but you could get the list via Hudsons script console and a three lines of Groovy:


activeJobs = hudson.model.Hudson.instance.items.findAll{job -> job.isBuildable()}
failedRuns = activeJobs.findAll{job -> job.lastBuild.result == hudson.model.Result.FAILURE}
failedRuns.each{run -> println(run.name)}

or a little bit longer (for explanations):


hudsonInstance = hudson.model.Hudson.instance
allItems = hudsonInstance.items
activeJobs = allItems.findAll{job -> job.isBuildable()}
 failedRuns = activeJobs.findAll{job -> job.lastBuild.result == hudson.model.Result.FAILURE}
 failedRuns.each{run -> println(run.name)}

Line 1: This is the main entry point to the running Hudson instance.

Line 2: Jobs are a special kind of item. Here are all stored items.

Line 3: Now we filter the whole list. We dont need disabled jobs as we dont want to fix them. So the closure checks if the job is buildable and therefore active.

Line 4: The next filter removes all jobs which were ok. The closure checks the last build result and selects the job if it was a failure.

Line 5: Now we pass an additional closure to the list for getting a nicer output (than a println(failedRuns) )

(Sorry for bad source code formatting – WordPress doesnt support Groovy.)

My first Hudson PlugIn

Posted 24. Juni 2009 by janmaterne
Categories: Hudson, en

I set up a Hudson instance in my company. But after adding the 20th job I lost the overview: which job depends on which other job? I need a visualization.
I need a plugin which reads the job dependencies and creates a dependency graph.

This problem consists of two parts: getting te dependencies and creating the graph. And I need to solve both for getting the plugin work …

I start with creating the graph. Shouldn’t be a problem – I am sure there are some libraries I could use as dependency graphs are an old and common ‘problem’. And searching for “java create dependency graph” gave me a nice hit: yFiles.
But this lib has an enourmous drawback: it is commercial and the plugin should be published as OSS.
But I remembered another tool: Graphiz. And via its homepage I found a Java layer on top of it: Linguine Maps.

First played a little bit:


String dotExeFileName = "C:/temp/ec-dep/Job Dependency Visualizer/lib/graphiz/dot.exe";
 // create the board for the graph
 Graph graph = GraphFactory.newGraph();
 // create the nodes
 GraphNode child   = graph.addNode();
 GraphNode me      = graph.addNode();
 GraphNode wife    = graph.addNode();
 GraphNode father  = graph.addNode();
 GraphNode mother  = graph.addNode();
 GraphNode grandma = graph.addNode();
 GraphNode grandpa = graph.addNode();
 GraphNode friend  = graph.addNode();
 // connect the nodes according to the dependencies
 graph.addEdge(me, child);
 graph.addEdge(wife, child);
 graph.addEdge(father, me);
 graph.addEdge(mother, me);
 graph.addEdge(grandma, father);
 graph.addEdge(grandpa, father);
 // nodes have a caption
 child.getInfo().setCaption("Child");
 me.getInfo().setCaption("Me");
 wife.getInfo().setCaption("Wife");
 father.getInfo().setCaption("Father");
 mother.getInfo().setCaption("Mother");
 grandma.getInfo().setCaption("Grandma");
 grandpa.getInfo().setCaption("Grandpa");
 friend.getInfo().setCaption("Friend");
 // highlight some nodes
 me.getInfo().setFillColor("red");
 grandma.getInfo().setModeDotted();
 grandpa.getInfo().setModeDotted();
 // save as gif
 File dotFileName = File.createTempFile("tmp.graph.", ".dot");
 GRAPHtoDOTtoGIF.transform(
 graph,
 dotFileName.getAbsolutePath(),
 "C:/temp/graph.demo-old.gif",
 dotExeFileName
 );

graph.demo-old

Oh, this is the library I want to use. So I wrote a class tayloring the graph creation (Java 1.4) to dependency graphs with use of Java >=5 and a fluent interface.


DependencyGraph
 .create()
 .addDependency("Child", "Me", "Wife")
 .addDependency("Me", "Father", "Mother")
 .addDependency("Father", "Grandma", "Grandpa")
 .addDependency("Friend")
 .nodeBorder("Grandma", BorderStyle.BORDER_DOTTED)
 .nodeBorder("Grandpa", BorderStyle.BORDER_DOTTED)
 .highlight("Me", Style.FILL_COLOR, "red")
 .toGif("C:/temp/graph.demo.gif");

My base concepts are:

  • a string could have dependency on other strings (me–>father,mother; wife:<none>)
  • the string is the caption of the graph node
  • the string is the unique node name
  • two strings can only be connected once

So I could store the dependencies as a map: Map<String,Set<String>>


public class DependencyGraph {

 /** Path to the Graphiz executable. */
 private String dotExeFileName = "C:/temp/ec-dep/Job Dependency Visualizer/lib/graphiz/dot.exe";

 /** The graph to create. */
 Graph graph = GraphFactory.newGraph();

/** All already created GraphNodes. */
 Map<String, GraphNode> nodes = new Hashtable<String, GraphNode>();

 /** Already created dependency lines. */
 Map<String, Set<String>> dependencies = new Hashtable<String, Set<String>>();

 public static DependencyGraph create() {
 return new DependencyGraph();
 }

/**
 * Adds a node and its dependencies to the graph. If the node exists
 * already, it is not created a second time.
 */
 public DependencyGraph addDependency(String node, String... parents) {
 Set<String> list = new HashSet<String>();
 Collections.addAll(list, parents);
 addDependency(node, list);
 return this;
 }

public DependencyGraph addDependency(String node, Set<String> parents) {
 createGraphNode(node);
 for (String parent : parents) {
 createGraphNode(parent);
 if (!dependencies.get(node).contains(parent)) {
 // create the link
 graph.addEdge(nodes.get(parent), nodes.get(node));
 // register the dependency
 dependencies.get(node).add(parent);
 }
 }
 return this;
 }

/**
 * Helper method for creating graph nodes and store them in the cache if the
 * don't exist before.
 */
 private void createGraphNode(String nodeName) {
 if (!nodes.containsKey(nodeName)) {
 GraphNode node = graph.addNode();
 node.getInfo().setCaption(mask(nodeName));
 nodes.put(nodeName, node);
 dependencies.put(nodeName, new HashSet<String>());
 }
 }

/**
 * Modify the shape style of a node.
 *
 * @param node the name of the node to style
 * @param style the new style
 * @return self
 */
 public DependencyGraph nodeStyle(String node, ShapeStyle style) {
 nodes.get(node).getInfo().setShape(style.value);
 return this;
 }

/**
 * Stores the graph as GIF image.
 * @param file the file to new image file
 * @return self
 * @throws IOException while file access
 */
 public DependencyGraph toGif(File file) throws IOException {
 File dotFileName = File.createTempFile("tmp.graph.", ".dot");
 GRAPHtoDOTtoGIF.transform(
 graph, dotFileName.getAbsolutePath(),
 file.getAbsolutePath(), dotExeFileName);
 dotFileName.delete();
 return this;
 }

}

public enum ShapeStyle {
 RECORD(0),
 BOX(1),
 ELLIPSE(2),
 CIRCLE(3),
 DIAMOND(4),
 TRIANGLE(5),
 INVTRIANGLE(6),
 HEXAGON(7),
 OCTAGON(8),
 PARALLELOGRAM(9);

 /** the LinguineMaps style value */
 int value;
 private ShapeStyle(int value) {
 this.value = value;
 }

}

After solving the second part of my problem I have to solve the first part too. I need information from Hudson:

  • List<Job> getAllJobs()
  • List<Job> getParentJobs(Job)

For creating a graph per view I also need

  • List<View> getAllViews()
  • List<Job>  getJobs(View)

And for keeping the graphs up to date I need to be informed about configuration changes.

  • void onJobConfigChange(Job)

I have to search for these…

I am working on the plugin part now – so I’ll write more if I have finished that.

Links for 2009-05-29

Posted 29. Mai 2009 by janmaterne
Categories: Uncategorized

Tags: , , ,

There is a new Windows 7 Blog by Microsoft MVPs

On DrBacchus is also a short article about the PHP MVC framework CakePHP. And a list of top-10 php frameworks on mustap.com

On MSDN there is a five part webcast series about programming with the task parallel library which will be part of Visual Studio 2010.

In his blog Jason Zander shows the highlights of VS 2010 and .NET 4.0.

Microsoft publishes a free tool for developing secure software.

JBoss 5.1 is out.

On Windowsblog@TheShop.at is a compatibility list of programs on Windows 7.

If you need an additional automat in your office (very nice for lunch) have a look at this video.

Links for 2009-05-12

Posted 12. Mai 2009 by janmaterne
Categories: Uncategorized

Tags: , , , ,

On 14.May there is free webinar about JSF 2.0 by Ed Burns.

Pearl provides with Tape2PC a hardware solution for copying from MC (music cassetes) to your PC (mp3).

With CamSpace you could control games, windows an applications via webcam. There is foto galery by chip.de.

Saros is a free available Eclipse plugin for pair programming. A demo online.

With Axum Microsoft releases a new programming language for parallel programming.

Links for 2009-05-06

Posted 6. Mai 2009 by janmaterne
Categories: Blogroll, en

Tags: , , , , , , , , , , , , , , , , ,

On Chip.DE I read an announcement of LicenseCrawler. It is a tool which scans your program for license information, so you could backup these more easily.

On CreateOrDie there is a list of 15 free available eBooks about web design.

Heise.DE provides a new podcast about software architecture. Havent listened to it yet, but could be in German.

According to Golem.DE Windows 7 comes with a built in VM for WindowsXP – a nice way for having BWC ;-)

I read an interesting announcement on TSS.com: OpenXava generates GUIs from JPA entities if they are marked with a few annotations more.

I also found a list of free java decompilers.

SE-Radio episode 133 was about Continuous Integration. It was very interesting …

It seems that JBoss provides with Tattletale an interesting tool for analyzing jar dependencies – including their version.

For PHP developers: the Zend framework 1.8.0 is released.

According to Win7 and its virtual XP: there is also a nice blog (German) which shows the use of its via two parallel running Outlooks of different version.

The German part Northrhine Westfalia (NRW) provides a ring tone (not sure about this term, in German it is “Klingelton”): it is a pop version of Beethovens 9th symphony – the theme used as Europe theme. It is a ‘commercial’ for the next election.

ZDNet.DE provides a link to a webcast by Microsoft: “10 Cool Things to Do With the Powershell”. The link is online. Content:
This webcast shows some of the neat things programmers can do with PowerShell with regards to server administration. There are 10 demos one can run through ranging from simple queries for services to enumerating information in Active Directory. Some of the scripts use lists of computers, some prompt for a computername. Each script shows a different thing programmers can do with PowerShell to quickly create administration scripts.

According to Entwickler.COM SpringSource and Google prepare a JSR for an annotation for DI.

Are you a .NET developer? Maybe this list of 10 open source tools is for you.

Links for 2009-04-06

Posted 6. April 2009 by janmaterne
Categories: en

Tags: , , ,

In Novells Wiki there is a nice code sample which tries to read all available security features from the request (Client Certificates, Encoding type, Cipher Suite, …)

Links for 2009-03-26

Posted 26. März 2009 by janmaterne
Categories: en

CreateOrDie has published the announcement of an ActionScript-Wiki by Joa Ebert.

According to an article on Heise.de Windows7 should block multimedia codecs by 3rd parties. ‘albain’, one the developer of ffdshow-tryouts, discovered that build 7057 of Windows 7 blocks alternative codecs of MPEG-4 and H.264.

Final steps for PHP 5.3. Interesting features are namespaces, lambda-functions and closures, nowdoc-syntax and optional garbage collection of cyclic references. Release Candidate 1 is available.

MS-Infoservice.ch provides a poster about the architecture of MS Exchange 2007.

Mark Melvin bloggt interestingly about new features of Antlately about string manipulating with Ant 1.7 string-resources and regexp-filterchain.

Links for 2009-03-05

Posted 5. März 2009 by janmaterne
Categories: Blogroll, en

There is a comparison of different RIA frameworks like GWR, Laszlo, Flex…

If you work with XPath, the XPath Workbook could be something for you. It is an Eclipse plugin for writing XPath constructs.

Galileo presents the free available OpenBook about Visual Basic 2008 (in German). It covers the Visual Studio IDE, the .NET framework, Visual Basic syntax and its OO concepts, WinForms, WPF and ADO.NET.

If you are developing web applications, CreateOrDie covers seven tools for cross-browser compatibility.

Software Engeneering Radios recent episode is about user interface design.

Links for 2009-02-24

Posted 24. Februar 2009 by janmaterne
Categories: Blogroll, en

Tags: , , , , ,

Entwickler.com published the announcement of CDT 6.0M5, an Eclipse plugin for C/C++ development.

A blog on MSDN shows the use of Contracts. mmh … why do I think of Java assert?

Chip.de sais that with CamSpace you could control Windows, applications and games via Webcam.

CInsk.org demonstrates the Artist Mode of EMacs. An easy way of drawing lines and rectangles in EMacs. Nice video …

MP3-Listing

Posted 24. Februar 2009 by janmaterne
Categories: en

Tags: , , ,

In the internal of my ensemble NiederrheinBrass we host the records of the last concerts. But I am lazy – why should I update that page each time I upload a new MP3? All required information are online: the URL of the file, title+record date as ID3 tags. And we have PHP…

So first a note: I am not a PHP programmer. I usually write in Java. So maybe these codings are not the best from a PHP view point. But that works …

I use the free Mphp3 script. But I just have seen that this not maintained any more… But you could the principles with another Read-ID3-Tag-Infos-PHP-Skript.


<?php
    // Einbinden der MP3-Funktionalität
    include('mphp3.php');
?>

Generating the content is easy – just


<?php
    // Einbinden der MP3-Funktionalität
    include('mphp3.php');

    $daten = new Daten();
    $daten->run();
?>

Ok – two things here: constructor and run() ;-)

I don’t have implemented a constructor. The run() method asks a factory for a Generator and then that generator for the HTML. I use the factory and different generators because I not only need the MP3 listing …


<?php
    // include MP3 functions
    include('mphp3.php');

    $daten = new Daten();
    $daten->run();

    //==============================================================================================        

    /**
     * Create HTML listing.
     */
    class Daten {
        /** Start-Funktion */
        public function run() {
            $generator = $this->createGenerator();
            $generator->ausgabe();
        }

        /**
         * Factory
         * @return generator
         */
        private function createGenerator() {
            if ($_GET['art'] == "aufnahmen") {
                return new Aufnahmen();
            }
            if ($_GET['art'] == "bandstuecke") {
                $rv = new Aufnahmen();
                $rv->art = "bandstuecke";
                return $rv;
            }
            if ($_GET['art'] == "noten") {
                return new Noten();
            }
            return new Fehler();
        }
    }
?>

So the the URL …./index.php?aufnahmen starts my MP3-Generator.

For the generator I use an (abstract) class and four concrete classes. The parent class is not really abstract. It is just not instantiated by the factory. The parent class contains four fields for

  1. what should be generated? (aufnahmen should generate the MP3)
  2. what is the HTML file? (caching the work)
  3. what subdirectories should be scanned?
  4. what is the working directory for the generator?

    /**
     * Baseclass for Generators.
     */
    class Generator {
        var $art;
        var $htmlFile;
        var $dirs = array();
        var $curDir;

        /**
         * Generates HTML.
         */
        public function ausgabe() {
            $this->htmlFile = $this->art . ".html";
            // Caching
            if ($this->needsGeneration()) {
                // Generate
                // Activate buffer and write into that.
                ob_start();
                $this->generate();
                // Write buffer to file
                $html = ob_get_clean();
                $fd = fopen($this->htmlFile, "w");
                fwrite($fd, $html);
                fclose($fd);
            }
            // Just/finally pass file content to client
            readfile($this->htmlFile);
        }

        /**
         * Checks timestamp of the html file against all timestamps of the files.
         * @return true if there a new files
         */
        protected function needsGeneration() {
            if (!file_exists($this->htmlFile)) {
                # No html file, generation required
                return true;
            } else {
                $mtimeHtml = filemtime($this->htmlFile);
                $mtimeDir  = $this->getMTime($this->art);
                if ($mtimeHtml < $mtimeDir) {
                    # HTML too old, new generation required
                    return true;
                } else {
                    # no new content, no generation required
                    return false;
                }
                return $mtimeHtml < $mtimeDir;
            }
        }

The easiest generator is the error generator. The factory creates that if there is an invalid URL argument. It prints an error message. It works as NullObject so I don’t need to check all arguments.


    /**
     * "Generator" for error text.
     * @see NullObject Design Pattern
     */
    class Fehler extends Generator {
        // Überschreibe ausgabe() anstelle generate(), um das Caching zu deaktivieren.
        function ausgabe() {
            echo "Fehlerhafte Auswahl. Eventuell ist der Link falsch.";
        }
    }

Before I show the implementation of the MP3-Generator, I show the directory layout used:

  • Under the index.php (with the generator) there is one directory for each concrete generator. So here aufnahmen.
  • The directory layout for each generator could be different. For this generator it contains directories with the date of the concert (yyyyMMdd).
  • Each concert directory could contain an ort.inc with the place where we played. I use its content as header.
  • Each concert directory could (should) contain *.mp3 files with ID3v2 tags.

    class Generator {
        /**
         * Generation. Subclasses should overwrite this.
         */
        protected function generate() {
            $this->dirs = $this->getDirs();
            $firstOpenElement = $this->dirs[0];

            $this->htmlHeader();
            $this->htmlBodyStart($firstOpenElement);
            $this->htmlNavigation();

            echo "
<div id=\"Inhalt\">\n";
            foreach($this->dirs as $dir) {
                $this->curDir = $this->art . "/$dir";
                $titel = $this->getTitel($dir);
                echo "
<div class=\"hideable\" id=\"id$dir\">\n";
                echo "
<h2>$titel</h2>
\n";
                @readfile($this->curDir . "/header.inc");
                echo "
<table border=\"1\" cellspacing=\"0\" cellpadding=\"4\">\n";
                $this->htmlInhaltUeberschrift();
                $files = $this->getFiles($this->curDir);
                foreach($files as $file) {
                    $this->htmlInhaltZeile("$this->curDir/$file", $file);
                }
                echo "</table>
\n";
                echo "</div>
\n";
            }
            echo "</div>
\n\n";

            $this->htmlFooter();
        }
    }

 //==============================================================================================        

    /**
     * Generator for MP3-Listing.
     */
    class Aufnahmen extends Generator {
        /** Construktor. */
        public function Aufnahmen() {
            $this->art = "aufnahmen";
        }

        /*
         * The newest record on top.
         * @Override
         */
        protected function sortDirs($array) {
            $rv = $array;
            sort($rv);
            return array_reverse($rv);
        }

        /* @Override */
        protected function getTitel($dir) {
            $aufnahmeDatum = ereg_replace("(....)(..)(..)", "\\3.\\2.\\1", $dir);
            $incFile = $this->art . "/$dir/ort.inc";
            if (file_exists($incFile)) {
                return $aufnahmeDatum . " " . file_get_contents($incFile);
            } else {
                return $aufnahmeDatum;
            }
        }

        /* @Override */
        protected function htmlInhaltUeberschrift() {
            echo "
<tr>
<th>Name</th>
<th>Interpret</th>
<th>Dauer</th>
<th>Größe</th>
</tr>
\n";
        }

        /* @Override */
        protected function htmlInhaltZeile($file, $filename) {
            ...
        }
    }

The generate() method of the abstract class generates the common HTML layout. Subclasses should overwrite only special methods and hook there content into the common layout. I dont want to write over the layout, the CSS for that or the javascript used there.

The real content should be in a <div> section with the id Inhalt. For each concert directory (subdirectory of the generator) a <div> with the titel and content is placed. The content is a HTML table contained of a header row and a data row for each file. The methods used are htmlInhaltUeberschrift() for the table header row and htmlInhaltZeile() for the data rows.

The header creates columns for name, actor, time and filesize of the file.


/* @Override */
protected function htmlInhaltZeile($file, $filename) {
    if (!$this->endsWith($filename, ".inc")) {
        echo "
<tr>";
        if ($this->endsWith($filename, ".mp3")) {
            $info = new mphp3();
            $info->load($file);
            if ($info->valid) {
                $titel   = ($info->v2_title)  ? $info->v2_title  : $info->v1_title;
                $artist  = ($info->v2_artist) ? $info->v2_artist : $info->v1_artist;
                $dauer   = $info->lengthstring;
                $anzeige = ($titel) ? $titel : $filename;
                echo "    ";
                echo "
<td><a href=\"$file\">$anzeige</a></td>
";
                echo "
<td>$artist</td>
";
                echo "
<td>$dauer min</td>
";
            }
            $size = round(filesize($file) / 1024 / 1024, 1);
            echo "
<td>$size MB</td>
";
        } else {
            echo "
<td colspan=\"3\"><a href=\"$file\">$filename</a></td>
";
        }
        echo "</tr>
\n";
    } else {
        // Keine Ausgabe der *.inc Dateien
    }
}
}

Basically this method

  • ignores *.inc files
  • for *.mp3
    • instantiate the MP3 reader
    • read title, artist and length via that reader (id3v2 if possible, id3v1 else)
    • read the filesize in MB
  • create a data row