Links for 2010-11-08

Verfasst 8. November 2010 von janmaterne
Kategorien: Blogroll, en

Tags: , , , , , , ,

TecChannel had a (German) article about one of my favorite tools: Launchy – a program for starting applications or opening web pages …

 

If you are a PHP programmer and interested in CakePHP, CakeFest was a conference for you. Not attended? There are slides online:

And according to Entwickler.COM the conference should be held in 2011 too…

 

If you are a Java programmer unit tests are a standard. Therefore mocking is also a discipline you know. If you are interested in the mock library Mockito, the DZone article „5 minute Mockito“ is worth reading.

 

 

Hudson: Review and ideas

Verfasst 8. November 2010 von janmaterne
Kategorien: en, Hudson

Tags: , ,

In June I had the pleasure to do a review of the first German book about CI with Hudson by Simon Wiest. The book is announced for December 2010 and it will cover topics from basic (setup, first jobs) up to deep knowledge like writing plugins for Hudson.

As with earlier reviews I had printed the book (draft), read it and made comments on the side (and Simon had to listen to them for hours 😉

But some comments weren’t for the author – they „just“ ideas which came into my mind. But because I won’t have the time for implementing them, I’ll write about them …

Capability Plugin

The main idea is automatic dispatching of jobs to slaves according to their needs. If I remember right, TeamCity has this feature too …

The Slaves

The slaves have some capabilities. There are two kinds of them:

  1. automatic capabilities: result from tool configuration, operating system, jdk versions, build tool name+version, …
  2. manual capabilities: the admin of the slave can define key-value pairs

The Jobs

For jobs you define requirements according to the provided capabilities. And like them there are two requirements:

  1. automatic requirements: selected jdk, build tool, operating system (if you use e.g. a Windows Batch …)
  2. manual requirement: define additional requirements (or overwrite the complete formula). Use key=value (aspectj.available=true) or comparisons (ant.version >= 1.8) and combine them (and, or, xor, …)

Dispatcher strategy

The dispatcher must select all slaves which fit the defined requirements. From this list it selects the one by looking at the build queue of the slave (does it have time for building?) and net-response-time (don’t forget that much data has to be transfered …)

 

Not sure if there is something like this now …. 😉

Another Admin job Hudson: ensure that there are not too many old builds …

Verfasst 14. August 2010 von janmaterne
Kategorien: en, Hudson

Tags: , , ,

Especially if you have a large number of jobs and they are running more often, you’ll come to a point, where your disk is full of old builds.

Hudson provides a configuration parameter for that: „discard old builds“. This will delete old builds according to the specified number of days or number of builds.

For Apache I wrote a script which ensures, that all jobs have „discard“ setting and that existing values are not higher than a defined maximum value.


/** Default-Setting for the "number of old builds" */
numberOfOldBuilds  = 10

/** Maximum of "number of days" */
maxDaysOfOldBuilds = 14

/** Should we override existing values? */
overrideExistingValues = true

/** Closures for setting default 'max number' */
setMaxNum = { job ->
   job.logRotator = new hudson.tasks.LogRotator(-1, numberOfOldBuilds)
}

/** Closures for setting default 'max number' */
setMaxDays = { job ->
 job.logRotator = new hudson.tasks.LogRotator(maxDaysOfOldBuilds, -1)
}

// ----- Do the work. -----

// Access to the Hudson Singleton
hudsonInstance = hudson.model.Hudson.instance

// Retrieve all active Jobs
allItems = hudsonInstance.items
activeJobs = allItems.findAll{job -> job.isBuildable()}

// Table header
col1 = "Old".center(10)
col2 = "New".center(10)
col3 = "Job".center(50)
col4 = "Action".center(14)
header = "$col1 | $col2 | $col3 | $col4"
line = header.replaceAll("[^|]", "-").replaceAll("\\|", "+")
title = "Set 'Discard old builds'".center(line.size())

println title
println line
println header
println line

// Do work and create the result table
activeJobs.each { job ->

 // Does the job have a discard setting?
 discardActive = job.logRotator != null

 // Enforce the settings
 action   = ""
 newValue = ""
 oldValue = ""
 if (!discardActive) {
 // No discard settings, so set the default
 setMaxNum.call(job)
 action   = "established"
 newValue = "$numberOfOldBuilds jobs"
 } else {
 // What are the current settings?
 oldDays = job.logRotator.daysToKeep
 oldNums    = job.logRotator.numToKeep

 if (oldNums > 0) {
 // We have a set value for 'numbers'
 if (oldNums > numberOfOldBuilds && overrideExistingValues) {
 // value is too large so set a new one
 setMaxNum.call(job)
 action   = "updated"
 newValue = "$numberOfOldBuilds jobs"
 oldValue = "$oldNums jobs"
 } else {
 // Correct value or we arent allowed to override.
 oldValue = "$oldNums jobs"
 }
 } else {
 // we have a value for 'days'
 if (oldDays > maxDaysOfOldBuilds && overrideExistingValues) {
 // value is too large so set a new one
 setMaxDays.call(job)
 action   = "updated"
 newValue = "$maxDaysOfOldBuilds days"
 oldValue = "$oldDays days"
 } else {
 // Correct value or we aren't allowed to override.
 oldValue = "$oldDays days"
 }
 }
 }

 // String preparation for table output
 oldValue = oldValue.padLeft(10)
 newValue = newValue.padLeft(10)
 jobname  = job.name.padRight(50)

 // Table output
 println "$oldValue | $newValue | $jobname | $action"
}
println line

// Meaningful output on the Groovy console
// (the console will output the result of the last statement)
printout = "Number of Jobs: $activeJobs.size"

In the first section I define the „constants“ (line 001-008). After that I define two closures which update a given Hudson job (line 010-018).
The basic structure is the one I used in earlier scripts
The work here is in lines 053-088. But that’s pretty easy: check the given values and eventually set new values using the pre defined closures.
New is the last line: I dont use a >x = „“< instruction for suppressing the output. I use a more meaningful message: the number of jobs.

Extend Windows-Search in Windows 7

Verfasst 13. August 2010 von janmaterne
Kategorien: en

Tags: , ,

I wanted to search for a string in Java properties files. I knew that there was one entry – somewhere deep in the codebase … But the search from Win7 didnt find it.

Sounds familiar? So read further …

I remembered that the search works only on particular files (doc, ppt …) and my properties files are not in that list. 😦
So I wanted to tune the search:

Open the system configuration via the start menu and type „search“ into the search field. You should find the „index options“ somewhere.

Press the „Advanced“ button (translated from the German „Erweitert“) and swith to the second „file types“ tab.

Here you can add additional file types by their suffix and specify their index option (only file information, file content).


Reindexing needs time …. but I hope after this I will be able to find my file 😉

OC4J Deployment: invalid archive: 6-Byte UTF8-Code not supported

Verfasst 5. August 2010 von janmaterne
Kategorien: en

Tags: , , , , , , , ,

If you get an error message like this while deploying an EAR into the Oracle Application Server 10.1.3.4 (oc4j) …

don’t waste the time checking the ZIPencoding. Have a look at the deployment descriptors first: do they contain characters which are not supported by the chosen UTF-8? Maybe some forgotten umlauts in a comment …

Probleme mit dem Gerüstbauer

Verfasst 31. Juli 2010 von janmaterne
Kategorien: Allgemein, de, en

Es ist ja doch interessant, was Gerüstbauer sich ausdenken, um die Kunden zu verar***

Wir haben unsere strassenseitige Hausfront gedämmt und uns vom Gerüstbauer Schürzeberg aus Viersen ein Gerüst aufstellen lassen. Es fehlten:

  • die abgemachte Aussockelung, damit die Verlängerung des Dachüberstands gemacht werden konnte
  • eine Leiter für den Einstieg
  • die Beleuchtung des Gerüsts
  • laut Putzer eigentlich sogar die „Boards“ – eine Absicherung gegen herunterfallene Kleinteile

Das feinmaschige Netz, was wegen der Putzarbeiten benötigt wird, kam auch erst nach telefonischer Nachfrage zwei Tage später – nachdem der Putzer zum ersten mal da war …

Aber berechnet wurde natürlich alles – inklusive einer Genehmigung der Stadt.

Mit dieser Genehmigung, die im Amtsdeutsch „Erlaubnis zur Sondernutzung des öffentlichen Strassenraumes“ heißt, hat es folgende Bewandnis: der Bauherr – also ich – muss diese bei einer Kontrolle des Ordnungsamtes vorweisen können. Hätte der Gerüstbauer diese also besorgt, hätte er sie mir aushändigen müssen. Ob er sie überhaupt eingeholt hatte …. wurde mir zumindest nicht nachgewiesen.

Glücklicherweise verfügte ich über ein zweites Angebot von Schürzeberg und konnte eine Differenz ermitteln, die ich dann zurückgehalten hatte. Dafür revangierte dieser sich, indem er ohne Rücksprache das Gerüst früher abbaute.

Aber da unser Putzer früh genug fertig war, kam uns das schon fast gelegen – hatten wir doch so früher das Teil vom Hals 😉

Links for 2010-07-20

Verfasst 20. Juli 2010 von janmaterne
Kategorien: Blogroll, en

Tags: , , , , , ,

According to Entwickler.COM Microsoft has published a free ebook about „Cloud Computing“ by Bob Muglia.

On Wakaleo the development of an open source book about Hudson: „Continuous Integration with Hudson„. First chapters are online …

Golem.DE has found a free German video workshop about Gimp 2.6.

On DZone Hudson creator Kohsuke Kawaguchi introduced his new startup, InfraDNA, which provides support and consulting for the Hudson Continuous Integration system.

Again on DZone there is a nice introduction into HtmlUnit. It provides a Java based WebClient which you can control via its API. With this you could write JUnit tests. But more easily you could write them with the additional assert-Methods:

@Test
public void testGoogle(){
 WebClient webClient = new WebClient();
 HtmlPage currentPage = webClient.getPage("http://www.google.com/");
 assertEquals("Google", currentPage.getTitleText());
}
@Test public void htmlunitAsserts() {
 // Load a page
 webClient.getPage("http://www.google.com/search?q=htmlunit");

 // JUnit asserts and WebClient API
 assertEquals(200,currentPage.getWebResponse().getStatusCode());
 assertEquals("OK",currentPage.getWebResponse().getStatusMessage());

 // HtmlUnit asserts
 WebAssert.assertTextPresent(currentPage, "htmlunit");
 WebAssert.assertTitleContains(currentPage, "htmlunit");
 WebAssert.assertLinkPresentWithText(currentPage, "Advanced search");

 // XPath Query
 assertTrue(currentPage.getByXPath("//h3").size()>0); //result number

 // Cookies
 assertNotNull(webClient.getCookieManager().getCookie("NID"));
}

According to Entwickler.COM Microsoft has published a bunch of Powerpoint-Templates for demonstrating the new features of PPT 2010.

If you ask yourself what Darth Vader and Yoda are doing after making the movies with George Lucas, GolemDE has found the answer: they are creating TomToms next voices … 😉

If you are updating to Java 1.6_21 and having problems with Eclipse, have a look at this blog entry: it show how to tune the JVM settings …

Also if you write JPA applications you should have a good test suite. So looking at the blog „Patterns for Better Unit Testing with JPA“ is not waste of time 😉

Hudson: Overview of the suggestd timeout settings

Verfasst 13. Juli 2010 von janmaterne
Kategorien: en, Hudson

Tags: , , ,

In my last post I explained why and how to check the timeout settings for Hudson jobs.

On our mailinglist for Hudson users at Apache there was a suggestion to get an overview of (computed) suggested timeout settings.

So here is the follow up to my earlier code …

hudsonInstance = hudson.model.Hudson.instance</pre>
allItems = hudsonInstance.items
activeJobs = allItems.findAll{job -> job.isBuildable()}
wrappableJobs = activeJobs.findAll{job -> job instanceof hudson.model.BuildableItemWithBuildWrappers}

jobsWithoutTimeout = wrappableJobs.findAll { job ->
 job.getBuildWrappersList().findAll{it instanceof hudson.plugins.build_timeout.BuildTimeoutWrapper }[0] == null
}

println "Suggested timeout values for jobs without any ($jobsWithoutTimeout.size in total):"
jobsWithoutTimeout.each { job ->
 defaultTimeout = Math.round(job.estimatedDuration * 2 / 1000 / 60)
 if (defaultTimeout < 10) defaultTimeout = 10
 String s = defaultTimeout
 s = s.padLeft(4)
 println "$s | $job.name"
}

x = ""

The new stuff is only the creation in the last few lines. Nothing special – apart from the conversion from Long to String for getting padLeft() work 😉

The result is a „table“ like this:

Suggestion of Timeout values per Job

How to check if all Hudson jobs have a timeout?

Verfasst 11. Juli 2010 von janmaterne
Kategorien: en, Hudson

Tags: , ,

At Apaches Hudson installation I have sometimes seen the situation where too many builds are stuck and therefore blocking the executors. And sadly for me – the executors my own jobs require …

Having a policy to use the „build timeout plugin“ and kill jobs which are running too long (thinking more of „not running any more“ 😉 is good. But having a program which checks this is better …

So I tried a little bit Groovy’in for the Groovy console:

hudsonInstance = hudson.model.Hudson.instance
allItems = hudsonInstance.items
activeJobs = allItems.findAll{job -> job.isBuildable()}
wrappableJobs = activeJobs.findAll{job -> job instanceof hudson.model.BuildableItemWithBuildWrappers}

jobsWithoutTimeout = wrappableJobs.findAll { job ->
 job.getBuildWrappersList().findAll{it instanceof hudson.plugins.build_timeout.BuildTimeoutWrapper }[0] == null
}

println "There are $jobsWithoutTimeout.size jobs without timeout:"
jobsWithoutTimeout.each { println "- $it.name" }

x = ""

In line 1 we get the reference to the Hudson singleton. Then we get the list of all item in line 2 which we filter in line 3 to get only buildable items, like our jobs. The line 4 contains the first thing special to this requirement: the item must be able to have a BuildWrapper.

But the most thing is done in line 5 which filters again with a closure: get all BuildWrappers for the job, but only if it is our TimeOut-Plugin. Because it can be registered only once, I check the first element of that list. It must be null for being a problem. Otherwise the job has a timeout setting.

After that, the last two lines are simply out … and the last line supresses the result output in the console.

Update:

Antoine Tulme had consulted Kohsuke Kawaguchi and he sees three possibilities of forcing the timeout setting:

  1. We cannot make the timeout field mandatory.
  2. We can create a plugin that presets the timeout field.
  3. We can iterate over the projects and set a value for the timeouts en masse.

Good, so I evaluate my „iteration solution“ a little more.

We have a list of all jobs without settings and so we only have to iterate over this list, instantiate and initialize the BuildTimeoutWrapper and add it to the jobs wrapper-list:

jobsWithoutTimeout.each { job ->
 defaultTimeout = 180
 defaultFailBuild = false
 plugin = new hudson.plugins.build_timeout.BuildTimeoutWrapper(defaultTimeout, defaultFailBuild)
 job.getBuildWrappersList().add(plugin)
}

BTW – If you want to work with a plugin, you could start with the Create Job Advances Plugin – maybe this requires code enhancement … and it will only for future jobs, not for existing one.

Update:

The last update of the script for setting the timeout value is this:


hudsonInstance = hudson.model.Hudson.instance
allItems = hudsonInstance.items
activeJobs = allItems.findAll{job -> job.isBuildable()}
defaultFailBuild = true

println "Cur   |  Est  | Name"
activeJobs.each { job ->
 // Get the Timeout-PlugIn
 wrapper = job.getBuildWrappersList().findAll{it instanceof hudson.plugins.build_timeout.BuildTimeoutWrapper }[0]

 // Get the current Timeout, if any
 currentTimeout = (wrapper != null) ? wrapper.timeoutMinutes : ""

 // Calculate a new timeout with a min-value
 defaultTimeout = Math.round(job.estimatedDuration * 2 / 1000 / 60)
 if (defaultTimeout < 10) defaultTimeout = 10

 // Update the timeout, maybe requires instantiation
 action = (wrapper != null) ? "updated" : "established"
 if (wrapper == null) {
 plugin = new hudson.plugins.build_timeout.BuildTimeoutWrapper(defaultTimeout, defaultFailBuild)
 job.getBuildWrappersList().add(plugin)
 } else {
 wrapper.timeoutMinutes = defaultTimeout
 }

 // String preparation for table output
 String defaultTimeoutStr = defaultTimeout
 defaultTimeoutStr = defaultTimeoutStr.padLeft(5)
 String currentTimeoutStr = currentTimeout
 currentTimeoutStr = currentTimeoutStr.padLeft(5)
 String jobname = job.name.padRight(40)

 // Table output
 println "$currentTimeoutStr | $defaultTimeoutStr | $jobname | $action "
}

x = ""

This updates all timeout settings and reports this like here:

Report of Timout-Setter

Ant: 2nd article

Verfasst 9. April 2010 von janmaterne
Kategorien: Ant, Blogroll, en

My 2nd (German) article about Ant is online. Here I write about subprojects and sandbox experiments like Ivy, props, autoconf, parallel executor, …