Archive for the ‘en’ category

Java-Cron-Jobs with Spring and Quartz

3. Februar 2012

We have several Jobs running the business. At the moment they are implemented as Spring-driven TimerTasks. But the next job should be run only nightly and a cron-like configuration syntax would be fine.

So I found Quartz and had to integrate that. For getting familiar with Quartz+Spring I wrote a small proof of concept.

The job implementation is just a POJO with a public void method. It should not throw any exception. The bean can be configured via usual Spring DI:

public class MyBean {

private String message;

public void doIt() {
System.out.println("Hello World: " + message);
}

public String getMessage() {
return message;
}

public void setMessage(String message) {
this.message = message;
}

}

So fine. But how to test? For this PoC a very simple test is ok – just wait for some Quartz run and check the output manually:

import static org.junit.Assert.assertNotNull;

import org.junit.AfterClass;
import org.junit.Test;
import org.springframework.context.ApplicationContext;
import org.springframework.context.support.ClassPathXmlApplicationContext;

public class QuartzTest {

@Test
public void context() {
System.out.println("Starting Spring and within that: Quartz");
ApplicationContext ctx = new ClassPathXmlApplicationContext("spring-config.xml");
assertNotNull(ctx);
}

@AfterClass public static void sleep() throws InterruptedException {
System.out.println("Wait for job output");
Thread.sleep(8000);
System.out.println("ready");
}
}

When using Spring, we have to provide the configuration.
First I define my Job-bean

<bean id="myBean">
<property name="message" value="TEST" />
</bean>

After that I configure Quartz:

<bean id="scheduler"
class="org.springframework.scheduling.quartz.SchedulerFactoryBean"
scope="singleton">
<property name="triggers">
<list>
<!-- trigger list -->
<ref bean="job1" />
</list>
</property>
</bean>

The binding between job and Quartz is done via a configured CronTriggerBean.

<bean id="job1">
<property name="cronExpression" value="0/2 * * * * ?" />
<property name="jobDetail">
<bean>
<property name="name" value="job1" />
<property name="group" value="nightlyJobs" />
<!-- Delegieren auf unsere Bean und unsere Methode -->
<property name="targetObject" ref="myBean" />
<property name="targetMethod" value="doIt" />
<property name="concurrent" value="false" />
</bean>
</property>
</bean>

This CronTriggerBean defines two values:
1. cronExpression specifies when to run the job. An explanation about that syntax can be found at http://www.quartz-scheduler.org/documentation/quartz-1.x/tutorials/crontrigger.
2. jobDetail specifies a JavaBean with information about the job. But we dont have to write this Bean, we just can configure the generic one from Spring:
- name and group for controlling the execution via control interfaces
- targetObject and targetMethod for the delegation to our JobBean
- specify if this job should run in concurrent mode (where concurrent means in one VM!)

Building this sample with Maven is easy. You’ll need just following dependencies:

  • org.springframework::spring-core::3.0.5.RELEASE
  • org.springframework::spring-context::3.0.5.RELEASE
  • org.springframework::spring-context-support::3.0.5.RELEASE
  • org.springframework::spring-tx::3.0.5.RELEASE
  • org.quartz-scheduler::quartz::1.8.5
  • junit::junit::4.9::{scope=test}

As far as I read you can’t use Quartz 2.x because Spring 3 doesnt support that. But I havent tried it.

When running you’ll get this output:

Starting Spring and within that: Quartz
... logs from Spring
... infos from SLF4J
INFO: Starting Quartz Scheduler now
Wait for job output
Hello World: TEST
Hello World: TEST
Hello World: TEST
Hello World: TEST
Hello World: TEST
ready

Update to the Hudson book

22. November 2010

News from the Hudson by Simon Wiest book I reviewed: it is now in the press and if you order it (e.g. at Amazon), you should be able to read it by Xmas ;-) (if you can read German :-O )

Links for 2010-11-08

8. November 2010

TecChannel had a (German) article about one of my favorite tools: Launchy - a program for starting applications or opening web pages …

 

If you are a PHP programmer and interested in CakePHP, CakeFest was a conference for you. Not attended? There are slides online:

And according to Entwickler.COM the conference should be held in 2011 too…

 

If you are a Java programmer unit tests are a standard. Therefore mocking is also a discipline you know. If you are interested in the mock library Mockito, the DZone article “5 minute Mockito” is worth reading.

 

 

Hudson: Review and ideas

8. November 2010

In June I had the pleasure to do a review of the first German book about CI with Hudson by Simon Wiest. The book is announced for December 2010 and it will cover topics from basic (setup, first jobs) up to deep knowledge like writing plugins for Hudson.

As with earlier reviews I had printed the book (draft), read it and made comments on the side (and Simon had to listen to them for hours ;-)

But some comments weren’t for the author – they “just” ideas which came into my mind. But because I won’t have the time for implementing them, I’ll write about them …

Capability Plugin

The main idea is automatic dispatching of jobs to slaves according to their needs. If I remember right, TeamCity has this feature too …

The Slaves

The slaves have some capabilities. There are two kinds of them:

  1. automatic capabilities: result from tool configuration, operating system, jdk versions, build tool name+version, …
  2. manual capabilities: the admin of the slave can define key-value pairs

The Jobs

For jobs you define requirements according to the provided capabilities. And like them there are two requirements:

  1. automatic requirements: selected jdk, build tool, operating system (if you use e.g. a Windows Batch …)
  2. manual requirement: define additional requirements (or overwrite the complete formula). Use key=value (aspectj.available=true) or comparisons (ant.version >= 1.8) and combine them (and, or, xor, …)

Dispatcher strategy

The dispatcher must select all slaves which fit the defined requirements. From this list it selects the one by looking at the build queue of the slave (does it have time for building?) and net-response-time (don’t forget that much data has to be transfered …)

 

Not sure if there is something like this now …. ;-)

Another Admin job Hudson: ensure that there are not too many old builds …

14. August 2010

Especially if you have a large number of jobs and they are running more often, you’ll come to a point, where your disk is full of old builds.

Hudson provides a configuration parameter for that: “discard old builds”. This will delete old builds according to the specified number of days or number of builds.

For Apache I wrote a script which ensures, that all jobs have “discard” setting and that existing values are not higher than a defined maximum value.


/** Default-Setting for the "number of old builds" */
numberOfOldBuilds  = 10

/** Maximum of "number of days" */
maxDaysOfOldBuilds = 14

/** Should we override existing values? */
overrideExistingValues = true

/** Closures for setting default 'max number' */
setMaxNum = { job ->
   job.logRotator = new hudson.tasks.LogRotator(-1, numberOfOldBuilds)
}

/** Closures for setting default 'max number' */
setMaxDays = { job ->
 job.logRotator = new hudson.tasks.LogRotator(maxDaysOfOldBuilds, -1)
}

// ----- Do the work. -----

// Access to the Hudson Singleton
hudsonInstance = hudson.model.Hudson.instance

// Retrieve all active Jobs
allItems = hudsonInstance.items
activeJobs = allItems.findAll{job -> job.isBuildable()}

// Table header
col1 = "Old".center(10)
col2 = "New".center(10)
col3 = "Job".center(50)
col4 = "Action".center(14)
header = "$col1 | $col2 | $col3 | $col4"
line = header.replaceAll("[^|]", "-").replaceAll("\\|", "+")
title = "Set 'Discard old builds'".center(line.size())

println title
println line
println header
println line

// Do work and create the result table
activeJobs.each { job ->

 // Does the job have a discard setting?
 discardActive = job.logRotator != null

 // Enforce the settings
 action   = ""
 newValue = ""
 oldValue = ""
 if (!discardActive) {
 // No discard settings, so set the default
 setMaxNum.call(job)
 action   = "established"
 newValue = "$numberOfOldBuilds jobs"
 } else {
 // What are the current settings?
 oldDays = job.logRotator.daysToKeep
 oldNums    = job.logRotator.numToKeep

 if (oldNums > 0) {
 // We have a set value for 'numbers'
 if (oldNums > numberOfOldBuilds && overrideExistingValues) {
 // value is too large so set a new one
 setMaxNum.call(job)
 action   = "updated"
 newValue = "$numberOfOldBuilds jobs"
 oldValue = "$oldNums jobs"
 } else {
 // Correct value or we arent allowed to override.
 oldValue = "$oldNums jobs"
 }
 } else {
 // we have a value for 'days'
 if (oldDays > maxDaysOfOldBuilds && overrideExistingValues) {
 // value is too large so set a new one
 setMaxDays.call(job)
 action   = "updated"
 newValue = "$maxDaysOfOldBuilds days"
 oldValue = "$oldDays days"
 } else {
 // Correct value or we aren't allowed to override.
 oldValue = "$oldDays days"
 }
 }
 }

 // String preparation for table output
 oldValue = oldValue.padLeft(10)
 newValue = newValue.padLeft(10)
 jobname  = job.name.padRight(50)

 // Table output
 println "$oldValue | $newValue | $jobname | $action"
}
println line

// Meaningful output on the Groovy console
// (the console will output the result of the last statement)
printout = "Number of Jobs: $activeJobs.size"

In the first section I define the “constants” (line 001-008). After that I define two closures which update a given Hudson job (line 010-018).
The basic structure is the one I used in earlier scripts
The work here is in lines 053-088. But that’s pretty easy: check the given values and eventually set new values using the pre defined closures.
New is the last line: I dont use a >x = “”< instruction for suppressing the output. I use a more meaningful message: the number of jobs.

Extend Windows-Search in Windows 7

13. August 2010

I wanted to search for a string in Java properties files. I knew that there was one entry – somewhere deep in the codebase … But the search from Win7 didnt find it.

Sounds familiar? So read further …

I remembered that the search works only on particular files (doc, ppt …) and my properties files are not in that list. :-(
So I wanted to tune the search:

Open the system configuration via the start menu and type “search” into the search field. You should find the “index options” somewhere.

Press the “Advanced” button (translated from the German “Erweitert”) and swith to the second “file types” tab.

Here you can add additional file types by their suffix and specify their index option (only file information, file content).


Reindexing needs time …. but I hope after this I will be able to find my file ;)

OC4J Deployment: invalid archive: 6-Byte UTF8-Code not supported

5. August 2010

If you get an error message like this while deploying an EAR into the Oracle Application Server 10.1.3.4 (oc4j) …

don’t waste the time checking the ZIP-encoding. Have a look at the deployment descriptors first: do they contain characters which are not supported by the chosen UTF-8? Maybe some forgotten umlauts in a comment …

Probleme mit dem Gerüstbauer

31. Juli 2010

Es ist ja doch interessant, was Gerüstbauer sich ausdenken, um die Kunden zu verar***

Wir haben unsere strassenseitige Hausfront gedämmt und uns vom Gerüstbauer Schürzeberg aus Viersen ein Gerüst aufstellen lassen. Es fehlten:

  • die abgemachte Aussockelung, damit die Verlängerung des Dachüberstands gemacht werden konnte
  • eine Leiter für den Einstieg
  • die Beleuchtung des Gerüsts
  • laut Putzer eigentlich sogar die “Boards” – eine Absicherung gegen herunterfallene Kleinteile

Das feinmaschige Netz, was wegen der Putzarbeiten benötigt wird, kam auch erst nach telefonischer Nachfrage zwei Tage später – nachdem der Putzer zum ersten mal da war …

Aber berechnet wurde natürlich alles – inklusive einer Genehmigung der Stadt.

Mit dieser Genehmigung, die im Amtsdeutsch “Erlaubnis zur Sondernutzung des öffentlichen Strassenraumes” heißt, hat es folgende Bewandnis: der Bauherr – also ich – muss diese bei einer Kontrolle des Ordnungsamtes vorweisen können. Hätte der Gerüstbauer diese also besorgt, hätte er sie mir aushändigen müssen. Ob er sie überhaupt eingeholt hatte …. wurde mir zumindest nicht nachgewiesen.

Glücklicherweise verfügte ich über ein zweites Angebot von Schürzeberg und konnte eine Differenz ermitteln, die ich dann zurückgehalten hatte. Dafür revangierte dieser sich, indem er ohne Rücksprache das Gerüst früher abbaute.

Aber da unser Putzer früh genug fertig war, kam uns das schon fast gelegen – hatten wir doch so früher das Teil vom Hals ;-)

Links for 2010-07-20

20. Juli 2010

According to Entwickler.COM Microsoft has published a free ebook about “Cloud Computing” by Bob Muglia.

On Wakaleo the development of an open source book about Hudson: “Continuous Integration with Hudson“. First chapters are online …

Golem.DE has found a free German video workshop about Gimp 2.6.

On DZone Hudson creator Kohsuke Kawaguchi introduced his new startup, InfraDNA, which provides support and consulting for the Hudson Continuous Integration system.

Again on DZone there is a nice introduction into HtmlUnit. It provides a Java based WebClient which you can control via its API. With this you could write JUnit tests. But more easily you could write them with the additional assert-Methods:

@Test
public void testGoogle(){
 WebClient webClient = new WebClient();
 HtmlPage currentPage = webClient.getPage("http://www.google.com/");
 assertEquals("Google", currentPage.getTitleText());
}
@Test public void htmlunitAsserts() {
 // Load a page
 webClient.getPage("http://www.google.com/search?q=htmlunit");

 // JUnit asserts and WebClient API
 assertEquals(200,currentPage.getWebResponse().getStatusCode());
 assertEquals("OK",currentPage.getWebResponse().getStatusMessage());

 // HtmlUnit asserts
 WebAssert.assertTextPresent(currentPage, "htmlunit");
 WebAssert.assertTitleContains(currentPage, "htmlunit");
 WebAssert.assertLinkPresentWithText(currentPage, "Advanced search");

 // XPath Query
 assertTrue(currentPage.getByXPath("//h3").size()>0); //result number

 // Cookies
 assertNotNull(webClient.getCookieManager().getCookie("NID"));
}

According to Entwickler.COM Microsoft has published a bunch of Powerpoint-Templates for demonstrating the new features of PPT 2010.

If you ask yourself what Darth Vader and Yoda are doing after making the movies with George Lucas, GolemDE has found the answer: they are creating TomToms next voices … ;-)

If you are updating to Java 1.6_21 and having problems with Eclipse, have a look at this blog entry: it show how to tune the JVM settings …

Also if you write JPA applications you should have a good test suite. So looking at the blog “Patterns for Better Unit Testing with JPA” is not waste of time ;)

Hudson: Overview of the suggestd timeout settings

13. Juli 2010

In my last post I explained why and how to check the timeout settings for Hudson jobs.

On our mailinglist for Hudson users at Apache there was a suggestion to get an overview of (computed) suggested timeout settings.

So here is the follow up to my earlier code …

hudsonInstance = hudson.model.Hudson.instance</pre>
allItems = hudsonInstance.items
activeJobs = allItems.findAll{job -> job.isBuildable()}
wrappableJobs = activeJobs.findAll{job -> job instanceof hudson.model.BuildableItemWithBuildWrappers}

jobsWithoutTimeout = wrappableJobs.findAll { job ->
 job.getBuildWrappersList().findAll{it instanceof hudson.plugins.build_timeout.BuildTimeoutWrapper }[0] == null
}

println "Suggested timeout values for jobs without any ($jobsWithoutTimeout.size in total):"
jobsWithoutTimeout.each { job ->
 defaultTimeout = Math.round(job.estimatedDuration * 2 / 1000 / 60)
 if (defaultTimeout < 10) defaultTimeout = 10
 String s = defaultTimeout
 s = s.padLeft(4)
 println "$s | $job.name"
}

x = ""

The new stuff is only the creation in the last few lines. Nothing special – apart from the conversion from Long to String for getting padLeft() work ;-)

The result is a “table” like this:

Suggestion of Timeout values per Job


Follow

Bekomme jeden neuen Artikel in deinen Posteingang.