At Apaches Hudson installation I have sometimes seen the situation where too many builds are stuck and therefore blocking the executors. And sadly for me – the executors my own jobs require …
Having a policy to use the „build timeout plugin“ and kill jobs which are running too long (thinking more of „not running any more“ 😉 is good. But having a program which checks this is better …
So I tried a little bit Groovy’in for the Groovy console:
hudsonInstance = hudson.model.Hudson.instance
allItems = hudsonInstance.items
activeJobs = allItems.findAll{job -> job.isBuildable()}
wrappableJobs = activeJobs.findAll{job -> job instanceof hudson.model.BuildableItemWithBuildWrappers}
jobsWithoutTimeout = wrappableJobs.findAll { job ->
job.getBuildWrappersList().findAll{it instanceof hudson.plugins.build_timeout.BuildTimeoutWrapper }[0] == null
}
println "There are $jobsWithoutTimeout.size jobs without timeout:"
jobsWithoutTimeout.each { println "- $it.name" }
x = ""
In line 1 we get the reference to the Hudson singleton. Then we get the list of all item in line 2 which we filter in line 3 to get only buildable items, like our jobs. The line 4 contains the first thing special to this requirement: the item must be able to have a BuildWrapper.
But the most thing is done in line 5 which filters again with a closure: get all BuildWrappers for the job, but only if it is our TimeOut-Plugin. Because it can be registered only once, I check the first element of that list. It must be null for being a problem. Otherwise the job has a timeout setting.
After that, the last two lines are simply out … and the last line supresses the result output in the console.
Update:
Antoine Tulme had consulted Kohsuke Kawaguchi and he sees three possibilities of forcing the timeout setting:
- We cannot make the timeout field mandatory.
- We can create a plugin that presets the timeout field.
- We can iterate over the projects and set a value for the timeouts en masse.
Good, so I evaluate my „iteration solution“ a little more.
We have a list of all jobs without settings and so we only have to iterate over this list, instantiate and initialize the BuildTimeoutWrapper and add it to the jobs wrapper-list:
jobsWithoutTimeout.each { job ->
defaultTimeout = 180
defaultFailBuild = false
plugin = new hudson.plugins.build_timeout.BuildTimeoutWrapper(defaultTimeout, defaultFailBuild)
job.getBuildWrappersList().add(plugin)
}
BTW – If you want to work with a plugin, you could start with the Create Job Advances Plugin – maybe this requires code enhancement … and it will only for future jobs, not for existing one.
Update:
The last update of the script for setting the timeout value is this:
hudsonInstance = hudson.model.Hudson.instance
allItems = hudsonInstance.items
activeJobs = allItems.findAll{job -> job.isBuildable()}
defaultFailBuild = true
println "Cur | Est | Name"
activeJobs.each { job ->
// Get the Timeout-PlugIn
wrapper = job.getBuildWrappersList().findAll{it instanceof hudson.plugins.build_timeout.BuildTimeoutWrapper }[0]
// Get the current Timeout, if any
currentTimeout = (wrapper != null) ? wrapper.timeoutMinutes : ""
// Calculate a new timeout with a min-value
defaultTimeout = Math.round(job.estimatedDuration * 2 / 1000 / 60)
if (defaultTimeout < 10) defaultTimeout = 10
// Update the timeout, maybe requires instantiation
action = (wrapper != null) ? "updated" : "established"
if (wrapper == null) {
plugin = new hudson.plugins.build_timeout.BuildTimeoutWrapper(defaultTimeout, defaultFailBuild)
job.getBuildWrappersList().add(plugin)
} else {
wrapper.timeoutMinutes = defaultTimeout
}
// String preparation for table output
String defaultTimeoutStr = defaultTimeout
defaultTimeoutStr = defaultTimeoutStr.padLeft(5)
String currentTimeoutStr = currentTimeout
currentTimeoutStr = currentTimeoutStr.padLeft(5)
String jobname = job.name.padRight(40)
// Table output
println "$currentTimeoutStr | $defaultTimeoutStr | $jobname | $action "
}
x = ""
This updates all timeout settings and reports this like here:
Jan's Blog 
