How to search for all Jenkins jobs with Daniel as e-mail recipient

In a working environment it’s not uncommon that some people find new jobs and new people come in to take their places. Ideally this transition goes smoothly, but sometimes details go unnoticed or get skipped completely. One of these details is the changing of email addresses for project members. I admit, it’s not a small thing and something you should do right away. In my defence, when you’re rushed or too focused on your own job this is apparently something that slips through.

In this case we are talking about the email recipient(s) of a project/job in Jenkins CI when something goes awry. Coincidentally this was only discovered after builds started going red on projects I didn’t get notified on, and nobody was fixing it.

The former employee took part in many projects, which meant that an equal quantity of edits would be needed to replace him as recipient in all projects he worked on. Luckily we live in times of automation, which allows us to efficiently and consistently change the old email address to the new one.

To my knowledge a suitable script wasn’t available back then, so I needed to write one myself. Creating a script for the first time takes some rewriting and debugging which often exceeds the time of changing all configurations by hand. Nevertheless it is worth to write a script, since the number of projects to check for the old address might get too dull to complete successfully. Another reason, which didn’t have any merit at that time, is the possibility to already include the new address during a time of transition.

The first step was to successfully find all projects that have a specific email address among the recipients. It took some puzzling, as I wasn’t too familiar with the Groovy Console of Jenkins yet.

import hudson.model.*
import hudson.maven.reporters.MavenMailer
 
Hudson.instance.items.findAll { job ->
   job.metaClass.hasProperty (job, 'reporters') &&
      job.reporters?.findAll{ it instanceof MavenMailer}.any{ reporter ->
	 reporter.recipients =~ 'daniel@example.com'  
         }
   }.each { job ->
      if (job)
         println "${job?.name} has Daniel as a recipient"
   }

The next step was to actually change the recipient to me.

import hudson.model.*
import hudson.maven.reporters.MavenMailer 

def from = 'oldguy@example.com'
def mine = 'me@example.com' 
def mvnmailer = { job -> (job.metaClass.hasProperty (job, 'reporters')) ? job.reporters?.findAll { it instanceof MavenMailer } : [] }
 
def projects = Hudson.instance.items.findAll { job ->
   job.metaClass.hasProperty (job, 'reporters') &&
      mvnmailer(job).any{ reporter -> reporter.recipients =~ from }
   }
println "$projects.size project(s) found"
if (projects?.size() > 0)
projects.each { job -> 
         mvnmailer (job).each {it.recipients = it.recipients.replaceAll(from, mine) }
         job.save() // pretty useless if you change it but you don't save it
      }

On line 6 you can see a closure, basically I needed a tasteful way to find all the MavenMailer instances of a job. At line 10 I’m using it to filter for the old recipient and at line 15 replace the old recipient with the new one. Finally it’s mandatory to actually persist your changes, otherwise it was just another futile exercise in wasting computing cycles.

The script did it’s job, though there is still room for optimizations. For instance, there is no need to cycle to the found jobs twice and probably in time there will come a moment where some clever use of Groovy makes the code even more elegant.

How to add a log rotator to multiple Hudson/Jenkins jobs

In my professional work environment I have used Hudson/Jenkins as a continuous integration tool and at the same time automate some tasks. Jenkins CI has become a valued part of the development track and over time others got to appreciate its usefulness.

Sadly, the more you use it, then more a need arises to maintain it. One aspect is the amount of logging and information you want to keep from past builds. This isn’t just build logs, but also artifacts and reports generated in the past. Jenkins has a feature to discard old builds. This feature is enabled on some jobs, but there was no guideline on the number of builds, artifacts or the number of days that logs and artifacts were kept.

So the first thing to do is figure out which projects actually had it enabled and the settings for those projects. Pasting and running the following groovy script into the Jenkins Groovy console did the trick.

import hudson.model.*
def jobs = Hudson.instance.items
 
jobs.findAll{ it.logRotator && !it.disabled }.each {
   println it.name;
   if (it.logRotator.daysToKeep >= 0){
      println "\t retains builds for ${it.logRotator.daysToKeep} day(s)"
      }
   if (it.logRotator.numToKeep >= 0){
      println "\t retains ${it.logRotator.numToKeep} build(s)"
      }
   if (it.logRotator.artifactDaysToKeep >= 0){
      println "\t retains artifacts for ${it.logRotator.artifactDaysToKeep} day(s)"
      }
   if (it.logRotator.artifactNumToKeep >= 0){
      println "\t retains artifacts of ${it.logRotator.artifactNumToKeep} builds"
      }
   }

There is no reason to list disabled jobs, so we will ask if the job has a logRotator and is not disabled. It’s important to know that the log rotator will return -1 or null for fields that aren’t set. The return values varied, but boolean operators in Groovy are null-safe. The result will be a sort of formatted overview of jobs and their settings for the log rotator.

With this overview I can decide what the default settings will be for the log rotator (just picking the longest retention time and the highest number of retained builds) and executed the following script to configure the log rotation of enabled jobs that didn’t have one.

import hudson.model.*
def jobs = Hudson.instance.items

jobs.findAll{ !it.logRotator && !it.disabled }.each { job ->
job.logRotator = new hudson.tasks.LogRotator ( 30, 40, 30, 10) // days to keep, num to keep, artifact days to keep, num to keep
   // println "$it.name" edited and changed to statement below
   println "$job.name"
   }

It took a while to figure out what to change, but programming the change is always more fun than going through the configuration file of each job to check if log rotation is enabled. I don’t know how much time I saved, but at least the change was consistent for each job.

Benefits of running your own test database

Setting up your own work environment proper and suiting your needs is an important and often undervalued part of software development. One of the things is to consider where and how to test the code. In big teams you have to conform to the standards or come up with a common understanding. I happen to have more freedom in this aspect, which made me change from using the “everything works on my pc” idea to the “keep everything centralized” one and then to the current one, a mix of both.

It has been only recently that I got it the way I wanted, more about that in another post. In the current configuration there is a production database, central test database that conforms with the latest changes in the version control system and a local test instance which has any changes which have not been committed yet. The main driver being that among others the continuous integration (CI) uses the central test database for testing and I want to continue development without breaking tests run by the CI server.

The advantages of my new configuration are:

  • the CI server can run its test without interference by developers
  • developers can their test without outside interference
  • speed, database access is less of a bottleneck

The disadvantages are:

  1. developers need their own test database to prevent interference
    1. the developer must update his test database schema with the latest changes from the central test database schema
    2. the developer must update the central test database’s schema, if necessary
  2. unrealistic performance expectations

Disadvantage 1 isn’t that hard to fix. For disadvantages 2.1 and 2.2 the solution is to restrict the number of developers on code that interacts with the database through a persistence subproject. And the last disadvantage might exist on several levels. The test database’s server might have different performance from the production server, which will certain differ between clients.

In conclusion I think I made my world a little better through these improvements. Another night with happy dreams ahead of me.