Using a Kubernetes CronJob + the Team City API to trigger a regular backup

“Time displayed on the screen of a MacBook on a white desk with a potted plant” by Markus Spiske on Unsplash

Are you spending a lot of time managing your Team City setup, and occasionally perform a manual backup, because Team City has no facility to schedule backups? Would you rather save your backups on storage that is not used by the server? Fortunately, all of this can be automated thanks to the Team City RESTful API, which exposes the backup resource and makes it simple to script starting a backup using a tool like cURL. In this post I will show how that can be done using a Kubernetes CronJob.

Requirements

This approach relies on the following:

1. A Docker image with cURL. I used byrnedo/alpine-curl:0.1.3
2. A Team City user with the “Change backup settings and control backup process” permission. For added security, create a dedicated user who only has this permission
3. Mounting the Team City data directory persistent volume into the job
4. Mounting a persistent volume for archiving backups into the job

CronJob manifest

What is going on?

Let me explain just a few of the key lines of code in the script above:

Pod affinity

affinity:
  podAffinity:
    requiredDuringSchedulingIgnoredDuringExecution:
      - labelSelector:
          matchExpressions:
          - key: app
            operator: In
            values:
            - teamcity
          - key: type
            operator: In
            values:
            - server
        topologyKey: kubernetes.io/hostname

The pod affinity section of the CronJob spec ensures that the job pod runs on the same host as the server. This may be important to allow the job to mount the server’s data directory. It assumes that the Team City server is labelled with “app=teamcity” and “type=server”. You can adjust this according to your specific needs. If your server is not managed by Kubernetes you may not have to do this, or perhaps use a node selector to bind the pod to the right host.

Start backup

curl -s -X POST --basic --user ${ADMIN_USERNAME}:${ADMIN_PASSWORD} "https://build.example.com/app/rest/server/backup?includeConfigs=true&includeDatabase=true&includeBuildLogs=false&fileName=CronJobTeamCityBackup"

This request, done with basic auth and relying on the environment variables set in the env property kicks off the backup. The parameters are pretty self-explanatory. The “fileName” parameter is actually just a prefix — Team City will add a timestamp to it. The resulting file may be “CronJobTeamCityBackup_20180531_123005.zip”

Backup status

# Wait for backup to complete
while : ;
do
  STATUS=$(curl -s --basic --user ${ADMIN_USERNAME}:${ADMIN_PASSWORD} https://build.example.com/app/rest/server/backup)
  echo "$(date) Backup is ${STATUS}"
  [[ "${STATUS}" = "Running" ]] && sleep 60s || break
done

In this loop we infinitely check the status of the backup process. Every time the status is “Running”, sleep for 60 seconds, otherwise stop.

Archive backups

cd /datadir/backup
BACKUP=$(ls -t | head -1)
cp ${BACKUP} /backups

In the last significant snippet go to the backups directory and determine the newest backup by sorting the output of ls by modification time (newest first), and taking the first result. That file is copied to the persistent volume holding the backup archives, and you’re done!