Introduction

First, you may be asking yourself what is gource? A video can demonstrate better than words:

This is a quick project you can put together to add a bit of zest to your team's meeting rooms or hallways. Having a prominent team status display is great for casually noticing trends and changes in the code and deployments. At my workplace we have used them to display pager duty schedules, Jenkins build status, commit activity (which this post demonstrates), and much more!

Requirements

Rendering Server

Most examples of gource show how to set up and configure it with an active display on the rendering server. Gource requires an active DISPLAY, but with some tricks we can give gource an in memory buffer and make it think it is running with an active display (with xvfb). It's also nice to render automatically on a schedule (with cron), and host the videos for easy retrieval. I kicked off a virtual machine in our VMWare cluster with plenty of RAM and an extra virtual core to handle the bursty work load. Though there are certainly ways to get faster rendering performance, this wasn't much of a concern for us since we only cared to update commit activity weekly and render on the weekends.

Install a distribution

I settled on Arch as my distro to handle the rendering, though it shouldn't really matter which distro you are using. Here's the important packages I installed to get the basic functionality in place:

Be aware not all distros will provide packages for these pieces of software, you may have to find alternative sources for your package manager or manually compile/install.

Since Arch uses a rolling release cycle, some packages interdependencies can shift around a bit more frequently than other distros. For completeness, here is a complete listing of packages on the system.

Add a user on the system

Keeping it simple, create an account named render with defaults and the usual user group of the same name: 

#!/bin/bash
useradd -D -U render

Configure ssh

Be sure to give the render user keys that can be used to access your git repositories. Either use an exiting (shared) set of keys (e.g. jenkins) or generate a new pair and upload the public key to your repository manager:

ssh-keygen

Bonus, if you are using a VM console window (which in my experience tends to be slow), you can now use any ssh client to perform the rest of the system configuration.

Create rendering scripts

This sample script demonstrates automatic cloning of repositories found on a gerrit server. And launching a gource render of the master branch for each project. One of my past posts discusses how to get gerrit set up.

gource-render.sh

#!/bin/bash
#For bash "strict mode" enable the following line, generally preferred but breaks this script for some reason
#set -euo pipefail
IFS=$'\t\n'

#Add a bit of color support to our script, thanks stack overflow
##########################################################################
black='\E[30;47m'
red='\E[31;40m'
green='\E[32;40m'
yellow='\E[33;40m'
blue='\E[34;40m'
magenta='\E[35;40m'
cyan='\E[36;40m'
white='\E[37;40m'

cecho ()                     # Color-echo.
                             # Argument $1 = message
                             # Argument $2 = color
{
local default_msg="No message passed."
                             # Doesn't really need to be a local variable.

message=${1:-$default_msg}   # Defaults to default message.
color=${2:-$black}           # Defaults to black, if not specified.

  echo -e "$color"
  echo "$message"
  tput sgr0                  # Reset to normal.

  return
} 
##########################################################################

#Optional step, retrieve images for avatars for contributors, alternatively lookup how to use gravatar in GOURCE_OPTS instead
${HOME}/get-avatars.sh
AVATAR_DIR=${HOME}/avatars

#Make this the location where you initially clone all of the projects you wish to render, they will be automatically updated
PROJECT_DIR=${HOME}/projects
VIDEO_DIR=${HOME}/videos

#Gource currently only supports these framerates: 25, 30, 60
FRAME_RATE=25
VIDEO_SIZE=1920x1080
START_DATE=$(date --date="2 weeks ago" +'%Y-%m-%d %T') #Date format passed to gource
END_DATE=$(date +'%Y-%m-%d %T')

#Mess with these settings however you wish, we decided not to be blinded by the bloom effect
GOURCE_OPTS=$(echo "--stop-at-end --auto-skip-seconds 1 --seconds-per-day 6 --file-idle-time 0 --max-files 0 --hide filenames --bloom-multiplier 0.1 --bloom-intensity 0.1 --start-date \"$START_DATE\" --stop-date \"$END_DATE\" --font-size 32 --user-image-dir ${AVATAR_DIR}")

GERRIT_SERVER=gerrit.toastedbits.com
GERRIT_USER=jenkins
PROJECTS=$(ssh -p 29418 ${GERRIT_USER}@${GERRIT_SERVER} gerrit ls-projects)

mkdir -p ${PROJECT_DIR}
mkdir -p ${VIDEO_DIR}
echo "" > ${HOME}/video_fail_list.log
echo "" > ${HOME}/video_list.log

pushd .
cd ${PROJECT_DIR}

for project in $PROJECTS; do
	BASE=$(basename $project)
	RENDER_NAME="$(echo $project | sed -e 's@/@_@g').mp4"

	if [[ ! -d ${PROJECT_DIR}/${BASE} ]]; then
		cecho "Encountered new project: $project (cloning)" $green
		git clone "ssh://${GERRIT_USER}@${GERRIT_SERVER}:29418/$project"
	fi

	pushd .

	cd $BASE
	git fetch
	git checkout master
	git reset --hard origin/master

#Rendering in two steps rather than direct pipe to ffmpeg intentionally - this is a workaround for ffmpeg rendering incorrectly while waiting for frames to buffer from gource
	cecho "Rendering video for project: $project to ${VIDEO_DIR}/${RENDER_NAME}" $green
	set -x
	eval "xvfb-run -a -s \"-screen 0 ${VIDEO_SIZE}x24\" gource -${VIDEO_SIZE} -r ${FRAME_RATE} --title \"$project\" ${GOURCE_OPTS} -o ${HOME}/gource.ppm"
	ffmpeg -y -r ${FRAME_RATE} -f image2pipe -vcodec ppm -i ${HOME}/gource.ppm -vcodec libx264 -preset ultrafast -pix_fmt yuv420p -crf 1 -threads 0 -bf 0 ${VIDEO_DIR}/${RENDER_NAME}
	set +x

	RESULT=$?
	if [[ $RESULT -eq 0 ]]; then
		cecho "Rendering succeeded for project: $project" $green
		echo "Rendering succeeded for project: $project" >> ${HOME}/video_list.log
	else
		cecho "Rendering failed for project: $project" $red
		echo "Rendering failed for project: $project" >> ${HOME}/video_fail_list.log
	fi

	popd
done

cd $VIDEO_DIR
#sometimes videos are rendered without any commit activity, dont list them
find $VIDEO_DIR -name "*.mp4" -size +1 -printf "%f\n" | sort > $VIDEO_DIR/video_list.txt

mv ${HOME}/video_list.log $VIDEO_DIR/video_list.log
mv ${HOME}/video_fail_list.log $VIDEO_DIR/video_fail_list.log
rm ${HOME}/gource.ppm

popd

Take extra care with ffmpeg, when I had originally written this script (about a year ago) I encountered a bug where piping direct to ffmpeg caused the rendered video to consist purely of only solid green frames. By manually buffering the output to a file and then sending the entire .ppm file to ffmpeg I was able to work around the issue. Unfortunately gource's ppm files can grow to be quite huge depending on the desired framerate, screen size, and duration. It is possible fixes have since been made to ffmpeg, or (speculation) perhaps direct rendering enhancements to gource itself (though I would find this fairly unlikely).

Note this script outputs a list of successfull renders to video_list.txt If a repository has no recent activity, the old video is destroyed and does not appear in the list. This is important to clients who wish to retrieve a list of available videos to play.

Setup nginx

I recommend setting up a simple symlink to the render folder from the web html directory. Special configuration to nginx may need to be made to allow following symlinks.

Setup cron

As the render user run crontab -e and add a line to run the gource-render.sh on the desired schedule. For weekly on saturdays:

0 0 * * 6 /home/render/gource-render.sh

Setup avatars

It's possible to configure gource to use a directory of photos for avatars instead of placeholder generic silhouettes. Below is an example script to scrape names from the git repository, you'll have to fill in the bit that actually knows how to retrieve images for the users though.

get-avatars.sh

#!/bin/bash

IFS=$'\n\t'

PROJECT_DIR=${HOME}/projects
AVATAR_DIR=${HOME}/avatars

for project in $(ls $PROJECT_DIR); do
	cd ${PROJECT_DIR}/$project
	for user in $(git log --pretty='format:%ae|%an' | sort | uniq); do
		userid=$(echo $user | sed -e 's/@.*//')
		username=$(echo $user | awk -F '|' '{print $2}')
		if [[ ! -s "${AVATAR_DIR}/${username}.jpg" ]]; then
			echo "In place of this line, I used the userid and username to query our corporate directory for a photo with curl"
		fi
	done
done

There are many tools at your disposal including wget, import.io (my favorite, perhaps deserves a future post!), and chrome plugins: Scraper and Kimono

Display client (Raspberry Pi)

More professional setups will not be dependent on such constrained environments as the Raspberry Pi. However if you can get video to play nice on this system, it should relatively be a breeze on more powerful setups. Note, if you are also wanting to launch a browser tab "slideshow" (via browser plugin, etc), I strongly recommend looking for something more powerful than a RPi. Though I have not tried this on the latest B+ model, so your mileage may vary

The Raspberry Pi fortunately supports hardware video accelleration, albeit only through special software, omxplayer, provided for raspberry pi distributions (e.g. raspbian).

You will also have to install the appropriate packages with apt-get. omxplayer, ssh, and cron should be enough to get started. Here's a full package list on my raspberry pi.

play-gource.sh

#!/bin/bash

HOST=gource.toastedbits.com
VIDEO_LIST=${HOME}/video_list.txt
curl http://$HOST/videos/video_list.txt 2> /dev/null > $VIDEO_LIST

export DISPLAY=:0 
CT_FILE=${HOME}/.vidct
VIDEO_CT=$(cat $VIDEO_LIST | wc -l)

CT=$(cat ${CT_FILE})
CT=$(expr $CT + 1)
IDX=$(expr $CT % ${VIDEO_CT} + 1)
echo $CT > $CT_FILE

VIDEO=$(sed -n "${IDX},${IDX}p" $VIDEO_LIST)
VIDEO=http://${HOST}/videos/${VIDEO}
echo "Playing video ${IDX}/${VIDEO_CT}: $VIDEO" | logger
omxplayer $VIDEO 2>&1 | logger
echo "Video finished playing" | logger

Configure the user on the system that drives the main display to run this script on a cron schedule every minute or so. The videos should stream from your gource server in round robin.

Conclusion

There are many approaches to this problem, the main thing is to not to overwhelm viewers with too many pieces of information. Keep the most important information on the screen for the longest ammount of time (if not constantly) and consider using multiple displays responsible for different views (Though elaborate setups can get expensive).

Also, be prepared for other people to take over your displays and/or hose your rendering server and clients. Having a display and monitoring solution in the office is a fun project to play with, but is naturally a second priority to getting real work done. Sometimes maintenance responsibilities are passed around or temporarily shrugged off. If this becomes a perpetual problem consider increasing (virtual and physical) security on both the rendering server and the display clients.