medicalnerds.com

Using subversion (or any other version-control system) to manage your working laboratory or research files is sensible. All changes can be tracked, and it is straightforward to review old versions of files. I store all work relating to research, including notes, papers, thesis chapters, statistical analyses and even data. If I were to make catastrophic changes (deliberately or non-deliberately) it is easy to roll-back changes. It’s like a “Track Changes” on steroids.

Using subversion is quite complex, but there are easy front-ends such as TortoiseSVN that anyone can use. This article discusses some more complex version control issues, and so if you are not familiar with command-line interfaces then I would suggest reading James’ up and coming article on the simple use of TortoiseSVN using Microsoft Windows.

It is recommended that software developers store project files within a common root organisational structure.


/trunk/
/branches/
/tags/


The current development work continues within the trunk directory, which may contain multiple projects organised into multiple subdirectories. Multiple developers can “checkout” this trunk and make local changes, “committing” their changes at the end of the day. Once development is finished, then the whole lot can be copied into the “tags” directory, labelling that constellation of files and directories as a “release” – for example version 1.2.

I ignored this structure when I created my subversion repository. I naively thought that these issues didn’t apply to research work but of course they do. My research work has a huge number of interdependencies and at certain times, all these separate systems need to work together to form a complete work, such as an ethics submission, a paper for publication, or a thesis. I have multiple files that depend on one another. For example, I have just submitted an article for publication. This is a dynamically generated PDF document using LaTeX and Sweave to run dynamic statistical analysis using R. It is dependent on data (held in a CSV file), the bibliography, a bibliography style file, some radiology images, and of course my entire suite of R pre-processing functions that take raw data and spin it summary statistics, graphs etc. I will obviously have the PDF generated, and this can be archived along with a paper copy. However, in the future, I should like to be able to repeat the analysis in the exactly same way as occured at the time of publication. In any normal system, I will update bibliographies and the R code, and this may change that old dynamic document. What I need to do is store a snapshot of the working system at the time of submission. The creation of a snapshot is called “tagging”, and is very straightforward.

Creating “tags” is very easy in subversion, and does not use up excessive diskspace as it uses a well designed but simple mechanism to ensure files are only stored once. However, one needs a sensibly organised directory structure to implement them. Tags need not be checked out, and do not need to hang around in your working folders (where they will take up space).

My problem is that my working repository has working files in the root of the repository. Creating a subdirectory of “tags” would work, but whenever the project is “checked out” of the repository, all of the tags would be copied too, leading to a dramatic waste of disk space, and risking committing changes to tags that should be regarded as “unchangeable”. One must re-organise the repository into the “trunk”, “tags”, “branches” paradigm, and instead checkout the trunk as a working copy:

Migrating to a sensible repository layout

• Review current structureCurrent working copy held at /home/mark/Documents/research

  This was originally checked out of my (locally held) repository with the command: svn co file:///home/mark/Repos/research

• Create the “trunk” and “tags” subdirectories
svn mkdir trunk
svn mkdir tags

• Move all working files into the trunk
svn mv research.bib trunk/
svn mv data trunk/
svn mv thesis trunk/
... etc..

• Commit your changes (with an appropriate comment)
  
svn commit -m "Created a sensible repository layout"

• Move the old working-copy somewhere out of the way
  Here I just rename my working-copy folder for safety, although really with subversion all files are securely backed up in the repository, and I could easily check out my last revision before any of these changes were made!
  
cd ..
mv research research-old

• Check out the new trunk back in the old location
svn checkout file:///home/mark/Repos/research/trunk research


  This time, I provide a label for the checkout command (“research”) so that the trunk is checked out to an appropriately named directory (and not the default, which would be “trunk”).

• Create a “tag” of this current trunkWe don’t need to have a working copy for this at all. One uses the svn copy command to “copy” the trunk as a tag:

  svn copy file:///home/mark/Repos/research/trunk file:///home/mark/Repos/research/tags/paper_for_publication -m "Tagging submitted paper to..."


  One can use any name for a “tag”. Software developers would commonly use a version number (e.g., V1.2), but in research this is not usually appropriate. Instead use an informative name. It is straightforward to delete tags in the future (although they can always be restored from the archive).