Apr 16-17, 2015
Instructors: Emmanouil (Manos) Farsarakis, Mario Antonioletti, Mark Stillwell
Helpers: Iain Emsley
ARCHER, the UK's national supercomputing service, offers training in software development and high-performance computing to scientists and researchers across the UK. As part of our training service we are running a 2 day Software Carpentry workshop.
Software Carpentry workshops help researchers become more productive by teaching software development skills that enable more to be done, in less time, and with less pain. We will cover skills including version control, task automation, good programming practice and automated testing. These are skills that, in an ideal world, researchers would master before tackling anything with "cloud" or "peta" or "HPC" in their name, skills that enable researchers to optimise their time and provide them with a secure basis to optimise and parallelise their code.
Who: The workshop is aimed at graduate students, post-docs and other researchers. You must have some experience of writing code or scripts and be familiar with programming concepts including conditionals, loops, arrays and functions. You should also be comfortable with using the bash shell. For an introduction to the shell, please see, for example, Software Carpentry's lessons on The Unix Shell.
Requirements: Participants must bring a laptop with a few specific software packages installed (listed below). They are also required to abide by Software Carpentry's Code of Conduct.
Contact: Please mail firstname.lastname@example.org for more information.
To register, or to get more information, please, visit the ARCHER training page.
|09:00||Introduction and software set-up|
|10:00||Version control with Git|
|11:00||Version control with Git|
|14:00||Building programs with Python|
|15:30||Building programs with Python|
|09:00||Automating tasks with Make|
|11:00||Automating tasks with Make|
|13:00||How (and how much) to test programs|
|15:00||How (and how much) to test programs|
|16:00||Best practices for scientific computing|
To participate in a Software Carpentry workshop, you will need working copies of the software described below. Please make sure to install everything (or at least to download the installers) before the start of your workshop.
Make sure you install, or have available, a text editor that you are comfortable with using. When you're writing code, it's nice to have a text editor that is optimized for writing code, with features like automatic color-coding of key words.
Bash is a commonly-used shell. Using a shell gives you more power to do more tasks more quickly with your computer.
Git is a state-of-the-art version control system. It lets you track who made changes to what when and has options for easily updating a shared or public version of your code on github.com.
Python is becoming very popular in scientific computing, and it's a great language for teaching general programming concepts due to its easy-to-read syntax. We teach with Python version 2.7, since it is still the most widely used. Installing all the scientific packages for Python individually can be a bit difficult, so we recommend an all-in-one installer.
Originally invented to manage compilation of programs written in languages like C, Make can be used to automatically update any set of files that depend on another set of files. This makes it a good solution for many data analysis and data management problems. While there are many build tools now in existence (e.g. ANT and CMake) they share the same fundamental concepts as Make.
nano is the editor installed by the Software Carpentry Installer,
it is a basic editor integrated into the lesson material.
Notepad++ is a popular free code editor for Windows. Be aware that you must add its installation directory to your system path in order to launch it from the command line (or have other tools like Git launch it for you). Please ask your instructor to help you do this.
Install Git for Windows by download and running the installer. This will provide you with both Git and Bash in the Git Bash program.
This installer requires an active internet connection
After installing Python and Git Bash:
Once you have installed Git Bash you can install Make by:
bindirectory where you installed Git Bash e.g.
C:\Program Files (x86)\Git\bin.
make, and press Enter.
make: *** No targets specified and no makefile found. Stop.This means that Make was successfully installed. Otherwise, you'll see this error message:
bash: make: command not found
The default shell in all versions of Mac OS X is bash,
so no need to install anything. You access bash from
the Terminal (found
/Applications/Utilities). You may want
to keep Terminal in your dock for this workshop.
Install Git for Mac by downloading and running the installer. For older versions of OS X (10.5-10.7) use the most recent available installer available here. Use the Leopard installer for 10.5 and the Snow Leopard installer for 10.6-10.7.
For OS X, version 10.9 (Mavericks) or above, download the Command Line Tools by doing:
For more information, see the OSX Daily blog.If you have an older OS X version and you do not already have access to
makefrom within your shell, you will need to install XCode (which is free, but over a gigabyte to download).
Once XCode has installed:
You will now be able to run
make within your shell.
Kate is one option for Linux users.
In a pinch, you can use
which should be pre-installed.
The default shell is usually
but if your machine is set up differently
you can run it by opening a terminal and typing
There is no need to install anything.
If Git is not already available on your machine you can try
to install it via your distro's package manager
We recommend the all-in-one scientific Python installer Anaconda. (Installation requires using the shell and if you aren't comfortable doing the installation yourself just download the installer and we'll help you at the workshop.)
bash Anaconda-and then press tab. The name of the file you just downloaded should appear.
yesand press enter to approve the license. Press enter to approve the default location for the files. Type
yesand press enter to prepend Anaconda to your
PATH(this makes the Anaconda distribution the default Python).
Make is a standard tool on Linux systems and should already be available.
As an alternative to the above, you can use a virtual machine (VM) rather than install all the software above. To use a VM:
To check you have the correct version of Python:
To check you have the necessary software and tools:
If you find yourself in a shell that you don't recognise, or in an editor that you can't get out of then see Recognising prompts and how to exit.
Software Carpentry online lessions:
Wilson G, Aruliah DA, Brown CT, Chue Hong NP, Davis M, et al. (2014) Best Practices for Scientific Computing. PLoS Biol 12(1): e1001745. doi:10.1371/journal.pbio.1001745.
Sandve GK, Nekrutenko A, Taylor J, Hovig E (2013) Ten Simple Rules for Reproducible Computational Research. PLoS Comput Biol 9(10): e1003285. doi:10.1371/journal.pcbi.1003285.
Noble WS (2009) A Quick Guide to Organizing Computational Biology Projects. PLoS Comput Biol 5(7): e1000424. doi:10.1371/journal.pcbi.1000424.
Ram K (2013) "git can facilitate greater reproducibility and increased transparency in science", Source Code for Biology and Medicine 2013, 8:7 doi:10.1186/1751-0473-8-7.
Glass, R. (2002) Facts and Fallacies of Software Engineering, Addison-Wesley, 2002. (PDF).