Brian Gough’s Notes
occasional GNU-related news
GNU Parallel – a map operator for the command line
GNU Parallel is a nice recent addition to my unix toolkit. It can do a lot of things, my favorite is as a “map” operator for the command-line, automatically parallelising over the arguments of a command.
The command
parallel command {} ::: arg1 arg2 arg3 ...
is equivalent to
command arg1 & command arg2 & command arg3 & ...
run in parallel. The {} is replaced by each argument in turn.
As part of the regular updates to GSRC I need to run “make check-update” for several hundred directories for all the gnu packages. With a for loop that takes a long time, as “check-update” examines remote urls to see if a new version of a package has been released. With parallel I can use
parallel make -C {} check-update ::: gnu/*/
to get the same effect but in parallel (by default 9 processes simultaneously – to change it use -j N).
The author of GNU Parallel, Ole Tange, is giving a talk at FOSDEM (Brussels) in the GNU Dev room on Saturday 5 Feb 2011, see http://www.gnu.org/ghm/2011/fosdem/ for details. Hope to see you there!