Mar 032010
Over the last two decades I have accumulated thousands of references that have travelled with me all the way from bibtex-mode through Endnote, Citavi and some more obscure packages until we finally came full circle and ended up in bibtex-mode again. To my mild surprise, my use of (some) keywords has been fairly consistent so that it was relatively easy (using make, bibtool and bibtex2html) to create a 380+ entries strong online bibliography on the Extreme Right in Western Europe. Enjoy.
Apr 102008
A couple of weeks ago, I posted an article on how make and Makefiles can help you to organise your Stata projects. If you are working in a unix environnment, you’ll already have make installed. If you work under Windows, install GNU make – it’s free, and it can make your Stata day. Rather unsurprisingly, make is also extremely useful if you have large or medium-sized latex project (or if you want to include tables and/or graphs produced by Stata) in a latex document. For instance, this comes handy if you have eps-Figures and use pdflatex. pdflatex produces pdf files instead of dvi files. If you produces slides with, this can save you a lot of time because you don’t have to go through the latex – dvips – ps2pdf cycle. However, pdflatex cannot read eps files: you have to convert your eps files with pstoedit to the meta post format, then use meta post to convert them to mps (which can be read by pdflatex). With this Makefile snippet, everything happens automagically:

#New implicit rules for conversion of eps->mp->mps
#Change path if you have installed pstoedit in some other place : %.eps
c:pstoedit/pstoedit.exe -f mpost $*.eps $*.mp

mpost $*.mp
mv $*.1 $*.mps
rm $*.mp

#Now specify a target

presentation.pdf: presentation.tex mytab1.tex myfig.mps

#Optional: if you want to create dataset x.eps, run
#Stata must be in your path
%.eps :
tab wstata -e do $<

Now type make presentation.pdf, and make will call Stata, pstoedit, metapost and pdflatex as required. If you need more figures, just write the do-file and add a dependency.

Social Bookmarks:
Makefile helps with latex, too 1Makefile helps with latex, too 2Makefile helps with latex, too 3Makefile helps with latex, too 4Makefile helps with latex, too 5Makefile helps with latex, too 6Makefile helps with latex, too 7Makefile helps with latex, too 8Makefile helps with latex, too 9Makefile helps with latex, too 10Makefile helps with latex, too 11Makefile helps with latex, too 12

Technorati Tags: make, latex, Makefile, Stata, political science, political, science, meta post, pstoedit, postscript, eps, pdf, pdflatex, beamer

Mar 202008
By relying on scripts (do-files), Stata encourages you to work in a structured, efficient and reproducible way. This text-based approach is familiar and attractive to anyone who has ever used a unix shell and the standard utilities. Actually, unix-flavoured utilities can make your stata experience even better. One non-obvious candidate is make, which is usually used for programming projects that require some sort of compilation.

Consider the following scenario. You have two ascii files of raw data, micro.raw and macro.raw. You want to read in both files, correct some errors, convert them to stata’s .dta format, merge them, apply some recodes, do a lot of preliminary analyses, and finally produce a postscript file and a table for inclusion in LaTeX. Of course, you could write one large do-File that does the job. But then, changing a tiny detail of the final graph would require you to repeat the whole procedure, which is time-consuming (especially when you work with very large datasets). Moreover, you often want to perform some interactive checks at an intermediate stage. So breaking down the large job in a number of smaller jobs that are much easier to maintain and produce a set of intermediate files seems like a good idea.

However, now running these jobs in the correct order (and determining whether a job needs to be re-run at all) becomes crucial. And this is where make can help you. A Makefile simply contains information on dependencies amongst your files as well as instructions for getting from A to B. Here is an example for the above scenario

#your comments here

#Define new implicit rule: if you want to create dataset x.dta, run
%.dta :
tab wstata -e do $<

micro.dta : micro.raw

macro.dta: macro.raw

combined.dta: micro.dta macro.dta

final.dta: combined.dta

finalmodel.ster: final.dta
tab wstata -e do

tabandgraph: finalmodel.ster
tab wstata -e do

The first two lines after the comment define a new rule: Whenever you ask make to make a datset, it will run Stata in batch-mode with a do-file of the same name as input. The next line specifies that micro.dta depends on the raw data (micro.raw) as well as on the code that transforms micro.raw into micro.dta. If either file changes, make knows that micro.dta has to be recreated, otherwise, it can be left alone. The same applies to macro.dta, combined.dta and final.dta. So if you type (at the command prompt) “make final.dta” will (re-create) combined.dta, micro.dta, and macro.dta if and only if this is necessary (i.e. if either the scripts or the raw data have changed. The same goes for your tables and graphs. If you change but none of the other files, only will be re-executed. If, on the other hand, you correct an error in or get a new release of micro.raw, all the intermediate datasets and the saved estimates will be re-created in the correct order before is re-executed.

Two fine points: a) instructions for making something start with a tabstop and b) -e (instead of /E) is required because MSDOS-like options confuse make. If your project is complex, makefiles can save your day. If you work in a *nix environment, make and its documentation is most probably installed on your system. In a Windows environment, you will have to install either Cygwin or MinGW to get make. The Wikipedia article is an excellent starting point for learning the syntax of makefiles.

Technorati Tags: stata, data, make political science

Social Bookmarks:
How Stata and a Makefile can make your day 13How Stata and a Makefile can make your day 14How Stata and a Makefile can make your day 15How Stata and a Makefile can make your day 16How Stata and a Makefile can make your day 17How Stata and a Makefile can make your day 18How Stata and a Makefile can make your day 19How Stata and a Makefile can make your day 20How Stata and a Makefile can make your day 21How Stata and a Makefile can make your day 22How Stata and a Makefile can make your day 23How Stata and a Makefile can make your day 24