Makefile: Difference between revisions

2,149 bytes added ,  13 years ago
Sometimes, "later" ''does'' come. I said "will continue tonight" and finished over 7 months later, but here it is. ;-)
[unchecked revision][unchecked revision]
(Sometimes, "later" ''does'' come. I said "will continue tonight" and finished over 7 months later, but here it is. ;-))
Line 10:
 
But 'make' requires a well-written Makefile to do these things efficiently and reliably.
 
= Makefile Tutorial =
 
Setting up a simple Makefile is easy. You can even go a long way ''without'' a Makefile, simply by using make's internal rulesets. Setting up a global project Makefile that works as intended is quite a bit harder.
 
The 'make' manual will tell you about the hundreds of things you can do with a Makefile, but it doesn't give you an example for a ''good'' Makefile. The following examples mainly use the [http://pdclib.rootdirectory.de/trac.fcgi/browser/trunk/Makefile PDCLib] Makefile as a template, and show some of the "tricks" used therein that may not be that obvious to the make beginner. It aims at the GNU variant of make, and uses some expressions specific to that particular make implementation (for example the patsubst function). Adaption to any special needs you might have should not be too hard once you got the idea, however.
 
The Makefile creates only one project-wide linker library, but it should be easy to expand it for multiple binaries/libraries.
 
== Basics ==
Line 36 ⟶ 28:
</pre>
 
Both ''dependency'' and ''command'' are optional. There might be more than one ''command'' line, in which case they are executed in sequence.
Note that the command ''must be tab-indented''. If your editor environment is set to replace tabs with spaces, you have to undo that setting while editing a makefile.
 
Note that the commandcommands ''must be tab-indented''. If your editor environment is set to replace tabs with spaces, you have to undo that setting while editing a makefile.
What makes makefiles so hard to read, for the beginner, is that we are not looking at an imperative program here that is executed top-down; 'make' reads the ''whole'' makefile, building a dependency tree, and then resolves the dependencies by hopping from rule to rule until the target you gave it on the command line has successfully been resolved.
 
What makes makefiles so hard to read, for the beginner, is that weit areis not lookingprocedural atin ansyntax imperative program here that is(i.e., executed top-down;), but functional: 'make' reads the ''whole'' makefile, building a dependency tree, and then resolves the dependencies by hopping from rule to rule as necessary until the target you gave it on the command line has successfully been resolved.
I won't go into further details. This is not a man page, but a tutorial, so I will show you how a makefile is built, and the ideas behind each line.
 
IBut wonlet'ts not go into furtherinternal details of 'make'. This is not a man pagetutorial, butnot a tutorialman page, so Iit will show you how a makefile is built, and the ideas behind each line.
 
= Makefile Tutorial =
 
Setting up a simple Makefile is easy. You can even go a long way ''without'' a Makefile, simply by using make's internal rulesets. Setting up a global project Makefile that works as intended is quite a bit harder.
 
The 'make' manual will tell you about the hundreds of things you can do with a Makefile, but it doesn't give you an example for a ''good'' Makefile. The following examples mainly use the [http://pdclib.rootdirectory.de/trac.fcgi/browser/trunk/Makefile PDCLib] Makefile as a template, and show some of the "tricks" used therein that may not be that obvious to the make beginner. It aims at the GNU variant of make, and uses some expressions specific to that particular make implementation (for example the patsubst function). Adaption to any special needs you might have should not be too hard once you got the idea, however.
 
The Makefile creates only one project-wide linker library, but it should be easy to expand it for multiple binaries/libraries.
 
=== Automated Testing, pt. 1 ===
 
As the Makefile presented in this tutorial takes care of automated testing in a special way, this approach will be explained up front, so the related parts of the tutorial will make sense to you.
 
As stated above, this tutorial is mainly derived from work done on the PDCLib project - which builds a single linker library. In that project, there is strictly one library function per source file. In each such source file, there is a test driver attached for conditional compilation, like this:
<pre>
#ifdef TEST
int main()
/* Test function code here */
return NUMBER_OF_TEST_ERRORS;
#endif
</pre>
Thus, when I compile that source is compiled with ''gcc -c'', Iit getresults in an object file with the library function code; when I compilecompiled with ''gcc -DTEST'', Iit getgives a test driver executable for that function. Returning the number of errors allows me to do a grand total of errors encountered (see below).
 
== File Lists ==
 
First, I assemble various "file lists" are assembled which Iare needneeded later in the Makefile.
 
=== Non-Source Files ===
Line 53 ⟶ 71:
</pre>
 
Further down Iwe will have a target ''"dist"'', which packages all required files into a tarball. I create listsLists of the sources and headers are created anyway,. butBut forthere are commonly some auxiliary files, which are not usedreferenced anywhere else in the Makefile but should still end up in the tarball,. IThese need this explicit list to haveare themlisted includedhere.
 
=== Project Directories ===
Line 59 ⟶ 77:
PROJDIRS := functions includes internals
</pre>
Those are my subdirectories. Itholding couldthe beactual yoursources. sub(Or projectsrather, orsearched whatever.for Isource couldfiles useautomatically, similarsee "autobelow.) detection"These forcould thembe assubprojects, Ior dowhatever. forWe mycould sourcesimply filesrecurse in furtherany downsubdirectory, but asif Iyou like to have temporary subdirectories in myyour project (for testing, keeping reference docssources etc.), that wouldn't work.
 
''Note that this is not a recursive approach; there is no Makefile in those subdirectories. Dependencies are hard enough to get right if you use one Makefile. GoogleAt forthe bottom of this article is a link to a very good paper on the subject titled "recursive make considered harmful"; not only is a single Makefile easier to maintain (once you learned a couple of tricks), it's also more efficient!''
 
=== Recursion ===
<pre>
SRCFILES := $(shell find $(PROJDIRS) -mindepthtype 1 -maxdepth 3f -name "\*.c")
HDRFILES := $(shell find $(PROJDIRS) -mindepthtype 1 -maxdepth 3f -name "\*.h")
</pre>
While itIt should be obvious to see what these two lines do, it took some experimentation to get it right given the typical GNU-style documentation of ''make'' (which is correct, but not very helpful). IWe now have a list of all source and header files in myour project directories.
 
''The -mindepth option keeps any top-level files out of the result set, like tmp.c or test.c or whatever you might have created for testing something on the fly. The -maxdepth option stops the 'find' from recursing into the CVS/ directories I used to have. You might want to adapt this to your requirements.''
 
=== Automated Testing, pt. 1 ===
I think it's time I explain my approach to testing. I write strictly one library function per source file. Then, I add a test driver to that source file:
<pre>
#ifdef TEST
int main()
/* Test function code here */
return NUMBER_OF_TEST_ERRORS;
#endif
</pre>
Thus, when I compile that source with ''gcc -c'', I get an object file with the library function code; when I compile with ''gcc -DTEST'', I get a test driver executable for that function. Returning the number of errors allows me to do a grand total of errors encountered (see below).
 
=== Object Files and Test Driver Executables ===
<pre>
OBJFILES := $(patsubst %.c,%.o,$(SRCFILES))
TSTFILES := $(patsubst %.c,%.t_t,$(SRCFILES))
</pre>
''OBJFILES'' should be clear - a list of the source files, with ''*.c'' exchanged for ''*.o''. ''TSTFILES'' does the same for the extensionfilename suffix ''*.t_t'', which Iwe will choseuse for myour test driver executables.
 
:''Note: This tutorial initially used ''*.t'' instead of ''*_t'' here; this, however, kept us from handling dependencies for test driver executables seperately from those for library object files, which was necessary. See next section.''
 
=== Dependencies, pt. 1 ===
Many people edit their Makefile every time they add/change an ''#include'' somewhere in their code. Did you know thatBut ''GCC'' can do this automatically for you? Yes, it can. Trust me.! Although the approach looks a little backward. For each source file, ''GCC'' will create a ''dependency file'' (which Iis canonically made to end in ''*.d''), which contains a Makefile dependency rule listing that source file's includes. (And more, but see below.)
 
We need two seperate sets of dependency files; one for the library objects, and one for the test driver executables (which usually have additional includes, and thus dependencies, not needed for the OBJFILES).
<pre>
DEPFILES := $(patsubst %.c,%.d,$(SRCFILES))
TSTDEPFILES := $(patsubst %,%.d,$(TSTFILES))
</pre>
 
Line 104 ⟶ 112:
</pre>
 
== Phony.PHONY ==
The next one can take you by surprise. When you write a rule for ''make clean'', and there happens to be a file named ''clean'' in your working directory, you might be surprised to find that ''make'' does nothing, because the "target" of the rule ''clean'' already exists. To avoid this, declare such rules as ''phony'', i.e. disable the checking for a file of that name. These will be executed every time:
<pre>
.PHONY: all clean dist testcheck testdrivers todolist
</pre>
 
Line 113 ⟶ 121:
If you thought ''-Wall'' does tell you everything, you'll be in for a rude surprise now. If you don't even use ''-Wall'', shame on you. ;)
<pre>
CFLAGSWARNINGS := -Wall -Wextra -pedantic -Wshadow -Wpointer-arith -Wcast-align \
-Wwrite-strings -Wmissing-prototypes -Wmissing-declarations \
-Wredundant-decls -Wnested-externs -Winline -Wno-long-long \
-Wuninitialized -Wconversion -Wstrict-prototypes
CFLAGS := -g -std=c99 $(WARNINGS)
</pre>
IIt suggestis yousuggested to add themnew onewarning byoptions to your project one at a time instead of all at once, to avoid getting swamped in warnings. ;) These flags are merely recommendations for C work. If you use C++, thereyou areneed moredifferent toones. beCheck usedout forthe C++GCC manual; each major compiler update changes the list of available warning options.
 
== Rules ==
 
Now come the rules, in their typical backward manner (top-level rule first). The topmost rule is the default one (if 'make' is called without an explicit target). It is common practice to have the first rule called "all".
 
=== Top-Level Targets ===
Line 132 ⟶ 141:
 
clean:
-@$(RM) $(wildcard $(OBJFILES) $(DEPFILES) $(TSTFILES) $(REGFILES) pdclib.a pdclib.tgz)
 
dist:
@tar czf pdclib.tgz $(ALLFILES)
</pre>
The ''@'' at the beginning of thea line tells ''make'' to be quiet, i.e. not to echo the executed commands on the console prior to executing them. The Unix credo is "no news is good news". You don't get a list of processed files with ''cp'' or ''tar'', either, so it's completely beyond me why developers chose to have their Makefiles churn out endless lists of useless garbage. One very practical advantage of shutting up ''make'' is that you actually get to ''see'' those compiler warnings, instead of having them drowned out by noise.
 
The ''-'' at the beginning of a line tells ''make'' to continue even in case an error is encountered (default behaviour is to terminate the whole build).
The awkward looking loop in the ''clean'' rule is to avoid confusing error messages if the files to be deleted don't exist.
 
The ''$(RM)'' in the ''clean'' rule is the platform-independent command to remove a file.
 
The ''$?'' in the ''pdclib.a'' rule is an internal variable, which ''make'' expands to list all dependencies to the rule ''which are newer than the target''.
 
=== Automated Testing, pt. 2 ===
<pre>
testcheck: testdrivers
-@rc=0; count=0; for file in $(TSTFILES); do ./$$file; rc=`expr $$rc + $$?`; \
for file in $(TSTFILES); do \
count=`expr $$count + 1`; done; echo; echo "Tests executed: $$count Tests failed: $$rc"
echo " TST $$file"; ./$$file; \
rc=`expr $$rc + $$?`; count=`expr $$count + 1`; \
done; \
count=`expr $$count + 1`; done; echo; echo "Tests executed: $$count Tests failed: $$rc"
 
testdrivers: $(TSTFILES)
Line 151 ⟶ 168:
Call it crude, but it works beautifully for me. The leading ''-'' means that 'make' will not abort when encountering an error, but continue with the loop.
 
IfThe ''echo " TST $$file"'' in there is useful in case you get a ''SEGFAULT'' or something like that, addfrom aone temporaryof ''echoyour $$file''test todrivers. the(Without loop to getechoing the namedrivers ofas thethey testare driverexecuted, thisyou happenswould in.be Forat ''normal''a testloss failures, add a diagnosticas to thewhich testone drivercrashed.)
 
=== Dependencies, pt. 2 ===
<pre>
-include $(DEPFILES) $(TSTDEPFILES)
</pre>
Further below, you will see how dependency files are ''generated''. Here, we ''include'' all of them, i.e. make the dependencies listed in them part of our Makefile. Never mind that they might not even exist when we run our Makefile the first time - the leading "-" again suppresses the errors.
This adds all the dependency rules auto-generated by ''GCC''. (see below)
 
=== Extracting TODO Statements ===
Line 164 ⟶ 181:
-@for file in $(ALLFILES); do fgrep -H -e TODO -e FIXME $$file; done; true
</pre>
This will ''grep'' all those ''TODO'' and ''FIXME'' comments from theyour files, and display them in the terminal. It's is nice to be remembered of what's is still missing before you do a release. To add another keyword, just addinsert another <code>-e keyword</code>. Don't forget to add <code>todolist</code> to your <code>.PHONY</code> list.
 
=== Dependencies, pt. 3 ===
 
Now comes the dependency magic I talked about earlier:. Note that this needs GCC 3.3 or newer.
<pre>
%.o: %.c Makefile
@$(CC) $(CFLAGS) -DNDEBUG -MMD -MP -MT "$*.d $*.t $*.o" -g -std=c99 -I./includes -I./internals -c $< -o $@
</pre>
Isn't it a beauty? ;-) Note that this needs GCC 3.3.x or newer.
 
The bunch of "M"-flags create a ''*.d-MMD'' fileflag alongsidegenerates the objectdependency file (%.d), which holdswill hold (in Makefile syntax) rules making all the ''generated'' filesfile (*%.o, *.t,in *.dthis case) depend on the source file ''and any non-system headers it includes''. That means the object file, the test driver, and the dependency file itself getgets recreated automatically whenever relevant sources are touched. (TheIf dependencyyou filewant requiresto recreatingalso toodepend ason thesystem sourceheaders edit(i.e., couldchecking havethem addedfor anotherupdates on each compile), use ''#include-MD'' instead. The ''-MP'' option adds empty dummy rules, which avoid errors should header files be removed from the filesystem.)
 
Compiling the object file actually looks like a side effect. ;-)
 
Note that the dependency list of the rule includes the Makefile itself. If you changed e.g. the ''CFLAGS'' in the Makefile, you want them to be applied, don't you? Using the $< macro ("first dependency") in the command makes sure we do not attempt to compile the Makefile as C source.
 
Of course we also need a rule for generating the test driver executables (and their dependency files):
 
=== The Rest ===
<pre>
%.t_t: %.c Makefile pdclib.a
@$(CC) $(CFLAGS) -DTESTMMD -std=c99MP -I./includes -I./internalsDTEST $< pdclib.a -o $@
</pre>
 
This is somewhat of an anticlimax in its "triviality". My test drivers need to link against the ''PDCLib'' itself, but that's already all. This is not really perfect; if ''any'' PDCLib function was touched (and, thus, pdclib.a updated), it recreates ''all'' test drivers, even when not necessary. Ah well, no Makefile ever is perfect, but I'd rather have too much compiled than missing a dependency.
Here you can see why test driver executables get a ''*_t'' suffix instead of a ''*.t'' extension: The ''-MMD'' option uses the basename (i.e., filename without extension) of the ''compiled'' file as basis for the dependency file. If we would compile the sources into ''abc.o'' and ''abc.t'', the dependency files would both be named ''abc.d'', overwriting each other.
 
== Summary ==
 
A well-written Makefile can make maintaining a code base much easier, as it can wrap complex command sequences into simple 'make' invocations. Especially for large projects, it also cuts back on compilation times when compared to a dumb "build.sh" script. And, once written, modifications to a well-written Makefile are seldom necessary.
 
= Advanced Techniques =
 
Some of the stuff we did above already ''was'' pretty advanced, and no mistake. But we needed those features for the basic yet convenient setup. Below you will find some even trickier stuff, which might not be for everybody but is immensely helpful ''if'' you need it.
 
== Conditional Evaluation ==
 
Sometimes it becomesis useful to react on the existence or content of certain environment variables. For example, you might have to rely on the path to a certain framework being passed in FRAMEWORK_PATH. Perhaps the error message given by the compiler in case the variable is not set isn'tis thatnot helpful, or it takes long until 'make' gets to the point where it actually detects the error. You want to fail early, and with a meaningful error message.
 
Luckily, 'make' allows for conditional evaluation and manual error reporting, quite akin to the C preprocessor:
Line 205 ⟶ 230:
endif
</pre>
 
= A Final Word =
 
Hope this helps you in creating your own Makefile, and then forgetting about it (as you should, because 'make' should lessen your workload, not add more).
 
--[[User:Solar|Solar]] 04:05, 21 January 2008 (CST)
 
 
= See Also =
448

edits