I immediately turned to the WWW and looked up all sorts of tools - but quite honestly I was rather disappointed. Mostly, they were what I would call _EM(Typing Aids) - instead of having to remember arcane incantations like _CODE(<a href="link">text</a>), you are given a button or a magic keychord like ALT-CTRL-j which remembers the syntax and does all that nasty typing for you.

_STRONG(Linux) to the rescue! HTML is built as ordinary text files and therefore the normal _STRONG(Linux) text management tools can be used. This includes the revision control tools such as _STRONG(RCS) and the text manipulation tools like _STRONG(`awk, perl, etc.') These offer significant help in version control and managing development by multiple users as well as in automating the process of extracting from a database and displaying the results (the classic _CODE(`"grep |sort |awk"') pipeline).

The use of these tools with HTML is documented elsewhere, e.g. see Jim Weinrich's article in _STRONG(Linux Journal) Issue 36, April 1997, "Using Perl to Check Web Links" which I'd highly recommend as yet another way to really flex those _STRONG(Linux) muscles when writing HTML.

What I will cover here is a little work I've done recently with using _STRONG(m4) in maintaining HTML. The ideas can probably be extended to the more general SGML case very easily.

I decided to use _STRONG(m4) after looking at various other pre-processors including _STRONG(cpp), the _STRONG(C) front-end. While _STRONG(cpp) is perhaps a little too _STRONG(C)-specific to be very useful with HTML, _STRONG(m4) is a very generic and clean macro expansion program - and it's available under most Unices including _STRONG(Linux).

Instead of editing _EM(*.html) files, I create _EM(*.m4) files with my favourite text editor. These look something like this: _CODEQUOTE(``m4_include(stdlib.m4) _HEADER(`This is my header') <P>This is some plain text<P> _HEAD1(`This is a main heading') <P>This is some more plain text<P> _TRAILER'')

The `format' is simple - just HTML code but you can now `include' files and add macros rather like in _STRONG(C). I use a convention that my new macros are in capitals and start with "_" to make them stand out from HTML language and to avoid name-space collisions.

The _STRONG(m4) file is then processed as follows to create an _STRONG(.html) file e.g. _CODEQUOTE(m4 -P <file.m4 >file.html) This is especially easy if you create a "makefile" to automate this in the usual way. Something like: _CODEQUOTE(.SUFFIXES: .m4 .html .m4.html: m4 -P $*.m4 >$*.html default: index.html *.html: stdlib.m4 all: default PROJECT1 PROJECT2 PROJECT1: (cd project2; make all) PROJECT2: (cd project2; make all)) The most useful commands in _STRONG(m4) include the following which are very similar to the _STRONG(cpp) equivalents (shown in brackets):

_CODE(``m4_include''):

includes a common file into your HTML (_CODE(`#include'))

_CODE(``m4_define''):

defines an _STRONG(m4) variable (_CODE(`#define'))

_CODE(``m4_ifdef, m4_ifelse''):

conditionals (_CODE(`#ifdef, #if'))

_CODE(``m4_changecom''):

change the _STRONG(m4) comment character (normally #)

_CODE(``m4_debugmode''):

control error disgnostics

_CODE(``m4_traceon/off''):

turn tracing on and off

_CODE(``m4_dnl''):

comment

_CODE(``m4_incr, m4_decr''):

simple arithmetic

_CODE(``m4_eval''):

more general arithmetic

_CODE(``m4_esyscmd''):

execute a _STRONG(Linux) command and use the output

_CODE(``m4_divert(i)''):

This is a little complicated, so skip on first reading. It is a way of storing text for output at the end of normal processing - it will come in useful later, when we get to automatic numbering of headings. It sends output from _STRONG(m4) to a temporary (internal) file number _STRONG(i). At the end of processing, any text which was diverted is then output, in the order of the file number _STRONG(i). File number -1 is the bit bucket and can be used to comment out chunks. File number 0 is the normal output stream. Thus, for example, you can _CODE(``m4_divert'') text to file 1 and it will only be output at the end.

_LINK_TO_LABEL(Contents) _H2(`Creating new text styles') Styles built into HTML include things like _CODE(<EM>) for emphasis and _CODE(<CITE>) for citations. With _STRONG(m4) you can define your own, new styles like this: _CODEQUOTE(``m4_define(`_MYQUOTE', <BLOCKQUOTE><EM>$1</EM></BLOCKQUOTE>)'') If, later, you decide you prefer _CODE(<STRONG>) instead of _CODE(<EM>) it is a simple matter to change the definition and then every _CODE(_MYQUOTE) paragraph falls into line with a quick _CODE(make).

The classic guides to good HTML writing say things like "It is strongly recommended that you employ the logical styles such as _CODE(<EM>...</EM>) rather than the physical styles such as _CODE(<I>...</I>) in your documents." Curiously, the _EM(WYSIWYG) editors for HTML generate purely physical styles. Using these _STRONG(m4) styles may be a good way to keep on using logical styles.

_LINK_TO_LABEL(Contents) _H2(`Sharing HTML elements across several page') In many "nests" of HTML pages, each page shares elements such as a button bar like this:

_LINK(nil,[Home]) _LINK(nil,[Next]) _LINK(nil,[Prev]) _LINK(nil,[Index])

With _STRONG(m4) we can more easily do this by putting the shared elements into an _CODE(m4_include) statement, just like _STRONG(C).

While I'm at it, I might as well also automate the naming of pages, perhaps by putting the following into an `include' file, say _CODE("button_bar.m4"): _CODEQUOTE(``m4_define(`_BUTTON_BAR', <a href="homepage.html">[Home]</a> <a href="$1">[Next]</a> <a href="$2">[Prev]</a> <a href="indexpage.html">[Index]</a>)'') and then in the document itself: _CODEQUOTE(``m4_include button_bar.m4 _BUTTON_BAR(`page_after_this.html', `page_before_this.html')'') The $1 and $2 parameters in the macro definition are replaced by the strings in the macro call.

_LINK_TO_LABEL(Contents) _H2(`Links that change for every page') Links are often the same on every page except for one small change - you want the link to the current page to be disabled or invisible. Here's an easy way to do it with _STRONG(m4).

Simply define this in each page: _CODEQUOTE(``m4_define(`_PAGE_1',1) m4_include(links.m4)'')

Then, in the file _CODE(links.m4):

_CODEQUOTE(``[ m4_ifdef(`_PAGE_1',`Page 1',_LINK(page1.html,`Page 1')) ] [ m4_ifdef(`_PAGE_2',`Page 2',_LINK(page2.html,`Page 2')) ] [ m4_ifdef(`_PAGE_3',`Page 3',_LINK(page3.html,`Page 3')) ] [ m4_ifdef(`_PAGE_4',`Page 4',_LINK(page4.html,`Page 4')) ]'')

In "Page 1" you would get:

_CODE([ Page 1 ] _LINK(nil,[ Page 2 ]) _LINK(nil,[ Page 3 ]) _LINK(nil,[ Page 4 ]))

which is, hopefully, what you'd expect. It's nice that the complexity is pushed into the included file _CODE(links.m4) leaving the calling pages relatively uncluttered.

_LINK_TO_LABEL(Contents) _H2(`Managing HTML elements that often change') It is very troublesome to have items change in multiple HTML pages. For example, if your email address changes then you will need to change all references to the new address. Instead, with _STRONG(m4) you can do something like this in your _CODE(stdlib.m4) file: _CODEQUOTE(``m4_define(`_EMAIL_ADDRESS', `MyName@foo.bar.com')'') and then just put _CODE(``_EMAIL_ADDRESS'') in your _STRONG(m4) files.

A more substantial example comes from building strings up with multiple components, any of which may change as the page is developed. If, like me, you develop on one machine, test out the page and then upload to another machine with a totally different address then you could use the _CODE(m4_ifdef) command in your _CODE(stdlib.m4) file (just like the _CODE(#ifdef) command in _STRONG(cpp)): _CODEQUOTE(``m4_define(`_LOCAL') . . m4_define(`_HOMEPAGE', m4_ifdef(`_LOCAL', `//127.0.0.1/~YourAccount', `http://ISP.com/~YourAccount')) m4_define(`_PLUG', `<A REF="http://www.ssc.com/linux/"> <IMG SRC="_HOMEPAGE/gif/powered.gif" ALT="[Linux Information]"> </A>')'') Note the careful use of quotes to prevent the variable _CODE(``_LOCAL'') from being expanded. _CODE(``_HOMEPAGE'') takes on different values according to whether the variable _CODE(``_LOCAL'') is defined or not. This can then ripple through the entire project as you _STRONG(make) the pages.

In this example, _CODE(``_PLUG'') is a macro to advertise _STRONG(Linux). When you are testing your pages, you use the local version of _CODE(``_HOMEPAGE''). When you are ready to upload, you can remove or comment out the _CODE(``_LOCAL'') definition like this: _CODEQUOTE(``m4_dnl m4_define(`_LOCAL')'') ... and then _EM(re-make).

_LINK_TO_LABEL(Contents) _H2(`Automatic date stamping') For simple, automatic datestamping of HTML pages I use the _CODE(`m4_esyscmd') command to maintain an automatic timestamp on every page: _CODEQUOTE(``This page was updated on m4_esyscmd(date)'') which produces:

This page was last updated on Fri May 9 10:35:03 HKT 1997

Of course, you could also use the date, revision and other facilities of revision control systems like _STRONG(RCS) or _STRONG(SCCS), e.g. _CODE(`$Da'`te$').

You can also use any system command, not just _STRONG(date). For example you can count the number of elements in a flat file database like this: _CODEQUOTE(``There are m4_esyscmd(wc -l filename.txt) items in the database today'') This might be dynamic enough for your purposes and avoids the overhead of a _STRONG(perl) invocation every time the otherwise static page is downloaded.

_LINK_TO_LABEL(Contents) _H2(`Automatic numbering') _STRONG(m4) has a simple arithmetic facility with two operators _CODE(m4_incr) and _CODE(m4_decr) which act as you might expect - this can be used to create automatic numbering, perhaps for headings, e.g.: _CODEQUOTE(``m4_define(_CARDINAL,0) m4_define(_H, `m4_define(`_CARDINAL', m4_incr(_CARDINAL))<H2>_CARDINAL.0 $1</H2>') _H(First Heading) _H(Second Heading)'')

This produces: _CODEQUOTE(``<H2>1.0 First Heading</H2> <H2>2.0 Second Heading</H2>'')

_LINK_TO_LABEL(Contents) _H2(`Generating Tables of Contents') Automatic numbering naturally leads us to this topic. Even if you don't want to understand the innards of how _STRONG(m4) does this, you can use the feature in my _CODE(stdlib.m4) for an accurate, fast and simple to use facility.

_STRONG(m4) allows you to `define' commonly repeated phrases and use them consistently - this appeals to me enormously as I hate repetition both because I am lazy and because I make mistakes. In HTML, a TOC involves repeating the heading title in the table of contents and then in the text itself. This is tedious and error-prone especially when you change the titles. There are specialised tools for generating tables of contents from HTML pages but the simple facility provided by _STRONG(m4) is irresistable to me. _H3(`Simple to understand TOC') The following example is a fairly simple-minded Table of Contents generator - the next section discusses a simple-to-use version that I use. It uses nicknames (or glossaries) for the headings instead of the headings themselves.

First, create some useful macros in _CODE(stdlib.m4): _CODEQUOTE(``m4_define(`_LINK_TO_LABEL', <A HREF="#$1">$1</A>) m4_define(`_SECTION_HEADER', <H2><A NAME="$1">$1</A></H2>)'') Then `define' all the section headings at the start of the page: _CODEQUOTE(``m4_define(`_DIFFICULTIES', `The difficulties of HTML') m4_define(`_USING_M4', `Using <EM>m4</EM>') m4_define(`_SHARING', `Sharing HTML Elements Across Several Pages')'') Then build the table: _CODEQUOTE(``<UL><P> <LI> _LINK_TO_LABEL(_DIFFICULTIES) <LI> _LINK_TO_LABEL(_USING_M4) <LI> _LINK_TO_LABEL(_SHARING) <UL>'') Finally, write the text: _CODEQUOTE(``. . _SECTION_HEADER(_DIFFICULTIES) . .'') The advantages of this approach are that if you change your headings you only need to change them in one place and the table of contents is automatically regenerated; also the links are guaranteed to work.

Hopefully, that simple version was fairly easy to understand.

_LINK_TO_LABEL(Contents) _H3(`Simple to use TOC') The Table of Contents generator that I normally use is a bit more complex and will require a little more study, but is much easier to use. It not only builds the Table, but it also automatically numbers the headings on the fly - up to a maximum of 4 levels of numbering e.g. section 3.2.1.3 - although this can be easily extended. It is, however, very simple to use as follows:

1. Where you want the table to appear, put _CODE(``Start_TOC'')
2. at every heading use _CODE(``_H1(`Heading for level 1')'') or _CODE(``_H2(`Heading for level 2')'') as appropriate.
3. After the very last HTML code (probably after </HTML>), put _CODE(``End_TOC'')
4. and that's all!

This works by using the _CODE(``m4_divert(1)'') command in _CODE(``_Start_TOC'') to send all the remaining text from the file to an (internal) temporary file - _STRONG(m4) just calls it "file 1".

From then on, whenever an _CODE(``_H1'') or _CODE(``_H2'') etc command is reached, the relevant header numbering variables are incremented (with _CODE(``m4_incr'')) and the Table of Contents entry is sent to the standard output (file 0) together with automatically generated pointers into the main text.

The diversion to file 1 is then resumed for the regular text and an automatically generated section heading with pointer target is added.

The _CODE(``_End_TOC'') statement _EM(must) be placed at the end of the file. When it is reached the text which was diverted to file 1 is read back to standard output.

The net result is that the Table of Contents appears near the start of the final file, with automatically generated pointers to the correct section in the later text.

Of course, if you plan on using the _CODE(``m4_divert'') command in your own text, you will have to check that it does not clash with the Table of Contents generator.

As a final note on this feature, the table of contents generator in the downloadable version of _CODE(stdlib.h) also indents the sections. For the sake of a simpler explanation this extra feature is not explained here (or in the words of my Analysis Lecturer, Prof. Noble, "it is left as an excercise for the student").

_LINK_TO_LABEL(Contents) _H2(`Simple tables') Other than Tables of Contents, many browsers support tabular information. Here are some funky macros as a short cut to producing these tables. First, an example of their use: _CODEQUOTE(``<CENTER> _Start_Table(BORDER=5) _Table_Hdr( ,Apples, Oranges, Lemons) _Table_Row(England,100,250,300) _Table_Row(France,200,500,100) _Table_Row(Germany,500,50,90) _Table_Row(Spain, ,23,2444) _Table_Row(Denmark, , ,20) _End_Table </CENTER>'')

...and now the code. Note that this example utilises _STRONG(m4's) ability to recurse: _CODEQUOTE(``m4_dnl _Start_Table(Columns,TABLE parameters) m4_dnl defaults are BORDER=1 CELLPADDING="1" CELLSPACING="1" m4_dnl WIDTH="n" pixels or "n%" of screen width m4_define(_Start_Table,`<TABLE $1>') m4_define(`_Table_Hdr_Item', `<th>$1</th> m4_ifelse($#,1,,`_Table_Hdr_Item(m4_shift($@))')') m4_define(`_Table_Row_Item', `<td>$1</td> m4_ifelse($#,1,,`_Table_Row_Item(m4_shift($@))')') m4_define(`_Table_Hdr',`<tr>_Table_Hdr_Item($@)</tr>') m4_define(`_Table_Row',`<tr>_Table_Row_Item($@)</tr>') m4_define(`_End_Table',</TABLE>)'')

_LINK_TO_LABEL(Contents) _H1(`Using m4 with Perl') This works too - you can have common header files for your HTML files and _STRONG(perl) CGI scripts. These might define common constants and look and feel.

e.g. if `_HEADER' and `_DocumentRoot' are defined as:

_CODEQUOTE(`` m4_define(`_HEADER',<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 3.2//EN"> <HTML> <HEAD> <TITLE>$1</TITLE> <META NAME="Author" CONTENT="Bob Hepple"> <META NAME="Description" CONTENT="$3"> <META NAME="Keywords" CONTENT="$4"> </HEAD>) m4_define(`_DocumentRoot',/cars) '')

then we can define the following in a _STRONG(perl) script

_CODEQUOTE(``m4_include(stdlib.m4) print "_HEADER(title,params,description,keywords)"; print "_LINK(_DocumentRoot/a_file.html,a link)\n";'')

and in a plain HTML page: _CODEQUOTE(``m4_include(stdlib.m4) _HEADER(title,params,description,keywords)'')

We then get the advantages of consistency between HTML and _STRONG(perl) scripts, easy global changes and most of the complexity moved off to the include-file.

_STRONG(perl)'s 'here document' construct is also amenable to macro expansion with _STRONG(m4). e.g. _CODEQUOTE(`` print <<Eof; _HEADER(title,params,description,keywords) Eof '')

I use this process in my own pages e.g. at _LINK(http://www.finder.com.au, The Finder) which is a commercial site for classic cars in Australia. All scripts and HTML pages are generated from _STRONG(m4) source and result in a consistent look and feel inherited from a single include-file.

_LINK_TO_LABEL(Contents) _H1(`m4 gotchas') Unfortunately, _STRONG(m4) is not unremitting sweetness and light - it needs some taming and a little time spent on familiarisation will pay dividends. Definitive documentation is available (for example in _STRONG(emacs' info) documentation system) but, without being a complete tutorial, here are a few tips based on my fiddling about with the thing. m4_define(_GOTCHA,0) m4_define(`_GOTCHA', m4_incr(_GOTCHA)) _H2(`Gotcha _GOTCHA - quotes') m4_changequote([,])_STRONG(m4's) quotation characters are the _STRONG(grave) accent ` which starts the quote, and the _STRONG(acute) accent ´ m4_changequote(`,')which ends it. It generally helps to put all arguments to macros in quotes, e.g. _CODEQUOTE(``_HEAD1(`This is a heading')'') The main reason for this is in case there are commas in an argument to a macro - _STRONG(m4) uses commas to separate macro parameters, e.g. _CODE(``_CODE(foo, bar)'') would print the _CODE(foo) but not the _CODE(bar). _CODE(``_CODE(`foo, bar')'') works properly.

This becomes a little complicated when you nest macro calls as in the _STRONG(m4) source code for the examples in this paper - but that is rather an extreme case and normally you would not have to stoop to that level.

_LINK_TO_LABEL(Contents) m4_define(`_GOTCHA', m4_incr(_GOTCHA)) _H2(`Gotcha _GOTCHA - Word swallowing') The worst problem with _STRONG(m4) is that some versions of it "swallow" key words that it recognises, such as "include", "format", "divert", "file", "gnu", "line", "regexp", "shift", "unix", "builtin" and "define". You can protect these words by putting them in _STRONG(m4) quotes, for example: _CODEQUOTE(``Smart people `include' Linux in their list of computer essentials.'') The trouble is, this is a royal pain to do - and you're likely to forget which words need protecting.

Another, safer way to protect keywords (my preference) is to invoke _STRONG(m4) with the _CODE(-P) or _CODE(--prefix-builtins) option. Then, all builtin macro names are modified so they all start with the prefix _CODE(m4_) and ordinary words are left alone. For example, using this option, one should write _CODE(m4_define) instead of _CODE(``define'') (as shown in the examples in this article).

The only trouble is that not all versions of _STRONG(m4) support this option - notably some PC versions under M$-DOS. Maybe that's just another reason to steer clear of hack code on M$-DOS and stay with _STRONG(Linux!)

_LINK_TO_LABEL(Contents) m4_define(`_GOTCHA', m4_incr(_GOTCHA)) _H2(`Gotcha _GOTCHA - Comments') Comments in _STRONG(m4) are introduced with the `#' character - everything from the `#' to the end of the line is ignored by _STRONG(m4) and simply passed unchanged to the output. If you want to use `#' in the HTML page then you would need to quote it like this - ``#''. Another option (my preference) is to change the _STRONG(m4) comment character to something exotic like this: _CODE(``m4_changecom(`[[[[')'') and not have to worry about ``#'' symbols in your text.

If you want to use comments in the _STRONG(m4) file which do not appear in the final HTML file, then the macro _CODE(``m4_dnl'') (dnl = _STRONG(D)elete to _STRONG(N)ew _STRONG(L)ine) is for you. This suppresses everything until the next newline. _CODEQUOTE(``m4_define(_NEWMACRO, `foo bar') m4_dnl This is a comment'') Yet another way to have source code ignored is the _CODE(``m4_divert'') command. The main purpose of _CODE(``m4_divert'') is to save text in a temporary buffer for inclusion in the file later on - for example, in building a table of contents or index. However, if you divert to "-1" it just goes to limbo-land. This is useful for getting rid of the whitespace generated by the _CODE(``m4_define'') command, e.g.: _CODEQUOTE(``m4_divert(-1) diversion on m4_define(this ...) m4_define(that ...) m4_divert diversion turned off'')

_LINK_TO_LABEL(Contents) m4_define(`_GOTCHA', m4_incr(_GOTCHA)) _H2(`Gotcha _GOTCHA - Debugging') Another tip for when things go wrong is to increase the amount of error diagnostics that _STRONG(m4) emits. The easiest way to do this is to add the following to your _STRONG(m4) file as debugging commands: _CODEQUOTE(``m4_debugmode(e) m4_traceon . . buggy lines . . m4_traceoff'')

_LINK_TO_LABEL(Contents) m4_define(`_GOTCHA', m4_incr(_GOTCHA)) _H2(`Gotcha _GOTCHA - perl in m4') There is a degree of syntax collision between perl, HTML and m4 and you need to be very careful in certain areas, particularly in the use of quotes. Perl's double quote mechanism (e.g. "this is $value\n") allows dynamic expansion of _STRONG(perl) variables. It may help to use the more obscure equivalent form of ", namely the qq{ .... } construct which is identical (e.q. _CODE(`qq{this is $value\n}')).

Of course _STRONG(perl)'s back quoting mechanism is also vulnerable and needs protection from _STRONG(m4). You can use _CODE(`qx{cmd}') instead of the more usual m4_changequote([,]) _CODE(`cmd`) to run any system command _STRONG(cmd). m4_changequote(`,')

_LINK_TO_LABEL(Contents) _H1(`Conclusion') "ah ha!", I hear you say. "HTML 3.0 already has an include statement". Yes it has, and it looks like this: _CODEQUOTE(``<!--#include file="junk.html" -->'') The problem is that:

• The work of processing the include is done on the server-side before downloading and adds overhead as the server has to scan files for ``include'' statements.
• Consequently most servers (especially public ISP's) have this feature deactivated.
• ``include'' is all you get - no macro substitution, no parameters to macros, no ifdef, etc, etc.

There are several other features of _STRONG(m4) that I have not yet exploited in my HTML ramblings so far, such as regular expressions and doubtless many others. It might be interesting to create a "standard" _CODE(stdlib.m4) for general use with nice macros for general text processing and HTML functions. By all means download my version of _CODE(stdlib.m4) as a base for your own hacking. I would be interested in hearing of useful macros and if there is enough interest, maybe a Mini-HOWTO could evolve from this paper.

Beyond _STRONG(m4), there are many additional advantages in using _STRONG(Linux) to develop HTML pages, far beyond the trivial help given by the typical _EM(WYSIWYG) tools.

I hope you enjoy these little tricks and encourage you to contribute your own. Certainly, this little hacker will go on using _STRONG(m4) until HTML catches up.

Happy hacking! _H1(`Files to download') You can get the HTML and the _STRONG(m4) source code for this article here (for the sake of completeness, they're copylefted under GPL 2): _PRE(_FTP(using_m4.html, using_m4.html) :this file _FTP(using_m4.m4, using_m4.m4) :_STRONG(m4) source _FTP(stdlib.m4, stdlib.m4) :Include file _FTP(makefile, makefile))

_LINK_TO_LABEL(Contents) _H1(`Bob Hepple') _EMAILME(Bob Hepple) has been hacking at Unix since 1981 under a variety of excuses and has somehow been paid for it at least some of the time. It's allowed him to pursue another interest - living in warm, exotic countries including Hong Kong, Australia, Qatar, Saudi Arabia, Lesotho and Singapore. His initial aversion to the cold was learned in the UK.

_LINK_TO_LABEL(Contents)

m4_dnl 800 on 25th Oct 01 _COUNTER(5, `since March 18th 1998')