Mo’ Betta Indexes

While websites often incorporate a standard look & feel across all of their pages, any generated index pages included in the site typically stick out like a sore thumb. So let’s if we can’t incorporate the look & feel of a sample site (barren though it may be) into a page that Apache will automatically generate.

Note: The term “Apache” typically refers to both the Apache Software Foundation and the Apache httpd (web server) project. To avoid any confusion, I will refer to the Apache Software Foundation simply as the “ASF”, and to the ASF's httpd project as Apache.

What Isn’t Covered

This article will not go into detail regarding the structure of Apache’s configuration file; both the project’s online manual and O’Reilly’s “Apache: The Definitive Guide, 3rd Edition” (Ben Laurie, Peter Laurie) provide a comprehensive reference. In addition, the inline documentation included in the default configuration file provides an excellent starting point.

Instead, I’ll focus specifically on the directives that we’ll be using to accomplish our task: to incorporate Apache’s auto–index pages into an existing website’s look and feel. If the term “directive” is foreign to you, then you may want to peruse these documents before continuing, since a detailed tutorial on how to get Apache up and running is beyond the scope of this article. However, links will be provided to the related section of Apache’s online manual (for both versions 1.3.x and 2.0.x) when appropriate.

What Is Covered

Because of the complexity of running a web server that can host hundreds of domains and the httpd daemon itself, most of Apache’s configuration directives relate to technical aspects such as security & authentication, hosting, administration, module management, and other aspects that aren’t readily apparent to a sites visitors. However, there are some simple directives that are accessible to anyone familiar with Apache that can make a dramatic difference in the appearance of your site’s auto–indexes.

Most commerical websites and desktop applications have thousands upon thousands of dollars poured into their design and interface. While Apache’s client–side and open–source cousin Firefox has recently undergone a facelift, Apache has had the same skin since around 1995 when it was still under the wing of its founder, the NCSA. And while it‘s true that visitors interact with their web browser and the pages of your site much more than the HTTP daemon that’s pulling the strings on your web server, the only real snapshot that they will receive is through your site’s auto–indexes (unless an error page rears its ugly head). So why not take some time to put some spit & polish into them?

Split Personality

Apache has been the net‘s most popular web server since early 1996, and has been available in two versions (1.3.x and 2.0.x) since 2001. As has been previously documented, migrating from the previous version to the latest isn’t quite as simple as stopping the old daemon and starting up the new. Some of the changes are esoteric, but most are due to changes in the architecture between versions; modules had to be re–written, configuration files will have to be modified, and settings will have to be tweaked.

Luckily, most of these changes do not apply to the work we’ll be doing today. In this article, we’ll make use of two modules: mod_autoindex and mod_dir (and a smidgen of mod_alias). The differences between these modules across Apache's two versions are mostly limited to new features that have been added in the latest releases, and those features will be marked as such.

Precautionary Steps

So we’re almost ready to go. But before we get started, there are two very important steps we must take:

  1. The single most important thing to do is to create a backup copy of the configuration flie in case we mangle it beyond repair and can’t get the daemon to start up again. To do so, type the following into your Terminal window: sudo cp /private/etc/httpd/httpd.conf /private/etc/httpd/httpd.conf.noindex (The command line is wrapped due to space constraints; be sure to hit “Return” only after entering/copying it in its entirety).

  2. The second most important step (which is relatively minor compared to #1) is to take a snapshot of Apache’s default index page to serve as the reference point in our "before" and "after" comparison. The page on your server may appear a little different from the following, but it should be pretty much identical to the Figure #1.

An example of Apache’s default auto–index page

Figure 1?: A typical index page, as created by Apache’s default configuration file on OS X.

Now let’s turn that eyesore into something a little more pleasing.

General Directives

Before we can start fiddling with Apache’s generated index pages, we have to be able to view a page that Apache pieces together when we make a request for a folder. If the folder contains a file that is speicified in a DirectoryIndex directive, then that file will be served to the client; otherwise Apache will create an auto–index page (as shown in Figure #1).

DirectoryIndex file1 [file2] ...

This directive specifies a list of files that should be sent to the client when a request is made for a directory’s URL. If multiple files are specified, then the first file takes precedence over the second, and so on down the line. index.html (or index.php if you’re using PHP) is the prototypical directory index page under Apache.

On the flip side of this coin are files that you don’t want to show up in your index pages; typically files that you wish to ignore are hidden or are temporary files such as those created by your editor (emacs saves buffers with “~” appended), or files that you don't want visitors to see (such as CVS subdirectories).

IndexIgnore pattern [pattern] ...

This directive allows you to specify files that shouldn’t be included in directory listings.

So the next step is to create a directory in our server’s document tree that will not contain any file specified in the DirectoryIndex directive. To add a little diversity, let’s populate our new directory with a sample of files of differing types so that we will be able to see our icons in action:

A view of our folder through a command–line interface

Figure #2: A CLI view of our folder in OS X’s Terminal application.

The first directive we’ll look at that alters the display of our generated index page is possibly the simplest and most important—it controls whether or not Apache’s index pages will contain...

FancyIndexing on|off | off

The FancyIndexing directive is pretty self–explanatory in nature; on will tell Apache to include the icons and columns in the index page, whereas off will simply generate an unordered list (ul) for the files.

The following image is what you see when FancyIndexing is turned off:

An index page without “fancy indexing”

Figure #3: An example of an index page without “fancy indexing.”

The next directive that we’ll look at can be considered the Swiss Army Knife of Apache’s configuration directives—it has so many parameters that can affect virtually every component of an auto–index page. So let’s just take a quick look at it, put it in the back burner, and move on to the first major section of our makeover: icons.

IndexOptions option1 [option2] [...]

We’ll see what the possible options for this directive can be shortly.

Icons & Alternate Text

Big–name software companies such as Apple, Microsoft, Adobe, and Macromedia have resources to pay graphic designers to coat their offerings in eye–pleasing pixels; however, open–source developers traditionally haven’t had the resources to provide the same appeal (although this situation is gradually starting to disappear, especially among the Mozilla foundation’s offerings such as Firefox and Thunderbird). The Apache httpd distribution includes a set of icons created by Kevin Hughes back when the project was under the wing of the NCSA, which are still used in the auto–index pages in default installations. And while these icons have served their purpose well, they certainly don’t compare with the quality of pixel art currently being created by professionals, such as The Iconfactory.

A sampling of the icons included in Apache’s distribution for use in serve–generated index pages.

Figure #4: A sampling of the icons included in Apache’s distribution for use in server–generated index pages.

So one of the easiest visual alterations that can be made to your auto–index pages is to change the icons, since this can be done by simply replacing the images rather than altering the configuration file itself; although creating the icons themselves using your image editor of choice (be it the GIMP, Photoshop, or something in between) is a different matter entirely. If you are running Apache in a development environment on your machine or somewhere else within your network, this is simply a matter of dragging & dropping (or mving, depending on your environment) the desired images to the location specified in your configuration file.

However, if you don’t have permissions to modify the images (which is most likely the case in a production environment) we’ll have to tell Apache to look for them somewhere else. This can be done with the Alias directive.

Alias url directoryOrFile

The Alias command, which is not strictly applicable to our indexes, allows files that are not located in your document root to be accessed as though they were. So to create a new place for our icons, we can use the command Alias /icons/ "/Library/WebServer/Icons/"

So we now have a place to store our icons; the next step is to copy some images into the folder:

An example location to store our custom icons for Apache on OS X

Figure #5: An example location to store our custom icons for Apache on OS X.

And now it‘s time to put our little creations to work. In order to tell Apache to use a specific icon for a given file (or series of files) in our document tree, we have to map each icon to the file(s) that will accompany the icon in our index pages. This process can be done through one of the following three directives:

AddIcon filename pattern [pattern] ...
AddIcon (ALT, filename) pattern [pattern] ...

Each pattern is a (partial) filename or wildcard expression that resolves to a filename (ex: *.html).

There are three special cases for the pattern:

AddIconByType file mime–pattern [mime–pattern] ...
AddIconByType (alt,file) mime–pattern [mime–pattern] ...

This directive is very similar to AddIconByType, except that you specify the appropriate icon based on the files’ MIME type (as listed in Apache's mime.types file), rather than its encoding.

AddIconByEncoding filename mime–encoding [mime–encoding] ...
AddIconByEncoding (alt,file) mime–encoding [mime–encoding] ...

This directive is similar to AddIcon, except that you specify the appropriate icon based on the files’ MIME encoding, rather than its name.

Each of the above directives has an optional syntax where it is possible to specify both the file to be used in the src attribute of the img tag and the alt text. However, it is also possible to set the alt text sepaarately, as we'll see shortly.

Because there is more than one method to specify the icon for any given file, there is a good chance that multiple directives will apply to some files in your directory tree. For example, the following two directives

  AddIconByType /images/text.png text/*
  AddIconByType /images/web.png text/html

will both apply to HTML files. So which icon is used? In this situation, the first directive that matches the file will take precedence. In order to achieve the desired effect, define the most specific directives first, and then fall back on generic types later in the configuration file.

In the event that none of our AddIcon directives match one or more files in our directory, then it would be nice if Apache serves up a default icon for our obscure file types. This is exactly what the DefaultIcon directive is suited for.

DefaultIcon filename

The default icon will be used when a file doesn’t match any of the directives specified in the configuration file(s). This concept is separate from the ^^BLANK^^ icon, which is used as a spacer image to indent the column headers.

That being said, let’s take a look at our progress:

A sample directory listing after adding our own icons

Figure #6: A sample directory listing after adding our own icons.

Definitely an improvement over our stock page. Now let’s use our spork to complement the directives that we introduced above:

IndexOptions [IconHeight[=height]] [IconWidth[=width]] [IconsAreLinks]

By including the first two parameters in the configuration file, Apache will include the width and height attributes in the img tags; it should be noted that it is perfectly acceptable to include or exclude one or the other. If included, IconsAreLinks instructs Apache to place the img tag itself within the a tag that links to a file in the directory.

If the parameters IconHeight and IconWidth are not included, then the img tags for the icons will not contain the corresponding attributes; whereas if the optional values height and width are not specified, then Apache will use the size of the icons that are included in the default distribution (which happen to be the rather odd size of 20 pixels wide by 22 pixels tall). So if your icons appear stretched or compressed in the index page, you’ll know where the culprit is.

The following screenshot illustrates the differences made by the previous parameters; the markup in the upper window is generated without including the IndexOptions parameters we just saw, while the lower source is:

The effects of several IndexOptions parameters

Figure #7: The effects of using the IndexOptions parameters that relate to icons; image sizes are now specified and the images themselves are now included in the <a> tags.

We can also see from the above image that although the width and height attributes of the img are specified for the icons, the alt attribute of the tags is not specified for some of the files. Fixing this problem is trivial; we can specify the alt attribute of an icon through analogous methods that we just used for the src attribute:

AddAlt string pattern [pattern] ...

This directive follows the same conventions as the AddIcon directive.

AddAltByType string mime_type [mime_type] ...

This directive follows the same conventions as the AddIconByType directive.

AddAltByEncoding string mime_encoding [mime_encoding] ...

This directive follows the same conventions as the AddIconByEncoding directive.

The following image is a screenshot of our index page in a text–browser after adding alt attributes for our icons:

Our auto–index page as seen in a text browser

Figure #8: Our directory listing (as seen in Lynx) after adding alt attributes for our icons.

Columns

Now that you’ve seen how to control how icons are embedded in the index, let’s take a look at the ways to control how the metadata columns are displayed (or not). Again, it’s time to break out the spork.

IndexOptions [NameWidth=n|*] [DescriptionWidth=n|*] [SuppressColumnSorting] [SuppressLastModified] [SuppressSize] [SuppressDescription]

The first two parameters specify the width of the “Name” and “Description” columns in characters; n will make the column n characters wide, whereas * will make the column accomodate the longest filename. Each Suppress... parameter instructs Apache to not include the corresponding column in the auto–index page.

By including one (or more) of the suppression parameters, the corresponding column(s) won’t be included in the index page; whereas specifying SuppressColumnSorting will remove all of the links from the column headers to prevent visitors from sorting the file listing by anthing other than the default value (which, not surprisingly, can also be changed).

By default, the order of the files in Apache’s index pages can be sorted by any of the four columns in ascending or descending order. Figure #9 is a screenshot of an index page with column sorting and the “Description” and “Last Modified” columns suppressed and the “Name” column decreased in size:

An example of altering the displayed columns

Figure #9: An example of altering the index columns through the parameters of the IndexOptions command.

At this point, no files in our index have a description attached to them, so sorting the index by description in ascending or descending order has no effect. The AddDescription directive, not surprisingly enough, allows us to add descriptions to files.

AddDescription string pattern [pattern] ...

AddDescription operates in virtually the same fashion as the AddIcon and AddAlt directives.

The description for a file can be virtually any markup that you can dream of ( including HTML tags). For example, the directive

AddDescription "<a href=\"http://www.apple.com/quicktime/\"
title=\"QuickTime Movie\">QuickTime movie</a>" *.mov 

will include a link to QuickTime's website so that visitors can download the plug–in.

As you can see from the above image, the files are sorted in ascending alphabetical order by default, meaning that A.txt comes before Z.txt, which in turn comes before a.txt. Should you need to alter this (if you wish to sort by size or by descending order, for example), then the IndexOrderDefault directive is what you need.

IndexOrderDefault Ascending|Descending Name|Date|Size|Description

The parameters to the command govern how the files will be sorted in auto–index pages by default; the first indicates which direction the values are sorted in, and the second inidicates which column contains the keys to sort by.

It is also possible to pass a parameter to the generated page to sort the columns based on the user’s preference. The parameter is passed as a key=value pair, and the key can be one of:

And the value can be one of:

For example, let's inspect the following example URLs:

In the event that an invalid parameter is passed (such as in the URL http://localhost/?Y=Z), then Apache will ignore the parameter and generate the file listing according to the directives included in its configuration file.

Headers & Footers

We now know how to control the portion of the page that lists the directory’s contents. The next step is to control the content that appears before and after the listing.

HeaderName filename

If a file with the given filename is present in the requested directory, the contents of it will be inserted into the index page before the file listing, but after the HTMLnot exist, then the top of the document will appear the same as if the directive wasn’t included in the configuration file.

This allows you to include special instructions in the index page that visitors may need to read before downloading the directory’s contents.

IndexOptions SuppressHTMLPreamble

One more step that can be made with respect to the header of the document is to insert our own fragment as the header of the page, which allows us to control everything that comes before the file listing.

By suppressing the HTML preamble, it is possible to incorporate your own site’s templates into your index pages; and it is now possible to use your site’s CSS stylesheets to tweak the page’s appearance and JavaScript library to govern its behaviour. For example, the following snippet is just one example of what can be inserted into a header file:

<html>
  <head>
    <title>Widget 3001 Downloads</title>
    <link rel="stylesheet" type="text/css" href="/c/lib.css"/>
    <link rel="icon" type="image/png" href="/icon.png"/>
    <script type="text/javascript" src="/j/lib.js"></script>
  </head>
  <body>
    <h1>Download Widget 3001!</h1>
    <p>
      Get the latest releases here:
    </p>
    <!-- file list will be appended here -->

By default, Apache will include a signature on its index pages, which looks something like Apache/1.3.29 Server at localhost Port 80 (the details of which will vary according to your site’s configuration, of course). It is possible to remove this signature and use your own markup in its place.

ReadmeName filename

The readme file behaves similar to the header file, except that it is appended to the page after the file listing has been included, rather than before.

If the ReadmeName file is found, then it is placed below the file listing in our auto–index page instead of the server’s signature, which typically looks something like (the values displayed will vary based on your version and host, of course) “Apache/1.3.29 Server at localhost Port 80” .

Odds & Ends

In addition to the directives we’ve already seen, there are some square pegs that don’t fit into our round holes (and not surprisingly, they are all tools that are only marginally useful); let’s take a quick look at them.

IndexOptions [FoldersFirst] [IgnoreCase] [ScanHTMLTitles]

The parameters shown above have the following effects:

The ScanHTMLTitles directive should be avoided for high–traffic sites, since every HTML file in the directory will be scanned to retreive its title.

Apache 2’s Additions

In addition to all of the configuration options that were introduced above, the 2.0.x branch of Apache also includes several auto–index directives that aren’t available in the 1.3.x family. Most of these additions are small utilities that are to be tacked on to our ever–expanding utility belt, so let’s take a quick peek at them:

IndexOptions [SuppressIcon] [SuppressRules] [VersionSort] [XHTML] [HTMLTable]

Anyone familiar with web standards (or atleast the HTML 4.1 DTD) will realize that Apache 1.3.x’s pages don’t validate for a couple of reasons:

Zeldman & Co. (myself included) will appreciate Apache 2 solely for the fact that its index pages are now able to validate; while this feature is relatively minor when compared to the other differences between 1.3.x and 2.0.x, it is one more step in the road to the semantic web. Making your pages validate can be accomplished in one of two ways:

The other advantage to using HTMLTable is that it frees you to use a proportionally–spaced font (such as Arial, Verdana, Helvetica, or any other sans-serif font), instead of a fixed–width font such as Courier or Monaco). Which means that you’re now able to use the same font face that is utilized throughout the rest of your site, making your index pages blend in that much more.

Other Options

Apache’s auto–indexes aren’t the only option; it is a simple matter to create a server–side script with PHP (or Perl/CGI if you prefer the old school) to generate a custom index page with links to the directory’s contents. However, as with any solution there are pros and cons with this method; the main advantage being that you can customize the page in ways that aren’t possible using Apache’s configuration directives, while the main drawback is the increased load placed on your web server required to generate the page (due to the reduced performance of interpreted languages as compared to their pre–compiled cousins). But the choice is up to you.

The Result

We have just seen the entire menagerie of directives that alter Apache’s auto– pages. Remember what Step #1 of this article was? It's time to go back and see what our original auto–page looked like. Now take a look at what this simple configuration file and CSS file produced:

The result of all our hard work

Figure #10: The result of all our hard work. Contrast this image with Figure #1 to see how our auto–indexes have progressed.

The above page is a relatively simple example of what can be done using Apache’s auto–index directives. If you've created a work of art using mod_autoindex’s directives, post a link in the Comments section and let’s see what kind of gallery we can create. Happy indexing.