Demon Homepages Mirroring


This Perl script wlll maintain a set of Demon homepages as a mirror of a local directory and will fetch the log files and statistics information from the server.

Updated for new homepages server in 2007.

Perl script called ftp-www: ftp-www.

How Pages Are Uploaded

As a Demon homepage user the web space appers as a directory that can be accessed by FTP. From 2007 the directory for the web pages has changed and is now called docroot rather than just being the default FTP location.

New Files

Any file that is newer than the time that the last batch of pages were uploaded will be uploaded this time. Any directories that are needed are also created at the same time. The time of the last update is recorded in the modification time of the file that contains the ls -lR listing.

Obsolete Files

The time will come when files need to be deleted, they have become obsolete. They can be recognised by their absence in the local directory and their presences in the ls -lR listing from last time. These will be deleted from the homepages server.

Disk Usage

The ls -lR output is fetched each time that a change is made and this is also used to calculate the amount of disk space occupied and this is reported.

Counters

If there is a counter on the homepage then the script will help track the number of visitors (although this is much inferior to the extra log files that are now available). The script downloads the count.txt file and adds the count to a log file count.log. The number of new visitors is also reported by the program.

Log Files

With the new homepages server in 2007 there is a directory that contains the web server access logs. The script will download any files that don't already exist into a directory called LOGS that it will create if needed.

Statistics

With the new homepages server in 2007 the graphs that are available on the homepage control panel (for number of hits but not for disk usage or bandwidth) are also available. The script will download the image files that have changed on the server into a directory called STATS that it will create if needed.

Example of Using ftp-www

In the example the local copy of the files are all maintained in a directory called /home/demon/www. The command to use to mirror these files onto Demon's homepage server and get the log files is simply
ftp-www -v -u hostname -p password
where hostname is the name of the Demon account (gedanken in my case) and password is the password for the web pages. There are two pieces of customisation might be needed, by editing the ftp-www program.
  1. Change the name of the directory to be mirrored.
  2. Change the list of files to exclude (perl regular expressions).
Note:
The first time that you run the program, it will fetch a list of files from the server (the ls-lR file) but will not update anything. The next time, it will use the ls-lR file to know what has to be updated.