Esolang:Wiki dumps

From Esolang
Jump to: navigation, search

A dump of the wiki's content in MediaWiki's XML format is available. It is updated every day at around 6:00 EDT/EST ("New York time").

You can download the compressed dump directly, but if you plan to update the dump regularly, zsync is strongly recommended, as it allows the dump to be updated incrementally, which decreases the bandwidth load on this server. You can install zsync on Debian-derived systems (such as Ubuntu) with apt-get install zsync (remember to run this as root, e.g. with sudo on Ubuntu). The following command will download or update the esolang.xml file in the current directory:

$ zsync http://esolangs.org/dump/esolang.xml.zsync

Downloading these dumps lets you easily access all the content (including page history and files), allowing you to back up the wiki in case of future failure, perform bulk analysis on the data, or set up a mirror.

Keeping updated with cron[edit]

If you're on a Unix-like operating system, you can create a crontab file (e.g. with crontab -e) containing a line like this:

30 6 * * * zsync -o /path/to/esolang.xml http://esolangs.org/dump/esolang.xml.zsync

Adjust the first two fields (minute and hour) according to your timezone, and the path according to where you want to keep the dump (for example, /home/elliott/esolang.xml).

The example line will incrementally update your copy of the dump every day at 6:30 your local time. It should only download one or two megabytes, rather than the 40 megabytes or so that the full dump takes up. Note that it will store the dump uncompressed on your computer, which takes up over 300 megabytes at the time of writing.

This method works best on a server; if your computer is not always on at 6:30 (or whatever time you have set), you can look into using something like anacron to schedule dumps.