Katlas installation notes: Difference between revisions

Revision as of 13:04, 6 November 2006

Okay... these are our notes while trying to setup the wiki on katlas.

Upgrading MySQL

For better or worse, I decided to upgrade MySQL to version 4.0 before doing anything else. I mostly followed these instructions.

Warning: do not install MySQL 4.1. It is incompatible with mediawiki. Install version 4.0.

For this I need three RPMs which are all available with only a little digging at mysql.com

MySQL-shared-compat-4.0.25-0.i386.rpm
MySQL-server-4.0.25-0.i386.rpm
MySQL-client-4.0.25-0.i386.rpm

The first of these is required so we don't break dependencies while upgrading. Install it first, with

rpm -iv MySQL-shared-compat-4.0.25-0.i386.rpm

Then upgrade the other two packages

rpm -U MySQL-server-4.0.25-0.i386.rpm

rpm -U MySQL-client-4.0.25-0.i386.rpm

Note the warning the server install gives --

PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
To do so, start the server, then issue the following commands:
/usr/bin/mysqladmin -u root password 'new-password'
/usr/bin/mysqladmin -u root -h katlas.math.toronto.edu password 'new-password'
See the manual for more instructions.

(note however that you need to upgrade the client package before mysqladmin is available to do this!)

I also deleted the local anonymous account, and put a password on the remote anonymous account. (Ask User:Scott for these passwords.)

Apache

I had to fix the server name, from katlas to katlas.math.toronto.edu in /etc/httpd/conf/httpd.conf

I added

AllowOverride FileInfo
AllowOverride AuthConfig

in the httpd.conf file, in the section for /var/www/html, so I could use .htaccess files to rewrite the wiki URLs nicely, and to disallow access to phpMyAdmin.

I added a robots.txt file, to protect the wiki internals from spiders. --Scott 12:44, 19 Aug 2005 (EDT)

I moved the document root from /var/www/html to /www/html. (Actually, I moved everything in /var/www to /www) This required the following steps

in /var/www, as root
```
cp -R --preserve * /www
```
in httpd.conf, replacing /var/www everywhere with /www
in LocalSettings.php replacing /var/www with /www in the $IP line
test everything seems to work!
in /var, as root,
```
rm -Rf www
```

--Scott 12:44, 19 Aug 2005 (EDT)

This messed up webalizer, however, so I modified the appropriate line in /etc/webalizer.conf. I also added a section in http.conf to make /usage world-readable. --Scott 13:03, 19 Aug 2005 (EDT)

Restarting apache is best achieved by (as root)

/usr/sbin/apachectl graceful

or

/etc/rc.d/init.d/httpd restart

--Scott 13:17, 19 Aug 2005 (EDT)

Error logs can be found at /var/log/httpd --Scott 10:33, 26 Aug 2005 (EDT)

We had a nasty (to diagnose!) problem with uploading large files. We had to edit /etc/httpd/conf.d/php.conf, modifying the LimitRequestBody line. --Scott 11:08, 23 Aug 2005 (EDT)

creating new accounts

As root, do something like

$ htpasswd -m /www/.htpasswd sally
New password: *******
Re-type new password: *******
Adding password for user sally

phpMyAdmin

Just unzip'd it into phpMyAdmin, editted the config.inc.php, created a .htpassword file and added a <Directory> entry to httpd.conf --Scott 10:34, 26 Aug 2005 (EDT)

MediaWiki

I added the namespace "Data", and later "Article", as per these instructions, adding

$wgExtraNamespaces =
       array(100 => "Data",
             101 => "Data_talk",
             102 => "Article",
             103 => "Article_talk");
# Enable subpages in the knot data namespace
$wgNamespacesWithSubpages[100] = 1;
$wgNamespacesWithSubpages[101] = 1;
$wgNamespacesWithSubpages[102] = 1;
$wgNamespacesWithSubpages[103] = 1;

to LocalSettings.php. Do not use spaces in the namespace names, only underscores.

Enable <math></math> processing

First, I made the /images directory writable.

Next, I installed ocaml (download the sources, ./configure, make world, make bootstrap, make opt, make install). What a pain! After that, go to the math directory under the wiki installation, and run make.

Finally, edit LocalSettings.php to enable TeX.

Later, we started modifying texvc, to allow more symbols from amssymb, and to allow use of Dror's dbnsymb. This required modifications to Texutil.ml, and recompiling texvc (just run make). --Scott 15:10, 29 Aug 2005 (EDT)

Installing the Font dbnsymb

Download the files dbnsymb.mf and dbnsymb.sty from [1].
Put dbnsymb.mf at "/usr/share/texmf/fonts/source/public".
Put dbnsymb.sty at "/usr/share/texmf/tex/latex/misc".
Run the command "texhash".

Allowing .zip and .gif uploads

In LocalSettings.php I added the line

$wgFileExtensions = array( 'png','gif','jpg','jpeg','ogg','mp3','pdf','zip','gz','nb','m','lhs');

Marking users as bots

See [2]. Note that all our tables have the prefix mw_. As far as I can tell, you have to guess the user number (or you can log in as that user, and go to Preferences).

Modifying the navigation box

Follow the instructions at [3]

Make changes to LocalSettings.php
edit the pages Mediawiki:todo, Mediawiki:todo-url, Mediawiki:rolfsen, Mediawiki:rolfsen-url, etc. (these pages are protected.)

Note that this will change in mediawiki 1.5, and become a lot easier.

Add "tour" sidebar box

In skins/MonoBook.php, right above the line beginning with

<?php if( $this->data['language_urls'] )

add lines as follows:

        <div class="portlet" id="p-tour">
          <h5><a href="/wiki/Tour_of_the_Knot_Atlas">tour</a></h5>
          <div class="pBody">
            <ul>
            <li><a href="/wiki/5_2">5 2</a> (edit me!)
            <li><a href="/wiki/5_2_Quantum_Invariants">5 2 Quantum Invariants</a>
            <li><a href="/wiki/L10n85">L10n85</a>
            <li><a href="/wiki/The_Multivariable_Alexander_Polynomial">Multivariable Alexander</a>
            <li><a href="/wiki/Knot_Atlas:About">About</a>
            <li><a href="/wiki/Data:5_2/Bridge_Index">Data:5_2/Bridge_Index</a>
            <li><a href="/wiki/The_Mathematica_Package_KnotTheory%60">Knot Theory`</a>
            <li><a href="/wiki/How_to_Edit_this_Manual...">How to Edit Manual...</a>
            <li><a href="/wiki/Modifying_Knot_Pages">Modifying Knot Pages</a>
            <li><a href="/wiki/Special:Recentchanges">Recent changes</a>
            <li><a href="/wiki/To_Do">To Do</a> (history!)
            </ul>
          </div>
        </div>

(this will surely change as we understand MediaWiki better)

Modifying the logo

Grab , resize it to 145x145, save it as /www/html/w/skins/common/images/katlas.png, and put the line

$wgLogo             = "$wgStylePath/common/images/katlas.png";

in LocalSettings.php. Also, grab , convert it to Windows .ico format and save it as favicon.ico at /www/html (not /www/html/w !).

Hacking mediawiki

I modified ImagePage.php so redirects on image pages don't produce a spurious copy of the image. There's a bug on bugzilla, where a different fix is being proposed. There's another (not so good?) solution as an extension.

Later, I modified Math.php, so if a math .png file is missing, but isn't immediately needed, it isn't regenerated. This let's us delete .png files from /w/images/math. To avoid creating too many such .png images in the first place, any robots uploading math formulas should have 'Leave TeX' set in their math preferences. --69.110.11.59 04:59, 21 Sep 2005 (EDT)

Fixing search

In file /www/html/w/skins/MonoBook.php within <form name="searchform" ... > add method="post".

Automatic "&action=purge"

In file /www/html/w/index.php change the line reading $action = $wgRequest->getVal( 'action', 'view' ); to $action = $wgRequest->getVal( 'action', 'purge' );.

Several Wikis

Allow logins on several wikis on the same server: Add the following line to LocalSettings.php after $wgSiteName = ...:

ini_set("session.name", "KnotAtlasSession" );

Spam blacklists

After surviving several annoying spam attacks on my private wiki, I learnt how to deal with it, and I'm applying what I've learnt here. See /w/extensions/SpamBlacklist for the spam blacklist extension, and LocalSettings.php for the changes required there. Things should just work. Even better, if you're a sysop, you can just edit the page Local spam blacklist to add regular expressions. I've downloaded the latest copy of the spam blacklist from meta, but we should update it more often. Updating it is simply a matter of running /www/html/w/extensions/SpamBlacklist/load_lists --Scott 13:20, 29 Sep 2005 (EDT)

Can we create a cron job for /www/html/w/extensions/SpamBlacklist/load_lists, or add it to our backup script? --Scott 13:20, 29 Sep 2005 (EDT)

Referer Spam

To combat referer spam, I added a referer's black list as here to /www/html/.htaccess, with some additions of my own following the log files. More should be added as needed. --Drorbn 09:33, 10 May 2006 (EDT)

Upgrading mediawiki

katlas hasn't been upgraded from mediawiki 1.4.7 since it was first set up. Scott's successfully upgraded several other wikis, however. Here's a recipe:

mv w w-old
gzip -d mediawiki-[version].tar.gz
tar xvf mediawiki-[version].tar
mv mediawiki-[version] w
chmod a+w w/config
Go to http://example.com/w/config/ and fill in all the fields. It's useful to have w-old/LocalSettings.php open as you do this, as many of the fields can be copied from this.
If installation is successful, follow the instructions: mv w/config/LocalSettings.php w/
Go through the new and old LocalSettings.php files, migrating changes across to the new LocalSettings.php
Copy across the contents of the images/ directory.
Remember to set permissions on the images/ directory (probably chmod -R g+w images).
If appropriate, copy across the math/ directory, containing our hacks (allowing use of dbnsymb, amongst other things).

Actually, this wasn't entirely successful. Dror subsequently discovered that LaTeX support was broken, and that new pages couldn't be created. --Scott 13:00, 28 Mar 2006 (EST)

The new page problem is because the spam blacklist extension installed is not compatible with mediawiki 1.5. We need to download a new copy of the spam blacklist extension for any wikis we upgrade to 1.5, from [4]. Alternatively, is should be safe to copy it from the omath wiki. The LaTeX support was broken because I hadn't made the images directory group writable (or, for that matter, copied across the images directory!) --Scott 13:28, 28 Mar 2006 (EST)

Server security

Everyday we have scripts come through trying to login through sshd, with various user/pass combinations. If we need to a solution we could either

move sshd to listen on another port
install on the the scripts described at http://www.linux.com/article.pl?sid=05/09/15/1655234

up2date and python

At some point I installed version 2.4 of python on katlas; we previously had version 2.2 installed, and something (I can't remember what) needed a more recent version. However, I never worked out how to properly install new python modules, and, as a result, there's various stuff missing from the python 2.4 installation, in particular the rpm module. This causes redhat's "up2date" program to fail, which is serious, as this is the main mechanism for automatically installing security updates from redhat. --Scott 08:36, 3 Apr 2006 (EDT)

Work around

In /usr/bin, you'll find files "python2.2", "python2.4" and "python". "python2.2" is a copy of the originally installed "python"; "python2.4" is a symbolic link to my installation of python 2.4. Currently, the file "python" is an exact copy of the file "python2.2". Things seem to work; in particular, "up2date" runs again.

If you find something that needs python 2.4 (happily, running up2date brought the honest version of python up to 2.3), you could try replacing "python" with "python2.4", but I really wouldn't recommend it. --Scott 08:36, 3 Apr 2006 (EDT)

Running up2date

Running up2date for the first time in a long while seemed to work just fine. It complained about two sendmail configuration files, and made new versions with .rpmnew appended, but there were no essential differences. --Scott 09:45, 3 Apr 2006 (EDT)

We should check with the toronto math department computer people to see if they're now happy with katlas.

Subversion

We have a working subversion server going now. (I had to find the RHEL3 rpm, but it was easy after that!)

We have several separate repositories, and it's easy (for root) to create more. Go to one of

to get started.

To create a new repository, type something like

svnadmin create RepositoryName /home/svn

then change the ownership and permissions to match the other repositories. (The easiest way to do this is to login as svn, before using svnadmin, then chmod -R g+w /home/svn/RepositoryName to allow commits via the web.)

There's read-only public access; to commit changes you'll need to create an account. For windows, download TortoiseSVN, for eclipse use subclipse.

I've set up fine grained access control, as per instructions in the SVN Book. You'll need to edit /home/svn/svnaccess.conf to grant people access to the repositories. --Scott 20:26, 9 Oct 2006 (EDT)

Some people have had trouble with Mathematica notebooks in version control, although so far it's been fine for us. There are several solutions people have come up with. Do be very careful when resolving conflicts, on the other hand!

http://www.itwm.fhg.de/as/asemployees/wichmann/nbcache.html A perl script for stripping out the cache data.
Using the ShowShortBoxForm option in Mathematica [5], which apparently makes notebooks less prone to arbitrary changes independent of real content changes.

iptables

Arno's iptables is the way to go. Just edit

/etc/iptables-firewall.conf

Make sure to have "DROP_IANA_RESERVED=0" to prevent the blocking of certain DSL/Cable providers.

To restart, use

/etc/init.d/rc.iptables restart

Fisheye and Jira

run /home/fisheye/server/bin/start.sh and /home/jira/server/bin/startup.sh to start them
to allow fisheye access to a new repository, see http://www.cenqua.com/fisheye/doc/1.1/admin/svnallow.html
I modified /etc/rc.d/rc.local to automatically start fisheye and jira.
getting the automatic proxying working on the new server required copying /etc/httpd/conf.d/jira.conf over.

Sendmail

Not too sure, but I think the critical step to getting sendmail working again was

cd /etc/mail
nano sendmail.mc
(... editing the line DAEMON_OPTIONS(`Port=smtp,Addr=128.100.68.50, Name=MTA')dnl
      by replacing 127.0.0.1 with the local IP ...)
m4 /etc/mail/sendmail.mc > /etc/mail/sendmail.cf
service sendmail restart

OS Upgrade, October 2006

On October 2006 we had a disk failure and had to replace it with a new disk and re-install the OS. After doing so, we had to modify the following OS files from the RedHat default:

/etc/iptables-firewall.conf
/etc/hosts.allow
/etc/fstab
/etc/passwd
/etc/shadow
/etc/group
/etc/httpd/conf/httpd.conf

Dror will be holding the old HD on his counter for at least a few months just to be sure everything is fine. To use it, open the box and reconnect it at the slot marked "Old HD", reboot, go to /mnt/oldhd and run "mountme" or simply enter the lines

mount /dev/Volume00/LogVol00 /mnt/oldhd/LogVol00
mount /dev/Volume00/LogVol01 /mnt/oldhd/LogVol01
mount /dev/Volume00/LogVol02 /mnt/oldhd/LogVol02
mount /dev/Volume00/LogVol04 /mnt/oldhd/LogVol04