Katlas installation notes: Difference between revisions

From Knot Atlas
Jump to navigationJump to search
m (Reverted edits by LapasOrsit (Talk); changed back to last version by Drorbn)
 
(102 intermediate revisions by 11 users not shown)
Line 1: Line 1:
On July 14, 2007 we've upgraded MediaWiki to version 1.10.1. The pre-upgrade installation notes are at [[MediaWiki 1.4 installation notes]].
Okay... these are our notes while trying to setup [http://katlas.math.toronto.edu/wiki/ the wiki on katlas].


==Upgrading MySQL==
===Fixing search===
In file <tt>/www/html/w/skins/MonoBook.php</tt> change <tt>&lt;form ... id="searchform"&gt;</tt> to <tt>&lt;form method=post ... id="searchform"&gt;</tt>.
For better or worse, I decided to upgrade MySQL to version 4.0 before doing anything else. I mostly followed [http://dev.mysql.com/doc/mysql/en/linux-rpm.html these instructions].


===Rendering PNG===
''Warning: do not install MySQL 4.1. It is incompatible with mediawiki. Install version 4.0.''
Follow the instructions at http://mywiki.ncsa.uiuc.edu/wiki/MediaWiki_Servers#Problems_Rendering_PNG_Images:

*Add to LocalSettings.php
For this I need three RPMs which are all available with only a little digging at [http://www.mysql.com/ mysql.com]
<pre>
* MySQL-shared-compat-4.0.25-0.i386.rpm
$wgMimeDetectorCommand = "file -bi";
* MySQL-server-4.0.25-0.i386.rpm
</pre>
* MySQL-client-4.0.25-0.i386.rpm
*then rebuild images

<pre>
The first of these is required so we don't break dependencies while upgrading. Install it first, with
# cd /www/html/w/maintenance
:rpm -iv MySQL-shared-compat-4.0.25-0.i386.rpm
# php rebuildImages.php
Then upgrade the other two packages
Processing image...
:rpm -U MySQL-server-4.0.25-0.i386.rpm
Finished image... 3 of 4 rows updated
:rpm -U MySQL-client-4.0.25-0.i386.rpm
Processing oldimage...
Finished oldimage... 2 of 2 rows updated
#
</pre>


===Creating nice thumbnails with ImageMagick===
Note the warning the server install gives --
Unfortunately, they've broken things in mediawiki 1.10, and creating thumbnails of transparent gifs is horrible. If you make the following changes to /includes/media/Bitmap.php, things should behave just like in mediawiki 1.4 (i.e. perfectly, for our purposes).
<pre>
<pre>
// replacing this all with something much simpler from around mediawiki 1.4
PLEASE REMEMBER TO SET A PASSWORD FOR THE MySQL root USER !
# $cmd = wfEscapeShellArg($wgImageMagickConvertCommand) .
To do so, start the server, then issue the following commands:
# " {$quality} -background white -size {$physicalWidth} ".
/usr/bin/mysqladmin -u root password 'new-password'
# wfEscapeShellArg($srcPath) .
/usr/bin/mysqladmin -u root -h katlas.math.toronto.edu password 'new-password'
# // Coalesce is needed to scale animated GIFs properly (bug 1017).
See the manual for more instructions.
# ' -coalesce ' .
# // For the -resize option a "!" is needed to force exact size,
# // or ImageMagick may decide your ratio is wrong and slice off
# // a pixel.
# " -thumbnail " . wfEscapeShellArg( "{$physicalWidth}x{$physicalHeight}!" ) .
# " -depth 8 $sharpen " .
# wfEscapeShellArg($dstPath) . " 2>&1";
// here's what we used to do, back in mediawiki 1.4, and it works great.
$cmd = wfEscapeShellArg($wgImageMagickConvertCommand) .
" {$quality} -background white -geometry {$physicalWidth} ".
wfEscapeShellArg($srcPath) . " " .
wfEscapeShellArg($dstPath) . " 2>&1";
</pre>
</pre>


===Modifying the navigation box===
(note however that you need to upgrade the client package before mysqladmin is available to do this!)
Edit [[MediaWiki:Sidebar]].


===Add "tour" sidebar box===
I also deleted the local anonymous account, and put a password on the remote anonymous account. (Ask [[User:Scott]] for these passwords.)
In skins/MonoBook.php, right above the lines
<?php
if( $this->data['language_urls'] ) { ?>
add lines as follows:<pre><nowiki>
<div class="portlet" id="p-tour">
<h5><a href="/wiki/Tour_of_the_Knot_Atlas">tour</a></h5>
<div class="pBody">
<ul>
<li><a href="/wiki/5_2">5 2</a> (edit me!)
<li><a href="/wiki/5_2_Quantum_Invariants">5 2 Quantum Invariants</a>
<li><a href="/wiki/L10n85">L10n85</a>
<li><a href="/wiki/The_Multivariable_Alexander_Polynomial">Multivariable Alexander</a>
<li><a href="/wiki/Knot_Atlas:About">About</a>
<li><a href="/wiki/Data:5_2/Bridge_Index">Data:5_2/Bridge_Index</a>
<li><a href="/wiki/The_Mathematica_Package_KnotTheory%60">Knot Theory`</a>
<li><a href="/wiki/How_to_Edit_this_Manual...">How to Edit Manual...</a>
<li><a href="/wiki/Modifying_Knot_Pages">Modifying Knot Pages</a>
<li><a href="/wiki/Special:Recentchanges">Recent changes</a>
<li><a href="/wiki/To_Do">To Do</a> (history!)
</ul>
</div>
</div>
</nowiki></pre>


==="Redraw Page" tab===
==Apache==
I had to fix the server name, from katlas to katlas.math.toronto.edu in /etc/httpd/conf/httpd.conf


Add to [[MediaWiki:Monobook.js]], following [http://en.wikipedia.org/wiki/Wikipedia:WikiProject_User_scripts/Scripts/Add_purge_to_tabs Wikipedia:WikiProject User scripts/Scripts/Add purge to tabs]:
I added
AllowOverride FileInfo
AllowOverride AuthConfig
in the httpd.conf file, in the section for /var/www/html, so I could use .htaccess files to rewrite the wiki URLs nicely, and to disallow access to phpMyAdmin.


addOnloadHook(function () {
I added a [http://meta.wikimedia.org/wiki/Robots.txt robots.txt] file, to protect the wiki internals from spiders. --[[User:Scott|Scott]] 12:44, 19 Aug 2005 (EDT)
var hist; var url;
if (!(hist = document.getElementById('ca-history') )) return;
if (!(url = hist.getElementsByTagName('a')[0] )) return;
if (!(url = url.href )) return;
addPortletLink('p-cactions', url.replace(/([?&]action=)history([&#]|$)/, '$1purge$2'),
'redraw page', 'ca-purge', 'Purge server cache for this page', '0');
});


===Spam filtering===
I moved the document root from /var/www/html to /www/html. (Actually, I moved everything in /var/www to /www) This required the following steps
Follow the directions at http://www.mediawiki.org/wiki/Extension:SpamBlacklist
# in /var/www, as root <pre>cp -R --preserve * /www</pre>
# in httpd.conf, replacing /var/www everywhere with /www
# in LocalSettings.php replacing /var/www with /www in the $IP line
# test everything seems to work!
# in /var, as root, <pre>rm -Rf www</pre>
--[[User:Scott|Scott]] 12:44, 19 Aug 2005 (EDT)


===Tweaking the job queue===
This messed up webalizer, however, so I modified the appropriate line in /etc/webalizer.conf. I also added a section in http.conf to make /usage world-readable. --[[User:Scott|Scott]] 13:03, 19 Aug 2005 (EDT)
Running jobs seems to slow everything down horribly. It's '''essential''' that you add

<pre>
Restarting apache is best achieved by (as root)
$wgJobRunRate = 0.01;
<pre>/usr/sbin/apachectl graceful</pre>
</pre>
or
to <code>LocalSettings.php</code>. It may prove necessary to add a cron job running <code>/w/maintenance/runJobs.php</code>, if the length of the job queue (see [[Special:Statistics]]) gets out of hand. This should only happen after edits to widely used templates.
<pre>/etc/rc.d/init.d/httpd restart</pre>--[[User:Scott|Scott]] 13:17, 19 Aug 2005 (EDT)

Error logs can be found at /var/log/httpd --[[User:Scott|Scott]] 10:33, 26 Aug 2005 (EDT)

We had a nasty (to diagnose!) problem with uploading large files. We had to edit /etc/httpd/conf.d/php.conf, modifying the LimitRequestBody line. --[[User:Scott|Scott]] 11:08, 23 Aug 2005 (EDT)


In an attempt to stop the job queue from processing latex, Scott added a line to <code>includes/JobQueue.php</code>.
===creating new accounts===
As root, do something like
<pre>
<pre>
$options = new ParserOptions;
$ htpasswd -m /www/.htpasswd sally
// turning off math rendering while processing jobs, because it's a terrible idea.
New password: *******
$options->setUseTeX( false );
Re-type new password: *******
Adding password for user sally
</pre>
</pre>


===Installing ParserFunctions===
==phpMyAdmin==
In order to use template-like parser functions, such as <code>{{#ifexist:<pagename>|<true>|<false>}}</code>, we need to enable the ParserFunctions extension, following the instructions at http://meta.wikimedia.org/wiki/ParserFunctions.
Just unzip'd it into phpMyAdmin, editted the config.inc.php, created a .htpassword file and added a <nowiki><Directory></nowiki> entry to httpd.conf --[[User:Scott|Scott]] 10:34, 26 Aug 2005 (EDT)


Download the files
==MediaWiki==
* [http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/ParserFunctions/Expr.php Expr.php]
I added the namespace "Data", as per [http://meta.wikimedia.org/wiki/Help:Custom_namespaces these instructions], adding
* [http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/ParserFunctions/ParserFunctions.php ParserFunctions.php]
* [http://svn.wikimedia.org/svnroot/mediawiki/trunk/extensions/ParserFunctions/ParserFunctions.i18n.php ParserFunctions.i18n.php]
and then add a line to LocalSettings.php:
<pre>
<pre>
require_once( "$IP/extensions/ParserFunctions/ParserFunctions.php" );
$wgExtraNamespaces =
array(100 => "Data",
101 => "Data talk");
$wgNamespacesWithSubpages =
array( -1 => 0, 0 => 0, 1 => 1, 2 => 1, 3 => 1,
4 => 0, 5 => 1, 6 => 0, 7 => 1,
8 => 0, 9 => 1, 10 => 0,11 => 1,
100 => 1, 101 => 1 );
</pre>
</pre>
to LocalSettings.php. (For some reason, the namespace "Knot Data" didn't seem to work?)


===Installing a PHP cache===
===Enable <nowiki><math></math></nowiki> processing===
Look at http://meta.wikimedia.org/wiki/PHP_caching_and_optimization.
First, I made the /images directory writable.


# Install PEAR, by saving http://go-pear.org/ as <code>/home/wiki/pear/go-pear.php</code>, then running <code>php go-pear.php</code> and following the prompts
Next, I installed ocaml (download the sources, ./configure, make world, make bootstrap, make opt, make install). What a pain! After that, go to the math directory under the wiki installation, and run make.
# Install APC, by running <code>/home/wiki/pear/bin/pecl install apc</code>
# Add <code>extension=apc.so</code> to <code>/etc/php.ini</code>


You can see APC's statistics at http://katlas.org/apc.php
Finally, edit LocalSettings.php to enable TeX.


Add
Later, we started modifying texvc, to allow more symbols from amssymb, and to allow use of Dror's [http://www.math.toronto.edu/~drorbn/projects/dbnsymb/dbnsymbman.html dbnsymb]. This required modifications to [[Texutil.ml]], and recompiling texvc (just run make). --[[User:Scott|Scott]] 15:10, 29 Aug 2005 (EDT)

====Installing the Font dbnsymb====

# Download the files dbnsymb.mf and dbnsymb.sty from [http://www.math.toronto.edu/~drorbn/projects/dbnsymb/dbnsymbman.html].
# Put dbnsymb.mf at "/usr/share/texmf/fonts/source/public".
# Put dbnsymb.sty at "/usr/share/texmf/tex/latex/misc".
# Run the command "texhhash".

===Allowing .zip and .gif uploads===
In LocalSettings.php I added the line
<pre>
<pre>
# See http://www.mediawiki.org/wiki/Manual:%24wgMainCacheType
$wgFileExtensions = array( 'png','gif','jpg','jpeg','ogg','mp3','pdf','zip');
$wgMainCacheType = CACHE_ACCEL;
</pre>
</pre>
to <code>LocalSettings.php</code>.


===Marking users as bots===
===Installing memcached===
In my continuing quest for faster page loads, I'm trying memcached. Follow the nice instructions at http://blog.ajohnstone.com/archives/installing-memcached/.
See [http://meta.wikimedia.org/wiki/Setting_user_rights_in_MediaWiki]. Note that all our tables have the prefix mw_. As far as I can tell, you have to guess the user number (or you can log in as that user, and go to Preferences).


Although it's not installed, it's not running, or being used. See http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/docs/memcached.txt?view=markup for instructions for actually running it. You'll probably need to uncomment some lines in LocalSettings.php too.


===Modifying the navigation box===
===Updating CSS and JS===
* Write [http://en.wikipedia.org/wiki/MediaWiki:Common.css MediaWiki:Common.css] on top of the local [[MediaWiki:Common.css]].
Follow the instructions at [http://meta.wikimedia.org/wiki/MediaWiki_FAQ#How_to_customize_your_own_navigation_bar]
* Write [http://en.wikipedia.org/wiki/MediaWiki:Monobook.css MediaWiki:Monobook.css] on top of the local [[MediaWiki:Monobook.css]].
# Make changes to LocalSettings.php
* Write [http://en.wikipedia.org/wiki/MediaWiki:Common.js MediaWiki:Common.js] on top of the local [[MediaWiki:Common.js]] being super-careful not to step on any of our extensions.
# edit the pages Mediawiki:todo, Mediawiki:todo-url, Mediawiki:rolfsen, Mediawiki:rolfsen-url, etc. (these pages are protected.)

Note that this will change in mediawiki 1.5, and become a lot easier.

===Modifying the logo===
Grab [[Image:The_Knot_Atlas.png|36px]], resize it to 145x145, save it as /www/html/w/skins/common/images/katlas.png, and put the line
$wgLogo = "$wgStylePath/common/images/katlas.png";
in LocalSettings.php. Also, grab [[Image:favicon.png|16px]], convert it to Windows .ico format and save it as favicon.ico at /www/html (not /www/html/w !).

===Hacking mediawiki===
I modified [[ImagePage.php]] so redirects on image pages don't produce a spurious copy of the image. There's [http://bugzilla.wikimedia.org/show_bug.cgi?id=539 a bug] on bugzilla, where a different fix is being proposed. There's [http://meta.wikimedia.org/wiki/User:Sysy/Joshua_Oreman%27s_icon_extension another (not so good?) solution] as an extension.

===Fixing search===
In file <tt>/www/html/w/skins/MonoBook.php</tt> within <tt>&lt;form name="searchform" ... &gt;</tt> add <tt>method="post"</tt>.


===Increasing PHP's memory_limit===
===Several Wikis===
In LocalSettings.php, change
Allow logins on several wikis on the same server: Add the following line to LocalSettings.php after <code>$wgSiteName = ...</code>:
ini_set("session.name", "KnotAtlasSession" );
ini_set( 'memory_limit', '20M' );
to
ini_set( 'memory_limit', '40M' );


===Fixing "Random Page"===
==Subversion==
We have a working subversion server going now. (I had to find the RHEL3 rpm, but it was easy after that!) Go to http://katlas.math.toronto.edu/repository/ There's read-only public access; to commit changes you'll need to [[#creating new accounts|create an account]]. For windows, download TortoiseSVN, for eclipse use subclipse.


The standard MediaWiki [[Special:Random|random page]] algorithm is strange, and anyway unsuitable for our purposes. See [[Fixing "Random Page"]] for the details and for our fix.
Mathematica notebooks are slightly hard to version control. There are several solutions people have come up with.
*http://www.itwm.fhg.de/as/asemployees/wichmann/nbcache.html A perl script for stripping out the cache data.
*Using the ShowShortBoxForm option in Mathematica [http://forums.wolfram.com/mathgroup/archive/2005/Mar/msg00175.html], which apparently makes notebooks less prone to arbitrary changes independent of real content changes.

Latest revision as of 17:10, 8 January 2009

On July 14, 2007 we've upgraded MediaWiki to version 1.10.1. The pre-upgrade installation notes are at MediaWiki 1.4 installation notes.

Fixing search

In file /www/html/w/skins/MonoBook.php change <form ... id="searchform"> to <form method=post ... id="searchform">.

Rendering PNG

Follow the instructions at http://mywiki.ncsa.uiuc.edu/wiki/MediaWiki_Servers#Problems_Rendering_PNG_Images:

  • Add to LocalSettings.php
$wgMimeDetectorCommand = "file -bi"; 
  • then rebuild images
# cd /www/html/w/maintenance
# php rebuildImages.php 
Processing image...
Finished image... 3 of 4 rows updated
Processing oldimage...
Finished oldimage... 2 of 2 rows updated
#

Creating nice thumbnails with ImageMagick

Unfortunately, they've broken things in mediawiki 1.10, and creating thumbnails of transparent gifs is horrible. If you make the following changes to /includes/media/Bitmap.php, things should behave just like in mediawiki 1.4 (i.e. perfectly, for our purposes).

                        // replacing this all with something much simpler from around mediawiki 1.4
                        # $cmd  =  wfEscapeShellArg($wgImageMagickConvertCommand) .
                        #       " {$quality} -background white -size {$physicalWidth} ".
                        #       wfEscapeShellArg($srcPath) .
                        #       // Coalesce is needed to scale animated GIFs properly (bug 1017).
                        #       ' -coalesce ' .
                        #       // For the -resize option a "!" is needed to force exact size,
                        #       // or ImageMagick may decide your ratio is wrong and slice off
                        #       // a pixel.
                        #       " -thumbnail " . wfEscapeShellArg( "{$physicalWidth}x{$physicalHeight}!" ) .
                        #       " -depth 8 $sharpen " .
                        #       wfEscapeShellArg($dstPath) . " 2>&1";
                        // here's what we used to do, back in mediawiki 1.4, and it works great.
                        $cmd  =  wfEscapeShellArg($wgImageMagickConvertCommand) .
                                " {$quality} -background white -geometry {$physicalWidth} ".
                                wfEscapeShellArg($srcPath) . " " .
                                wfEscapeShellArg($dstPath) . " 2>&1";

Modifying the navigation box

Edit MediaWiki:Sidebar.

Add "tour" sidebar box

In skins/MonoBook.php, right above the lines

<?php
                if( $this->data['language_urls'] ) { ?>

add lines as follows:

        <div class="portlet" id="p-tour">
          <h5><a href="/wiki/Tour_of_the_Knot_Atlas">tour</a></h5>
          <div class="pBody">
            <ul>
            <li><a href="/wiki/5_2">5 2</a> (edit me!)
            <li><a href="/wiki/5_2_Quantum_Invariants">5 2 Quantum Invariants</a>
            <li><a href="/wiki/L10n85">L10n85</a>
            <li><a href="/wiki/The_Multivariable_Alexander_Polynomial">Multivariable Alexander</a>
            <li><a href="/wiki/Knot_Atlas:About">About</a>
            <li><a href="/wiki/Data:5_2/Bridge_Index">Data:5_2/Bridge_Index</a>
            <li><a href="/wiki/The_Mathematica_Package_KnotTheory%60">Knot Theory`</a>
            <li><a href="/wiki/How_to_Edit_this_Manual...">How to Edit Manual...</a>
            <li><a href="/wiki/Modifying_Knot_Pages">Modifying Knot Pages</a>
            <li><a href="/wiki/Special:Recentchanges">Recent changes</a>
            <li><a href="/wiki/To_Do">To Do</a> (history!)
            </ul>
          </div>
        </div>

"Redraw Page" tab

Add to MediaWiki:Monobook.js, following Wikipedia:WikiProject User scripts/Scripts/Add purge to tabs:

addOnloadHook(function () {
    var hist; var url;
    if (!(hist = document.getElementById('ca-history') )) return;
    if (!(url = hist.getElementsByTagName('a')[0] )) return;
    if (!(url = url.href )) return;
    addPortletLink('p-cactions', url.replace(/([?&]action=)history([&#]|$)/, '$1purge$2'),
                   'redraw page', 'ca-purge', 'Purge server cache for this page', '0');
});

Spam filtering

Follow the directions at http://www.mediawiki.org/wiki/Extension:SpamBlacklist

Tweaking the job queue

Running jobs seems to slow everything down horribly. It's essential that you add

$wgJobRunRate = 0.01;

to LocalSettings.php. It may prove necessary to add a cron job running /w/maintenance/runJobs.php, if the length of the job queue (see Special:Statistics) gets out of hand. This should only happen after edits to widely used templates.

In an attempt to stop the job queue from processing latex, Scott added a line to includes/JobQueue.php.

$options = new ParserOptions;
// turning off math rendering while processing jobs, because it's a terrible idea.
$options->setUseTeX( false );

Installing ParserFunctions

In order to use template-like parser functions, such as <false>, we need to enable the ParserFunctions extension, following the instructions at http://meta.wikimedia.org/wiki/ParserFunctions.

Download the files

and then add a line to LocalSettings.php:

require_once( "$IP/extensions/ParserFunctions/ParserFunctions.php" );

Installing a PHP cache

Look at http://meta.wikimedia.org/wiki/PHP_caching_and_optimization.

  1. Install PEAR, by saving http://go-pear.org/ as /home/wiki/pear/go-pear.php, then running php go-pear.php and following the prompts
  2. Install APC, by running /home/wiki/pear/bin/pecl install apc
  3. Add extension=apc.so to /etc/php.ini

You can see APC's statistics at http://katlas.org/apc.php

Add

# See http://www.mediawiki.org/wiki/Manual:%24wgMainCacheType
$wgMainCacheType = CACHE_ACCEL;

to LocalSettings.php.

Installing memcached

In my continuing quest for faster page loads, I'm trying memcached. Follow the nice instructions at http://blog.ajohnstone.com/archives/installing-memcached/.

Although it's not installed, it's not running, or being used. See http://svn.wikimedia.org/viewvc/mediawiki/trunk/phase3/docs/memcached.txt?view=markup for instructions for actually running it. You'll probably need to uncomment some lines in LocalSettings.php too.

Updating CSS and JS

Increasing PHP's memory_limit

In LocalSettings.php, change

ini_set( 'memory_limit', '20M' );

to

ini_set( 'memory_limit', '40M' );

Fixing "Random Page"

The standard MediaWiki random page algorithm is strange, and anyway unsuitable for our purposes. See Fixing "Random Page" for the details and for our fix.