Upload Queues

2008-07-13T18:34:44Z

ZelmoNrelb: delacelracd

cmoncnat
This page keeps track of the progress of various robots uploading data to the KnotAtlas.

If you look at the source of this page, you'll see a line that says <pre>{{:ScottDataRobot Queue}}</pre>This line 'transcludes' the page [[ScottDataRobot Queue]], showing its contents here. You can edit it to control the ScottDataRobot. You can create new queues by creating a new page in the same format. There's no need to transclude it here, but perhaps you should, so it's easy to see what's going on. To start uploading data in a queue, see the [[#Instructions]] section below.

==Broken queue items==
*"HOMFLYPT", "TorusKnots[20]"
*"HOMFLYPT", "TorusKnots[40]"
*"Kauffman", "TorusKnots[20]"
*"Kauffman", "TorusKnots[40]"
**For some reason the HOMFLYPT and Kauffman polynomials that get uploaded for TorusKnots are broken -- powers of a all replaced with $Failed^-1.

==ScottDataRobot Queue==
{{floating edit link|ScottDataRobot Queue}}
{{:ScottDataRobot Queue}}

==QuantumInvariants Queue==
{{floating edit link|QuantumInvariants Queue}}
{{:QuantumInvariants Queue}}

==Testing Queue==
{{floating edit link|Testing Queue}}
{{:Testing Queue}}

==Completed Work==
{{floating edit link|Upload Queues Completed Work}}
{{:Upload Queues Completed Work}}

==Instructions==
To upload data from a queue, you can either use an interactive Mathematica session, or a command line script.

===Command line===
In the <code>KnotTheory/scripts/</code> folder, you should find a script called upload_script.sh, and upload_script_nice.sh. These both do the same thing, but upload_script_nice runs in the background, with lower priority than anything else, survives after you logout, and writes its output to nohup.out, rather than the tty. Both scripts require three arguments:

<pre>
./upload_script.sh username password queue
</pre>

The queue should be the wiki name for the queue you want to use; something like <code>ScottDataRobot_Queue</code>. You must replace any spaces in the name with underscores. Optionally, you can specify a fourth argument, the path to your <code>math</code> executable, if it's not already on the path. This path must be terminated with a slash.

For example, to start uploading, I use
<pre>
./upload_script_nice.sh ScottDataRobot password ScottDataRobot_Queue /scratch/bin2/
</pre>

(If you're not sure how to kill a process you started this way, using ps and kill, you might just try <code>kill `ps -C MathKernel -o pid=`</code>. Strongly not recommended if you have another Mathematica session running! :-)

===Mathematica session===
Suppose you want to use the "YourDataRobot Queue". You'll need to load KnotTheory`, probably with <code><<KnotTheory`</code>, and then login, using something like
<pre>
CreateWikiConnection[
"http://katlas.math.toronto.edu/w/index.php",
"YourDataRobot",
"password"
]
</pre>
Now issue the command <code>ProcessKnotAtlasUploadQueue["YourDataRobot Queue"]</code>. Options <code>Repeat->numberOfRepeats</code> (defaults to infinity) and <code>Timeout->numberOfSeconds</code> (defaults to 12 hours) can be used to control how many items will be processed, and the maximum amount of time spent on each.

===Format of queue pages===
The queue pages should contain one line for each task, in the form
<pre>
*"Invariant Name", "Mathematica expression that evaluates to a list of knots"
</pre>
The expression evaluating to a list of knots is subjected to a very strict sanity check before it's evaluated, to protect against malicious code. At the moment, all you can use is AllKnots, AllLink, and Take. In particular, you can't write the presumably quite useful <code>Select[AllKnots[14], BraidIndex[#]<=4&]</code>.

Knot Atlas - User contributions [en]

Upload Queues