Upload Queues: Difference between revisions

From Knot Atlas
Jump to navigationJump to search
No edit summary
m (Reverted edits by ZelmoNrelb (Talk); changed back to last version by Scott)
 
(14 intermediate revisions by 2 users not shown)
Line 1: Line 1:
This page keeps track of the progress of various robots uploading data to the KnotAtlas.
This page keeps track of the progress of various robots uploading data to the KnotAtlas.


If you look at the source of this page, you'll see a line that says <pre>{{ScottDataRobot Queue}}</pre>This line 'transcludes' the page [[ScottDataRobot Queue]], showing its contents here. You can edit it to control the ScottDataRobot. You can create new queues by creating a new page in the same format. There's no need to transclude it here, but perhaps you should, so it's easy to see what's going on. To start uploading data in a queue, say "YourDataRobot Queue", you'll need to login, using something like
If you look at the source of this page, you'll see a line that says <pre>{{:ScottDataRobot Queue}}</pre>This line 'transcludes' the page [[ScottDataRobot Queue]], showing its contents here. You can edit it to control the ScottDataRobot. You can create new queues by creating a new page in the same format. There's no need to transclude it here, but perhaps you should, so it's easy to see what's going on. To start uploading data in a queue, see the [[#Instructions]] section below.

==Broken queue items==
*"HOMFLYPT", "TorusKnots[20]"
*"HOMFLYPT", "TorusKnots[40]"
*"Kauffman", "TorusKnots[20]"
*"Kauffman", "TorusKnots[40]"
**For some reason the HOMFLYPT and Kauffman polynomials that get uploaded for TorusKnots are broken -- powers of a all replaced with $Failed^-1.

⚫
==ScottDataRobot Queue==
⚫
{{floating edit link|ScottDataRobot Queue}}
⚫
{{:ScottDataRobot Queue}}

==QuantumInvariants Queue==
{{floating edit link|QuantumInvariants Queue}}
{{:QuantumInvariants Queue}}

==Testing Queue==
{{floating edit link|Testing Queue}}
{{:Testing Queue}}

⚫
==Completed Work==
⚫
{{floating edit link|Upload Queues Completed Work}}
{{:Upload Queues Completed Work}}

==Instructions==
To upload data from a queue, you can either use an interactive Mathematica session, or a command line script.

===Command line===
In the <code>KnotTheory/scripts/</code> folder, you should find a script called upload_script.sh, and upload_script_nice.sh. These both do the same thing, but upload_script_nice runs in the background, with lower priority than anything else, survives after you logout, and writes its output to nohup.out, rather than the tty. Both scripts require three arguments:

<pre>
./upload_script.sh username password queue
</pre>

The queue should be the wiki name for the queue you want to use; something like <code>ScottDataRobot_Queue</code>. You must replace any spaces in the name with underscores. Optionally, you can specify a fourth argument, the path to your <code>math</code> executable, if it's not already on the path. This path must be terminated with a slash.

For example, to start uploading, I use
<pre>
./upload_script_nice.sh ScottDataRobot password ScottDataRobot_Queue /scratch/bin2/
</pre>

(If you're not sure how to kill a process you started this way, using ps and kill, you might just try <code>kill `ps -C MathKernel -o pid=`</code>. Strongly not recommended if you have another Mathematica session running! :-)

===Mathematica session===
Suppose you want to use the "YourDataRobot Queue". You'll need to load KnotTheory`, probably with <code><<KnotTheory`</code>, and then login, using something like
<pre>
<pre>
CreateWikiConnection[
CreateWikiConnection[
Line 9: Line 54:
]
]
</pre>
</pre>
and then issuing the command <code>ProcessKnotAtlasUploadQueue["YourDataRobot Queue"]</code>.
Now issue the command <code>ProcessKnotAtlasUploadQueue["YourDataRobot Queue"]</code>. Options <code>Repeat->numberOfRepeats</code> (defaults to infinity) and <code>Timeout->numberOfSeconds</code> (defaults to 12 hours) can be used to control how many items will be processed, and the maximum amount of time spent on each.


===Format of queue pages===
The queue pages should contain one line for each task, in the form
The queue pages should contain one line for each task, in the form
<pre>
<pre>
*"Invariant Name", "Mathematica expression that evaluates to a list of knots"
*"Invariant Name", "Mathematica expression that evaluates to a list of knots"
</pre>
</pre>
The expression evaluating to a list of knots is subjected a very strict sanity check before it's evaluated, to protect against malicious code. At the moment, all you can use is AllKnots, AllLink, and Take. In particular, you can't write the presumably quite useful <code>Select[AllKnots[14], BraidIndex[#]<=4&]</code>.
The expression evaluating to a list of knots is subjected to a very strict sanity check before it's evaluated, to protect against malicious code. At the moment, all you can use is AllKnots, AllLink, and Take. In particular, you can't write the presumably quite useful <code>Select[AllKnots[14], BraidIndex[#]<=4&]</code>.


==ScottDataRobot Easy Queue==
⚫
{{:ScottDataRobot Easy Queue}}

⚫
==ScottDataRobot Queue==
⚫
{{:ScottDataRobot Queue}}

⚫
==Completed Work==
⚫
{{:Upload Queues Completed Work}}

Latest revision as of 00:02, 14 July 2008

This page keeps track of the progress of various robots uploading data to the KnotAtlas.

If you look at the source of this page, you'll see a line that says 

{{:ScottDataRobot Queue}}

This line 'transcludes' the page ScottDataRobot Queue, showing its contents here. You can edit it to control the ScottDataRobot. You can create new queues by creating a new page in the same format. There's no need to transclude it here, but perhaps you should, so it's easy to see what's going on. To start uploading data in a queue, see the #Instructions section below.

Broken queue items

  • "HOMFLYPT", "TorusKnots[20]"
  • "HOMFLYPT", "TorusKnots[40]"
  • "Kauffman", "TorusKnots[20]"
  • "Kauffman", "TorusKnots[40]"
    • For some reason the HOMFLYPT and Kauffman polynomials that get uploaded for TorusKnots are broken -- powers of a all replaced with $Failed^-1.

ScottDataRobot Queue

[edit ScottDataRobot Queue]
  • "Dowker-Thistlethwaite Code", "AllKnots[13]"
  • "Dowker-Thistlethwaite Code", "AllKnots[14]"
  • "Dowker-Thistlethwaite Code", "AllKnots[15]"
  • "Dowker-Thistlethwaite Code", "AllKnots[16]"
  • "HOMFLYPT", "AllKnots[13]"
  • "HOMFLYPT", "AllKnots[14]"
  • "HOMFLYPT", "AllKnots[15]"
  • "Rational Khovanov Polynomial", "AllKnots[13, NonAlternating]"
  • "Rational Khovanov Polynomial", "AllKnots[14, NonAlternating]"
  • "Rational Khovanov Polynomial", "AllKnots[15, NonAlternating]"
  • "Kauffman", "AllKnots[13]"
  • "Kauffman", "AllKnots[14]"
  • "Alexander", "AllKnots[13]"
  • "Alexander", "AllKnots[14]"
  • "Alexander", "AllKnots[15]"
  • "Signature", "AllKnots[13]"
  • "Determinant", "AllKnots[13]"

QuantumInvariants Queue

[edit QuantumInvariants Queue]


Testing Queue

[edit Testing Queue]


Completed Work

[edit Upload Queues Completed Work]
  • "Rational Khovanov Polynomial", "AllKnots[12, NonAlternating]"
  • "Kauffman", "AllKnots[12]"
  • "HOMFLYPT", "AllKnots[12]"
  • "HOMFLYPT", "TorusKnots[17]"
  • "QuantumInvariant/G2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A4/0,1,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A3/1,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "Alexander", "AllKnots[12]"
  • "QuantumInvariant/A1/6", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A1/6", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A3/1,0,1", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A1/2", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A1/2", "Select[AllKnots[{3,11}],First[BR[#]]<=4&]"
  • "QuantumInvariant/A1/8", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A3/1,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=4&]"
  • "QuantumInvariant/A3/1,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A3/0,1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/B2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A1/1", "Select[AllKnots[{3,11}],First[BR[#]]<=4&]"
  • "QuantumInvariant/A1/3", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A4/0,1,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "Signature", "AllKnots[12]"
  • "QuantumInvariant/A2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=5&]"
  • "QuantumInvariant/A1/1", "Select[AllKnots[{3,11}],First[BR[#]]<=5&]"
  • "QuantumInvariant/A3/1,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=5&]"
  • "QuantumInvariant/A2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/B2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A2/2,0", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/D4/0,1,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A1/2", "Select[AllKnots[{3,11}],First[BR[#]]<=5&]"
  • "QuantumInvariant/A1/1", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A4/1,0,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "Determinant", "AllKnots[12]"
  • "QuantumInvariant/A1/7", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/B2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=4&]"
  • "QuantumInvariant/A1/1", "Select[AllKnots[{3,11}],First[BR[#]]<=6&]"
  • "QuantumInvariant/D4/1,0,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=4&]"
  • "QuantumInvariant/A2/1,1", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/D4/1,0,0,0", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A1/5", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/G2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A1/1", "Select[AllKnots[{3,11}],First[BR[#]]<=7&]"
  • "QuantumInvariant/A3/0,1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A3/1,0,1", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A1/1", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/G2/0,1", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A1/5", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A2/2,0", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A1/4", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A1/2", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/B2/0,1", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/G2/1,0", "Select[AllKnots[{3,11}],First[BR[#]]<=4&]"
  • "QuantumInvariant/A1/3", "Select[AllKnots[{3,11}],First[BR[#]]<=4&]"
  • "QuantumInvariant/A1/4", "Select[AllKnots[{3,11}],First[BR[#]]<=4&]"
  • "QuantumInvariant/A1/4", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A1/3", "Select[AllKnots[{3,11}],First[BR[#]]<=2&]"
  • "QuantumInvariant/A2/1,1", "Select[AllKnots[{3,11}],First[BR[#]]<=3&]"
  • "QuantumInvariant/A1/2", "Select[AllKnots[{3,11}],First[BR[#]]<=6&]"

Instructions

To upload data from a queue, you can either use an interactive Mathematica session, or a command line script.

Command line

In the KnotTheory/scripts/ folder, you should find a script called upload_script.sh, and upload_script_nice.sh. These both do the same thing, but upload_script_nice runs in the background, with lower priority than anything else, survives after you logout, and writes its output to nohup.out, rather than the tty. Both scripts require three arguments: 

./upload_script.sh username password queue

The queue should be the wiki name for the queue you want to use; something like ScottDataRobot_Queue. You must replace any spaces in the name with underscores. Optionally, you can specify a fourth argument, the path to your math executable, if it's not already on the path. This path must be terminated with a slash.

For example, to start uploading, I use 

./upload_script_nice.sh ScottDataRobot password ScottDataRobot_Queue /scratch/bin2/

(If you're not sure how to kill a process you started this way, using ps and kill, you might just try kill `ps -C MathKernel -o pid=`. Strongly not recommended if you have another Mathematica session running! :-)

Mathematica session

Suppose you want to use the "YourDataRobot Queue". You'll need to load KnotTheory`, probably with <<KnotTheory`, and then login, using something like 

CreateWikiConnection[
  "http://katlas.math.toronto.edu/w/index.php",
  "YourDataRobot",
  "password"
]

Now issue the command ProcessKnotAtlasUploadQueue["YourDataRobot Queue"]. Options Repeat->numberOfRepeats (defaults to infinity) and Timeout->numberOfSeconds (defaults to 12 hours) can be used to control how many items will be processed, and the maximum amount of time spent on each.

Format of queue pages

The queue pages should contain one line for each task, in the form 

*"Invariant Name", "Mathematica expression that evaluates to a list of knots"

The expression evaluating to a list of knots is subjected to a very strict sanity check before it's evaluated, to protect against malicious code. At the moment, all you can use is AllKnots, AllLink, and Take. In particular, you can't write the presumably quite useful Select[AllKnots[14], BraidIndex[#]<=4&].

Retrieved from "https://katlas.org/index.php?title=Upload_Queues&oldid=1691790"