API for Server Data Queries: Difference between revisions

From Eterna Wiki

(Two labs gone missing from past_labs)
(Started outline documenting queries)
Line 1: Line 1:
<p>The intent for this page is to document the RESTful interface to the Eterna data.</p>
<p>The EteRNA servers implement a <a href="http://en.wikipedia.org/wiki/Representational_state_transfer">RESTfu</a>l API for much of the data behind both the game and lab.&nbsp; This data is available to both in-game scripts and browser-based toold</p>
<p>__TOC__</p>
<p>&nbsp;</p>
<p>&nbsp;</p>
<p>==Overview==</p>
<p>==Query types==</p>
<p>===type=active_labs===</p>
<p>===type=groups===</p>
<p>===type=lab===</p>
<p>===type=past_labs===</p>
<p>===type=pending_labs===</p>
<p>===type=puzzle===</p>
<p>===type=script===</p>
<p>===type=solutions===</p>
<p>===type=user===</p>
<p>===type=votes===</p>
<p>==Issues surrounding recent changes==</p>
<p>For the moment,&nbsp; this is focused on documenting the 9/26 changes that broke a bunch of existing scripts.</p>
<p>For the moment,&nbsp; this is focused on documenting the 9/26 changes that broke a bunch of existing scripts.</p>
<p>&nbsp;</p>
<p><strong>Update: Justin says he'll return the old API, to preserve compatibility with existing scripts.&nbsp; Thanks, Justin!</strong></p>
<p><strong>Update: Justin says he'll return the old API, to preserve compatibility with existing scripts.&nbsp; Thanks, Justin!</strong></p>
<p>Friday (5:50 PM PDT) update: My Data Mining Tool, which was broken by the change to the type=lab query, now works again.&nbsp; But I think Justin was probably especially careful about fixing the queries it uses.<strong>&nbsp; I advise anyone reading this to test any scripts they really care about, and if it is still broken, post here.&nbsp; Please describe the query URL, and what about the JSON is missing.</strong></p>
<p>Friday (5:50 PM PDT) update: My Data Mining Tool, which was broken by the change to the type=lab query, now works again.&nbsp; But I think Justin was probably especially careful about fixing the queries it uses.<strong>&nbsp; I advise anyone reading this to test any scripts they really care about, and if it is still broken, post here.&nbsp; Please describe the query URL, and what about the JSON is missing.</strong></p>
Line 9: Line 22:
<p>My favorite script by Stlne is broken. The one that states how many votes and design slots I have left in a lab. Here is its URL: http://eterna.cmu.edu/web/script/3126111/ and here is the message I get back: Return : jsonOut.data.labs is undefined</p>
<p>My favorite script by Stlne is broken. The one that states how many votes and design slots I have left in a lab. Here is its URL: http://eterna.cmu.edu/web/script/3126111/ and here is the message I get back: Return : jsonOut.data.labs is undefined</p>
<p><span id="docs-internal-guid-11878caa-7a0b-be1d-9337-e321c9b11e1e" style="font-size: 15px; font-family: Arial; color: #000000; background-color: #ffffff; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">-- [[User:Eli Fisker|Eli Fisker]] ([[User talk:Eli Fisker|talk]]) 16:42, 2 October 2013 (UTC)</span></p>
<p><span id="docs-internal-guid-11878caa-7a0b-be1d-9337-e321c9b11e1e" style="font-size: 15px; font-family: Arial; color: #000000; background-color: #ffffff; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">-- [[User:Eli Fisker|Eli Fisker]] ([[User talk:Eli Fisker|talk]]) 16:42, 2 October 2013 (UTC)</span></p>
<p>&nbsp;</p>
<p><span style="font-size: 15px; font-family: Arial; color: #000000; background-color: #ffffff; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">I confirmed this has been fixed.</span></p>
<p><span style="font-size: 15px; font-family: Arial; color: #000000; background-color: #ffffff; font-weight: normal; font-style: normal; font-variant: normal; text-decoration: none; vertical-align: baseline;">[[User:Omei|Omei]] ([[User talk:Omei|talk]]) 18:50, 16 October 2013 (UTC)</span></p>
<p>---</p>
<p>---</p>
<h3>type=past-labs: New fields are needed to access the thumbnail images<strong> (change)<br /></strong></h3>
<h3>type=past-labs: New fields are needed to access the thumbnail images<strong> (change)<br /></strong></h3>

Revision as of 18:50, 16 October 2013

The EteRNA servers implement a <a href="http://en.wikipedia.org/wiki/Representational_state_transfer">RESTfu</a>l API for much of the data behind both the game and lab.  This data is available to both in-game scripts and browser-based toold

 

==Overview==

==Query types==

===type=active_labs===

===type=groups===

===type=lab===

===type=past_labs===

===type=pending_labs===

===type=puzzle===

===type=script===

===type=solutions===

===type=user===

===type=votes===

==Issues surrounding recent changes==

For the moment,  this is focused on documenting the 9/26 changes that broke a bunch of existing scripts.

Update: Justin says he'll return the old API, to preserve compatibility with existing scripts.  Thanks, Justin!

Friday (5:50 PM PDT) update: My Data Mining Tool, which was broken by the change to the type=lab query, now works again.  But I think Justin was probably especially careful about fixing the queries it uses.  I advise anyone reading this to test any scripts they really care about, and if it is still broken, post here.  Please describe the query URL, and what about the JSON is missing.

Omei (talk) 01:30, 28 September 2013 (UTC)

 

My favorite script by Stlne is broken. The one that states how many votes and design slots I have left in a lab. Here is its URL: http://eterna.cmu.edu/web/script/3126111/ and here is the message I get back: Return : jsonOut.data.labs is undefined

-- Eli Fisker (talk) 16:42, 2 October 2013 (UTC)

I confirmed this has been fixed.

Omei (talk) 18:50, 16 October 2013 (UTC)

---

type=past-labs: New fields are needed to access the thumbnail images (change)

I note two related changes here, but I wouldn't consider asking Justin t change it for my scripts.  They seem like a reasonable step in moving forward.

  1. The thumbnail images have been moved to Amazon's S3 service.  The thumbnails don't exist at their old location.  This breaks scripts that displayed the thumbnails, like my took that creates the list of past labs here on the wiki.  But moving them to Amazon seems good, so I'm just going to update that script.
    Update: To my pleasant surprise, the thumbnails are back at the old URL on the Eterna servers, as well as thei new location in the Amazon cloud.  Not only that, but thumbnails for the newest labs have been added, too.  So I don't have to update the script right away.  For me, this is striking evidence that Eterna really is serious about supporting player-led developments.  A big thanks to the Eterna devs in general and (presumably) to Justin in particular.
  2. The format of the type=past-labs query (e.g. <a href="http://eterna.cmu.edu/get/?type=past_labs&skip=0&size=21">http://eterna.cmu.edu/get/?type=past_labs&skip=0&size=21</a> response has changed.  It may well be completely backward compatible; I'm not sure because I don't have a complete record of what it was.  But as it stands now, a solution has a field named "puzzles", which is an array of objects.  Each element of the array is an object which has a field also called "puzzles", which is also an array.  Each element of this array is a nid which seems to correspond to a secondary structure, because the query "https://s3.amazonaws.com/eterna/puzzle_mid_thumbnails/thumbnail<nid>.png" returns a thumbnail image of the target structure for that lab. I think this new level of indirection is there to support labs that allow sequences with differernt lengths and target structures. The reproducibility lab is one example, but I remember Rhiju talking about supporting this, long before the reproducibility lab came into being.

Omei (talk) 01:30, 28 September 2013 (UTC)

---

type=lab: Doesn't return anything meaningful for labs in last synthesis round

Apparently Justin restored the behavior of the lab query for existing labs, but didn't cary that over to new labs.  A query for a new lab returns

{"data":{"lab":null,"comments":[],"supercomments":[],"num_synthesized":0,"follow":[],"num_slots":false,"sum_picks":null,"num_solutions":"0","my_votes":null,"uid":"57675"},"memcache":true}

So any existing scripts that use the lab query are still broken for new labs.  I'll raise this issue with Justin.

Resolution: Justin pointed out that I was incorrect; the lab query for the latest round returns the same results it used to.  The cause of my confusion is that the in-game URLs can be misleading.  If you go to the past labs page and choose the Reprodicibility Lab, you'r be taken to the URL http://eterna.cmu.edu/web/lab/3376330/.  But 3376330 is not the lab ID; it is the project (a new construct)  ID.  To determine the lab ID, you have to click through to the browse page, whose URL is http://eterna.cmu.edu/web/browse/3293294/. Here we see the lab ID, and if you use that in a type=lab GET query, you'll get the results you expect.

Hopefully the project/lab terminolgy will get straigthened out.

Omei (talk) 23:45, 30 September 2013 (UTC)

----

Missing Semicirle - 2 bends and Semicircle - 5 bends in past labs.

This is a strange one.  I don't know that it is related to the recent server changes, but I'll note it here.

The two labs Semicircle - 2 bends and Semicircle - 5 bends, which were part of Cloud Lab 8, have disappeared from the past_labs query and replaced by Semicircle - 3 bends and Semicircle - 4 bends, which were part of Cloud Lab 9.  The easiest way to confirm this is to look at the <a href="/wiki/index.php5?title=List_of_EteRNA_Labs&diff=2932&oldid=2839">most recent diff</a> for the list of labs.  Search for "2 bends".  I verified that the GET query for past_labs isn't returning them, but the queries for solutions and lab still do.

Omei (talk) 01:57, 3 October 2013 (UTC)