2018.03.19 Dev Chat
: hi everyone
: a warm cozy 82 degrees here
: heya rhiju
: 59F here, California cold lol
: Wish I were there
: thanks for coming!
: Greetings, all.
: I keep telling myself spring is coming… Soon
: 8 and sunny in KY
: its 19 degrees F here
: glad to see so many of you here
: we’re at another super-interesting juncture for eterna
: Quite an exciting topic today
: a bunch of papers will be out with you (eterna players) as coauthors
: or even as lead authors
: I knew my time to recognize my brilliance was coming
: so here’s the deal –
: how do we best get feedback from players on these publications, which list their names & logins?
: how do we ensure that credit goes to players who contribute?
: and how do we especially honor players who have taken a lot of extra time to write papers?
: user handles are unavoidable for the sake of continuity with the eternagame side of things
: give them their ancient labcoin?
: haha yes
: well let me quickly lay out the papers
: lmao Jieux…
: there are two kinds
: (Working on it…)
: (Would just rather not break the site in the process)
: one style of paper is bring written by ‘experts’ here at stanford, but making use of contributions of players
: example: https://www.biorxiv.org/content/early/2018/01/31/227611
: that is a neural net method that was made possible through player solutions to openTB and switch puzzles
: ooh Cold Spring Harbor…nice
: Hm, here’s a thought - are the Eterna “certificates” still being updated? I’m guessing not
: and written by a student in my lab (michelle wu)
: @LFP6 yes, let’s remember that –
: Adrien is still listed as a lead researcher on them
: the other style of paper is being written by players themselves
: example is https://www.ncbi.nlm.nih.gov/pubmed/26902426
: curiosity peeked
: it’s good to see Fisker & Jander in the spotlight.
: and at least 2 more should be going out this year in that style. @omei is writing one on poly(A) SHAPE protections
: they deserve it for sure.
: and @brourd has recently promised me one on DMSSD
: with a little help from our friends
: and nando is writing one too. (and hekping brourd with DMSSD paper)
: these are particularly special papers
: thumbs up slinking away into the background to graph misc college lab data
: heya Brorud… happy slinking.
: no other scientific discovery project has led to publications written by citizen scientists
: though they should!
: It’s quite a powerful concept
: experts tend to suffer from tunnel vision
: furthermore, for the papers coming out this year, the only authors on the papers will be players (my lab may help write companion papers that test ideas)
: @cynwulf28 my suspicion is that other scientific discovery projects are not as well-structured to allow player-driven science
: eterna’s weird in it doesn’t just crowdsource tasks by ‘expert’ scientists
: also the community on eterna is super special
: I know that Foldit for example does not provide any data back to players
: nor is the publishing sceene usually citizen-scince friendly
: witness eternacon
: @jandersonlee your paper with elifisker etc. is helping change the way publishers think about citizen science
: my wife’s in special ed and she says that about me all the time
: @LFP6…that was led to migration from FoldIt to Eterna…that and my undergraduate research into RNA-Editing
: i know this based on conversations with editors of publications as well as peer reviewers
: will forthcoming papers be open to peer/player review and possible additions?
: @dizzywings that’s one of the 2 major topics
: hi masterstormer!
: maybe via forums or a google drive?
: the other major topic is what @jieux and @lfp6 pointed out about labcoin/certificattes/etc. – how do we give special recognition to player authors
: @dizzywings yes that’s a great idea
: I wouldn’t mind finding myself on NCBI…that Science Cred is always good
: how about this – each paper written by eterna players first goes on to a preprint server
: for ‘peer’ review?
: (that also would hold for papers written by ‘expert’ scientists that include eterna players as contributors)
: then we announce in game
: then we set up one forum thread for each paper
: I suspect you have a platform in mind
: and ask the lead author of the paper to respond to suggestions. in some cases, suggestions might require too much extra work, but the lead author points that out and player commenters accept it and/or offer help.
: Google docs allows selective sharing with comment privs
: @jandersonlee TY
: @cynwulf28 i was thinking of getsat
: @jandersonlee google docs is a fantastic idea.
: GDocs is nice because it allows you to suggest inline
: i agree
: maybe both as necessary
: yes both
: does it have to be selective sharing? perhaps we could start with just allowing anyone with the link to comment inline
: depends on prupbulishin requirements
: Doing a link share with comment privileges would work well
: good point jandersonlee
: @jandersonlee what sort of requirements were you thinking about?
: What about players like myself who do the labs but are not very sciency and will likely have nothing to offer to to paper-writing
: @jieux that’s no problem
: we want to just make sure there is a procedure to get comments from players.
: Re in-line comments, that has worked very well for the authoring phase. ut we had to convert to a more “standard” format (e.g. Word) to submit to bioRxiv.
: there are some players who already have made interesting suggestions on some of the papers
: Could GetSat be for general discussion and GDocs be for refined?
: In regards to the publishers themselves and their VERY specific requirements/demands…how would those demands that directly relate to the player-side writing the Doc transverse through Eterna HQ to us players?
: some journals require that they be the first public publication, which any with link might contradict if the link was published in a public forum like eterna
: @omei but probably we could keep a separate google doc version up
: maybe we could even advertise the google doc in the ABSTRACT for the preprint?
: is that too crazy
: or we could link to the getsat post or a link into eternagame.org that then lays out groundrules and has a link to the gdoc
: Or there’s some procedure to request a link
: Or a direct share even
: maybe the link to eternagame.org is in the biorxiv abstract
: Google Docs makes it easy to request access to a non-public link
: but only players who have logins to eternagame.org can see it
: Should be able to translate the current version back into Google docs, perhaps without taking the effort to redo-all the formatting.
: Good point @ja
: @omei right
: @rhiju point taken
: Maybe you have to be logged in to get the link? Which still makes it public because anyone can register for an account on Eterna?
: @Omei You should be able to import/export without too much trouble normally
: @omei what do you think of @ja’s idea of allowing requests?
: Depends on how much formatting was done
: That seems right.
: then, e.g. for the polyA paper, you’d have to OK requests to access.
: but that might be nice.
: a filter is a sound idea
: OK, sounds good.
: Question…In regards to the publishers themselves and their VERY specific requirements/demands…how would those demands that directly relate to the player-side writing the Doc transverse through Eterna HQ to us players?
: Should the link on the bioRxiv abstract be directly into the gdoc or into an eterna page?
: I doubt I’ll get inundated with requests. But if so, I can live with it. :-)
: @cynwulf28 the demands/questions probably wouldn’t go thorugh eterna HQ, but to the lead author of the paper
: @rhiju would that link be from an Eterna page?
: e.g., for the RiboNet paper posted above on bioRxiv, comments would be addressed (or rejected) by michelle.
: ah, ty
: @cynwulf28. the bioRxiv abstract would go to http://eternagame.org/current_papers/ (or something) and that page would have links to all active preprints and gdocs.
: @rhiju, again TY
: actually we could even include the annotated gdoc as supporting information for review of the paper when we submit it to a journal
: the question is whether we even need the https://eternagame.org/current_papers/
: as an intermediate
: I’m trying to follow so maybe you’ve sorted this out, but here’s my question: do you want it so that people can make a eterna account to participate but they haven’t really been playing they just want this part of it?
: That or a wiki page is a useful jump-off point
: that second paper has “eternal players” as an author mentions the EteRNA100 benchmark… are the “eterna players” listed in the paper? That would seem unwieldy
: a dedicated URL might be nice
: @Rhiju I don’t think you’d need to link to it from BioRxiv, but it would be nice for us players to view past and ongoing papers
: i guess the nice thing about having a central page inside eternagame.org is that we could also include a link to it in COMMUNITY in the menubar
: Players are listed in sipplemental data if I recall
: and someone going there would also see other papers that are under active review by players
: @jieux, you are listed, in the supporting files.
: great, my mama will be proud :)
: @dizzywings @lfp6 yes you said it
: confirming what @jandersonlee said
: check out supplemental material of the biorxic link
: I imagine if only a subset of puzzles or such should be used, that list of players should be limited to those involved with said puzzles?
: also check out jade shi’s sentRNA preprint:
: its on the physics preprint server arxiv
: @Jieux, as someone who has read hundreds of sdcience papers I can tell you that none of them are wieldy…they all are forced to be cumbersome due to the sheer weight of their content…usually found in the supplementary content
: That’s what the poly(A) paper does – contributors to the specific puzzles used in the paper.
: @Jieux *Science
: @Omei Did you generate that list by hand? Do you think we need some sort of standard script/util to help generate these?
: @lfp6 yes in each these papers the lead authors have made a call as to who to formally list. e.g. for jade’s paper on setnRNA he put the list of folks who contributed solutions that he actually used to train the neural net.
: Yeah I’m seeing that now
: I guess my question is that like getting the ‘lab’ section unlocked so you can participate, you are looking at something like that for this? I’m more arts and science savvy, not tech as much.
: of course the gdoc review process would allow players to suggest changes
: @rhiju, to hit on a early mention by yourself…what if players wish to have their real names included in a publication?
: to suggest changes in who is listed
: @LFP6 It would have been great to have that, for sure. Mainly because we took data from over 100 puzzles.
: one more thing on the review process:
: it would be nice to have someone (or a small group) of trusted players who could give a final stamp of approval from players
: I second the question about the option of real names
: most journals require a statement that ‘all authors ahve reviewed the manuscript and approve it’
: I’m certainly willing to do that
: @jandersonlee was also willing to take on that role
: I finally understand it all
: even if our names are just shoved in a supplemental file?
: both @jandersonlee and @cynwulf28 could perhaps be player representatives
: I may be willing to in the future when I’m farther along, maybe there can be an application process.
: and confirm to lead authors of papers that they are OK with things and that concerns from players in forums or in gdocs are suitably addressed
: @monarch that’s great.
: At some point we could have a profile field for “real name for publications”, but that probably wouldn’t be a high priority for the near term
: my guess is that these representatives would probably need to switch in and out
: please. :)
: it might actually be best to have a more formal nomination process at eternacon
: No, it’s a life term for Jeff. :-)
: On a tangential note, lroppy had brought up the point that it would be good to have some sort of player committee interacting with the dev team in terms of making sure player concerns and priorities are heard
: leave it to Lroppy to create a committee.
: There could be some relation with the nomination process
: *relationship in
: true, I do recall his comments
: player headed peer review committee
: yes, i have been waiting to hear from @lroppy for more on that – if any of you have an lroppy sighting tell him to repond to my e-mails
: ok, this has already been a superproductive community chat
: or not - different committees for different tasks
: the other topic(s) relate to recognition for players
: I think Lroppy is looking for funding for EteRNAcon South 3.0
: @cynwulf28 you brought up an interesting point
: I don’t remember if he had mentioned more specifics than that, but I remember it being an interesting thought. Will give him a nudge if I see him
: @jandersonlee Right, I intended to say the process may be similar
: some players would like these eterna publications to be associated with their ‘actual’ scientific authorial entities
: Is there a way for a player to find out if one of their puzzles were used in a paper?
: @monarch good point.
: i think in the gdoc-syle review, players can suggest this to authors if they didn’t do that already
: I think it would be good to have a regulated pipeline of authors specifying puzzles and/or designs used, which can be both reflected in the paper and on individual users’ certificates
: for jade’s sentRNA paper another player already contacted me and jade about having a list of the 724 training puzzles, their authors, and eterna IDs.
: I know that somehow I was in one of these, it might have been a viability of using citizen scientist but I know is that I was referenced a while back.
: @monarch great.
: upon any submission of info to a paper for review there could be an option like a checkbox/radio button to say, “use my player name” or “use my real name”?
: I think in the first paper, all Eterna users were used as coauthors
: @lfp6 you’re right
: I found myself buried in supplement 2 of the 2nd paper.
: here’s one proposal
: @Dizzywings I’d personally think having a field on the player profile for real name, then including both (via standard pipeline) would be the ideal
: first, for folks like @cynwulf28 who want publications to help them build their own scientific CVs,
: As well as Eterna UID
: we could ask players if they have ORCID ids
: this is a new system that is rapidly becoming pervasive in the science literature
: Oh cool, I have not yet looked into this .
: are user ID’s tied to UUID’s just a geek question @LFP
: I just don’t know if it was like questionnaire data or puzzles. If players feel like they are contributing more there will be more involvement.
: it allows a person to have a persistent numerical ID that they can associate with all their scientific efforst like papers
: @dizzywings UIDs (what you see in your profile link) are unique on Eterna
: *within Eterna
: we could allow players to associate their ORCID id with their profiles, if they want
: And usernames are unique, I believe
: thanks :) gotcha
: some may not want that – but those who do, can do it
: every number from 0 up has been used for player UIDs…
: @rhiju Agreed, definitely
: @cyn Well, not 0, that’s used for anonymous users :)
: then when a paper goes out with a bunch of player IDs, we allow the authors of the paper to also get any ORCID ids associated with those polayer names
: @cyn after you look into ORCID i’d love to hear what you think
: i bet you’ll come up with a better proposal than above
: @rhiju I’m guessing not, but could this be done retroactively?
: Can a ORCID be associated with an anonymous Eterna Name or only real names?
: @jandersonlee i don’t know
: Ie, I may not have a reason to be within ORCID now, but maybe I do later in my academic career
: They have explicitly said pseudonyms are OK, I think.
: @LFP6 yes
: i suspect that a person could set up two ORCIDs for their pseudonymous eterna presence and for their separate official scientific presence
: An you can associate aliases with an ORID I believe
: the choice would be up to them
: @ja alias association would be great
: If yo want to tie them together
: @ja Info on how such a retroactive process would work would be of interest to be
: I might need that for some of my 20+yr old papers
: @lfp6 as far as retrospective ORCID, i’m not sure if that’s possible. it might be actually.
: @rhiju do you know of an option to merge 2 ORCIDs?
: ORCID’s philosophy seems to be pretty well attuned with Eterna’s.
: @omei hmm yea
: @lfp i’m looking at my orcid https://orcid.org/0000-0001-7497-0972
: it does list papers that were published way before ORCID was even a thing
: so retrospective is possible
: It may only do it by name in the set of authors
: Rhiju did you want to bring up setting up Eterna as an ORCID organization?
: My concern would be if I don’t list my real name, PLUS I’m only in the supplemental material
: what might not yet be possible: journals list ORCID ids for authors on their papers. those might not update if a player later in life claims authorship via ORCID.
: That might make jandersonlee@Eterena easier
: but it might be possible in some journals
: @ja yes great idea
: I’d definitely want to look into it with ORCID and the respective journals
: I could see wanting to associate with real name/ORCID only later as a common occurence
: so that’s several ORCID-related ideas that i’d never considered. make eterna an ORCID consortium/organization; ask ORCID about retroactive authorship assignation
: very very interesting
: there’s an ORCID workshop happening in the bay area in May. @ja has graciously volunteered to go with me to ask these weird questions for eterna.
: I’m not sure how updating the supplemental material would (or would not) work
: On top of ORCID
: ORCID must have thought about some of these issues as there are papers from ‘big science’ areas like particle physics which have ~1000 authors
: OK, let’s move beyond ORCID
: BTW, lfp6 would you be able to record the chatlog and post?
: OK, beyond ORCID
: how about honoring players & contributors in the game?
: Rhiju: Yep, I have a window up waiting for a copy paste
: first idea, special badges
: broad topic …beyond badges and coin?
: @omei i got permission from sharif to show drafts of badges
: The badge mockups I saw looked great
: @lfp6 right
: @lfp6 can you post sharif’s mockups?
: here in chat?
: while lfp is doing that
: Uno momento - let me go throw them on Drive or smth
: I know that there are some badges which work through a progression which many players have supassed only to find that there are not more badges
: as evidenced by the badges on hoglahoo’s page: http://www.eternagame.org/web/player/36921/
: @cyn – actually anyway you coudl start a getsat thread and post those problems?
: we had some problems with badges, but sharif discovered technical fixes
: so we could correct those issues with badges, probably
: I believe he was referencing needing higher “levels” of some badges
: ah ok, let’s start a forum thread
: more badges should be easy.
: for the overly ambitious
: while lfp is getting the images up
: the idea is the following
: we have separate badges for each paper that comes out of eterna
: and they will be at three levels
: one for contributor (those listed in supplemental table)
: one for author
: cynwulf wants all his badges ready for when he achieves them :D
: and another for lead author (the person who actually organizes the writing and submission)
: Ok, here we go: https://drive.google.com/file/d/1daffIUECNNXCnHKbs329CdmlWYw05Jdu/view?usp=sharing
: so, e.g., jandersonlee would get a lead author, author, and contributor for the 2016 JMB paper
: Very pretty
: Those look great!
: This brings up another issue we’re running in to - the sheer number of badges that we have now
: the question then is do we honor paper authorship more than that.
: lab coin?
: And their more formal nature reflects the more formal nature of the contribution
: for example, do player’s default chat colors change with the highest authorship badge that they’ve earned?
: We need to compress a little bit, I have some thoughts that I’ll have to work out at some point
: should there be a separate ‘rankings’ in the list of players for not just ‘most richest’ but ‘most paper contributions’?
: maybe we can start a badge financial market boom…
: and for good lab results
: @Rhiju Re: Colors, there was a consensus at one point amoung a few that icons would be more optimal https://github.com/EteRNAgame/HTML-Chat/issues/14
: @lfp6 do we really have too many badges?
: @rhiju interesting thought for papers column
: can there be a certificate you can print and use as something you can add to your CV when applying to help in real labs? (I’m a university student)
: if you help author something
: @astro interesting idea. we currently have badges for # syntheses, but not much special for scores of syntheses.
: give some folks are colorblind, Icons would be terrer
: @monarch certification would be great
: @Rhiju I’d like to see badge “levels” grouped together, and maybe the author/lab submission badges could be grouped so that you see “10 badges of this type earned”, and then you click through to see the individual breakdown
: @lfp6 also brought that up above
: we actually had something like that, which has not been updated for a bit
: sharif and i discussed now bringing that back up to date
: here’s an example of my certificate: http://www.eternagame.org/certificate/20
: paper contributions are listed on the eterna cert
: Yeah, that needs to be brought back
: And maybe have a more nicely formatted version
: …as an option
: we would set up a proper database table in the game for papers accomplishments – to ensure badges, review, certificate update, etc.
: for a CV the recognized publications help (eg ORCID)
: @lfp6 yes for the certificate i think we can create nice graphics too with help from sharif
: @jandersonlee hey that’s an interesting idea – separate for ORCID
: Can’t wait for Eterna 4 when this process of adding stuff to the DB won’t be so much of a pain ^.^
: my id has 6 digits (:
: The look of the certificate could use some work, make it look more formal, but I think it would be a good thing
: yea Rhiju is 20…and he has those Legacy badges
: @monarch do yoi have an example?
: everyone i have to go
: Not handy but they shouldn’t be hard to do.
: @Rhiju: Re: A new point category. What I had suggested at one point was reworking the point system to be a more generalized statistic, including stats not just from puzzle solves and labs, but also papers, scripts, etc etc
: will look forward to more discussions on forums and PMs
: Perhaps kinda like StackExchange
: take care rhiju
: maybe. Thank you Rhiju for your time
: @lfp6 let’s discuss offline! and perhaps then make a concrete proposal in a community chat
: yes, thank you
: Thanks Rhiju, have a good one!
: thanks for the chat
: Thanks! great chat
: thanks everyone very much for attending. (monarch PM me links to certificates if you see any you like)
: ok, I will