The hidden nature of peer review makes it hard to know what is normal. Do I review more than others? How detailed am I expected to make my review? Do others recommend rejection as often as I do?
Today we’re releasing a feature that answers these questions and more. Publons reviewer profiles now show a range of metrics and graphs (eg) that highlight interesting characteristics about review activity, and compares these against the average of the reviewer’s university and all Publons users.
These include reviewer acceptance rate (i.e. how many of the reviewed manuscripts have been published), reviews per month, the average word count of reviews, and the days-of-the-week distribution (interesting fact: people review on Sundays twice as much as they do on Fridays).
You can access your own stats by clicking on your profile image and ‘View profile’ from any page on Publons, or by signing up if you haven’t already.
These are the first in a series of metrics and graphs we are releasing over the next month to provide greater insight into peer review. Stay tuned for more next week!
Exciting news! We are piloting a partnership with Wiley to give Wiley reviewers official recognition for peer review work. This partnership creates a direct feed between Publons and select Wiley journals, allowing you to have your reviews for these journals automatically added to your Publons…
[

For those of you that don’t know, Publons, the company I co-founded, was part of the 2013 Lightning Lab cohort.
Last week I had the pleasure of joining Rollo Wenlock (CEO of Wipster) at the Lab to talk to the current batch of companies. We discussed what it’s like to pitch at demo day and then go through the due diligence process on your way to raising money.
I don’t feel I was able to fully convey how much of a distraction that process is. We talked about it within Publons and Daniel (my co-founder) pointed out that it’s evident in our git commit logs.
So we went back and took a look. We found that during the three months of the lab we made a cumulative 317 commits; in the month we were raising money we made 23 – roughly as many as we made in the two weeks around Christmas.
We effectively stopped working on the product for that month. With the benefit of hindsight, I don’t think that was avoidable, at least not for the founding team. The fact is we still had a lot of work to do to figure out who we were and where we were going. The diligence period helped us to do that; a good group of investors provides a lot of value.
At the same time, it does speak to a potential weakness in the Lightning Lab model. What happens after demo day and before the money is in the bank? If we’d had the resources to build a team during that period then perhaps we’d be further along today.
Just a thought…
It’s always fun to share what you’ve learned. For the record, here are the points I touched on:
At Publons we strive to give peer reviewers credit for all their peer review work, including the peer review they do for journals. To that end, we’re very excited to announce an official partnership with the innovative open-access publisher PeerJ.
All reviews submitted to PeerJ can now be…
Ever wondered what life at Publons looks like? Well, wonder no more!
Here’s the first of what will be an ongoing series of posts about our time here - these particular images were taken by our awesome new designer Rudi Theunissen.
Enjoy!
This talk was originally delivered at the NZAS conference in April 2012.
When we ask if emerging scientists have a future in New Zealand I think we’re really asking a few separate, but related questions. I sometimes wonder, especially in the context of a conference like this, if we’re making implicit assumptions about the answer to each of them, so I’m going to try to separate them out and answer them individually. First, we’re asking if emerging scientists actually want a future in New Zealand. Assuming they do, we’re asking if they can actually have a future here. Finally, we’re asking how we can help to make it happen.
We can ask those questions in a couple of different ways. We can ask if, in terms of public policy, we want emerging scientists to have a future in New Zealand, if can we make that happen, and how to ensure it happens. We can also ask those same questions on a personal level. Do emerging scientists, like you or I, actually want a future here? And if so, can we make it happen?
Let’s talk public policy for a minute. I’m sure that at some point today someone has presented a set of numbers and figures to the effect that an emerging scientist is a public good; each additional scientist in New Zealand confers some large (but hard to measure) benefit to the country as a whole. I’m sure they also provided evidence that the cost of convincing scientists to stay is much less than the return. If you haven’t heard that here today, then you will have heard it elsewhere. Paul Callaghan, for example, discussed this extensively in Wool to Weta.1 I don’t disagree. Numbers never lie. However, one thing I rarely hear discussed is the opportunity cost. Scientists are smart people, and they would be smart people even if they weren’t scientists. When we convince someone to work in the basic sciences we’re also convincing them not to work on something else. What, I wonder, are we giving up?
I don’t have an answer to that. It’s just a thought.
What I actually want to discuss today is the personal side of all this. So you understand where I’m coming from I’m going to tell you a bit about myself. I was born in South Africa and moved here when I was nine. I grew up in a town called Whakatane and moved to Wellington for university. I stayed for a PhD and then moved to Boston University to do a postdoc. Last year I quit my job and moved back here to start a company.
Technically that probably means I’m not an emerging scientist anymore, but I do know what it’s like to leave New Zealand, and what it’s like to return. Also, I am working in a related field. Let me tell you a little about my company, Publons.2 Basically, we scientists prove our value with what we publish. A good publication record is the bedrock for any job or grant application. I’m sure you all agree that system is not perfect but, like democracy, it’s the least bad system we’ve tried. Actually that’s not true. We haven’t tried any other systems. The basic technology we use to publish and review scientific research hasn’t really changed in the three hundred plus years since the Royal Society started the first journal.
Publons is an attempt to change that. It’s an experiment at allowing you to rate, review, and discuss any publication in any journal in one place, and an attempt to decouple your reputation as a scientist from journals and journal articles. I want to emphasize the experimental part of this project. We really are trying to approach this as a way to test a hypothesis. I doubt that would come naturally to me without a background in science. In that sense it’s an example of how it’s possible to do science without doing “science”.
Anyway, publons.com. Please take a look. If you have any thoughts or suggestions then I’d love to hear them.
Ok, with that out of the way, let’s get back to the original question: do emerging scientists want a future in New Zealand. I don’t, not particularly. New Zealand is a beautiful place, and Wellington is a great city. My family is here and a lot of my friends are here; I’ll always spend time here. However, New Zealand is a beautiful place in a world full of beautiful places, and Wellington is a great city in a world that contains Copenhagen, Barcelona, Cape Town, San Francisco, Boston, Chicago, New York, Shanghai, London, and Tokyo. In an ideal world I’d get to experience living in all of those places. And in a log cabin in the mountains of Montana.
However, I’m willing to accept that I’m in the minority. A good friend of mine just moved here because he fell for Wellington during a visit. I work with some people that moved back here from Hong Kong specifically because they wanted to live in Wellington. There seem to be a lot of emerging scientists that do want a future in New Zealand.
My only point is that it’s up to you to decide on your future. It’s ok to want to leave, and it’s ok to want to stay.
For those of you that want to stay, can you? If science is your priority then the answer is probably no. My postdoctoral work focused on soft x-ray spectroscopy. Basically, that means I shot x-rays at various materials and looked at what came out. To do that you need a synchrotron, which is basically a particle accelerator, and costs on the order of 100 million dollars to build. I could not have come back to New Zealand if I’d wanted to keep doing that.
There is no chance you’ll ever be able to do that kind of science in New Zealand. Nor is it likely that we’ll attract many superstars. If you want to be surrounded by the very best scientists and students then you need to be in Boston, or the Bay Area, or somewhere like that. I’m sure that there’s been plenty of talk about virtual networks today, but you can’t beat physical presence.
However, the answer is also yes. If staying in New Zealand is your priority then there are plenty of things you can do here. I’m not going to list them; I trying to make a more general point. If we want to claim that we are brilliant scientists, capable of pushing forward the boundaries of human knowledge, then surely we are also capable of finding a way to stay in New Zealand. There’s been a lot of talk today about how we need help from businesses, universities, and government. If you really want to stay in New Zealand then perhaps you should ignore that and figure it out for yourself. You might not have the same choices of what to work on, but it’s certainly possible.
Don’t get me wrong, I’m not saying that’s easy.
I’ll never forget what it was like to wake up in the days after quitting my postdoc. There is nothing you have to do; you’re completely free to do whatever you want. No boss, no papers to write, no requirements. Sounds great, right? It’s actually terrifying. What I’ve come to realize is that freedom is a scary thing. Having a job, a PhD to complete, a talk to present, these things give you validation.
A couple of years ago William Deresiewicz gave a talk to the plebe class at West Point. The title was “Solitude and Leadership”.3 The transcript is definitely worth reading; you’ll find it in The American Scholar. In it he makes the case that leadership is not about being popular with people, but that leadership is about setting a course that is true to what you believe; it’s about solitude.
The thing is that as we’ve trained to become expert scientists we’ve also trained to become experts at meeting the requirements of a bureaucracy. We go to school and learn how to pass tests. We graduate and learn how to get papers accepted into journals. We start working and learn how to write funding proposals.
I wonder if we shouldn’t also try to train leaders, people that can imagine something beyond the bureaucracy. Maybe that sort of creative thinking is what New Zealand needs most of all.
Thanks.
Thanks to Daniel Johnston for reading a draft of this talk.
Wool to Weta: Transforming New Zealand’s Culture - Paul Callaghan; Auckland University Press 2009. ↩︎
Solitude and Leadership - William Deresiewicz; The American Scholar 2010. ↩︎
Those who assume hypotheses as first principles for their speculations…may indeed form an ingenious romance, but a romance it will still be.
Roger Cotes, 1713
I wrote a manifesto once. It asserted that the world around us is explained by societal evolution through natural selection.
The rough argument is that societies can have different cultural norms. These norms are the primary determinant of societal wealth. Richer societies have the resources to grow and displace others. Therefore our culture, which displaced many others, is one that is optimized for wealth generation.
So what makes us rich? Trade. Trade allows us to work together, to specialize, and to gain from our comparative advantages.
At its simplest, trade is an agreement between two parties to co-operate on some matter. After the agreement each party has a choice about whether to keep their word or cheat. In that sense, trade is equivalent to the Prisoner’s dilemma.
The Prisoner’s dilemma is interesting because the (Nash) equilibrium outcome (mutual cheating) is not one of the (Pareto) efficient outcomes. Societies that succeed at changing the equilibrium outcome are able to trade more and get richer.
Historically, we’ve found a number of different solutions, which are basically different ways of turning the one-shot trading game into an iterated Prisoner’s dilemma.
The limit of each solution tends to be the number of people we can reliably trade with – the size of our sphere of trust. As technology advances new solutions become available. Our sphere of trust increases, the economy expands, and previous solutions fade in relevance.
That’s the basic idea. Next up: a history of the system of the world…
(via mermaidsbones)
theprestonator writes:
In todays day and age, ones job seems almost inseperable from ones self. Most times when someone meets you for the first time one of the first questions they will ask is: what is your job?
This fact, and the crippling lack of attention payed to the necessities of life (mostly because they are handed to us on a platter) means that job choice is one of the seemingly biggest decisions you make in terms of your place in society.
This pisses me off. I want to try loads of different shit, I want to be a musician for a month, then maybe next month I’ll be a teacher… Then maybe a month later I wont work.
Alot of people who give you advice against this kind of lifestyle will say your not thinking in the long term, don’t you want a comfortable lifestyle?
Man, fuck your comfortable lifestlye. I think this kind of thinking needs to be nurtured, not knocked down. Theres too much out there to commit to one thing for too long. Perhaps I’m a special case, maybe I’m way to impatient or have some deep seated fear of commitment but new shit just grabs my attention more than old shit.
Here’s what I think.
When someone asks you what you do you may hear them asking what your job is, but often they’re just trying to find what makes you interesting. For better or worse you spend a substantial fraction your time focusing on your vocation so it is a good proxy for what makes you interesting.
Sure, the world is full of interesting people and places but there is more to life than novel experiences. The problem with doing something for just a month is that while it is interesting it doesn’t really help you to develop. Anyone can be a musician for a month but it took The Beatles a lot longer than that.
When you decide what to work on you’re making a decision about your personal development — the direction you want to grow, the skills you’re going to develop, and the options you’re going to have in the future. You’re choosing the person you will become.
Experiences and development are not mutually exclusive either. I’ve found that interesting experiences tend to follow from taking on tough challenges, so optimise for that. Travel is fun and you should do it, but know that true fulfilment comes from working on hard things and eventually succeeding at them.
What would you work on if you didn’t have to worry about money? Your career should be a chain of things that were the most interesting things you could possibly work on at the time. That’s why you should start working on something. It is a commitment, and I always worried about that, but it’s less of a commitment than you think. You can always move on to something else when you figure out what it is, but in my experience one opportunity tends to lead to the next.
In summary, I think you should try all of those things you want to try, but I also think you should find ways to make them a side effect of your personal development — a side effect of doing the things that make you interesting. Life is short and you only get one chance; we don’t get enough time to do all the things we want to do, but waiting for the best option to come along is just another way to waste time.
I’ve been meaning to start a blog for a while but never quite had enough to say.
That’s strange because I have an opinion about everything and I do write a lot. A lot of email in particular.
Then my brother started blogging and I found myself wanting to respond to every post. That’s when I realised that I write best when I’m writing to someone.
It’s easier.
It’s easier to respond than it is to initiate. It’s easier to critique than it is to create. It’s easier to give advice than it is to build. And it’s certainly easier to give a knee-jerk reaction than any sort of original action.
Might as well accept it…