Thursday, May 11, 2006

We should see other people

Lance adds more thoughts on the subject of rejection, and he talks about rejection letters. On this subject, Oded had interesting thoughts in an earlier essay. Oded was writing about papers rejected from conferences, but perhaps his point applies more broadly. If I may paraphrase, Oded says that a rejection letter should contain constructive criticism, as appropriate, but it should not explain the reason of the decision. In Oded's view (reiterated in his more recent essay), rejection of papers from conferences and decisions not to hire certain people are simply decisions about the allocation of scarce resources. Typically, they have no sensible rationale, and, in any case, they are not meant to be a judgement on the value of the paper or of the person. It is confusion between allocation decisions and value statements that creates much needless frustration, and it should be avoided.

Offering specific reasons for rejecting a paper or passing on a candidate is then, in Oded's opinion, a step in the wrong direction, because it makes the decision sound indeed more like a value judgement.

Making a job offer is, for a department, a significant long-term commitment, and so it is for a candidate to accept an offer. Deciding where to work greatly affects one's life, and deciding whom to hire has a great effect on a department's life as well. It is no wonder that the comparison to romantic relationships always comes to mind. Indeed, if one thinks of an academic job interview as a blind date set up by common friends, everything makes sense.

So, I was wondering, what does Oded's ideal rejection letter sound like when translated to the setting of romantic relationships? And then it hit me:
It's not you, it's me.

9 Comments:

  1. Anonymous Anonymous
    5/11/2006 06:07:00 AM

    I've always been much happier to get turned down with an explanation than without one (whether by a date or a STOC PC is irrelevant).

     
  2. Anonymous Anonymous
    5/11/2006 10:36:00 AM

    I don't understand why not to give a reason. Even if the reason is arbitrary or wrong, as long as the author/rejectee can recognize it as such then it provides useful feedback. (For example, maybe the Introduction to a paper needs to be changed to make the contribution more clear.)

    Furthermore, something lost sight of in this entire discussion (that applies both to candidate hiring and PC-member decisions) is that although some decisions are arbitrary, most are not. Failing to provide feedback so as not to hurt the feelings of a minority, actually ends up hurting the majority and gives a less efficient system overall.

     
  3. Anonymous Anonymous
    5/11/2006 11:04:00 AM

    Luca, I like your comparison. "It's me" is the worst rejection you can really get on a date or from a committee. It does not help you in self-improving, and it may crush your self-esteem. (This paper was so lousy we even did not care to comment on it.)

     
  4. Anonymous Anonymous
    5/11/2006 12:34:00 PM

    although some decisions are arbitrary, most are not.

    That's simply wrong.

     
  5. Anonymous Anonymous
    5/11/2006 03:01:00 PM

    is that although some decisions are arbitrary, most are not.

    If you read the dictionary definition of arbitrary, almost all are. If on the other hand what you wanted to say was: "some decisions are wrong, [but] most are not" then I would agree in the case of conferences and disagree in the case of interviews. Interviews are blind dates where you can get rejected for completely non-academic reasons e.g. he likely wouldn't stay here for long.

     
  6. Anonymous Anonymous
    5/11/2006 05:27:00 PM

    If you read the dictionary definition of arbitrary, almost all are.

    I found this definition: arbitrary: depending on individual discretion. (This is not typical usage of the word.)

    So, then, what I meant was: "some decisions are random, most are not." I am thinking in particular of conferences: the first half of papers that get cut are being cut for a reason; it's when you have to pick 30 accepted papers out of the top 40 that things get noisy.

    BTW, the same holds for faculty positions among all applicants (not necessarily among the tiny minority that actually get called for an interview --- there is a lot more randomness there).

    In any case, what I am trying to say is this: if a school were to tell me that I didn't get an interview because my publication record wasn't strong enough, I would not take that as an objective statement about the quality of my work. I would take it as an indication that maybe I need to publish more, or at better venues, or start getting those journal papers out, etc. And if a program committee tells me that they didn't understand what my paper was about, I don't take this as an objective statement that the paper is worthless or that I am a bad writer. I do take it as an indication that I better work on making the Introduction more clear, etc.

    I see no reason why an applicant/author would not want every piece of information they can get their hands on, if only to improve their application/paper the next time around.

     
  7. Anonymous Anonymous
    5/11/2006 06:03:00 PM

    the first half of papers that get cut are being cut for a reason; it's when you have to pick 30 accepted papers out of the top 40 that things get noisy.

    I agree.

    BTW, the same holds for faculty positions among all applicants

    I don't agree. For one, universities will routinely pass on top candidates that "weren't quite in the right area" as if this last was a requirement etched in stone and some internal made up goal.

    To be sure, if you have candidate A who is slightly weaker than candidate B but in an area where there is more need one should prefer candidate A. This is not what I'm talking about. I'm refering to truly outstanding candidates who get rejected for not being in the right area and instead much inferior researchers are prefered.

    A department which was truly concerned about academic matters would generally hire the best candidate each year, from any area, and let the laws of averages take care of the area distribution. After all what are the chances that you will get four top notch theory candidates in a row? And if you happen to do, then bingo you now have one of the top 10 theory departments in the country. I can think of a lot worse problems that a department might have.

     
  8. Anonymous Anonymous
    5/11/2006 09:07:00 PM

    ...then bingo you now have one of the top 10 theory departments in the country. I can think of a lot worse problems that a department might have.

    Apparently you haven't spoken to any of the systems faculty in my dept. They would consider the above a nightmare scenario. =)

     
  9. Anonymous Anonymous
    5/12/2006 02:23:00 AM

    The selection of candidates to interview for faculty positions is extremely noisy compared with conference reviewing. The ratios are typically well more than 20 to 1 between applicants and interview slots (40 to 1 may be more like it). Moreover, there is often very little overlap in 'reviewers' (letter writers and such) for candidates so side-by-side comparisons outside of the interview process itself are often impossible. The choice of whom to interview can be based in part who argues most forcefully about bringing candidates in or political considerations.

    Different institutions have different interests, priorities, styles, etc., and they see candidates under very different circumstances, with different interviewing practice, etc. On the other hand there can be a 'herd' mentality about whom to interview.

    It seems ridiculous to expect well-reasoned and objective feedback from such a flawed process.

     

Post a Comment

<< Home