Sure you do! Check out this new book (via Inside Higher Ed) by Michèle Lamont, How Professors Think: Inside the Curious World of Academic Judgment. The author, a sociologist, was permitted to observe the review of grant applications at several different and prominent funding agencies, and she has concluded that:
As for excellence, that quality that peer review theoretically promotes, Lamont isn’t so sure it exists. It may be invoked all the time, she said in an interview, but her examination of the process suggests no way to measure it. “I think excellence means nothing,” she said, suggesting that panels be honest about the criteria they use. “I think you have to give the criteria. Typically it’s originality, feasibility, and also the social and intellectual significance.” There is nothing wrong with those definitions per se, she said, but people shouldn’t pretend they equate with some scientific measure of excellence, as other criteria could be used as well.
The most common flaw she documents is a pattern of professors applying very personal interests to evaluating the work before them. “People define what is exciting as what speaks to their own personal interest, and their own research,” she said.
That’s probably not so surprising–most of us who win grants probably didn’t write proposals that were all that much better than the unfunded proposals–we just got luckier in terms of who read our applications and helped move them on up the line. There are an awful lot of smart, hardworking people out there (like my commenters!), and there aren’t all that many grant programs or fellowships. Perhaps more interestingly, Lamont found that “professors in different disciplines take very different approaches to decision making. The gap between humanities and social sciences scholars is as large as anything C.P. Snow saw between the humanities and the hard sciences.”
Many humanities professors, she writes, “rank what promises to be ‘fascinating’ above what may turn out to be ‘true.’ ” She quotes an English professor she observed explaining the value of a particular project: “My thing is, even if it doesn’t work, I think it will provoke really fascinating conversations. So I was really not interested in whether it’s true or not.”
In contrast, Lamont quotes a political scientist on what he values in proposals he reviews: “Validity is one, and you might say parsimony is another, I think that’s relatively important, but not nearly as important as validity. It’s the notion that a good theory is one that maximizes the ratio between the information that is captured in the independent variable and the information that is captured in the prediction, in the dependent variable.”
Uhhh. . . yeah. Who the hell talks about their research this way? I don’t know if she quoted this guy because she (as a social scientist) admired his clear and precise language (hint: I’m kidding here!), or because she thought his statement is laughable on its face, but I’d like to stand up for supporting interesting projects that may or may not lead directly to the exact research findings and arguments as laid out in the proposal. Yes, I value the “fascinating” over the “true,” because I assume that we’re all adults here, and that we understand that the nature of “truth” is contingent and consensual. I don’t go looking for “truth” in scholarship–just for arguments that are backed up by deep and creative research and due diligence with the historiography. It is entirely responsible and reasonable to change one’s argument as one completes more research–in fact, that’s the only honest way for a scholar to proceed–even if it doesn’t “maximize the ratio between the information that is captured in the independent variable and the information that is captured in the prediction, in the dependent variable.” People who ask “fascinating” questions tend to come up with fascinating answers, even if they’re not the ones they thought they’d come up with originally. And that’s the “truth.”
This just in last night from the Historiann mailbag, from Jen Kirkman, the comedian who narrated 
