(Editor’s Note:After readingGolf Digest’s May issue that included its biannual listing of America’s 100 Greatest Golf Courses, we draftedthe post below. When we realized it contained more questions than answers, we submitted those questions toGolf Digest. The magazine’s editor-in-chief Jerry Tarde responded via e-mail. Though we reserved the right to edit for brevity, Jerry’s answers here run in full. The editor didn’t need editing.)
Styled to a Tee: How do you get to be a "rater"? Write a letter? Know somebody at Golf Digest? Qualify with USGA handicap index? Inherit the position in a will?
Jerry Tarde: All of the above except the last. Some get suggested by raters, editors, people we know. Some people suggest themselves. We send them an application form, which asks about their skill level, golf experience and competitive success, knowledge of design, courses they’ve played and willingness/ability to travel. A key factor is where they live. If it’s New York or Florida, forget it. If it’s North Dakota, we’re listening. We try for geographic balance, so our panel covers the country. A 9-handicapper in Montana has a better chance of getting invited than a 3-handicapper in California, but generally we’re looking for accomplished amateur golfers with handicap indexes south of 5, extensive experience in playing good courses and a sense that they understand good architecture. And we’re not limited to amateurs; some professionals are on the panel, too. But no architects.
Styled to a Tee: Are there special qualifications or training?
Jerry Tarde: We used to avoid "training" because we didn’t want to influence panelists too much, engage in groupthink, beyond giving them aGolf DigestCourse Ranking Handbook that explained the seven criteria we use in evaluating courses. Over the years we came around to thinking that training would only make the process better, so we began a series of panelist summits about five years ago. We have national meetings with more than 100 participants and many guest speakers and we have regional meetings that are not so grand. This month we have a meeting in Palm Beach with about 60 panelists attending, a sellout. They listen to lectures by Ron Whitten, Topsy Siderowf, Dean Knuth, Bob Carney on the process. A wide range of speakers have shared their views — architects like Tom Fazio, Rees Jones, Tom Doak, Bill Coore, Pete Dye, Jerry Pate, Jack Nicklaus, Tom Weiskopf, Geoff Shackelford, Steve Smyers et al. The panelists typically play a course and rate it, and then engage in a postmortem with Whitten, who asks them to defend their votes. Between meetings, Dean Knuth, who developed the Slope Handicapping System over his 18 years with the USGA, reviews all the balloting and "rates the raters." Those panelists who are "extreme outliers," which is a mathematical designation based on their votes falling at least two points off the standard deviation, are identified for further trainingeither individually or through attendance at summits. We take the process very seriously and so do our volunteer panelists. It’s both a privilege and responsibility to serve on the panel.
Styled to a Tee: The number [of raters] appears to be growing, from 700-plus (two years ago) to 900-plus this year. Do you have an interest in adding raters? What would be an ideal group size?
Jerry Tarde: The number is about right. We’re always looking for good new raters, but I don’t see us growing over 1,000.
Styled to a Tee: Does a rater have to rate a minimum number of courses? (I ask this because if a rater filed one or two ratings every year, his outlook and basis for comparison would seem less than ideal.)
Jerry Tarde: Panelists are required to rate at least five courses a year. The average panelist rates nine courses a year. We also think it’s important that panelists have experienced as many 100 Greatest courses as possible to have a sense of perspective on "greatness."
Styled to a Tee: Do raters visit anonymously or do they call ahead and identify themselves? Or is there no policy? Do they pay their way or can they expect to play as a guest if courtesy is extended?
Jerry Tarde: Panelists must call in advance to arrange tee times, so they normally identify themselves as panelists unless playing as a guest of a member (and in that case they may be anonymous). They must presume that they will pay for the round, but usually green fees are waived as a courtesy. Panelists have ID cards that they present to the courses.
Styled to a Tee: Winged Foot East and West make the list. That club is famous for two things: hosting U.S. Opens and membership that includes many low-single-digit-handicap golfers. Surely, there are some raters in the club. Is there a policy regarding rating your home course or club?
Jerry Tarde: We have no policy against rating a course where you are a member or even an employee. Would you rather listen to the evaluation of someone who plays a course dozens of times or who has played it only once? Frankly, there’s a minimum of 45 ballots required for 100 Greatest and we exercise a mathematical formula to eliminate "extreme outliers" (see question #2 above), so the effect of one member is negligible. By the way, I’m a member at Winged Foot, but no editor including me is on the panel, so we don’t vote. I suspect there are members at Winged Foot who may be on our panel, but offhand, I don’t know of any.
Styled to a Tee: Why doesn’tDigest, in an effort to offer and achieve greater transparency, list or enumerate the number of raters who rate each course that is ranked? In other words, list the number be it 68 or 85 or 17 — who have played and rated the course in the last two years? Is there a minimum number of votes or ratings that must be qualified for a course to be ranked? Surely, if three raters rank a course in a two-year span, it wouldn’t qualify for 100 Greatest.
Jerry Tarde: In this past ranking period, we reduced the number of ballots considered from the last 10 years to the last 8 years, and I think that’s where it will remain for the foreseeable future. In other words, an evaluation lives in the database for eight years. Although looking at only the last two years would be more current, it would not supply enough evaluations for a fair judgment. We require a minimum of 45 ballots to become eligible for 100 Greatest. Once on the list, a course must have at least 30 ballots to remain eligible for future rankings. We have not made public the number of ballots in the database, but maybe we should.
Styled to a Tee: (Flip side of question above.) Why not publish (on the web) the number of courses each rater rates? For example: Rater No. 235, Austin, Texas: 16 courses.
Jerry Tarde: We have a policy against identifying the panelists. Years ago, we published their names in the magazine, but we found they were solicited especially by owners and architects of new courses. It became a nuisance, so we discontinued the practice. As for identifying panelists by a number and listing the number of courses they play, I don’t think it would add value.
Thank you.
