Policy and planning committee |
Minutes of the Faculty of Arts and Sciences Meeting October 17, 2013 A DISCUSSION OF THE ACADEMIC REVIEW PROCESS Jack Snyder, chair of the Policy and Planning Committee, called the group to order shortly after noon in 555 Lerner. About 50 people were present. Introduction. Prof. Snyder introduced discussion of the theme of the meeting by saying the vision of the A&S Academic Review Committee process is a triumph of hope over experience. The hopeful message is that the ARC process affords an opportunity to think holistically about what an academic department does, with a close look from both the inside and the outside, leading to recommendations that may enable administrators to allocate additional resources. The voice of experience warns that these reviews include an onerous self-study, and can result in calls from external cheerleaders for more faculty when there are no funds to hire them. The follow-up is often weak, the recommendations unimplemented. Prof. Snyder asked how the ARC process can be done better, with less suffering. He outlined four ideas, partly his own, partly a PPC effort. 1. Lighten the burden of the initial self-study by asking the A&S administration to do more of the data analysis. 2. Improve the timing of reviews, by giving priority to departments that are nearing some kind of turning point. 3. Make sure there is a preliminary three-way conversation among the department chair, the dean, and the ARC chair about the key issues to take up in the impending review. 4. Focus ARC’s attention on a holistic consideration of issues usually considered piecemeal. Budget decisions about, say, adjuncts and faculty lines should be made together, with full consideration of the trade-offs. These ideas might strengthen the opportunity that the ARC process offers, Prof. Snyder said—the chance for hope to overcome experience. Remarks from David Madigan, EVP for Arts and Sciences. EVP Madigan referred to a graph he had distributed at the start of the meeting showing the number of ARC reviews completed in each of the last 17 years. EVP Madigan said the process is clearly burdensome, with many reviews needing two years to complete, with only one or two reviews completed in most years. He said the process has to be streamlined, and that his office is prepared to provide data needed for self-studies. He said the current procedure, which waits till internal reviews are done before inviting external reviewers in, is unnecessarily time consuming. He said the scheduling should be changed, though each change also requires care. Remarks from former ARC chair Robert Shapiro. Prof. Shapiro recalled his stint as chair of the A&S executive committee a decade ago, when the ARC process was a gung ho operation recently launched by A&S VP David Cohen. He recalled one major ARC review from that period, led by Charles Tilly on the natural sciences. He said that was also a transitional period, with the funding model in the Ph.D. program being overhauled, and the expectation that new junior faculty hired would go through reviews with the possibility of tenure. He recalled that the process could also be very slow. He recalled a self-study of his own department that had to be postponed when he was chair. He said a rethinking of the review process is in order, with particular attention to the timing of reviews, and to advance preparation involving department chairs. He said the external reviews may vary in importance. The requirement of an external report can be a source of huge delays, which can be particularly vexing for departments poised to move up in the ranks. Remarks from current ARC chair Thomas DiPrete. Prof. DiPrete recalled that in 2004 he wrote the self-study for Sociology’s ARC review. He wanted to add without sounding defensive that ARC had accomplished a good deal the year before, completing reviews of Slavic Languages, Music, Chemistry, Anthropology, and Earth and Environmental Sciences (DEES). ARC also planned external reviews for three units that had completed self-studies. He said he agreed with much of what the previous speakers had said, adding that it’s worth keeping variance in mind, along with the mean. Some reviews are more useful than others, some departmental visions clearer than others. Prof. DiPrete said he found the process valuable, with a wide range of voices heard, but it was also burdensome. He agreed that statistical information should be kept current for departments by Arts and Sciences. It’s much easier for a department to focus on its 25-page narrative if it does not also have to produce the extensive appendices that have in recent years been part of departmental self-studies. He said external reports vary in value. Sometimes they are excellent and so the internal report largely repeats the recommendations of the external report. The external report is completed before the internal report is completed, and the internal committee always looks at the external study before it makes its own report. Sometimes the external report has shortcomings. The goal of the internal report is to combine the insights of the external committee with the greater knowledge of the university available to the internal committee to produce an internal report that leads to a productive final report by the ARC The slow parts of the process are the self-study, especially the appendices to the report, and the scheduling of the visit by the external review committee. Discussion. James Zetzel said there are huge differences among types of information required for different departments. There is a variety of needs, and ways to meet them. He said it is a mistake to review teaching in terms of departments. In a time of finite resources, more review committees can deal with cross-departmental issues. He said the allocation of resources is crucial, and a more flexible forum is needed for this purpose. It would be useful, for example, to have internal reviews that cover larger areas. Prof. Snyder said some larger issues, like Prof. Zetzel’s suggestion of an overlap of teaching reviews among multiple departments, might be profitably taken up by the Educational Policy and Planning Committee. Susan Pedersen, chair of the EPPC, said she had taken part in only one review, about a decade ago, as a member of the internal committee. That review had taken a very long time, and it struck her as strange that her group had been expected to revise its report several times and to incorporate what they chose from the externals' report, rather than simply letting the external report stand independently. She was puzzled about the purpose of all this effort. Was it to write the most beautiful report? Since the most valuable information the ARC process can provide is insight from prominent scholars outside Columbia about how Columbia departments compare within their disciplines, she suggested that it might make more sense to dispense with the internal committee and just to have an external visiting committee, as is common among our peers, and then to concentrate on figuring out what to do with their advice. The weight given to the internal committee means the process tends to be too much about Columbia norms and culture and not enough about how we look nationally and internationally, and how to address any problems. EVP Madigan said his own experience as an external reviewer was that there seemed to be no corresponding internal reviewers. Christopher Peacocke said that the last external review in which he participated (at Harvard) had five external members, and his experience was that the comments of each reviewer had an effect on the reviewed department’s policies. He said it is important to have a large number of external perspectives, from experts with diverse areas of expertise and concern (curriculum, graduate program, ways of encouraging the best research, and so forth). Patricia Kitcher agreed with Prof. Pederson. She said the department responds to the eternal report. Shouldn’t ARC do an action document? Prof. DiPrete said that in the past year the external group went first, and met with the internal committee. There is no internal report until the external committee produces its report. The internal group responded to the external review with an action document, which served as the first draft of the ARC report. He said the recommendations of the external committee are sometimes right on, and the internal group seconds them. Another professor said the internal review and the departmental self-study are different. EVP Madigan said the internal committee is an ARC subcommittee. Robert Pollack asked where in the budget process the ARC recommendations reach the administration. Do budgetary realities affect the report early on? EVP Madigan said the ARC reports can be used in the budget process. The problem is that some are so out of date that they have no budgetary relevance. That’s one of the problems to be fixed. Prof. Snyder recalled being told by David Cohen not to write the ARC report based on any particular budgetary constraint, but to use common sense in considering substantive recommendations. He said the process was supposed to work in something like the following way: Knowing the budgetary situation, David Cohen responds to the ARC review with a letter to the department chair promising three bullet points, on condition that the department make certain improvements. The chair writes back, agreeing. Prof. DiPrete said ARC includes faculty, deans, and the A&S EVP. The internal committee is there as well. If the budget is playing an inhibiting role, that will emerge. Overall, he said, it’s a mixed process. William Zajc said he was troubled by the description of David Cohen’s mandate. Part of the ARC review has to be an assessment of what is needed to make a department world-class. It’s important for both departments and the EVP to know that, since it’s essential information when establishing priorities in strategic planning. Prof. Snyder said he was not fundamentally disagreeing with this understanding. He was just adding some emphasis to the role of common sense in tempering recommendations. Prof. Shapiro said it was common when he chaired ARC for departments to measure themselves against a world-class standard and to provide a plan for becoming one of the top 10 or five departments. Prof. DiPrete said that’s still true. Prof. Snyder said that if the A&S administration provides the data for the self-study, departments may react with an urge to write their own appendix. Is there a norm for regulating that urge? Prof. DiPrete said the external report typically goes to the department chair, who responds. The department chair also responds to any factual inaccuracies he may find in the first draft of the ARC report. The final document typically incorporates all these stages. Prof. Shapiro said the problem of data for the department and for the university more broadly comes up in different ways. EVP Madigan said data are sometimes used for a single purpose. Prof. Kitcher said students in her department—Philosophy—are so tied to their mentors that they can’t provide a critical assessment. The external reviewers have to examine the situation. Rosalind Morris said she had participated in several reviews, some within her own department, and three involving other programs and institutes. These reviews had been very different experiences, because of different perceptions of the relationships among the parts, particularly of discontinuities between the stages of the review and its final outcome, which are sometimes exacerbated by external budgetary factors such as university-wide shortfalls or competition among departments for resources. Prof. Morris said one common perception is that a review is a success when resources are forthcoming (because no new allocations seem to be made without ARC reviews), and a failure when they are not. How can the relationship between reviews and resources be made more rational and transparent? Would it help to change the time periods for which needs are identified, by requiring three- and five-year plans in reviews as a matter of course? Are there other ways to apply for resources, so that a department’s critical self-study doesn’t get distorted by that process? GSAS dean Carlos Alonso said that most people consider an ARC review as a transitive exercise: What will we get from the ARC process? He said the review is an important self-referential moment for a department, enabling it to achieve a degree of self-knowledge that will support later requests for resources—when there are resources. He argued for disconnecting the assessment from the expectation of resources. He said the exercise of looking inward will be its own reward. Prof. Darcy Kelley said ARC reports are sometimes out of date. She said it makes sense to prioritize the timing of the reviews. That would help the department chair see the value of the exercise. EVP Madigan agreed. He said he was called upon to apply the lessons of some 2004 ARC reports last spring. Prof. Snyder asked what the institutional mechanism should be in following up ARC recommendations. Should decision making revert to normal channels, for example, using ARC reports as background when the Executive Committee makes decisions on departmental hiring requests? Or should it be institutionalized through the ARC itself, with a role in follow-up discussion? Prof. Kelley said it is not wise to use ARC to address old cases. Responsibility for the budget is with the EVP; ARC’s role is advisory. Prof. Kelley suggested that the EVP meet with chairs from departments whose ARC reviews are out of date to go over changes prior to the spring budget (IBS) meetings. Madeleine Zelin asked if there is a way to design the questionnaire so as to capture the externalities of department decisions, and add those findings to the data collection. EVP Madigan said one advantage of the ARC process is broad representation of the A&S faculty in looking into the inner workings of departments. Prof. DiPrete said that in his experience that input from each and every department that has a close connection with the department under review was not collected systematically, but the ARC attempted to get this input via the selection of appropriate members to the internal committee. It was helpful, for example, to have the perspective of Darcy Kelley, a neuroscientist, in the ARC review of the Psychology Dept. Prof. Peacocke said it would be good to have an expert consultant from outside the department advising on the choice of external reviewers. This approach has worked in selecting letter writers for the Promotion and Tenure Committee. It could also help to produce more informative and authoritative departmental reviews. EVP Madigan brought the discussion to a close at 1 pm, saying he hoped to have more discussions like this one. Respectfully submitted, Tom Mathewson
|
