CU Home

Search & Directories | AZ Index | Help

 

Columbia University in the City of New York

Home Publications Workshop Bibliography EBP Course TOC Critical Appraisal Skills CourseTOC Systematic_Review_Course_TOC

 

 

 

 

 

 

 

 

 

 

 

 

 

General Course Information:

SOCW T7501.003 CLINICAL CASE EVALUATION    
                

Instructor Information:

Edward Joseph  Mullen
E-mail: ejm3@columbia.edu
Office Hours: By Appointment

Prerequisites

3 points. Prerequisites: T6501, T6502. Required for all students in Advanced Clinical Practice.

Assumptions:

  • Students already have a foundation in both research methods and social work practice methods
  • Students are in field placements where concepts discussed in this class can be examined & illustrated in current clinical practice cases
  • Students have accepted the social work professional ethical standards that require them to use research & best evidence in their practice
  • This is not a practice course the course does not have as an objective teaching how to do evidence-based practice, empirical practice or scientific practice but rather the objective is to teach research methods relevant to engaging in practice. This course together with the practice courses and field work are designed to provide students with the knowledge & skills needed to be evidence-based practitioners. The course is designed to help students gain the competencies needed to be critical users of the clinical practice research literature and to use systematic research methods to study their work with clients.
  • Graduate social work students are adult learners. Accordingly, teaching and learning methods appropriate for adult learners are used in this course, especially problem-based methods.

Course Objectives

Building on principles of social scientific inquiry and basic research methodology covered in T6501, this course will reinforce and extend understanding of a scientific, analytic approach to building knowledge for practice and evaluating service delivery.  Students are expected to draw on previously acquired knowledge about problem formulation, measurement, and research design as they learn to formulate and analyze research questions using methodologies relevant to clinical practice.

 

Core themes:

  • Understanding principles of clinical research methodology
  • Critical thinking skills needed to formulate research questions
  • Use of data reported in published sources to examine these research questions
  • Understanding of procedures and technologies for data analysis in clinical research
  • Ability to communicate findings to clients, colleagues and others to advance mutual learning and decision making

Strict adherence to ethical standards of scientific inquiry will be emphasized, with particular attention to protecting and promoting the wellbeing of vulnerable and oppressed populations. An analytic approach to knowledge building and evaluation relevant to clinical practice is reinforced by other advanced curriculum areas. These include clinical practice courses (T7113 & T7114), practice platform courses, electives and fieldwork. Learning to use advanced clinical research methods will reinforce and extend advanced content.

 

Upon successful completion of this course, students are expected to be able to:

  • Gain access to and critically read research literature
  • Understand the strengths and limitations of research methods used in clinical research
  • Identify obstacles that may be encountered in clinical research and know how to deal with these obstacles
  • Identify the probable benefits and costs of conducting and using research in clinical practice
  • Understand distinctions between using research findings for clinical practice decision making, using research methods to learn from one's own practice, and using research methods to conduct research to answer research questions
  • Use computer based resources for locating relevant clinical research findings
  • Communicate findings from research to facilitate clinical practice decision making and advance understanding of clinical issues
  • Understand ethical guidelines pertaining to clinical research
  • Use appropriate methodologies to evaluate clinical practice
  • Avoid biases in clinical research with clients with varying ethnic, age, gender, sexual orientation, socioeconomic, physical and/or mental ability characteristics, and/or from high-risk, vulnerable and other disadvantaged groups.

This section of T7501 takes an evidence-based practice approach to teaching and learning of clinical research methods. Accordingly, this section focuses on the development of specific research competencies needed to engage in evidence-based practice. Competencies #1 through #5 and # 8 listed below are the focus of this course. Competencies #6, #7 and #9 are taught in practice courses rather than in the research course (marked with asterisk). The 9 core evidence-based practice competencies are as follows:

 

  1. Ability to examine a case situation and to identify uncertainties or dilemmas about how best to assess, intervene or monitor.
  2. Ability to make a judgment about each uncertainty:
    1. Relative importance to achieving outcomes of importance to client how important is it to address & resolve the uncertainty to achieving an outcome of importance to the client?
    2. How typical is the uncertainty among cases seen at the agency? uncertainties that occur frequently in practice may be ranked more important to address & resolve
    3. How likely is it that the uncertainty can be resolved to your and the client's satisfaction by:

                                                               i.      Asking your supervisor to provide you with the best resolution

                                                             ii.      Asking a consultant to provide you with the best resolution

                                                            iii.      Asking professional colleagues at the agency to provide you with the best resolution

                                                           iv.      Reviewing conceptual-theoretical discussions pertaining to the area of uncertainty

                                                             v.      Adopting whatever resolutions traditionally used at the agency for such uncertainties

                                                           vi.      Using your common sense and intuition

                                                          vii.      Reviewing empirical research reports pertaining to the area of uncertainty

  1. Ability to transform those areas of uncertainty where appropriate into answerable evidence-based practice questions (not all areas of uncertainty can be resolved empirically).
  2. Ability to locate information pertinent to each evidence-based practice question.
  3. Ability to critically evaluate the information located and to weigh the information relevance to each evidence-based practice question.
  4. Ability to integrate the evidence located from all sources.*
  5. Ability to make a decision with client about how best to proceed with the uncertainty (considering possible alternative courses of action) taking into account client values and preferences, resources, and preferred outcomes.*
  6. Ability to monitor what happens when the intervention is implemented.
  7. Ability to modify the chosen intervention based on the information gained through the monitoring process.*

For a discussion of core evidence-based practice competencies see the statement by the Evidence-based Behavioral Practice Council.

Method of Instruction

The class will take a problem-oriented approach to teaching and learning. Typically, each class will begin with an opportunity to discuss the assigned topic and readings. Since this is an advanced course students are expected to read widely about the assigned topics. While the instructor has provided suggested readings and resources students should not limit themselves to only those resources. Much is available on the internet regarding each topic and students are encouraged to search out and explore the wide range of resources available. As adult, life-long learners this is the learning mode that you will need to develop as future up-to-date autonomous clinicians.

The class will be subdivided into teams of approximately six students each. These teams will work throughout the semester on assignments. Teams may be organized around areas of common interest to the extent possible such as field of practice, population or problem. During the first class students will be offered the opportunity to form teams on their own. Most classes will provide break-out time for students to work in their teams on in-class problems. Students are encouraged to assist one another in completion of the in-class problems as well as the assignments.

Method of Evaluation

  • The course grade will be based on quizzes (50%) and in-class team assignments completed by each team (50%). Team members will share the team grade.

    Teams will post their weekly in-class assignments  on the Discussion Board prior to the beginning of the following class (teams can begin to work on the assignment during class in the breakout groups and then continue working on them during the week). Please see Discussion Board link for more information.
  • An elective bonus assignment is offered to students wishing to elect this additional option (see assignment section of CourseWorks for details).

Session

Topic

Session 1

Introduction to Course

Session 2

Forming questions of importance to your client's welfare & making decisions about services

Session 3

Evaluating Practice Outcomes

Session 4

Locating evidence to answer your question

Session 5

Critically Appraising Effectiveness Questions: Criteria for Inferring Effectiveness

Session 6

Effectiveness Questions: Critically Appraising Experiments

Session 7

Effectiveness Questions: Critically Appraising Quasi-Experiments: Non-equivalent Comparison Groups Designs

Session 8

Critically Appraising Quasi-Experiments: Time-series Designs

Session 9

Critically Appraising Quasi-Experiments: Single-Case Designs

Session 10

Election-day Holiday

Session 11

Critically Appraising Systematic Reviews & Meta-Analyses

Session 12

Critically Appraising Descriptive-Background Questions: Predictors of Risk; Prognosis; Etiology; Experiential

Session 13

Critically Appraising Qualitative Studies for All Questions

Session 14

Critically Appraising & Selecting Assessment & Diagnostic Instruments

Session 15

Evaluating Process & Outcomes and Monitoring Client Progress

Session 1

Topic:
Introduction to Course

Class Plan:

  • Review of course objectives, syllabus, resources, and assignments.
  • Defining evidence-based practice in social work as described by Leonard Gibbs.
  • The importance of forming clinically relevant questions of importance to your client as a first step in evidence-based practice - COPES questions.
  • Taking a multidisciplinary view of evidence-based practice and defining evidence-based behavioral practice.
  • Defining the steps of evidence-based practice.
  • Review of anticipated evidence-based practice competencies to be attained by clinicians as viewed from an interdisciplinary perspective.
  • Review of anticipated EBP Competencies taught in this course and where training is provided in the social work curriculum for other competencies (PowerPoint file below in files section). 
  • Discussion of how this course builds on the foundation research course and links with other second year courses.
  • Review of clinical research in social work current emphases including evidence-based practice as a new organizing framework.
  • Discussion of the following questions: (1) Why is it important for social work students to learn evidence-based approaches?; (2) What are some of the benefits to the field and to clients or populations that social workers work with?; (3) How is EBP viewed in allied disciplines? (4) Why is it important for students to know how to critically assess research evidence?
  • The class will watch an example of evidence-based practice selected from one of the following: (1) Hospital Interactive Team Thinking exercise (HITTT) from Gibbs, L. E. (2003); (2) School-based Clinical Practicum project from REACH-SW CD-ROM; (3) Women's Shelter Clinical Practicum project from REACH-SW CD-ROM; (4) Task Force on Mental Health and Aging project from REACH-SW CD-ROM (the REACH-SW CD-ROM beta version is on reserve for subsequent viewing).

Break-out Teams:

  • Time will be made available for students to circulate and to engage in team formation. Teams should be composed of approximately 6 members each and formed around a common interest such as field of practice, population or problem area. These teams will work together throughout the semester.
  • Following team formation teams should discuss the case video: (1) Identify the steps the practitioners went through to reach a decision; (2) What EBP competencies did the practitioners demonstrate; (3) If you were the social worker what would you have done, if anything, to improve the decision-making process?
  • Regroup & report to class your answers to #1 & #2.

 

Assignments Due:

  • Completion of in-class team oral reports (time permitting)
  • Completion of a one page team report on the in-class assignment to be posted on the CourseWorks Discussion section under todays topic to be submitted before the beginning of next class.
  • Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin (2007), Preface, chapters 1 & 2 :

 

Gibbs, L. E. (2003). Chapter 1. :

     

Evidence-based Behavioral Practice Project white paper entitled Definitions and Competencies of Evidence-Based Behavioral Practice :

Evidence-based Behavioral Practice Project white paper entitled An Inter-Professional Model of Evidence-Based Practice :

Recommended Readings:

Straus, S.E., et. al. (2005). Preface & Introduction : .

Roberts & Yeager (2006). Chapters 1,2 & 3. :

 

Evidence-based Behavioral Health Project at Northwestern University web site videos of experts talking about EBBP http://www.ebbp.org/index.html :

Danya International, Inc REACH-SW beta CD-ROM : On restricted library reserve for this class's use only.

Files:

An Inter-Professional Model of EBP

Defining EBP Powerpoint

EBP Competencies

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 2

Topic:
Forming questions of importance to your client's welfare & making decisions about services

Class Plan:

  • While the EBP process typically begins by forming a question emerging from practice there is an even earlier process that needs to be considered, namely, identification of some "uncertainty" or "lack of knowledge" aboutt a case situation that needs to be resolved to provide effective service to the client. Once the uncertainty is understood and stated then a question can be formed that if answered will dimension or resolve the uncertainty and provide guidance to practitioners and clients about how to proceed to provide effective services. Information from research can help to answer some questions but others can best be answered through other means such as consultation with a supervisor. An important competency is the ability to identify uncertainties and related questions followed by an ability to determine the best way of answering these questions, especially knowing which questions can be answered by looking to research evidence. The class will begin with a discussion of the importance of being able to identify uncertainties in practice that result in dilemmas about how to proceed with clients.
  • Possible ways to resolve these dilemmas will be reviewed.
  • Qualities of uncertainties that lend themselves to resolution through examination of research evidence will be discussed.
  • The class will discuss how to convert uncertainties into answerable questions which can structure an evidence search.
  • Videos of cases demonstrating identification of uncertainties in practice and subsequent problem formulation will be viewed from the REACH-SW Video Case Studies CD-ROM (continued from last class) and from the Evidence-based Behavioral Practice Training Modules web site.
  • Teams will meet to begin work on the in-class assignment.

At the end of this class students will have gained knowledge about how to formulate COPES/CIAO/PICO questions.

 

Assignments Due:

(1) One to two page team reports due prior to next class posted in the CourseWorks Discussion section. These reports should address the following five questions.

The evidence-based practice process begins with identification of an uncertainty or dilemma and formulation of a practice question of practical importance to a problematic situation that practitioners face with specific clients. This class examines how such questions can be formed. Also, EBP requires motivation to engage in such practice as well as organizational support.

 

Answer the following questions about the cases shown in today's video clips.

  • What were the uncertainties and resulting questions identified in those cases? Discuss and identify these uncertainties and questions in your group.
  • Did the practitioners appear to be motivated to engage in EBP? What things shown in the videos suggest to you that they were or were not motivated?
  • Did the practitioners appear to have the necessary organizational support to engage in EBP? What things shown in the videos suggest to you that they were or were not motivated?
  • What type of questions did the practitioners formulate? Remember that evidence-based practice questions can be:

         1.      About what intervention might have the best effect with a client? - Effectiveness and prevention questions.

2.      About what factors might best predict desirable or undesirable outcomes with a client? - Prognosis and risk questions.

3.      About what a client is experiencing (what is it like to have had the client's experiences)? - Descriptive questions.

4.      About what assessment tool should be used to gather information about a client's characteristics or condition? - Assessment questions.

  • Formulate four additional questions that your team thinks could have been asked based on any of the cases shown in the video-clips. Form an effectiveness/prevention question; a prognosis/risk question; a descriptive question; an assessment question.

(2) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin, A. (2007). Chapters 2 & 3 :

Gibbs, L. E. (2003). Chapters 2 & 3 :

Recommended Readings:

Straus, S. E., Richardson, W. S., Glasziou, P., & Haynes, R. B. (2005). (Introduction & chapter 1) :

 

Guyatt, G., & Rennie, D. (2002). (Front Matter & 1A) :

R. M. Grinnell & Y. A. Unrau, Chapter 25 :

Danya International, Inc REACH-SW CD-ROM : On restricted library reserve for this class's use only.

Files:

Background and Foreground Questions

Uncertainties

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 3

Topic:
Evaluating Practice Outcomes

Class Plan:

In today's class we will:

  • Review key concepts in the readings about monitoring in EBP
  • Watch a 15-minute video of an EBP case presented in the REACH-SW video pertaining to a women's shelter for abused women.
  • Work in break-out teams to assess and propose a monitoring plan for this case.

 

 

Review of Methods for Evaluating Practice Process & Outcomes

 

As we have seen the last step (5th step) in the EBP process is evaluation. We think this is such an important step that we present it as second to problem formulation, and use this class to examine the role and methods of evaluation in EBP. Students should be anticipating from the start how they will evaluate their evidence-based practice process and outcomes. This class will discuss how evaluation has been proposed in evidence-based medicine and in social work.

 

(1) Self-evalution & reflecting on how one is doing as an evidence-based practitioner (based on Strauss, et al, 2005, chapter 8)

  • How am I doing in asking answerable questions?
  • How am I doing is searching for evidence?
  • How am I doing in critical appraisal of the evidence?
  • How am I doing in integrating evidence and client values/preferences?
  • Is our practice improving?
  • Is our practice becoming more evidence-based?
  • How am I doing sharing what I have learned from the practice with others in my team?

(2) Monitoring Client Progress & the Practitoner Friendly Designs (based on Rubin, 2007, chapter 12.

 

(3) Evaluationg Practice Outcomes & Single-System Designs (based on Thyer & Myers, 2007, chapters 1, 2, & 3).

 

Break-out Team Assignment

Teams will work on in-class assignment.

 

Assignments Due:
(1) One to two page team reports due prior to next class posted in the CourseWorks Discussion section. These reports should address the following questions.

Answer the following questions about any one of the cases shown in the video clips presented during today's class or in the first two classes ( you need to answer these questions for only one of the cases and you may select any one of the three):

  • What evaluation plans were used or proposed by the practitioners in the video clip?
  • If you were one of these practitioners what evaluation plan would you have been comfortable proposing? Describe this plan for any one of the cases.

(2) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material. Since the Thyer & Meyer book is not yet on reserve the quiz will not include content in that text not covered in the class lecture.

Required Readings:

Rubin (2007), Chapter 12 :

Straus, Richardson, Glasziou & Haynes (2005), Chapter 8. : Although this is written for physicians the treatment of this topic is easily adaptable for social workers.

Recommended Readings:

Thyer & Myers (2007), Chapters 1-3 :

This material should already be familiar to you since it would typically have been presented in the first year research course. You may wish to read the entire book to refresh yourself on this topic. It is a brief and easily readable treatise on the topic of evaluating practice outcomes for social work practitioners.

Files:

Figure 12 from Rubin Text

Monitoring in EBP PowerPoint

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 4

Topic:
Locating evidence to answer your question

Class Plan:

Introduction to Locating Evidence

Following formulation of a practice question the next EBP step is to locate relevant evidence by searching internet sources as well as published and unpublished reports of evidence.  This class provides opportunity to discuss methods for conducting such searches as well as some of the more useful sources with special attention to internet-based sites which publish systematic reviews, meta-analyses, guidelines, reviews of assessment instruments, and individual research study reports. The skills you develop for locating evidence will be used by the teams to locate articles to complete team in-class assignments for classes 5 through 14.

 

Topics examined will include:

  • Types of evidence sources (computerized decision support systems; synopses such as found in journal abstracts written for practitioners; syntheses; individual studies as discussed by Strauss, et all (2005), chapter 2)
  • Major databases available through the CU library for locating evidence & some suggests for efficient searching
  • Designing search strategies to achieve sensitive and specific results including specification of key concepts, limiting terms and quality control filters.

Break-out Team Assignments

Teams will break-out to work on the in-class assignment.

 

Assignments Due:

(1) During the in-class break-out each team should identify an evidence-based practice question relevant to one of the members current cases. Each team should then design a search strategy for locating evidence to answer the question. The strategy should include the databases to be searched and the search terms to be used including any quality control filters and limiting terms.

(2) One page team reports due prior to next class posted in the CourseWorks Discussion section. These reports should address the following tasks:

  • Describe in one paragraph the key elements of the case you have selected (remember the four components of evidence-based questions).
  • Present the evidence-based question formed by the team for this case.
  • Identify the databases that you would use for this search.
  • Describe the search terms including key concepts, limiting terms and quality control filters.

(3) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin (2007), Chapter 2, pp. 20-27 :

Gibbs (2003), Chapter 4 & Web Site :

Gibbs' book web site  (select the tabs for plan a search, access a database, and access a database - the forms on this web site can be used for this assignment) 

Recommended Readings:

REACH-SW Module 2 :

Evidence-based Behavioral Practice Project Training Resource Module on Searching for Evidence :

Grinnell & Unrau (2007), Chapter 22. :

Straus, et al (2005). Chapter 2 :

Greenhalgh (2001). Chapter 2 :

Petticrew & Roberts (2006). Chapter 4 :

Littell, Corcoran & Pillai (2008). Chapter 3 & Appendix D :

Files:

5 "S" Evolution of Information Services

Examples of Search Strategies

PowerPoint Instructions for Planning Evidence Search Strategy

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 5

Topic:
Critically Appraising Effectiveness Questions: Criteria for Inferring Effectiveness

Class Plan:

The next five classes are designed to provide opportunity to develop critical assessment skills for articles that report evidence about effectiveness questions typically using various experimental and quasi-experimental designs. Today's class provides opportunity for discussion of criteria for inferring effectiveness which are common to all of the subsequent designs discussed in the following 4 classes. Topics for consideration in today's class include:

  • Internal Validity (Threats to Internal Validity, Selectivity Bias, Random Assignment)
  • Measurement Issues
  • Statistical Chance
  • External Validity
  • Effect size measures including risk, absolute risk reduction, relative risk reduction, number needed to treat, number needed to harm as well as measures of difference (e.g., standardized and unstandardized mean differences), measures of association (correlations), and, measures of differences between non-continuous, catagorical data (e.g., odds ratios).

Break-out Team Assignment

 

Assignments Due:

(1) During the in-class break-out each team should develop a plan and make team member assignments for tasks needing to be completed for the remaining classes. The remaining classes will be used to learn critical appraisal skills need to assess reports of research studies using a wide variety of designs (look ahead on the syllabus to see what designs will be examined).

  • Beginning with next class each team is required to bring one article to class which reports a study using the design for that classes' topic. For example, the topic for next class is skills needed to critically assess studies reporting the use of an experimental design. For the next class search the internet for articles reporting the use of experimental designs and bring these articles to the next class (or view them on-line if you all have computers and can access full-text articles).
  • During the break-out session for next class (and the remaining classes) each team should use the time to critically assess the articles and prepare a written report on one of the articles.The Rubin text provides guidelines for assessing most types of designs that we will be examining. In addition the assignments section of CourseWorks for this class includes guidelines for each type of design that we will be examining and the team can use these guidelines for the team assessment.
  • Use today's break-out team time to develop a plan for how your team will allocate responsibility among team members and to specify tasks so that these assignments can be completed to everyone's satisfaction. For example, does your team want to select a topic such as a specific evidence-based practice concern of mutual interest and search for articles that relate to that topic? Or, would you rather just search for interesting practice-relevant articles related to a broad topic? Once you have decided on your focus then allocated team member responsibilities. For example, will one or two members take responsibility for locating articles for each of the remaining class and become the lead reviewer for each class (specialize by design type)? Or, will all team members conduct a search for every class and then let the team select the article from the batch that you want to work on for the assignment?

(2) One page team reports due prior to next class posted in the CourseWorks Discussion section. These reports should describe the team plan for completing the remaining in-class assignments including specification of the topic (or topics) and team member tasks.

(3) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin, A. (2007). Chapter 4 :

Gibbs, L. (2003). Chapter 5 :

CONSORT Statement :

Because there has been great variation in how researchers have reported the results of random controlled studies examining effectiveness questions an evidence-based minimum set of reporting has been published specifying what should be included in such reports. This can be found at the home page for the CONSORT group at: http://www.consort-statement.org/QUOROM.pdf . CONSORT, which stands for Consolidated Standards of Reporting Trials, encompasses various initiatives developed by the CONSORT Group to alleviate the problems arising from inadequate reporting of randomized controlled trials (RCTs).

Thyer & Myers (2007). Chapter 4 :

Recommended Readings:

Straus, S. E. (2005). Chapters 5 & 6, cards 3A & 3B, 8A & 8B :

Guyatt, G., & Rennie, D. (2002). 1B, 1B1, 1B2 :

Shadish, Cook & Campbell (2002) :

Experimental and Quasi-experimental Designs for Generalized Causal Inference is an advanced, authoritative resource for experimental and quasi-experimental designs. If you have questions that are not addressed in the required reading browse this text for answers.

Files:

PowerPoint Dr Lin Fang

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 6

Topic:
Effectiveness Questions: Critically Appraising Experiments

Class Plan:

This class will provide opportunity to discuss effectiveness questions and to develop knowledge and skills for critically appraising articles that report studies using experimental designs. Designs and issues for discussion include:

  • Classic Pretest-Posttest Control Group Design
  • Posttest-Only Control Group Design
  • Solomon Four-Group Design
  • Alternative Treatment Designs
  • Dismantling Designs
  • Placebo Control Group Designs
  • Experimental Demand and Experimenter Expectancies
  • Obtrusive Versus Unobtrusive Observation
  • Compensatory Equalization and Compensatory Rivalry
  • Resentful Demoralization
  • Treatment Diffusion 
  • Treatment Fidelity
  • Practitioner Equivalence
  • Differential Attrition

Break-out Team Assignment

 

Assignments Due:
(1) During the in-class break-out each team should review the articles located for this classes' team assignment (as planned at the last class).

(2) Team reports due prior to next class posted in the CourseWorks Discussion section. These reports should present the team's critical appraisal of one of the articles examined in today's class using the guideline posted in the Assignment section of CourseWorks for this design type.

(3) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin, A. (2007). Chapter 5 :  

Gibbs, L. (2003). Chapter 5 :

Recommended Readings:

Straus, S. E. (2005). Chapters 5 & 6, cards 3A & 3B, 8A & 8B :

Guyatt, G., & Rennie, D. (2002). 1B, 1B1, 1B2 :

Files:

Experimental Designs Lecture_A. Kapadia

PowerPoint Overview of Critically Appraising Articles on Effectiveness Question Experimental Designs

PowerPoint Overview of Experimental Designs

Today's Lecture PowerPoint

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 7

Topic:
Effectiveness Questions: Critically Appraising Quasi-Experiments: Non-equivalent Comparison Groups Designs

Class Plan:

This class continues the discussion of effectiveness questions. This class provides opportunity for developing knowledge and skills for critically assessing articles that report studies using nonequivalent comparison group designs. Designs and issues for consideration include:

  • Nonequivalent Comparison Groups Designs (Are the Groups Comparable? Grounds for Assuming Comparability)
  • Additional Logical Arrangements To Control For Potential Selectivity Biases (Multiple Pretests, Switching Replications, Nonequivalent Dependent Variables)
  • Statistical Controls for Potential Selectivity Biases (When the Outcome Variable is Categorical, When the Outcome Variable is Quantitative)
  • Pilot Studies

Mid-semester class evaluation

How is the course going for each of you? What should we continue doing that has worked for you? What should we change to improve learning opportunities?

Break-out Team Assignment

 

Assignments:
(1) During the in-class break-out each team should review the articles located for this classes' team assignment.

(2) Team reports due prior to next class posted in the CourseWorks Discussion section. These reports should present the team's critical appraisal of one of the articles examined in today's class using the guideline posted in the Assignment section of CourseWorks for this design type.

(3) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin, A. (2007). Chapter 6 :

Gibbs, L. (2003). Chapter 5 :

Recommended Readings:

Straus, S. E. (2005). Chapters 5 & 6, cards 3A & 3B, 8A & 8B :  

Guyatt, G., & Rennie, D. (2002). 1B, 1B1, 1B2 :

Files:

Experimental Designs_A. Kapadia

Guideline for Assessing Nonequivalent Comparison Group Studies

PowerPoint for Today's Class

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 8

Topic:
Critically Appraising Quasi-Experiments: Time-series Designs

Class Plan:

This class provides opportunity for developing knowledge and skills for critically assessing articles that report studies using time-series designs. Designs and issues for consideration include:

  • Simple Time-Series Designs
  • Multiple Time-Series Designs

Break-out Team Assignment

 

Assignments Due:
(1) During the in-class break-out each team should review the articles located for this classes' team assignment.

(2) Team reports due prior to next class posted in the CourseWorks Discussion section. These reports should present the team's critical appraisal of one of the articles examined in today's class using the guideline posted in the Assignment section of CourseWorks for this design type.

(3) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin, A. (2007). Chapter 7 :

 

Gibbs, L. (2002). Chapter 9 :

Recommended Readings:

Guyatt, G., & Rennie, D. (2002). 2B1 : 

Straus, S. E. (2005). page 172-175 :

Files:

Guide for Assessing Articles Reporting Studies UsingTime-series or Single-case Designs

PowerPoint on Critically Assessing Time-Series & Single-Case Designs

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 9

Topic:
Critically Appraising Quasi-Experiments: Single-Case Designs

Class Plan:

This class provides opportunity for developing knowledge and skills for critically assessing articles that report studies using single-case designs. Designs and issues for consideration include:

  • Single-Case Designs
  • N=1 RCTs

 

Assignments Due:
(1) During the in-class break-out each team should review the articles located for this classes' team assignment.

(2) Team reports due prior to next class posted in the CourseWorks Discussion section. These reports should present the team's critical appraisal of one of the articles examined in today's class using the guideline posted in the Assignment section of CourseWorks for this design type.

(3) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin, A. (2007). Chapter 7 :

Thyer & Myers (2007). Chapter 3 :

Files:

Single-Subject Designs: Bledsoe & Bellamy

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 10
Open for Discussion

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 11

Topic:
Critically Appraising Systematic Reviews & Meta-Analyses

Class Plan:

The most efficient and comprehensive method for locating evidence about a clinical practice question is to locate a high quality systematic review of relevance to your

question. This class provides opportunities to discuss systematic reviews. Relevant topics include:

 

  • Advantages Of Systematic Reviews And Meta-Analyses
  • Risks In Relying Exclusively On Systematic Reviews And Meta-Analyses
  • Where to Start?
  • What To Look For When Critically Appraising Systematic Reviews (Bias, Cochrane and Campbell Collaborations, Inclusion and Exclusion Criteria, Does the Review Critically Appraise the Quality of Included Studies?, Comprehensiveness, Transparency)
  • What Distinguishes A Systematic Review From Other Types Of Reviews?
  • What to Look for When Critically Appraising Meta-Analyses (Effect-Size, Correlating Effect Size with Other Variables)
  • Statistical & Clinical Significance

Specific Plan for Today's Class:

  • Instuctor will provide brief overview of topic & comment on class resources & PowerPoints
  • We will look at PubMed section on Clinical Queries as an example of how to find systematic reviews that are clinician friendly
  • We will go to the Campbell Collaboration web site to view available systematic reviews in social work and social welfare, criminal justice and education http://www.campbellcollaboration.org/index.asp .
  • We will look at a Campbell Collaboration systematic review protocol (Risk of child maltreatment: A systematic review of the predictive validity of instruments by Aron Shlonsky, aron.shlonsky@utoronto.ca, University of Toronto, Canada, Mike Saini, and Ulla Jergeby View Documents
  • We will look at a completed Campbell Collaboration systematic review:

Multisystemic therapy for social, emotional, and behavioral problems in children and adolescents aged 10-17 by Julia H. Littell, jlittell@brynmawr.edu, Bryn Mawr College, USA, Melanie Popa, and Burnee Forsythe

View Documents

  • Break-out team work on in-class assignment.

 

Assignments Due:
(1) During the in-class break-out each team should review the articles located for this classes' team assignment.

(2) Team reports due prior to next class posted in the CourseWorks Discussion section. These reports should present the team's critical appraisal of one of the articles examined in today's class using the guideline posted in the Assignment section of CourseWorks for this design type.

(3) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin, A. (2007). Chapter 8 :  

Gibbs, L. (2003). Chapter 6 :

Recommended Readings:

Greenhalgh T. (1997) :

How to read a paper: Papers that summarise other papers (systematic reviews and meta-analyses). BMJ 1997;315:672-5.

Littell, J. H., Corcoran, J., & Pillai, V. (2008). :

Systematic reviews and meta-analysis. New York: Oxford. (Written for social work applications)

Petticrew, M. & Roberts, H. (2006). :

This is a very readable overview of systematic review methods as applied to the social sciences including social work.

Files:

Article on the Role of Systematic Reviews in EBP

Article Presenting a Guide for Assessing Systematic Reviews in Context of EBP

Guide for Assessing Articles that Report Systematic Reviews

PowerPoint on Advanced Meta-analysis Methods

PowerPoint on Meta-analysis Basics

PowerPoint on Various Methods for Synthesizing Evidence in Systematic Reviews

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 12

Topic:
Critically Appraising Descriptive-Background Questions: Predictors of Risk; Prognosis; Etiology; Experiential

Class Plan:

This class provides opportunity to examine how to critically assess evidence about descriptive questions. Descriptive questions ask about what is known about characteristics of people like your client. These include questions about predictors or risk & protective factors of social problems or conditions like the one's experienced by your client. They include prognostic questions which ask about what you might expect to occur with your client's problem, need or situation if no intervention is provided. They also include questions about the etiology of problems or conditions like those experienced by your clients. They also include questions about how clients like yours may be experiencing something in their lifes including how they might be experiencing the interventions that you or another professional are providing. Sometimes these are called background questions (in the medical literature). Basically, these questions do not ask about the effects or outcomes of interventions that you might consider but rather seek answers to questions that come up when you engage in an assessment of the client's condition, situation or problems. The study designs generally are prospective or retrospective correlational designs although experimental designs may be used as well (such as in a control condition). Experience questions often used qualitative designs which we will examine later in the semester. Typical designs and issues to be examined are:

 

  • Surveys
  • Cross-Sectional and Longitudinal Studies (Cohort Studies and Panel Studies; Case-Control Studies)
  •  
  • Reporting Guidelines
    STROBE - reporting of observational studies in epidemiology
    EQUATOR Network - collection of reporting guidelines

 

 

Assignments Due:

(1) During the in-class break-out each team should review the articles located for this classes' team assignment.

(2) Team reports due prior to next class posted in the CourseWorks Discussion section. These reports should present the team's critical appraisal of one of the articles examined in today's class using the guideline posted in the Assignment section of CourseWorks for this design type.

(3) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin, A. (2007). Chapter 9 :

Gibbs, L. (2003). Chapter 8 :  Gibbs' chapter also includes a discussion of qualitative research which we will discuss later in the semester

Recommended Readings:

Straus, S., et al. (2005). Chapter 4 & card 7A :

Guyatt, G., & Rennie, D. (2002). 1D :

Op-Ed Contributor: The Bad News First :

An interesting op-ed about the use of prognoses in medical practice:

OPINION   | August 24, 2007

By NICHOLAS A. CHRISTAKIS
For reliable prognoses to become a routine part of medical care, they must become a priority of medical research and education.

Files:

PowerPoint Surveys, Cohorts, Case-Control

Student Example Cohort Study

Student Example Survey Study

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 13

Topic:
Critically Appraising Qualitative Studies for All Questions

Class Plan:

This class provides opportunity for consideration of articles that report qualitative research findings relevant to clinical practice questions. Increasingly researchers are using both qualitative and quantitative methods to examine clinical practice questions. These methods are complimentary. Qualitative research methods can inform all four types of evidence-based practice questions but these designs are especially useful for descriptive questions. Topics for considerations include:

  • Qualitative Observation 
  • Qualitative Interviewing (Life History, Focus Groups)
  • Qualitative Sampling
  • Grounded Theory
  • Frameworks for Appraising Qualitative Findings (Empowerment Standards, Social Constructivist Standards, Contemporary Positivistic Standards)

Break-out Team Assignment

 

Assignments Due:

(1) During the in-class break-out each team should review the articles located for this classes' team assignment.

(2) Team reports due prior to next class posted in the CourseWorks Discussion section. These reports should present the team's critical appraisal of one of the articles examined in today's class using the guideline posted in the Assignment section of CourseWorks for this design type.

(3) Completion of quiz prior to next class (on-line quiz in the Test & Quiz section of CourseWorks available for completion from after today's class until the beginning of next week's class; failure to complete the quiz during this time period will result in 0 points for this quiz). The quiz will be based on material covered in today's class as well as required readings and web material.

Required Readings:

Rubin, A. (2007). Chapter 10 :  

Gibbs, L. (2003). Chapter 8 :

Recommended Readings:

Greenhalgh T, Taylor R. (1997) :

How to read a paper: Papers that go beyond numbers (qualitative research) . BMJ 1997;315:740-3.

Cochrane Qualitative Research Methods Group Web Site :

Review the resources at this web site. Especially check out the tab for "Tools to Assist Qualitative Researchers". For today's topic #3 "Critical Appraisal of Qualitative Studies" tab includes several relevant readings that can be downloaded.

Files:

Guide to Assess Qualitative Studies

PowerPoint for Critically Appraising Qualitative Research

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 14

Topic:
Critically Appraising & Selecting Assessment & Diagnostic Instruments

Class Plan:

This class provides opportunity to examine how to critically assess evidence about assessment and diagnostic questions. These questions ask about the soundness and relevance of various instruments that you might consider using to assess your client's needs or characteristics. Typical designs and issues to be examined are:

  • Reliability (Internal Consistency Reliability; Test-Retest Reliability; Interrater (Interobserver) Reliability)
  • Validity (Face Validity, Content Validity, Criterion Validity, Construct Validity)
  • Sensitivity (including Cultural Sensitivity)
  • Specificity
  • Likelihood Ratios
  • Positive and Negative Predictive Values
  • Pre-test and Post-test Probabilities and Odds
  • Feasibility
  • Sample Characteristics
  • Locating Assessment and Diagnostic Instruments

Break-out Team Assignments

 

Assignments Due:

(1) During the in-class break-out each team should review the articles located for this classes' team assignment.

(2) Team reports due prior to next class posted in the CourseWorks Discussion section. These reports should present the team's critical appraisal of one of the articles examined in today's class using the guideline posted in the Assignment section of CourseWorks for this design type.

Required Readings:

Rubin, A. (2007). Chapter 11 :  

Gibbs, L. (2003). Chapter 7 :

Recommended Readings:

Greenhalgh T. (1997) :

 How to read a paper: Papers that report diagnostic or screening tests . BMJ 1997;315:540-3.

 

Straus, S., et al. (2005). Chapter 3 & card 1A&B, 2A&B :

Guyatt, G., & Rennie, D. (2002). 1C :

Files:

Cochrane Diagnostic Test Accuracy Guidelines

Cochrane Diagnostic Test Accuracy Plots

PowerPoint on Measuring Instument Assessment

https://courseworks.columbia.edu/cms/cu_images/topbg.gif

 

Session 15

Topic:
Evaluating Process & Outcomes and Monitoring Client Progress

Class Plan:

The previous classes have focused on helping you develop knowledge and skills for critically appraising research studies for evidence-based practice questions. Based on what you find it is expected that these appraisals will be used by you in an evidence-based practice process with clients. It is important that when you use your appraisals in practice you monitor what happens with individual clients. What was found to work in the research literature may not work with your individual client! You need to monitor implementation and use what you and your clients find to modify your subsequent interventions. These skills will contribute to your EBP competiences:

  • Competency: Ability to monitor what happens when the intervention is implemented.
  • Competency: Ability to modify the chosen intervention based on the information gained through the monitoring process.

Also, practitioners can contribute to practice knowledge by evaluating their work with clients and reporting the results to others. Single-case and group designs can be used for this purpose.

 

In this class we review a few skills for monitoring client progress and for evaluating process and outcomes of interventions. Topics include:

  • Practitioner Friendly Design: The B+ Design
  • Feasible Assessment Techniques (What to Measure?, Who Should Measure?, With What Measurement Instrument? When and Where to Measure?)
  • Single-case designs (revisited)
  • Group designs (revisited)

Class evaluation

We will use time at the end of class to evaluate how the class went for each of you. What was good and what should be changed for future classes? Did you learn what you had hoped to learn? Do you feel that you have achieved the competencies we set for this course?

 

 

Assignments Due:

On-line course evaluations PLEASE!

Required Readings:

Rubin, A. (2007). Chapter 12 :  

Thyer & Myers, (2007) :

Assignments Due:

Elective Bonus Assignment

Files:

Rubin Figures Chapter 12

Short PowerPoint

Single-case PowerPoint

https://courseworks.columbia.edu/cms/cu_images/topbg.gif