An Intensive Primary-Literature–based Teaching Program Directly Benefits Undergraduate Science Majors and Facilitates Their Transition to Doctoral Programs

INTRODUCTION

The influential report, BIO 2010: Transforming Undergraduate Education for Future Research Biologists (National Research Council, 2003), emphasizes that “to successfully undertake careers in research after graduation, students will need scientific knowledge, practice with experimental design, quantitative abilities, and communication skills” (p. 2). Although much of the report focuses on the importance of engaging students in laboratory experiences, researchers have recognized that student participation in structured journal clubs is a highly effective approach to active learning (Glazer, 2000). This type of experience also contributes to the report's identified goals of increasing students' scientific knowledge and critical thinking abilities as well as improving their communication skills. In 1998, a unique collaboration between the UCLA undergraduate College of Letters and Science and the School of Medicine was established to engage highly motivated undergraduates in a weekly journal club and research meeting that would supplement their laboratory experiences. The overarching goals of this program were to expose students to a broad range of biomedical research topics, help them develop the skills necessary for critically analyzing scientific research articles, and allow them to practice effective scientific communication.

The program explored two hypotheses: 1) Training undergraduates in the primary literature is an effective tool for increasing scientific literacy; and 2) literature training will significantly facilitate the students' transition to postgraduate study. A number of assessment techniques have been used to gauge the outcomes of program participation. Annual student evaluations have been distributed and analyzed since the program's inception. In addition, during the 2004–2005 academic year, program alumni were contacted to participate in an e-mail survey, and interviews were held with a number of professors who served as advisors to multiple program participants. Findings from the three assessments indicate that participation in the program is advantageous to students not only during their undergraduate experience but also during the application and interview process for graduate and medical school and throughout their graduate training.

BACKGROUND INFORMATION

An examination of recent literature on undergraduate science education emphasizes the importance of actively engaging students in the practice of scientific exploration and communication. BIO 2010 (National Research Council, 2003) recommends that students should pursue independent research as early as possible in their education and that “seminar-type courses that highlight cutting-edge developments in biology should be provided on a continual and regular basis” (p. 9). The report emphasizes that “independent research gives students a real world view of life as a researcher” (p. 4). This is supported by a recent study by Lopatto (2004), who surveyed 1135 undergraduate students about the effects of participating in an undergraduate research experience. He found that these students gave very high rankings to the gains they made in “understanding of the research process” (4.13 on a scale of 1 [no gain] to 5 [very large gain]), but lower rankings to “understanding primary literature” (3.68) and “skill in oral presentation” (3.42). These results indicate that participation in a research experience alone is effective in developing some scientific skills more than others.

Muench (2000) reports on the “unique potential” (p. 255) of primary literature to familiarize students with the nature of scientific reasoning and communication. Smith (2001) states that scientists must be able to read, comprehend, and discuss primary literature, but warns that “many science majors reach their junior and senior years fearing and being intimidated by having to read and interpret primary literature” (p. 466). A number of instructors have addressed this deficiency in the standard curriculum by using primary literature as part of an undergraduate lecture class. DebBurman (2002) found that the utilization of this pedagogical approach successfully imparts strong scientific subject content and develops sophisticated scientific and intellectual skills, and Mulnix (2003) indicated that her students reported gains in self-confidence, communication skills, and their ability to read and understand the methods and results sections of journal articles.

Whereas it is clear that these two active processes provide different benefits to students, there appears to be little or no research on the effect of engaging students in the complementary processes of conducting research and participating in an intensive journal club experience. In addition, none of the studies cited above examine the long-term effects of these programs on students as they move from undergraduate to graduate-level study. This article addresses this gap in the literature by reporting on the structure and outcomes of an innovative program at UCLA.

In 1998, the Howard Hughes Medical Institute (HHMI) provided a grant to UCLA's undergraduate College of Letters and Science to support a number of projects designed to engage undergraduate students in research and encourage them to pursue graduate education in the sciences. Part of this funding was used to create the Howard Hughes Undergraduate Research Program (HHURP), a collaboration between the College and the University's Medical School. Under this competitive program, students who are engaged in research in a UCLA lab and are interested in pursuing graduate degrees (M.S., Ph.D., M.D., or M.D./Ph.D.) are encouraged to apply in the fall of their junior year to become Howard Hughes scholars. Applications are judged on the basis of the students' academic accomplishments, career goals, and research plans; a letter of support from their research mentor; and an interview with the three program coordinators. Approximately seven students are accepted each year for this 18-mo program that offers the following components: a weekly journal club, research presentations, seminar speakers, career guidance, and a scholarship paid out over the course of the program. Thus, the aggregate class size each year for juniors and seniors is typically 14 students.

The students selected for the program represent a range of academic interests and backgrounds. Over the last 5 years, 23 males and 20 females have participated. They encompass many ethnicities and include underrepresented minorities, but these parameters are not formally monitored. Their majors, in order of popularity, are as follows: molecular, cellular, and developmental biology; microbiology; neuroscience; biochemistry; psychobiology; ecology and evolutionary biology; and physiological sciences.

Three faculty members are dedicated to different aspects of the program, not including the students' research mentors. This low student-to-faculty ratio is essential for realizing the program goals. The involvement of the Medical School faculty, including Dr. Carey and Dr. Colicelli as program coordinators, distinguishes this program in three ways. First, the program coordinators, based on their experience teaching at UCLA's Medical School, recognized that the lack of exposure to primary literature is a major weakness of incoming graduate students. A survey of first-year students in the UCLA interdepartmental graduate program for biomedical sciences found that only a third had any prior experience in a journal club learning environment. Instruction in medical and graduate school programs requires students to interpret and build on the primary literature, and these skills are also essential for students to continue learning independently. Consequently, a portion of the graduate curriculum is devoted to what might be considered remedial learning (development of skills needed to evaluate the primary literature). The HHURP is intended to remedy this situation by allowing them to develop these skills as undergraduates. Second, the coordinators designed the journal club, both in terms of the articles selected and the level of discussion expected, to treat the HHURP scholars like graduate students rather than undergraduates. They felt this was the best approach for preparing students for a graduate school environment that relies on student–mentor and peer–peer learning interactions. Third, the coordinators serve as mentors to the students, providing academic and career counseling as well as feedback on scientific knowledge and presentation skills.

PROGRAM COMPONENTS

HHURP is a multifaceted program, designed to complement the students' laboratory experience and prepare them for other demands of graduate school. All HHURP participants, as a condition of maintaining their scholarship, are required to be involved in ongoing research for the five quarters between winter of their junior year and spring of their senior year and during the intervening summer. Before being selected as an HHURP scholar, they typically have been working on a project for anywhere from 3 to 12 mo. Most of the students are involved in National Institutes of Health R01-funded projects under the direction of a faculty member or a postdoctoral fellow and work alongside current graduate, and possibly other undergraduate, students. The undergraduates are invited to participate in lab group activities, such as weekly meetings, although their attendance and level of participation may be limited by their other class obligations or their lack of experience or confidence.

Weekly Journal Club

The central component of the HHURP is student participation in a weekly seminar entitled, “Topics in Contemporary Biology.” Most of the class sessions are run as a journal club, with one student presenting the results of a published research article. The remaining sessions are entitled “research seminars” and provide students with the opportunity to present the results of their own research. Each student is expected to present both a journal article and his or her own research once each academic year. The sessions are held on Friday afternoons from 4 pm to 6 pm.

For the journal club presentations, students can select an article from a list drawn up by the coordinators from journals such as Science and Nature, or they can get faculty approval for an article selected on their own. The week before their presentation is to be made, the students meet with one of the coordinators for 1 to 2 h to discuss any questions they have about the article and to receive input on their electronic presentation. The students are also responsible for creating a list of questions focused on key techniques and scientific principles discussed in the article. These questions, along with the selected research article, are distributed in advance to the other HHURP scholars, who are each assigned one question to which they must respond during the presentation. This process ensures that all students become “experts” in one aspect of the presentation and that they are actively engaged in each of the presentations.

A UCLA faculty member with expertise in the subject area of the journal article being discussed is invited to serve as a resource to the student making the presentation and to attend the class to provide feedback and insights based on his or her own related research. This individual's presence is intended not only to provide additional content expertise, often including unpublished advances in the field, but also to increase the students' comfort level in discussing scientific research with future colleagues.

After the presentation, the students participate in a round-table discussion during which they react to a series of focus points distributed in advance of the class. This discussion allows the students to evaluate the presenter's conclusions and provide constructive suggestions about the research described. At the end of the class, the faculty coordinators meet privately with the presenter to provide them with critical feedback, both to point out effective content and presentation techniques and to guide them in areas that can be strengthened for future presentations.

Research Presentation Opportunities

HHURP scholars are provided with a number of opportunities for presenting their own research. As discussed earlier, each student presents findings from his or her own research projects once a year during a “research seminar.” As with the journal club presentations, students meet with a coordinator before their presentation to discuss content and visuals, and they are expected to use electronic presentation software. Background research papers or relevant review articles may be distributed in advance to familiarize the other scholars with the presenter's area of research, because the student's research has not been published. The students' research mentor and other lab personnel are invited to attend these presentations. Students receive feedback on their presentations from the coordinators after the session.

The scholars are also required to participate in UCLA's undergraduate Science Poster Day, which is held each spring. This event is open to all undergraduates who have completed one or more quarters of research on a life, biomedical, or physical science project. HHURP students use their peers and the program facilitators as resources in preparing their posters. During Poster Day, students make themselves available at assigned times to respond to questions from other students, faculty, and assigned judges. Dean's Prizes are awarded for outstanding research. Students are also encouraged to attend national scientific conferences and are eligible for funding to cover meeting expenses if they submit an abstract for a poster presentation. They also become involved in editing and contributing to The UCLA Undergraduate Science Journal, a student-run academic journal featuring original research articles, review articles, and faculty interviews. Finally, during the annual spring banquet honoring HHURP scholars and their mentors, the graduating seniors give formal research presentations.

Seminar Speakers

To enhance the students' access to exceptionally accomplished biomedical researchers and to give them the chance to interact informally with these individuals, the scholars select and invite a number of speakers to campus each year. These individuals are asked to give a seminar presentation and to join the HHURP scholars for lunch. Research articles from the invited speakers are incorporated into the journal club presentations to promote greater interaction between the scholars and the guest. During a lunch, the students are able to talk with the guest about their own research projects and to get input and advice on career plans. One of the seniors introduces the guest speaker at the start of the seminar, and other students host the visiting scholar throughout the day, escorting him or her around campus.

Career Guidance

One of the three program faculty meets individually with each student during the year to provide him or her with feedback on performance in the program as well as career guidance and advice on graduate school admissions. The program-related conversations are focused on helping the students develop skills for working as part of a group. In terms of career advice, the faculty coordinators have experience on admissions committees at the UCLA Medical School and interdepartmental graduate program and are able to provide insightful guidance about what graduate programs look for in successful applicants. One of the coordinators conducts an M.D./Ph.D. workshop that brings faculty from UCLA's School of Medicine to speak with students about the different postgraduate tracks available at the school. Students are also encouraged to speak with other researchers in a variety of career paths. The faculty track students during their graduate application and interview processes and are available to provide letters of reference and advice as the students move through this process. During the journal club sessions, the seniors who are in the midst of applying to medical and graduate school are able to share their experiences with their peers.

PROGRAM OBJECTIVES

The long-term goal of the HHURP is to prepare students to successfully transition to graduate or medical school by immersing them in a learning approach that is used commonly at the graduate level but rarely in undergraduate classes, and by holding them to similar performance levels in terms of knowledge and presentation skills that the faculty would expect from first-year graduate students. This overarching aim is grounded in three specific, short-term goals that have guided the design of the program.

First, the program exposes students to a broad range of biomedical research topics at a sophisticated level well beyond what is taught in lecture classes. The intent is to give them a sense of how scientists in different areas of biology identify relevant questions and test their hypotheses. This is done through an examination of current primary literature, not simply by having students summarize the results and conclusions of other scientists. This emphasis on discovery complements the students' concurrent experience in the laboratory.

Second, the students are expected to develop the skills necessary for critically analyzing scientific research articles. This process not only strengthens their ability to evaluate others' research as they move forward in their own scientific careers, but ultimately affects their own efforts as researchers. The critical analysis of research is a crucial skill for graduate students and scientists. The types of issues that the students are instructed to examine as they prepare their journal club presentations include the following:

• Why is this research important?

• What is the intellectual framework for the problem or area being covered in the article?

• What is the experimental design for the research reported in the article?

• How do the data support the conclusions and fit into the model?

• What are the future problems or directions in this field?

Third, the students practice effective scientific communication by doing two presentations during the course of the academic year—one on a journal article of their choosing and one on their own research—and by observing their classmates' presentations. Preparing and delivering basic research and journal club presentations helps students to understand how basic science is performed and requires that they master the content of the articles to the extent that they can coherently relay the central ideas to the group. Through meetings with the faculty coordinators and a set of instructions available to all the HHURP scholars, the students are given guidance on the basic techniques for making presentations understandable and engaging. These range from simple guidelines about making text and figures sufficiently large to be read from the back of the room to more sophisticated advice about focusing on key points and being prepared for the questions and discussions that their presentation might evoke.

The structure of the journal club ensures broad-based participation. When students do not actually make the presentations, they are assigned a technical question to answer, which means that all students read the article and each one of them becomes the “expert” in one small area addressed in the article. Finally, students are expected to present in front of noncourse faculty, specifically their mentors and subject area specialists, as well as their peers. They are also able to talk informally with the seminar speakers who visit the campus. These interactions with experienced researchers build the students' confidence and convey that they are actually part of a community of scholars.

ASSESSMENT PROCESS AND RESULTS

Three types of assessments have been used to evaluate the effectiveness of the HHURP, and specifically the journal club, in meeting these goals. Sundberg (2002), Creswell (1998), and others have emphasized the importance of collecting information from a range of sources and using a number of different methods to collect data. Isaac and Michael (1981) state that “the triangulation of measurement process is far more powerful evidence supporting the proposition than any single criterion approach” (p. 92). These three assessments incorporate multiple perspectives and include survey questions, open-ended questions, and interviews. Since its inception in 1998, the program directors have had students complete an annual evaluation, which includes both closed- and open-ended questions. During the 2004–2005 academic year, two additional assessment processes were undertaken: interviews with four of the UCLA professors who have served as advisors to multiple HHURP scholars, and an e-mail survey of the alumni who graduated from the program between 1999 and 2004. These additional evaluations provided two new perspectives on the program. The first was from the advisors who worked most closely with the HHURP students on their research and were able to observe the students' intellectual growth over a period of years. The second was a long-term perspective from the program alumni who were able to reflect on the impact of the program as they applied for and became engaged in their graduate-level training programs.

In interpreting these assessment results, it is important to be aware that these students had an impressive record of academic achievement before their acceptance into the HHURP. In addition to their participation in the program, each of them also benefited from their involvement in laboratory research and the mentoring provided by the professors and graduate students with whom they worked. Although these assessment efforts have attempted to differentiate the effects of the program from the impact of other types of support they received, this cannot be done definitively without a comparison to a control group. Such a group was not identifiable on the campus. Therefore, these findings should be interpreted in that context.

Annual Evaluations

Since the 1998–1999 academic year, the faculty coordinators have distributed an annual student evaluation to the HHURP scholars during the spring quarter. The surveys used a 1–9 scale, with 1 indicating “disagree strongly” and 9 indicating “agree strongly.” The use of the 9-point range is largely historical and based on the evaluation scales used by many UCLA departments for seminars and didactic courses. In particular, the Biological Chemistry Departmental Teaching Committee feels that an expanded scale allows students to more accurately express their positive or negative feelings about a course and helps the Teaching Committee to distinguish above average to exceptional performance in the class. Over this period, 64 responses were received, including students who completed the evaluations twice (during their junior and senior years). All students completed the annual evaluation.

Table 1 presents the mean scores for the evaluations conducted between 1998–1999 and 2003–2004.

Overall, these responses have proved to be highly reliable. An analysis of variance showed that the results did not differ significantly (using a 0.05 significance level threshold) from year to year for any of the questions, except for “overall evaluation of the research seminars,” where the significance level was 0.005. This lack of consistency may be attributable to an improvement in this component. The first year this was evaluated (2000–2001), the rating was 7.2; in later years, the rating ranged between 8.2 and 8.7. The instructors have been experimenting with techniques to bring about greater involvement of the students in their colleagues' research, which might account for the higher scores in later years.

Cumulatively, these results provide a uniformly positive assessment of the program in terms of the goals of reading and presenting scientific research. It should be noted that a self-assessment of being “more comfortable” reading and presenting papers is likely to reflect not just gaining skill in these tasks but also becoming familiar with something with which they previously had little experience. The interesting discrepancy between the students' high ranking for presenting their own research (8.68) and the lower overall ranking for the research seminars (8.19) might be attributed to the relatively low 2001–2002 score (as detailed above) or the fact that students are involved in a variety of research projects that are unfamiliar to their colleagues, so the presentations may be more challenging to understand, especially given the lack of a formal paper to read in advance of the presentation. From a practical standpoint, these differences are minimal, with all rankings averaging above 8.0. The ratings also reflect positively on the inclusion of outside experts as commentators during the journal article presentations, which the course faculty believe significantly increases the level of discussions and helps the students to evaluate the real contribution of the article to the field, both of which are essential skills necessary for scientific success.

Advisor Perspective

Four UCLA faculty members have served as research advisors to more than one HHURP scholar during the first 7 years of the program. These individuals were interviewed to ascertain their perceptions of the HHURP program in general and to determine specifically whether they were able to attribute any academic or personal growth observed in these students to their participation in the HHURP. The advisors emphasized that each of the scholars with whom they worked were already outstanding students before they were chosen as HHURP scholars, but the faculty were able to perceive some additional advantages that the program afforded the students.

The faculty identified two benefits to students participating in the HHURP. First, they saw a clear boost in the self-confidence of the students selected for the program. Second, they were aware that the students benefited from a substantial amount of assistance and guidance with their presentation skills. However, one professor mentioned that he addressed presentation skills in his weekly lab meetings, so, in his case, it was unclear whether his students' improvement in this area was attributable to HHURP participation or to their involvement in the lab meetings.

As a general confirmation that the journal club discussions were targeted at a high level, one of the professors, who had attended a number of the journal club presentations as the subject area expert, remarked, “I thought the programs were exceptional. They were very much like at least first- or second-year graduate students.”

Their comments provide additional anecdotal evidence about the value of the program, although the small number of professors interviewed and the informal means of communication limit our ability to draw broad conclusions from these observations.

Alumni Perspective

Since the 1998–1999 academic year, 40 of UCLA's science majors graduated from the university as HHURP scholars. All of these alumni were contacted by e-mail during the 2004–2005 academic year and were asked to respond to a survey that gave them the opportunity to assess the long-term value of the program from the perspective of a graduate or medical student. The survey included both closed- and open-ended questions and focused particularly on the effect of participating in the weekly journal club. The survey was sent twice by e-mail, and responses were received from 33 of the alumni, for a response rate of 82.5%.

The alumni were asked to rate the impact of their journal club participation on 11 different areas using the following scale: 1 = negative effect; 2 = no effect; 3 = somewhat positive effect; 4 = positive effect; and 5 = very positive effect.

The choice of a five-point Likert scale is consistent with much of the education assessment literature (Sundberg, 2002). The scale was designed to be positively skewed, based on the cumulative responses to the annual surveys that had been conducted over the prior 6 years. Students' assessment of the program had averaged higher than 8.0 on the 9-point scale, and the researchers were concerned that a fully balanced scale (1 = very negative effect to 5 = very positive effect) would decrease the variation in responses because it would only allow students to choose from two, rather than three, positive choices. The inclusion of open-ended questions allowed the researchers to examine whether the responses to the scale questions were consistent with the narration provided for the open-ended questions.

The mean responses from the alumni are provided in Table 2.

Responses to 10 of the 11 questions indicated that the alumni felt their participation in the journal club had between a “positive” and “very positive” effect on the specified skill. Not surprisingly, given the format and goals of the journal club, the highest scores were reported for improvements in participants' abilities to develop and deliver presentations, followed by an improved ability to understand scientific literature and scientific presentations, critique scientific research, and interact effectively with others in a research seminar. Not only do these indicate that the short-term goals of the journal club are being met but that they produce positive benefits for the participants as they move on in their academic studies, as is confirmed in the responses to the open-ended questions. Responses to the question about the impact of the journal club on the students' ability to design and implement their own research averaged between “somewhat positive” and “positive.” This lower score may be explained by the fact that students' research projects were under way when they became HHURP scholars, and the research presentations they made through the journal club were designed to strengthen their presentation and analytical skills rather than their research skills, which are the responsibility of the students' mentors.

The alumni were then asked to reflect on, “How, if at all, did your experience as a Howard Hughes Scholar and journal club participant assist you with your graduate/medical school applications and interviews?” Alumni responses fell into four main categories. First, alumni indicated that the journal club enhanced their ability to discuss their own research in an articulate, sophisticated manner, which was very helpful to them in the interview process. Second, the range of articles presented and discussed in the journal club broadened participants' scientific knowledge base beyond their own area of expertise, which allowed them to converse comfortably with scholars in a number of fields during their interviews. One alumnus said, “Having presented my research to the HHURP, I was very confident and skilled in discussing my research with both seasoned researchers and lay people. I was also able to quickly understand others' research and formulate thoughtful questions to ask them. These skills helped me in interviews with scientists, physicians, and medical students.” Third, a number of the alumni mentioned that they received letters of recommendation from the program coordinators. Finally, the responses indicated that their association with the HHMI caught the attention of admission committee members. A number of responses included comments similar to the following: “It seems as though being a Howard Hughes Scholar greatly distinguishes me from the thousands of other applicants.”

Another question asked respondents to relate whether their participation in the journal club provided them with any relevant skills they have been able to use in graduate studies. Every person enrolled in a graduate program responded affirmatively to this question. In fact, a number of them indicated that the format of the journal club was similar to their graduate courses, so it gave them an advantage over their classmates because they had already learned some of the skills required for participation in an interactive seminar. One person went so far as to say that in the journal club, “I was around people smarter than me and more ambitious than me, which inspired and pushed me. As a result, the first year of grad school was a step back.” Some of the specific benefits resulting from participation in the journal club were the ability to read research articles quickly and efficiently while extracting the most important information from them, increased comfort in “conversing scientifically with colleagues,” confidence in dissecting and criticizing a research paper, and the ability to organize and present scientific data to an audience. One of the alumni reported, “The analytical skills I learned in the program are still used in my education at the graduate level. However, in addition to specific skills, the most important thing I gained from HHURP is just an overall increased intellect and ability as a scholar.”

The alumni were also asked a general question about the primary benefits they received from participating in the journal club. In addition to the skills they acquired, as described above, a number of them mentioned several affective outcomes of the program. First, they found a supportive community with a cohort of like-minded students, or as one alumnus mentioned, “a very bright group of people who had fun talking about science over pizza.” Alumni indicated they enjoyed getting to know both the faculty and students involved in the journal club. Second, the journal club fostered a sense of excitement about science and research. One alumnus indicated that “it got me excited about research because it pushed me to think critically,” whereas another stated, “it made me love basic science research.” Third, participants gained confidence in their own abilities as consumers and presenters of research. “My participation gave me more confidence in presenting my research, thinking about research questions, and conversing with faculty about research.” One additional benefit mentioned repeatedly was the exposure they received to scientific studies and ideas outside their major or main area of research. Alumni felt this broadened their knowledge base and allowed them to communicate with a wider array of scientists.

One additional question asked alumni for their suggestions for improving the organization or format of the journal club, from their perspective as a graduate student. Many of the responses indicated that the alumni would change nothing about the program, and a number of suggestions from the early cohorts of participants, such as utilizing electronic presentations, had already been addressed by the program coordinators. However, there were two areas in which the alumni offered constructive suggestions for the program. The first was, perhaps, an inevitable consequence of a program in which the academic level of the participants is so high. Some alumni felt that participants were afraid to ask clarification questions for fear of looking unprepared or ignorant. “I worried sometimes that students were intimidated by their peers and would therefore avoid asking questions when they were confused.” Another person emphasized “the importance of fostering an environment where students feel comfortable sharing their views and/or asking questions.” A number of the alumni did indicate they felt that the journal club provided a supportive environment for students, but that an extra effort should be made to encourage the active participation of shy students. The second suggestion was to consider doing away with assigning questions for students to respond to during the journal article presentations and instead allow students to come up with their own questions about the articles. “I would suggest a more open discussion that encourages everyone to participate on their own, and less of a direct questions approach.” The common thread in these suggestions seems to be an interest in increasing student involvement.

The alumni overwhelmingly felt that their experience with the journal club provided them with many benefits, including analytical and presentation skills, knowledge outside their area of research, confidence in their own abilities as researchers, assistance in the application and interview process for graduate or medical school, and a preview of the graduate school experience.

CONCLUSION

As prescribed in BIO 2010, the students who have been part of the HHURP have been actively engaged in the discovery process of scientific research, both through their own work in a laboratory and through their extensive interaction with primary literature. As undergraduates, they indicate that the program enhanced their research and increased their ability to read and present primary scientific research and to present their own research. After graduating from UCLA, they found that their role as an HHURP scholar helped them in securing admission to the M.D., Ph.D., or medical scientist training program of their choice and provided them with an advantage over their graduate school peers in participating in the interactive seminars that are the foundation of graduate education. The students' research mentors perceived in their students an increase in confidence and the ability to present research that they were able to attribute to participation in the HHURP.

Above and beyond the perceptions of the students, advisors, and alumni of the program, perhaps the strongest indicator of the success of the program is the record of achievement of its participants at UCLA and after graduation. In 2003, 2004, and 2005, a total of 321 undergraduates participated in UCLA Science Poster Day. HHURP scholars received 11 of the 28 Dean's Prizes awarded at those events. After graduating, 70% of the alumni moved on to Ph.D. (11 alumni), M.D. (13 alumni), or combined M.D./Ph.D. (five alumni) programs at top universities, including Massachusetts Institute of Technology, Harvard, UCLA, Cal Tech, Yale, Columbia, and the University of Chicago. Five of the students enrolled in medical scientist training programs and two more in other types of master's programs. The others are either working in biomedical research or pursing other postbaccalaureate studies. They partially credit their HHURP experience for these outcomes. As one alumnus said of the journal club, “it was one of the best educational experiences I had as an undergrad in terms of preparing me for a future in medicine and science.”

We can conclude that an intensive literature-based training program increases student confidence and scientific literacy during their undergraduate years and facilitates their transition to postgraduate study. Although we used a team of three faculty to train 14 undergraduates per year, elements of our program, i.e., those that enhance student participation, should be applicable to undergraduate journal clubs led by individual faculty members. A key issue in the future will be determining how to extend this approach to the many undergraduate students who would benefit from primary literature-centered training at universities with large undergraduate enrollments.

REFERENCES