Skip to main content

Outcomes Data


Assessment of Learning Objective 1  (Fundamental knowledge in the diverse fields of Chemistry)

Criteria 1: CHEM 4990 capstone course exit exam.

Each of the sixty exit exam questions has been mapped to specific learning objectives associated with the core chemistry degree program courses. This allows the department to track overall performance in the six major areas of chemistry associated with the degree program (General, Organic, Inorganic, Analytical, Physical and Biochemistry).  It also allows feedback to faculty teaching the specific courses as to areas requiring additional emphasis to enhance student performance.

Results from chemistry exit examination (by specific chemistry area)


Chem 4990- Exam Results for 2017
General Chem Questions  mean    55.3   STD= 28.6
Inorganic Questions  mean    40.0   STD= 20.1
Organic Questions  mean    64.0   STD= 19.9
Analytical Question  mean    54.0   STD= 27.6
Physical Questions  mean    44.0   STD= 28.8
Biochem Questions  mean    42.7   STD= 21.6
Overall Exam Score  mean    50.0   STD= 25.2

Exam Results (5 Year Average)
General Chem Questions   mean    64.3   STD= 8.8
Inorganic Questions   mean    49.7   STD= 6.9 
Organic Questions   mean    55.6   STD= 5.1
Analytical Questions   mean    57.6   STD= 2.8
Physical Questions   mean    44.1   STD= 3.1
Biochem Questions   mean    51.4   STD= 5.7
Overall Exam Scores   mean    54.5   STD= 3.1

Criteria 2: Other ACS examinations covering a specific chemistry area in depth.

Several courses have implemented the appropriate ACS standard examination as a final exam.  Data below is an example of the spring 2016 results for students taking the most current 2009 version of the ACS Instrumental Analysis examination as part of CHEM 5640 (a core chemistry class required for the chemistry degree).

Data below show the Spring 2017 and historical results.

2017 Average score of 31/50 = 85th percentile nationally

Historical Exam Performances For Reference

2016 Average score of 30/50 = 82nd percentile nationally

2015 Average score of 28/50 = 73rd percentile nationally

2014 Average score of 32/50 = 88th percentile nationally

2013 Average score of 30/50 = 82nd percentile nationally

2012 Average score of 28/50 = 73rd percentile nationally

The nationally normalized ACS Instrumental Analysis examination is used as the final exam for CHEM 5640 (a core senior-level chemistry class required for the chemistry degree).  Reviewing the results for specific exam questions showed students generally performed better on questions related to concepts and applications of spectroscopy and chromatography. As has been seen in the past, this is typical as this material is stressed in both the class and the associated laboratory. When students take both the class and laboratory concurrently (which is common and recommended), this tends to reinforce the class concepts and likely leads to better learning outcomes.

Assessment of Learning Objective 2  (Proficiency in basic laboratory techniques and laboratory safety; experience with modern lab instrumentation and experience working as a member of a team.)

These objectives are assessed in low enrollment laboratory courses run under the direct supervision of a faculty member and ensure students have developed proficiency in basic laboratory techniques and laboratory safety prior to graduation.

One measure is the performance of students in less structured laboratory environments such as faculty-directed Undergraduate Research Problems (CHEM 4800).  This is an elective course and not all undergraduates take this course, although those that do are required to submit a detailed written project report at the end of the semester. 

Another assessment occurs in the less structured laboratory experience that is part of the Chemistry degree core curriculum, the CHEM 5650 laboratory. In CHEM 5650, students perform eight structured labs and write detailed reports for each.  During the final month of the semester, students collaborate on a multi-week, team centric project. Student groups (typically 3 students per group) choose and design their own project, the results of which are presented in a detailed written final report.

For 2016, all of the submitted final projects and their reports were judged to be good to excellent. The average score for the final projects in the class was 87%.

For 2017, two of the three final projects and their associated reports were judged good or excellent. One of the final projects and its associated report was judged satisfactory. The overall average score for the final projects in the class in 2017 was 80%, which is below what is normally observed in a typical year.  This was primarily due to the small number of groups (three) in the 2017 course and the lower performance on the final project by one of the groups

Assessment of Learning Objective 3  (Communicate in written and oral formats scientific information to chemists and non-­chemists.)

Student proficiency in these skills is developed and assessed in several courses and laboratories during a students degree program.

  1. Writing skills are developed and assessed via the university designated communication intensive (CI) courses that the department requires to earn a chemistry degree.  Click here for more information on CI course requirements.

Required CI courses for chemistry degrees include: CHEM 3080/3090 (Physical Chemistry Laboratory I & II) and CHEM 4990 (Undergraduate Seminar Capstone class). As part of the two semester CHEM 3080/3090 CI courses, students are required to write a detailed laboratory report for each of a semester's experiments. This is typically eight laboratory reports per semester of 8-10 pages each.  For the CHEM 4990 seminar course, students attend all departmental seminars from visiting scholars are required to write a minimum of five, 1-2 page critiques of each seminar.  The critiques require that the student follow the material being presented in the seminar and to be able to coherently discuss what was learned from the presentation in a written format. The seminars also provide examples of various presentation styles that students can try to emulate (or not) in their own presentations.

  1. Oral communication skills of seniors are further developed, and assessed, in the Chemistry degree capstone course (CHEM 4990). Students are instructed on how to properly research a scientific topic of their choosing using a variety of library and Internet database tools.  Students are also instructed on how to present research results to a general scientific audience. Students do multiple in-class practice seminar presentations and receive feedback after each presentation to improve the quality of their presentations.  Finally, students are required to present both an oral presentation and a poster-based presentation to their undergraduate peers, selected faculty members and graduate students who provide constructive feedback on their presentations. The poster presentations (typically 90 minutes) are given during a departmental poster session and allows for a less formal presentation that is typical of many science conferences and allows students to interact one-on-one with those attending the poster session.

For the spring 2016 oral and poster seminar presentations, all students were deemed to have done very well to excellent on their final presentations. All students received combined scores of between 90% and 95% on their two presentations.

Assessment of Learning Objective 1:  Acquiring Fundamental Knowledge

  1. Students taking the capstone Chem 4890 class (a requirement of the major, taken during the senior year) are given the ACS Biochemistry exam.
  2. Each of the ACS exam questions have been mapped to one of the six objectives supported by the American Society of Biochemistry and Molecular Biology (ASBMB) (click here to view).  Transitioning to these objectives will facilitate future efforts to gain accreditation from the ASBMB.  In future years we will adopt the ASBMB exit exam in     place of the ACS Biochemistry exam.
  3. Each year the student scores will be compared to the national average score for the same questions reported for the ACS exam
  4. The 60 questions in the most recent ACS exam map as shown in Table 3.
Table 3
  1. The mean raw score achieved by the 2016-17 class on the ACS Biochemistry exam was 51% compared to the national mean percentile of 50%.  For comparison, the 2015-16 class scored in the 45th percentile.
  2. Figure 1 shows how the ACS exam results mapped onto the ASBMB objectives.
ACS 2016-2017 ACS Biochemistry Exam Results

Assessment of Learning Objective 2: Read, present, and analyze scientific literature and data

  1. These are skills are honed and assessed in Chem 5700/5710 (Biochemistry), Chem 5720 (Biochemistry lab) and Chem 4890 (Seminar Capstone class)
  2. In Chem 5700/5710, new short writing assignments have been included in the syllabi. Grading rubrics include content, grammar, and scientific correctness.  Average student scores increased in 5700 and 5710 by 7% and 9% over the semester, respectively.
  3. In Chem 5720, students are required to orally defend a research proposal and write extensive lab reports. In 2015 a new rubric for the lab reports was devised in   conjunction with the STEM center to assess learning based on scientifically validated criteria. For the 2016-2017 class, rubric scores increased by 26% over the course of the semester (from 68% for the first lab report to 86% for the last lab report).
  4. In Chem 4890, students are required to read the literature and to do several presentations that include both poster and verbal formats.  Rubrics for assessment are the same as those used in the graduate seminar class. Poster and verbal presentations by USU students in the spring of 2016-17 were scored at 90% or greater.


Assessment of
Learning Objective 3: Design experiments; choose and apply appropriate techniques to achieve a research goal; and demonstrate that they have good quantitative skills

  1. This objective is assessed using Chem 5720, the Biochemistry Laboratory course. The course is designed to give students a genuine research experience, and includes a two-month research project that requires students to develop their own protein purification protocol for a “mystery protein.” They are required to develop preliminary data on the protein expression and write a proposal for purifying the protein to at least 90% purity. The students are then required to give an oral defense of their proposal to a group that includes the instructor and several graduate students. Once the proposal is approved, the students are given two weeks in the lab to purify their protein. This is a difficult project for many students in part because it requires them to (1) clearly understand topics presented in lecture and described in the primary literature, and (2) apply the material in a real-life setting.
  2. As a means of numerically assessing student skills, the following rubric is used for the projects:
  • Project proposal.(150 points) This is a detailed description of your project justification, strategy, and timeline. The protein expression experiment must be fully described as part of the project justification/preliminary data.  An itemized list of required chemicals and reagents is also required. The TA and/or the instructor may request revisions before students are allowed to begin the experiment.
  • Proposal Q&A.(20 points) As a group, you will be required to meet with the TA and/or instructor to answer specific questions about your proposal. Each member of the group will receive a separate grade.
  • Lab notebook.(25 points) As with the other labs, you will be expected to keep a lab notebook each throughout the project. Each member of your group will turn in their own notebook at the end of the project.
  • Protein purification.(50 points) Full credit will be given to those who obtain 90% purity or better.
  • Final report.(135 points) The report will be similar to other lab reports, but only   one report will be submitted for each group. Consequently, it is expected that the report will be more detailed than those submitted for the other labs.
  • Team evaluation.(20 points) Each member of the group will evaluate the contributions of every other member of the group. If it becomes apparent that some individuals contributed more to the project than others (as judged by the evaluations and observations by the TA and instructor), the final grades for the project will be adjusted accordingly.
  1. The performance of students in the 2017 calendar year is shown in Figure 2. The average grade amongst 36 students was a 91%.
Chem 5720 Group Project Grades 2017

Assessment of Learning Objective 4: Overall preparedness to successfully gain entrance into high quality graduate schools, admission to professional schools, and securing quality careers in the chemical sciences.

Tracking of our graduates (where they are hired, and graduate/professional school acceptances) represents a major deficiency in our program.  While we maintain contact with some students (acceptance into high quality graduate and professional schools such as UCLA, Penn State, Duke, New York School of Medicine, University of Utah School) we are lacking in a comprehensive understanding of how the whole of our student body meets their post baccalaureate challenges.

Chemistry Teaching Major Outcomes Data

Prior to student teaching all teaching majors must meet a State of Utah mandated passing score in the ETS PRAXIS exam in their content area.  Knowledge of the content area for chemistry teaching majors as assessed by the PRAXIS exam for the past five years is below. 

 

PRAXIS Physical Sciences Content Exam Scores 2011-2015

2011 Chemistry, Physics and General Science: Attempted 0; Passed 0

2011 Physical Science - Content Knowledge: Attempted 1; Passed 1

 

2012 Chemistry, Physics and General Science: Attempted 1; Passed 1

2012 Physical Science - Content Knowledge: Attempted 1; Passed 1

 

2013 Chemistry, Physics and General Science: Attempted 1; Passed 1

2013 Physical Science - Content Knowledge: Attempted 0; Passed 0

 

2014 Chemistry, Physics and General Science: Attempted 0; Passed 0

2014 Physical Science - Content Knowledge: Attempted 6; Passed 5

 

2015 Chemistry, Physics and General Science: Attempted 0; Passed 0

2015 Physical Science - Content Knowledge: Attempted 0; Passed 0

2016 Chemistry, Physics and General Science: Attempted 1; Passed 1

2016 Physical Science - Content Knowledge: Attempted 2; Passed 2

 

Chemistry teaching majors take the chemistry major core classes.  Their outcomes data for the chemistry core are included in the data for the chemistry major.

Physical Science Composite Teaching Outcomes Data

Outcomes Data

PRAXIS Physical Sciences Content Exam Scores 2011-2015

2011 Chemistry, Physics and General Science: Attempted 0; Passed 0

2011 Physical Science - Content Knowledge: Attempted 1; Passed 1

 

2012 Chemistry, Physics and General Science: Attempted 1; Passed 1

2012 Physical Science - Content Knowledge: Attempted 1; Passed 1

 

2013 Chemistry, Physics and General Science: Attempted 1; Passed 1

2013 Physical Science - Content Knowledge: Attempted 0; Passed 0

 

2014 Chemistry, Physics and General Science: Attempted 0; Passed 0

2014 Physical Science - Content Knowledge: Attempted 6; Passed 5

 

2015 Chemistry, Physics and General Science: Attempted 0; Passed 0

2015 Physical Science - Content Knowledge: Attempted 0; Passed 0

2016 Chemistry, Physics and General Science: Attempted 0; Passed 0

2016 Physical Science - Content Knowledge: Attempted 0; Passed 0