ACE Assessment Instructor Guide

All ACE courses are assessed by applying an ACE-specific rubric to a signature assignment for each student in that course. While the minimum of a single assignment should be assessed, an instructor may wish to assess multiple assessments given the ease of integration with Canvas and the increased feedback that provides to an instructor.

This process facilitates aggregate reporting of ACE achievement across courses all the way up to the institution level. Furthermore, and equally important, the ACE-specific rubrics facilitate revision at the course-level as instructors consider alignment with the ACE-specific rubrics.

Options for Submission Type

There are three different options for setting up ACE assessment of a signature assignment.

  1. Students submit product online
    Students upload a product. It can be a word processor file, a pdf, or an image file. This is appropriate for most courses.
  2. Instructor’s complete rubric without online submission
    An instructor completes an assessment for each student based on a paper submission or an oral submission.
  3. Scores on elements in a Canvas-based “Quiz”
    This can be an exam in the Exam Commons, or quiz outside of class. One or more questions, which can be an automatically graded question such as multiple choice, are mapped onto each row of the ACE rubric.
  4. Matrix entry
    Complete a web-based matrix of the number of students in the course that have satisfied each entry in the rubric. This is based on the assessment tool of the instructor’s choice. This is entered completely separate from Canvas.

Setting up ACE assessment (option 1 and 2 above)

  1. Go to an existing assignment in Canvas. While viewing that assignment, click on “+ Rubric”.
  2. Click on “Find a Rubric”. On the left-hand side choose “University of Nebraska – Lincoln” and choose the appropriate ACE rubric for that course. Note, click on name of rubric in far right window to select.
    1. Optionally, add additional rows to the rubric used to determine the assignment grade in the course. Click on the pencil icon on the upper-left. Use “+ Criterion” to add rows. Select the checkbox “Use this rubric for assignment grading”. Note, ACE rows will not be included in the grading score for this assignment, only the new rows you added.

Scoring ACE assessment rubrics (option 1 and 2 above)

  1. After receiving student submissions, click on the name of the Assignment and click on “Speed Grader”.
  2. Click on “View Rubric” for each student. Click on the appropriate cells. Be sure to click “Save”. Note, you can drag the middle bar to change the size of the left-hand and right-hand window (especially useful if there is no online submission). Depending on how the assignment was set up, the grade can be assigned in this same window or the grade can be calculated based on additional rubric rows. Note, rows can be left unassessed if they do not apply to the course or assignment (see additional notes below).
  3. After clicking on “Save”, use the arrows to click through each student. Complete the above step.

Setting up ACE assessment for “Quiz” option (option 3)

  1. In Canvas go to Outcomes (in the left-hand side, typically grayed out).
  2. Click on “Find” then choose “Account Standards” and navigate to “University of Nebraska – Lincoln”. Click on ACE and choose the appropriate ACE group. With that group highlight, click “Import” to import all outcomes for that group.
  3. In Canvas go to Quizzes (left-hand side) and click the gear button in the upper-right, choosing “Manage Question Banks”.
  4. Click on “Add Question Bank” and give the bank a name. Note, you will want a bank for each row within the ACE rubric.
  5. Click on the name of that bank.
  6. In the right-hand side click on “Align Outcome”.
  7. Navigate to the appropriate outcome, click on it and choose “Import”. That outcome will now be listed in the right-hand column.
  8. Add one or more questions to this bank. Note, only a single question needs to be added. Questions can be an automatically graded question like Multiple Choice or Multiple Answers.
  9. Repeat the above steps for each outcome, i.e. each row in the rubric.
  10. In Canvas go to Quizzes and click on “+ Quiz”.
  11. Fill in the appropriate details such due date and available dates, viewing of responses and answers, etc.
    1. To use the Exam Commons, click on “Filter IP Addresses” and click on the magnifying glass icon to choose the UNL Testing Centers. Details at: http://its.unl.edu/dlc.
  12. Click on the “Questions” tab at the top.
  13. Click on “New Question Group”
  14. Click on “Link to a Question Group”. Select the appropriate bank.
  15. Click on “Create Group”
  16. Repeat the above steps for each outcome, i.e. each row in the rubric.
  17. Click on “Save” or “Save & Publish”.

Viewing results for your course

  1. In Canvas/Settings choose the “Feature Options” tab. Enable “Learning Mastery Gradebook”.
  2. Complete (a) and (b) below unless you previously set up a “Quiz” via option C above. In that case you have already completed this and you can skip ahead to the next step.
    1. In Canvas go to Outcomes (in the left-hand side, typically grayed out).
    2. Click on “Find” then choose “Account Standards” and navigate to “University of Nebraska – Lincoln”. Click on ACE and choose the appropriate ACE group. With that group highlight, click “Import” to import all outcomes for that group.
  3. In Canvas/Grades choose the top tab “Learning Mastery”
  4. The very top line will display the course average (it can be changed to mode or median). Mouse over the words in the Learning Outcome row to pop-up a summary for each outcome. Below is listed the outcome performance for each student.
  5. The “Export report” link in the right-hand column will download a csv file listing the rubric-row score for each student. The rubric scores are in every other column, which end with the word “result”.

Advanced Notes

  1. Turnitin can be used to check plagiarism. If the “Enable Turnitin Submissions” check box exists under the “Online” submission type, then TurnItIn can easily be used following the above directions for option A. However, that checkbox may go away (as the API integration with TurnItIn is depreciated in favor of the LTI integration). In the absence of the checkbox, the rubric has to be added before linking it to TurnItIn; follow these directions:
    1. Create an assignment in Canvas with the Submission Type as “No Submission”. Save the assignment.
    2. While viewing that assignment, click on “+ Rubric”, and follow the main directions above in “Setting up ACE Assessment (option 1 and 2)”
    3. After the rubric has been added, click on Edit at the top to edit the assignment. Change the Submission type to “External Tool”.
    4. Click “Find” and scroll the inner box to “TurnItIn LTI”. Click on that option and choose Select. Click Save at the bottom of the assignment edit page.
    5. Click on “Speed Grader” and follow the directions on “Scoring ACE assessment rubrics (options 1 and 2)”. Scoring of ACE rubrics must be completed in Speed Grader.
  2. One can assess a subset of rows from the rubric and leave others unassessed. However, the most elegant approach is to build a custom rubric that contains only the desired rows.
    1. Add the ACE rubric to the assignment, then edit each rubric to delete the rows not in use for a given assignment.
  3. One can use multiple assignments for ACE assessment.
    1. You can apply the rubric to multiple assignments and fully assess each assignment.
    2. Alternatively, you can use different assignments for different rows of the rubric. Simply add the ACE rubric to each assignment, then edit each rubric to delete the rows not in use for a given assignment.
  4. One can use the ACE outcomes as direct contributions to an assignment score, but the point contribution for each row is constrained to be 4 points.
    1. In Canvas go to Outcomes (in the left-hand side, typically grayed out).
    2. Click on “Find” then choose “Account Standards” and navigate to “University of Nebraska – Lincoln”. Click on ACE and choose the appropriate ACE group. With that group highlighted, click “Import” to import all outcomes for that group.
    3. Go to an existing assignment in Canvas. While viewing that assignment, click on “+ Rubric”.
    4. Click on “Find Outcome”. Navigate to one of the outcomes you just imported above.
    5. Keep the check-box by “Use this criterion for scoring.” The 4 points for that row will now be used in calculating the total points for that assignment. Note, it has to be 4 points.
    6. Repeat the above two steps for each row of the ACE rubric.
  5. If a course is certified for two ACE outcomes (for example ACE 5 and 9), the easiest option is to have two different assignments, each assigned to a different outcome and hence a different rubric. The other option is to combine both rubrics into a single rubric for a single assignment:
    1. Click on “Find” then choose “Account Standards” and navigate to “University of Nebraska – Lincoln”. Click on ACE and choose the one of the appropriate ACE groups. With that group highlighted, click “Import” to import all outcomes for that group.
    2. Go to an existing assignment in Canvas. While viewing that assignment, click on “+ Rubric”.
    3. Click on “Find a Rubric”. On the left-hand side choose “University of Nebraska – Lincoln” and choose one of the appropriate ACE rubrics for that course (i.e. the one you did not choose above).
    4. Click on the pencil icon on the upper-left.
    5. Click on “Find Outcome”. Navigate to one of the outcomes you just imported above.
    6. Uncheck the box in front of “Use this criterion for scoring”. Click Import.
    7. Repeat the above two steps to include all of the rows from this second ACE.
    8. It is recommended you change the Title of the rubric to reflect its hybrid nature.
    9. Click Update Rubric.
  6. Using an ACE rubric makes it easy to assess work from all students in a course. However, faculty should assess as much student work as needed to support a meaningful reflection across time and to sufficiently answer any question of their interest. As a guideline, if faculty choose to sample student work and there are more than 35 students in a course, one can manually choose 35 students at random for which to complete the ACE rubric, leaving the remaining students unassessed. The minimum of 35 students was based on a power analysis to achieve sufficient resolution for discerning changes in rubrics scores over time