For Faculty/Staff
Assessment Plans:
History and Development of Administrative Assessment Plans and Report
History
In 2000 a 2-year assessment plan and reporting cycle was developed to document our ongoing institutional effectiveness efforts. These plans and reports included program mission and goals based on the University's mission and goals along with the Strategic Plan; intended outcomes consistent with the program's mission and goals; methods for assessing each intended outcome, a summary of data collected related to each intended outcome and the use of the results of these evaluations to improve educational programs, services and operations.
Old Dominion University recently licensed WEAVEonlineSM, which is a web-based assessment management system, to facilitate the process of developing and posting assessment plans and results. WEAVE is a web-based, information system to support the development of an effective fully-integrated planning, budgeting, and assessment process, one that will facilitate communication across the campus and beyond.
WEAVE is an assessment cycle in which you:
· Write expected outcomes/objectives.
· Establish criteria for success.
· Assess performance against criteria.
· View assessment results.
· Effect improvements through actions.
As a "learning organization" it is essential to develop a common and widely-distributed method of developing objectives, evaluating outcomes, and making improvements based on those outcomes. WEAVE provides the mechanisms to accomplish this and to monitor:
· Improvements in student learning, both inside and outside the classroom, that is, the outcomes established for our academic programs and educational / administrative support units,
· Old Dominion University's institutional effectiveness,
· Performance measurements as required by the new budget planning process,
· the success of the 2005-2009 Strategic Plan,
· compliance with the requirements of the Southern Association of Colleges and Schools - Commission on Colleges (SACS/COC), and
· departmental accomplishments through the production of annual reports.
Developing and Implementing Administrative Assessment Plans
top
1. Become a learning organization - Organize for assessment - Assessment of student learning for the purpose of continuous quality improvement (CQI) is a requirement for SACS/COC accreditation and many specialized accrediting bodies. Assessment is also required by SCHEV. Locally, assessment is required to evaluate progress on the 2005-09 Strategic Plan and support performance based budgeting. Last, and most importantly, setting goals, collecting and analyzing data, and using the data to improve our programs is part of what we do as professionals in a learning organization. While one person in a department may take the lead in assessment for the department, all members of the department should participate in the assessment process in their department.
2. Define the mission and goals of the department - The mission of the department is the starting point for developing assessment plans. Presumably the department mission will dovetail with the missions of the vice presidential area and the University as well as the goals of the 2005-09 Strategic Plan. Mission statements often take the following form: "the mission of the (department) is to (primary purpose) by providing (primary functions and activities) to/for (stakeholders / customers)." For example, "The Office of Institutional Research and Assessment (IRA) serves as the analytical arm of Old Dominion University's central administration. As a part of its function, the office assists in systematically planning the University's future, managing its resources, and assessing its performance….The Assessment Program focuses on assessing … (3) learning outcomes of each academic program and other outcomes of each administrative unit with an Assessment Planning and Reporting Process …. Results are consistently used for program improvement." Goals are broad and abstract statements that flow out of the mission.
3. Define the department outcomes - Outcomes are more focused and concrete statements than goals that flow out of the mission and goals, as well as from professional organizations. Outcomes in assessment should NOT be confused with the tasks that need to be completed in order to accomplish the goal. It is the outcomes that are assessed. Outcomes often take the following form: "to (action verb) (subject) by (action)." For example, "To assist all academic/instructional departments and administrative/support units to assess their impact on student learning and contributions to the strategic plan by collecting and publishing Assessment Plans and Assessment Summary Reports." Departments should select a minimum of 3-5 outcomes (more if necessary) that represent areas in which they would like to improve in the near future.
4. Define the intended student learning outcomes of the department - While the primary focus of assessment is improving student learning, that is a difficult goal for many administrative units to impact directly. Thus, many administrative departments will focus on outcomes that help them become more efficient or effective and improve customer service; in short, to focus on improving their operations.
5. Inventory existing and needed assessment methods and measures and select the most valid - What kinds of assessments are you already doing, that is, what information are you already collecting? What types of assessment measures are you familiar with and comfortable using? What kinds of assessments are recommended by your professional organizations? Are there any benchmarking assessments sponsored by your professional organizations? Assessment measures can be both quantitative or qualitative, direct or indirect, and objective or subjective. They range from tests of knowledge to rubrics and prompts for scoring student work, as well as data base queries, surveys and focus groups. Choose a method(s) that will accurately measure your outcome. Finding one reliable and valid measure of an outcome is difficult so multiple measures are recommended. Multiple methods will offer multiple perspectives of the same outcome. In addition, some measures can be used to assess more than one outcome.
6. Identify assessment targets -Once you have chosen the measures, identify a target for achievement; e.g., 80% customer satisfaction or 80% accuracy.
7. Collect and analyze the data - In addition to collecting the data, departments need to plan how and where to store it as well as identify who will be responsible for it. Data needs to be collected systematically and in an unbiased manner according to your research plan. It needs to be stored in a safe and secure place, especially if it contains UIN or other identifying information, and backed-up. Rather than writing extensive reports, compile the data into short meaningful reports (tables and bullet observations) that colleagues can use to make decisions about how to improve the department.
8. Provide feedback to department colleagues - Share the results with your colleagues in the department and engage them in a discussion of the results and what they mean. When the data are understood by all, engage in a discussion about potential improvements. Communicate assessment results frequently to all concerned stakeholders.
9. Identify improvements, create an action plan, and implement changes - This step is referred to as "closing the loop." After establishing several outcomes, collecting and analyzing data related to those outcomes, and discussing those results with colleagues in the department, identify areas in which the department excels (celebrate them) and areas which need improvement. Establish a process for implementing the improvements including training for all staff who will be involved in the implementation and communication with other departments that will be impacted.
10. Start the process over again (continuous quality improvement). Maintain the same outcomes and measures in order to assess the impact of the changes and/or create new outcomes for the next assessment cycle. Develop measures, collect data, and analyze results with an eye on improvement.
* Adapted from work by Julia Pet Armacost (University of Central Florida) and presented by Bob Smallwood (University of North Florida) at the SACS Institute on Quality Enhancement and Accreditation, July 2005
