Academic Quality Improvement Program - Assessment of Student Learning Action Project
September 14, 2004
AQIP Action Project Update
-
Describe the past year's accomplishments and the current status of this Action Project.
-
Spring 2003 Assessment of Student Learning surveys were distributed to the campus community and analyzed to establish benchmarks for the College's level of assessment implementation
-
Web page for the Assessment of Student Learning at Augsburg College was developed (http://www.augsburg.edu/assessment/)
-
Assessment Committee teams met with or contacted all departments and programs in the Academic and Student Affairs Division
-
Committee members attended IUPUI assessment conference and The Higher Learning Commission's Annual Meeting
-
Augsburg Core Collaboratives were formed to develop assessment plans for the general education program
-
Assessment Committee continued its discussion about the need for and structure of the assessment director position and internal assessment consultant positions
-
Assessment Director was appointed for Summer 2004 to provide feedback to departments and programs on annual assessment reports
-
Spring 2004 Assessment of Student Learning surveys were distributed to:
a) Faculty and Staff,
b) Students, and
c) College Council, Board of Regents, and Deans
-
Spring 2004 Assessment of Student Learning Faculty and Staff surveys were analyzed; results were compared to Spring 2003 survey results
-
Faculty and staff overwhelmingly felt a need for more communication, particularly about assessment components and structures. But faculty and staff also felt the need for further information about what is meant by assessment of student learning, guidance on how to proceed, and timely, regular feedback about their assessment plans and activities.
-
Many faculty and staff identified problems with compliance and institutional support. These problems included situations where departments may delegate assessment to one person who does not communicate with others, finding common time to work together on assessment, and feeling perhaps that assessment right now remains a concern and practice primarily at the individual rather than the program level. Several respondents wanted to see resources dedicated to assessment, including personnel, to bolster guidance, collation, and reporting.
-
Faculty and staff also noted the problem of time - not having enough of it to do assessment thoroughly at the program level and not having a clear sense of an assessment cycle or timetable at the College
-
There is much more assessment activity than expected and the understanding of assessment of student learning is growing across campus
-
Augsburg Core Curriculum planning and collaboratives helped jumpstart assessment in the Humanities and Fine Arts
-
Departments are doing a good deal of informal assessment work that guides curriculum decisions - the next step for some is to formalize the link between assessment of student learning and decision-making about curriculum and programs. Other departments and programs have provided good evidence that they are already using assessment data in the decision-making process.
-
Common misperceptions:
-
Assess everything every year,
-
Focus on exit assessment exclusively, and
-
Don't deviate from the assessment plan template
-
-
Challenges to keep in mind:
-
As a small college that values interaction, how do we draw on informal exchanges among colleagues and students as part of the assessment process, without so formalizing those exchanges that we stultify them?
-
Need to find good models for qualitative assessment approaches
-
-
Some departments are expressing a tension between time on assessment and other professional responsibilities
-
Departments need guidance on how to move beyond course-based assessment to program-level assessment
-
Departments would like additional feedback from the Assessment Committee about what type of data would be sufficient
-
All departments in Professional Studies are at least in the second phase of development, meaning they have established a plan and are in the process of collecting data
-
Many examples of excellent course-based assessment methods
-
Many examples of well-written student learning objectives
-
Some assessment liaisons have been working mostly in isolation (need to involve faculty colleagues in the development and implementation of plans)
-
Some department assessment plans need to be revised to:
-
Focus on student learning objectives that reflect department priorities about assessment of student learning and are less complicated,
-
Include assessment methods that go beyond an emphasis on course grades or grades for an entire test or assignment, and
-
Make the number of assessment methods more manageable
-
-
Co-Curricular programs have similar areas of development and concern:
-
Differentiating between assessing student learning and assessing programs,
-
How best to measure student learning using direct and indirect measures,
-
Problems interpreting the assessment template, and
-
Need for more resources on the assessment of co-curricular programs.
-
Summary of accomplishments:
The 2003 Assessment of Student Learning Faculty and Staff survey results framed the College's assessment work in 2003-2004. Key results included:
Faculty and staff were able to identify their own efforts in the assessment of student learning, but saw it less so at the department and program level, and significantly less at the level of the college. The Assessment Committee suggested two interpretations of this pattern. First, faculty and staff respondents may have been less aware of efforts toward assessment in areas of the institution that are further away from their daily work lives. Less knowledge about assessment plans at the level of the College may simply be a function of proximity and direct exposure. Second, there may have been some confusion between classroom-level and program assessment and how the two relate to each other.
While a small number of respondents seemed reasonably satisfied with their understanding of Augsburg's assessment goals, plans, and process, we found three main areas of concern contained in the two survey questions that allowed written responses:
In response to the request for more communication about assessment the Assessment Committee developed a web page containing information about Augsburg's assessment process. In addition, three Assessment Committee members attended the Assessment Institute at IUPUI in the fall of 2003 to gather resources and information about the assessment of student learning in curricular and co-curricular programs.
Responding to the request from departments and programs for more assessment information and feedback, the most important work completed by the Assessment Committee in 2003-2004 was its contact with all departments and programs in the Academic and Student Affairs Division. The Assessment Committee formed teams of two to review assessment plans and reports in their field of expertise. The teams found that good progress is being made on assessment of student learning. Departments and programs fell along a continuum of assessment activity from very early stages to using data to make curriculum/program decisions. A small number of departments have not yet developed a formal assessment plan, though it is clear that assessment activities are occurring within these programs or departments.
Each Assessment Committee team offered summary reflections on their meetings with departments:
There is a proposal pending that would provide funding for a member of the faculty to serve as the Assessment Director during the academic year and summer. Internal assessment consultants would support the work of the Assessment Director and identify discipline-specific resources to include on the Assessment of Student Learning web page and in the assessment resources library. The Assessment Director and internal consultants would serve on the Assessment Committee. The Assessment Director who served during the summer of 2004 has agreed to continue in the role if the position is funded. Her follow-up over the summer on annual reports was timely and helpful to departments and programs. The Director discovered that there is more work on closing the loop in the assessment cycle than had been reported in annual assessment reports. She also confirmed the need to change the assessment plan template because it is causing confusion about how to report actual results and the meaning of results.
The Assessment of Student Learning survey was distributed again in Spring 2004 and is being used to determine assessment priorities for 2004-2005. An analysis of response patterns in the 2004 survey show that faculty and staff are more aware of, and are engaging in, assessment practices more than they were in 2003. These results indicate some modest but notable gains in perception and involvement in assessment practices at Augsburg College.
Faculty/Staff Assessment of Student Learning survey questions 1, 2, 3, and 5 highlight elements of institutional culture and shared responsibility:
1. "I understand the meaning, goals, characteristics, and value of the College's assessment of student learning."
2. "I'm knowledgeable about the College's assessment program in terms of its structures, components, and timetable."
3. "Assessment of student learning is valued across the institution."
5. "My department/program considers assessment of student learning to be an integral part of its functioning, not an add-on."
The College experienced slight gains in 2004 over the 2003 benchmark results. Questions 1, 2, and 3 address an understanding of assessment at the college-level. With the exception of Question 1, there were no majority responses to these survey questions, which is similar to the pattern found in 2003. There is, however, an increasing percentage of people answering in agreement or disagreement, and less answering that they don't know, than last year. The Assessment Committee's interpretation is that respondents remain comparatively unaware of various dimensions of the assessment process at the level of the college, but that this is slowly changing. Responses to Question 1 indicate that faculty and staff understand the meaning and value of the College's assessment efforts in the abstract, but not its particular features.
Question 5 emphasizes assessment work at the department level. As in 2003, all responses show a majority agreement to departmental/program efforts toward assessment of student learning. Faculty and staff indicate that departments and programs view assessment as integral, they have created structures for ongoing assessment, and are in the process of collecting data. It is clear from survey results that more faculty and staff are involved in assessment practices within their departments and programs.
Ninety percent (53/59) of the departments and programs in the Academic and Student Affairs Division have developed assessment of student learning plans. The total number of departments required to submit plans has been modified from 61 to 59 since September of 2003. Twenty-seven percent (14/53) of departments and programs with an assessment plan are using the data to make decisions. Fifteen percent (8/53) are collecting data and fifty-eight percent (31/53) have yet to collect data.
Fifty-seven percent (30/53) of departments and programs with plans submitted an annual assessment report in the summer of 2004. Given that forty-three percent have not yet submitted an annual report the percentages noted above on the collection and use of data may be artificially low at this time. We expect to receive additional annual reports after posting this Action Project Update. Last year the percentage of annual assessment reports received after posting the Action Project Update increased from fifty-six percent to sixty-eight percent during the fall term. Consequently, the stretch target of having at least half of the departments using data by the end of Year 2 of the Action Project will be re-evaluated this fall.
Assessment work for the general education program was supported by a grant from the Bush Foundation that was focused on the implementation of the new Augsburg Core Curriculum that was introduced Fall 2003. Thirteen collaboratives were formed that focused on:
1) Specifying shared learning outcomes,
2) Selecting and implementing common classroom assessment techniques,
3) Developing and implementing assessment rubrics,
4) Identifying and implementing best practices, and
5) Using evidence of student learning to guide improvements.
-
- Describe how the institution involved people in work on this Action Project.
The Assessment Committee includes nine faculty and staff from the Academic and Student Affairs Division. Members are available to meet with department and program representatives to discuss assessment questions. An Assessment Director worked with departments and programs during the summer of 2004 to discuss feedback on assessment of student learning annual reports. These reports were also made available to members of the Assessment Committee who were available during the summer to provide feedback to the Assessment Director. Ninety percent of departments and programs in the Academic and Student Affairs Division are engaged in the assessment of student learning.
Augsburg Core Curriculum collaboratives engaged over sixty faculty and staff in the work of developing assessment plans for the components of the new Augsburg Core Curriculum. The Bush Foundation funded this work through a 2003-2004 grant. The College will continue this important work on the assessment of general education through a two-year renewal grant received from the Bush Foundation.
A total of 481 people responded to the 2004 Assessment of Student Learning surveys (64 faculty and staff, 406 students, and 11 representatives from College Council, Board of Regents, and Associate Deans). This is a 154% increase due in large measure to the increased return rate from students.
- Describe your planned next steps for this Action Project.
- Meet with division chairs to discuss their concerns and identify division needs
- Revise the assessment plan template to avoid confusion about expectations
- Serve as advocates for a proposal to provide resources for a Director of Assessment and internal assessment consultants
- Distribute the analysis for Year 2 of Assessment of Student Learning surveys and identify current campus assessment needs
- Organize brown bag lunches/workshops to discuss good assessment practices at Augsburg
- Maintain Assessment of Student Learning web page
- Complete follow-up work on department/program 2004 annual assessment reports and evaluate results in terms of the AQIP Action Project Year 3 goals
- Work with Director of the Center for Teaching and Learning to identify professional development opportunities for faculty and staff
- Seek external professional development opportunities for Assessment Committee members
- Examine assessment of student learning practices as part of the development of the AQIP Systems Portfolio - Criterion One
- Work with Director of General Education, Director of the Center for Teaching and Learning, and the Augsburg Core Curriculum Assessment Coordinator to develop and monitor the general education assessment plan
- Distribute Assessment of Student Learning surveys - Year 3 (Spring 2005)
4. Describe "effective practice(s)" that resulted from your work on this Action Project.
Assessment Committee members formed teams to meet with departments and programs in their areas of expertise (e.g., Fine Arts and Humanities, Social and Behavioral Sciences, Graduate Programs, Professional Studies, Natural Sciences and Mathematics, and Co-Curricular). This opportunity to meet with all programs and departments to discuss their 2003 annual report and current assessment plans was a fruitful experience for all involved. The Assessment Committee learned a great deal; these contacts influenced its views about priority assessment needs. Departments and programs received positive feedback about their progress and had an opportunity to ask specific questions they had about assessment. This experience led to the appointment of an Assessment Director to work with departments and programs during the summer of 2004 and informed the Committee's proposal for resources for an Assessment Director and internal consultants.
The Assessment Committee utilized the Assessment of Student Learning Faculty and Staff survey results to guide decision-making about priorities and application of resources
5. What challenges, if any, are you still facing in regards to this Action Project?
Key lessons learned in 2003-2004 that are informing the Assessment Committee's work in 2004-2005:
- Augsburg needs an assessment director (proposal pending)
- Augsburg is making progress in developing an assessment culture
- Augsburg needs to look for opportunities for departments to collaborate on assessment and curriculum development (e.g., REL 100 and ENG 111)
- Assessment Committee needs to manage complementary efforts of departments working on Augsburg Core Curriculum and major field assessment
- Departments and programs need access to additional electronic resources on assessment of student learning that are program or discipline-specific
- Assessment Committee should set 2007-2008 as the year when all departments/programs would be fully engaged in assessment of student learning, that is, able to demonstrate that assessment evidence is being used in the decision-making process
- Assessment Committee should frame assessment in a way that allows departments to see the benefit of assessment to them if this work is going to be fully integrated across the campus
- More information and clearly articulated expectations from the Assessment Committee (role of proposed Assessment Director)
- More specific feedback and guidance from the Assessment Committee (role of proposed Assessment Director)
- Access to more resources, such as examples of ideal assessment plans from other departments, models/templates, and specific methods/techniques of assessment (role of proposed internal consultants)
- More inter-departmental communication across a larger division (proposed brown bag lunches and workshops)
These recommendations from the Spring 2004 Faculty and Staff survey respondents are also guiding the Assessment Committee's work in 2004-2005:
Additionally, progress needs to continue on the assessment of student learning in the new Augsburg Core Curriculum.
6. If you would like to discuss the possibility of AQIP providing you help to stimulate the progress on this Action Project, explain your need(s) here and tell us who to contact and when.
We do not need additional help at this time.
7. Who can see this?
Anyone