Draft
7/23/02
Bemidji
State University
NCA Monitoring Report
Please
forward comments and corrections to
Dr.Joann Fredrickson
Vice President for Academic and Student Affairs
jfredrickson@bemidjistate.edu
Note:
This is a draft, and some items are not completed.
CONTENTS
- Introduction:
A Culture of Assessment and Planning
- Assessment:
Balance and Consistency in Departmental Assessment
- Planning:
Assessment to Planning - A Closed Loop
- Conclusion:
Focus and Commitment
(Attachments and Appendixes may be viewed in the Office of Academic
& Student Affairs.)
Attachments
- University Assessment Plan: A Resource Manual for Department and Program
Review Guidelines and Bemidji State University Assessment Plan
- Academic Program Review Guidelines
- Summary of Bemidji State University Assessment Plan: Dimensions of
Student Learning, Outcomes and Student Learning Objectives
- III.- A Academic Profile Data Results & Summary
- Bemidji State University Data Book 2000-2001
- Report on the Assessment of Computer Technology Utilization, Expectations
and Student Knowledge
- Student Opinion Survey, Fall Semester, 2001 VII. "Telling Our
Story: Outreach and Partnership Efforts at Bemdiji State University"
Appendixes
- Reviewers Questionnaire (External Consultant)
- Five-Year Review Schedule for Service Departments
- College Assessment Coordinator Reports C-1 Academic Department Profiles
- Liberal Education Course Forms D-1 Assessment of Liberal Education
Area VIII: People and the Environment
- Dr. Cecelia Lopez, Assessment Conference
- IFO Liberal Education Committee Charge
- Director of Liberal Education Position Description
- Liberal Education Curriculum
- Thesis/Research Paper Committee Members Assessment Of Student Performance
In Oral Defense
- Graduate Student's Assessment Of Graduate Program
- Master's Degree Exit Survey
- Planning Committee Document Format
- Initiative Generation and Approval Flow Chart
- Planning Committee Recommendations to the University Council
- Revenue Sharing Proposal to Support Development of Web-Based Courses
at Bemidji State O-1 Online Course Self-Study & Agreement Draft
- Budget Information Sheets
- Budget Proposal Template
- Budget & Resources Committee Grant Writing Quality Improvement Process
- Budget & Resources Committee SWOT Analysis
- 2001-2002 BSU Student Employment Telephone Survey
I. Introduction:
A Culture of Assessment and Planning
NCA Evaluation
April 9-12, 2000 the North Central Association of Schools and Colleges
conducted a comprehensive evaluation for continued accreditation of Bemidji
State University. The review was based on NCA's General Institutional
Requirements and the five Criteria for Accreditation. The evaluation also
included review of the University's Experimental Planning Initiative,
which was recognized through a Memorandum of Agreement between North Central
and the University.
Recommendation for Full Accreditation
In their "Report of a Visit to Bemidji State University," the
evaluators recommended that the University be awarded full accreditation:
"The current NCA team has concluded that the University has satisfied
all the North Central General Institutional Requirements (p. 4). Furthermore,
the team members were "unanimous in their view that Bemidji State
University does meet" the Criteria for Accreditation (p. 5):
- Continue the accreditation of Bemidji State University.
- Schedule the next comprehensive evaluation for 2009-2010.
The team also required a monitoring report to NCA on assessment and operational
planning by September 30, 2002 (p. 50).
Monitoring Report
The evaluators' 2000 Report identifies two areas to be addressed in the
Monitoring Report, assessment and planning:
The written (monitoring) report must specifically address what the
team found to be an unevenness of implementing the University's approved
Assessment Plan. While the campus culture pertaining to assessment is
still evolving, it has not yet become an institutional priority or a
way of life. Some programs have defined appropriate and measurable objectives
for program educational goals; others are currently functioning at a
more elementary stage in the process; and still other areas have yet
to use the data collected in a manner that helps them determine if the
measures are adequate and appropriate. The report will need to also
address the University's need to develop a well articulated plan, or
set of procedures to meaningfully integrate the experimental planning
initiatives with other critical areas of the institution, i.e., program
assessment plans, five year program review schedules, resource allocation,
and the new committee and council infrastructure. Institutional operational
plans (i.e. Information Technology Plan, a Campus Master Plan or a Financial
Plan for the University), are not yet fully developed and must also
be integrated into the overall University Plan (p. 50).
University Progress
Bemidji State University has addressed the assessment and planning challenges
raised by the self-study team in 2000. That the many accomplishments discussed
in sections II and III were made concurrent with changes in a number of
senior administrative positions is a testament to the overall strength
of the University's commitments. The changes in leadership since the 2000
Self-Study visit include the naming of a new President and a new Vice
President for Academic and Student Affairs, and the filling of all open
deanships with permanent deans. The current Acting Vice President for
Administrative Affairs will be replaced by a permanent vice president
in 2002-2003.
Summary of Assessment Progress (See Section II for details.)
- The five-year review process, including assessment, is integrated
into the University Assessment Plan.
- In 2000, 10 of 27 programs were on schedule with reviews and assessment.
As of spring 2002, 27 of 27 programs are on schedule.
- All department assessment plans include ongoing activities.
- Academic assessment feedback is incorporated into planning at the
department, college and university levels.
- The University has increased its support for assessment planning and
activities.
Summary of Planning Progress (See Section III for details.)
- Membership on the planning committees has been established.
- The planning committees, the University Council and the Executive
Council meet on a regular basis.
- Protocols for committee processes and action have been established.
- Committee minutes and other documents are posted regularly on a planning
committee campus web site (http://info.bemidjistate.edu/News/planning/index.html).
- Processes for integrating planning "with other critical areas
of the institution" (Report, p. 50) are evidenced in the planning
committee profiles which include descriptions of initiatives, their
origins, and actions taken..
II. Assessment:
Balance and Consistency in Departmental Assessment
Contents
Page - Background, including progress, supportive measures, and culture
of assessment.
Page - Quick-glance summary, by college, of department five-year review
and assessment reporting status.
Page - Quick-glance summary, by college, comparison of five-year review
and assessment status, Spring 2000 to Spring 2002. Page - Quick-glance
summary, by college, areas assessed.
Page - Sample academic department profiles.
Page - Liberal Education Assessment.
Page - Graduate Studies Assessment.
Page - University-Wide Assessment.
Page - Student and University Services Assessment
SIDEBARS:Assessment
as "an institutional way of life": faculty, staff, and service
department activities additional to ongoing academic assessment.
Background
As summarized in Section I, and discussed below, the University has made
considerable progress in its goal of nurturing a culture of assessment
that is reflected in the University Assessment Plan (Attachment I) and
is evenly implemented in academic areas and across the University at large.
Progress since the 2000 visit
- Five-year review and assessment guidelines, approved by NCA in 1996,
and now integrated with the University Assessment Plan (Attachment I)
have been published by the Academic Affairs Committee and are used by
departments for review and assessment planning and implementation (Academic
Program Review Guidelines, Attachment II, and Summary of Bemidji State
University Assessment Plan, Attachment III).
- An external consultant report form developed by the Academic Affairs
Committee is utilized by all external consultants (Reviewers Questionnaire,
Appendix A).
- Approved, viable plans: All departments have current, approved five-year
plans, including assessment plans consistent with the University's published
assessment guidelines (Attachment I). The plans are re-submitted for
review as part of every five-year review cycle.
- All departments are on time with implementation of their assessment
plans, and all plans include ongoing activities.
- Findings from assessment are utilized in departmental planning and,
through the deans, are carried forward through the planning process
at the college and university levels.
- University service areas are engaged in five-year review cycles per
plans developed by the Student and University Services committee (Appendix
B).
A Commitment to Assessment
The University employed a number of measures, including allocation
of resources, to accomplish, in the past two years, full compliance with
its assessment plan:
- Each of the three colleges has an Assessment Coordinator, at .33 FTE
credits of release time per semester, per college. The coordinators
report to their respective deans (College Assessment Coordinator Reports,
Appendix C).
- Additional resources have been allocated to the Office of Research
and Assessment.
- Through the office of the Vice President for Academic and Student
Affairs, funding is available to departments for external consultant
reviewers.
- Departmental five-year review schedules, including assessment, are
required; departments are on-time within their cycles.
- The "Academic Program Review and Guidelines" and the "Summary
of (the University's) Assessment Plan," noted above provide step-by-step
guidance to departments in the development of assessment plans and preparation
of five-year reviews (Attachments II and III).
Culture of Assessment: Ongoing Departmental Assessment Activities
At the time of the University's next scheduled comprehensive self-study
accreditation review, 2009-2010, all departments will have completed at
least two five-year review and assessment cycles. Departmental progress
is tracked by the college assessment coordinators and the Office of Research
and Assessment, which is developing an on-line departmental assessment
portfolio web site. Departmental assessment activity is also tracked through
a Mid-Cycle Report submitted to the dean (Attachment II, page 9), and
the status of every department's five-year review is reported in the University's
Data Book 2000-2001 (Attachment IV) which is published annually.
In their site visit report in 2000, the evaluators noted that Bemidji
State "is making progress in implementing its assessment program
and thus it appears to be at Level Two in the NCA continuum" (p.
32). Evaluators noted some "unevenness of implementing the University's
approved Assessment Plan," particularly at the department level (p.
50). As indicated above, and evidenced in the following charts and profiles,
assessment is now implemented evenly across the academic spectrum.
Quick-Glance Summary by College
Departmental Five-Year Review and Assessment Reporting Status
The following profiles were developed through review of records, new
input from departments, and interviews with department chairs by the Monitoring
Report editor.
Assessment reports are a required component of five-year reviews. For
ongoing assessment activities and actions taken as a result of assessment
see department profiles, following.
Mid-cycle progress reports are a required component in the five-year
cycle (Attachment II, page 9).
| Department/ Program
|
Last Comprehensive
Review |
Next Comprehensive
Review |
| College of Arts & Letters |
| English |
Spring 2001-Fall 2002 |
2006-2007 |
| History |
Spring 2001-Fall 2002 |
2006-2007 |
| Mass Communication |
2000-2001 |
2005-2006 |
| Modern Languages |
Spring 2001-Fall 2002 |
2006-2007 |
| Music |
Spring 2001-Fall 2002 |
2006-2007 |
| Philosophy |
1998-1999 |
2003-2004 |
| Theatre and Speech Communication |
2000-2001 |
2006-2007 |
| Visual Arts |
2000-2001 |
2006-2007 |
| College of Professional Studies |
| Accounting |
2000-2001 |
2005-2006 |
| Business Administration |
2000-2001 |
2005-2006 |
| Physical Education, Health & Sport |
2000-2001 |
2005-2006 |
| Industrial Technology |
2000-2001 |
2005-2006 |
| Professional Education |
2000-2001 |
2005-2006 |
| College of Social & Natural Sciences |
| Biology |
2000-2001 |
2005-2006 |
| Chemistry |
1998-1999 |
2003-2004 |
| Computer Science |
2000-2001 |
2005-2006 |
| Criminal Justice |
2001-2002 |
2006-2007 |
| Economics |
2001-2002 |
2006-2007 |
| Environmental Studies |
1998-1999 |
2003-2004 |
| Geography |
1998-1999 |
2003-2004 |
| Mathematics |
2000-2001 |
2005-2006 |
| Nursing (8-year accreditation cycle) |
1995-1996 |
2003-2004 |
| Physics and Science |
2000-2001 |
2005-2006 |
| Political Science |
2000-2001 |
2005-2006 |
| Psychology |
1999-2000 |
2004-2005 |
| Social Work |
1998-1999 |
2003-2004 |
| Sociology |
1997-1998 |
2002-2003 |
Quick-Glance Summary, by College
Comparison of Five-Year Review and Assessment Status
Spring 2000 to Spring 2002
| On Schedule at Time of 2000 Self-Study |
On Schedule at Time of 2002 Monitoring Report |
| College of Arts & Letters |
| Philosophy |
English |
| |
History |
| |
Mass Communication |
| |
Modern Languages |
| |
Music |
| |
Philosophy |
| |
Theatre and Speech Communication |
| |
Visual Arts |
| College of Professional Studies |
| Physical Education, Health & Sport |
Accounting |
| Professional Education |
Business Administration |
| |
Professional Education |
| |
Industrial Technology |
| |
Physical Education, Health and Sport |
| College of Social & Natural Sciences |
| Chemistry |
Biology |
| Environmental Studies |
Chemistry |
| Geography |
Computer Science |
| Nursing |
Criminal Justice |
| Psychology |
Economics |
| Sociology |
Environmental Studies |
| Social Work |
Geography |
| |
Mathematics |
| |
Nursing |
| |
Physics and Science |
| |
Political Science |
| |
Psychology |
| |
Sociology |
| |
Social Work |
Quick-Glance Summary, by College
Areas Assessed (Selected)
| |
Majors |
Alumni |
| College of Arts & Letters |
|
|
| English |
X |
X |
| History |
X |
X |
| Mass Communication |
X |
X |
| Modern Languages |
X |
X |
| Music |
X |
X |
| Philosophy |
X |
|
| Theatre and Speech Communication |
X |
X |
| Visual Arts |
X |
|
| College of Professional Studies |
|
|
| Accounting |
X |
X |
| Business Administration |
X |
X |
| Industrial Technology |
X |
X |
| Physical Education, Health and Sport |
X |
X |
| Professional Education |
X |
X |
| College of Social & Natural Sciences |
|
|
| Biology |
X |
X |
| Chemistry |
X |
|
| Computer Science |
X |
X |
| Criminal Justice |
X |
|
| Economics |
X |
X |
| Environmental Studies |
X |
X |
| Geography |
X |
|
| Mathematics |
X |
X |
| Nursing |
X |
|
| Physics and Science |
X |
X |
| Political Science |
X |
|
| Psychology |
X |
X |
| Social Work |
X |
X |
| Sociology |
X |
X |
Academic Department Assessment Profiles, by College
An assessment profile has been created for all academic departments.
A sample profile from each college follows. Profiles for all departments
are in Appendix C-1.
Contents of Profiles
- Department Statistics
- Five-Year Review Dates
- Ongoing Assessment Activities
- Assessment Outcomes
- Actions Taken
- Challenges with Assessment
- Modifications to Plan
- Summary of Assessment Plan
- Questions asked
- Dimensions of student learning
- Student learning outcomes
- Assessment strategies
- Data sources
Departments
| College of Arts & Letters |
College of Professional & Graduate Studies |
College of Social & Natural Sciences |
English
History
Mass Communication
Modern Languages
Music
Philosophy
Theatre and Speech Communication
Visual Arts |
Accounting
Business Administration
Professional Education
Industrial Technology
Physical Education, Health and Sport |
Biology
Chemistry
Computer Science
Criminal Justice
Economics
Environmental Studies
Geography
Mathematics
Nursing
Physics and Science
Political Science
Psychology
Sociology
Social Work |
NOTE for Draft 07/23/02
Profiles for CAL, English; COPS, Physical Education, Health & Sport;
and SNS Geography will be included here. They are currently being proofread
by the departments.
Liberal Education Assessment
Approved Liberal Education Assessment Plan
The University's approved Assessment Plan (Attachment I) calls for two
primary Liberal Education assessment activities (page 19):
- General student surveys conducted by the Office of Research &
Assessment;
- Assessment and review of Liberal Education course offerings.
The Liberal Education Director and Committee continue to pursue a comprehensive
approach to Liberal Education assessment, and to cooperate with departments
on direct assessment of Liberal Education courses.
General Student Surveys
ETS Academic Profile (Office of Research & Assessment)
As noted elsewhere in this document (in University-wide Assessment Activities,
below), the first Academic Profile cycle of testing occurred in the fall
of 2001 (FY 2002) with a sample of juniors and seniors who are scheduled
to graduate with degrees in natural sciences and nursing. Preliminary
analysis indicates that the results for Bemidji State University are comparable
to other similar comprehensive institutions. As expected, upper classmen
(seniors and juniors) score higher than freshmen on all the scales (Attachment
III-A, Academic Profile Data Results & Summary).
Subsequent student learning outcomes testing will occur with samples
of students in arts and letters, education, business and accounting, and
social sciences. Preliminary data from the mathematics portion of the
exam indicate that students perform slightly above the national average,
as anticipated by the Mathematics Department.
SIDEBAR OPPOSITE ETS ACADEMIC PROFILE:
Preliminary analysis, Fall 2001 ETS Academic Profile
(Attachment III-A, Academic Profile Data Results & Summary)
- Bemidji State University scores are higher than the published
average scores for a national sample of comprehensive universities;
- Mean scores for the upper class cohort are higher than for the
freshmen cohort in reading, writing, mathematics and critical
thinking.
- Even though the "proficiency scores" are higher at
upper class levels, the absolute percentages of students scoring
at the proficient level is exceptionally low. This is also true
at the higher proficiency levels of writing and mathematics.
The Liberal Education Committee and Director will address these
results in 2002-2003. |
Assessment and Review of Liberal Education Course Offerings
Courses are approved for the Liberal Education program by the Liberal
Education Committee. Course proposals are submitted on forms (Appendix
D), and include identification of learning outcomes and objectives, and
methods of assessment (Attachment III). Course offerings in Liberal Education
are additionally reviewed by area on a rotating basis (year one, Areas
One and Two, etc.).
Additional Assessment Activities
In addition to the activities called for in the University's Assessment
Plan, some departments carry out assessment of Liberal Education courses
as part of their program assessment.
Mathematics Placement (Mathematics Department)
Incoming freshmen take the CAAP math placement test, which determines
their placement in developmental or intermediate algebra classes.
Composition Assessment (English Department)
Two writing across the curriculum assessments were conducted in the last
four years by Liberal Education. The results were inconclusive; a new
approach is being developed by the English Department.
People and the Environment Assessment
The People and the Environment component of Liberal Education (which is
in both the previous and the revised curricula) is assessed every semester
(Appendix D-1).
Outcomes
Students have moved in the direction expected: They are more aware of,
and more knowledgeable and concerned about environmental issues.
Changes to course based on assessment
- Content has been modified to engender more hope in students (who can
become depressed by the seriousness of what they learn), while still
provoking awareness.
- Active learning activities have been added (discussion, small groups,
field study).
- Dynamics of the interdisciplinary team have been modified.
Modification of assessment instrument
- Increased number of questions about cognitive content.
- Instrument modified to better identify attitude and behavioral changes
in students.
SIDEBAR IN LIB ED SECTION
Departmental Liberal Education Assessment Exemplar: Geography.
Starting 2001, the Geography Department began a comprehensive evaluation
of student learning outcomes in all of its liberal education courses.
This evaluation required the development of a general knowledge
instrument. After establishing their liberal education goals and
learning objectives, the department identified specific knowledge
items that were desired outcomes of their courses, and they identified
in which courses these knowledge items were to be conveyed. These
items became the basis for a general survey instrument of geography
knowledge that was administered according to a pretest—posttest
quasi-experimental design. The department's methodology and sampling
of students allowed them to identify both geography majors and non-majors,
together with the number of geography courses that the students
had taken prior to the exam.
Assessment findings include the following:
- Departmental faculty and the college dean's approval of assessment
methodology, and perceived value of the results.
- A plan for continued assessment (stage three) of selected courses
using the same format.
- Identification of goal areas that need modification (e.g. changes
in course outlines, additional articulation among faculty teaching
toward the same specific goals, inclusion of goals not currently
reflected in courses, etc.).
- Interest in presenting basic assessment plan at a regional
or national conference.
Utilization of results:
- Integration of more technology into Liberal Education courses.
- Creation of a Geo Center room which includes additional computers
for student use.
- Renovation and redefinition of the map library including the
addition of access to external map libraries.
- Inclusion of laboratory time in class time to allow educational
offerings that would otherwise require the hiring of additional
personnel.
Copies of the Geography Department assessment are available on
request. |
Implementation
General Student Surveys
As noted above, the ETS Academic Profile has been implemented.
Assessment and Review of Liberal Education Course Offerings
The process for evaluating Liberal Education courses has been functioning
since before 1996.
Institutional Culture
Dr. Cecelia Lopez visited the campus March 9, 2001 (Appendix E ) Session
topics included student learning outcomes, pre and post testing, and assessment
strategies. In addition to attendance by University faculty and staff,
representatives were present from the region's technical and community
colleges.
As noted above, proposals for courses to be considered for inclusion
in Liberal Education must include assessment measures.
Assessment is included in the charge of the IFO Liberal Education Committee
(Appendix F) and in the Director of Liberal Education position description(Appendix
G).
Departments are including Liberal Education courses in their assessment
plans. This year additions were made in English, History, Music, Visual
Arts, and Geography.
The University's Liberal Education Director receives .33 FTE release
time. The position description was revised Spring 2002 (Appendix E), in
accordance with its charge (Appendix E) and a request from the Faculty
Senate. As in the past, in addition to coordination of approval and ongoing
review of Liberal Education courses, and development and implementation
of assessment, the Director's duties include attendance and presentations
at assessment conferences. A new Director will be announced fall semester
2002. Outgoing Director Dr. Kay Robinson attended the following conferences
(asterisk indicates Dr. Robinson made a presentation):
- Council for the Administration of General and Liberal Studies--Chicago,
November 2000
- Association of American Colleges and Universities
- New Orleans, January 2001
- MnSCU CTL Workshop--Duluth, April 2001
- Theatre in Higher Education--Chicago, August 2001
- Canadian Studies in the United States
Use of Information
General Student Surveys
As noted above, the ETS Profile has been implemented.
Assessment and Review of Liberal Education Course Offerings
New or renewing courses that meet the requirements for Liberal Education,
including the use of Dimensions of Student Learning and assessment measures,
are included in the Liberal Education curriculum.
Assessment Loop
Liberal Education at Bemidji State is included in departmental, college,
and university decision-making, including action by planning committees.
For example, the newly instituted First-Year Experience (FYE) seminar
originated in part as the result of information received from the follow-up
Noel-Levitz Inventory (see FYE sidebar, page __). FYE was approved through
the planning committees, the University Council and the Executive Council.
Freshman students are currently advised into FYE; a proposal to include
the year-long FYE seminar in the University's graduation requirement is
in progress. The proposal was not implemented with the program in part
due to state mandated reconfiguration of the University's general education
program (see "Challenges" below).
Challenges for Liberal Education Assessment
In the Fall of 2001 Bemidji State was notified by MnSCU that it must realign
its Liberal Education offerings to conform with the Minnesota Transfer
Curriculum. In order to meet this requirement, the Liberal Education Committee
suspended review of new course proposals, and of courses within the existing
Liberal Education categories. In response to a directive from the Faculty
Senate, in 2002-2003 the Liberal Education Committee and Director will
study the revised curriculum, and will develop new review and assessment
strategies to match the Transfer Curriculum format (previous and new program
categories: Appendix H).
| SIDEBAR:
First-Year Experience
A major retention effort of the University for Fall 2002 is a two-semester
comprehensive First-Year-Experience course. The one-credit per semester
seminars will share enrollment with Freshman Writing sections in
an effort to create the same sense of cohort at Bemidji State as
is seen in private institutions.
Although the First-Year-Experience draws much from Towson State
University in Maryland, which consistently achieves in excess of
80% freshman to sophomore retention, it also has features unique
to Bemidji State. For instance, every FYE students will receive
instruction directly from professionals in career services, counseling,
records, informational technology, the library, and other student
support offices.
The FYE seminars are taught by faculty from across the disciplines.
The strong academic focus includes field excursions, student governance
and club fairs, instruction in electronic portfolios and WebCT,
and an outside speaker series. In addition, a common literary text
(Mango Street by Sandra Cisneros) will be read and discussed by
the approximately ___ students registered for FYE fall semester
2002.
Assessment
The First-Year Experience will be assessed (EACH SEMESTER? ANNUALLY?
FOR SATISFACTION? FOR DIRECT LEARNING OUTCOMES? ) |
Graduate Studies Assessment
Evidence of Approved Plan
The Graduate Studies Assessment Plan is a component of the University’s
NCA approved Assessment Plan (Attachment I, page 20-24).
Implementation of the Approved Plan
The plan has five components that are administered in all programs:
- Comprehensive examinations, in two parts, written and oral. Departments
are currently being asked to review, analyze, and act on the results
of these examinations.
- Demonstration of a competency within the discipline (such as statistics,
or a second language). These competency requirements are listed in the
department entries in the Graduate Catalog.
- A thesis reporting substantial work of research or creativity.
- An instrument evaluating student competence as a reflection of the
graduate curriculum administered to orals committee evaluators at the
time of the oral exam (Appendix I).
- An instrument evaluating the University’s graduate study strengths
and weaknesses administered to students at the time of the oral exam
(Appendix J).
Two additional assessment activities are currently in practice:
- A Graduate Studies Master’s Degree Exit Survey is mailed to
students with their diplomas (Appendix K).
- Graduate Studies graduation rates are analyzed on a regular basis.
Assessment by Departments:
The comprehensive exams, the thesis, and the competency exam (items 1,
2, and 5) are administered by the departments. As noted above, a plan
is in development for the review, analysis and act on these assessments.
Assessment by the Office of Graduate Studies:
The evaluating instruments and the exit survey (items 4, 6 and 7) are
administered by the Graduate Office.
Institutional Culture
As noted above, the Graduate Studies assessment plan is included in the
University’s Assessment Plan, and assessment activities are carried
out by academic departments and by the Office of Graduate Studies.
Department assessment plans are approved by the Academic Affairs Committee
which includes a member of the Graduate Studies Committee. The Graduate
Studies Committee charge includes responsibility for graduate study policy.
Graduate Committee - one person to Academic Affairs, which approves all
University assessment plans.
Use of Information
Assessment carried out by the Graduate Studies Office (items 4, 6 and
7 above) has resulted in the following changes:
- To encourage timely completion of their degrees, and to insure that
student work is current in the field, graduate students are held to
the seven year window for application of graduate courses toward a graduate
degree;
- Revision of candidacy forms;
- Development of additional forms to assist students and advisors in
planning and tracking their progress through established procedures;
- Development (in progress) of continuous enrollment for students actively
working on their degrees;
- Presentation of workshops to the Graduate Student Association on
candidacy and other graduate study processes.
Assessment Loop
The numerous changes in Graduate Studies forms and procedures made in
the last two years, which were identified through the assessment efforts
noted above, are expected to assist students through enrollment, study
and graduation. All of the new processes were approved by the Graduate
Studies Committee which has representation from all departments offering
graduate courses.
University-Wide Assessment
Office of Research & Assessment
Academic Affairs Committee
As reflected in its charge. the Academic Affairs Committee if continually
addresses assessment activities university-wide:
The Academic Affairs Committee will oversee both the five-year academic
program review cycle as well as all the University's student outcomes
assessment efforts. It will identify academic objectives and priorities
within the context of the University's mission, and assist in the planning,
development, and refinement of both programmatic assessment and student
outcomes assessment for the University. This includes all academic areas,
advancement programs, service areas, and Liberal Education. The Academic
Affairs Committee may also be called upon to review and make recommendations
about academic policies.
See the Academic Affairs Committee profile, page ___, for details.
ETS Academic Profile
As indicated in the 2000 Self-Study, COMP was discontinued by ACT, but
the University has begun a program utilizing the Academic Profile published
by ETS for the general education assessment of mathematics, reading/critical
thinking and writing. The first cycle of testing occurred in the fall
of 2001; analysis indicates that the results are comparable to those for
other similar comprehensive institutions. A sample of juniors and seniors
in natural sciences and nursing is scheduled. As noted in the Liberal
Education section above, subsequent outcomes testing will occur with samples
of students in arts and letters, education, business and accounting, and
social sciences (Attachment III-A, Academic Profile Data Results &
Summary).
Technology Utilization Survey
The Office of Research and Assessment, working with the Computer Technology
and Learning Resources Committee and other planning committees, the offices
of Computer Services and Department of Computer Science, conducted a study
of faculty and student computer technology preparedness. The results are
documented in the “Report on the Assessment of Computer Technology
Utilization, Expectations and Student Knowledge” (Office of Research
and Assessment, January, 2002). Indications include “that there
is a strong statistical relationship between faculty and staff computer
technology utilizations and their expectations for student utilization...”;
this could mean “that the higher the commitment to instructional
computer literacy, the higher the return in student computer literacy”
(Attachment, page 4). The results provide a baseline for further study
of technology utilization at Bemidji State.
SIDEBAR OPPOSITE TECHNOLOGY UTILIZATION SURVEY
The Computer Technology Knowledge Examination was administered
to a cross-section of freshmen through senior students and the related
Computer Technology Utilization Inventory was administered to all
faculty and staff in the fall of 2001. The purpose of these tests
was to determine the extent of student general education computer
knowledge and the degree of faculty utilization of computer technology
at four levels of behavior and within 17 areas of technology (Attachment
V). Analyses of the data yielded the following findings:
- A direct, statistically significant relationship between student
knowledge and faculty/staff utilization was found (r=.64 and .61);
that is, for every increment in faculty utilization there was
a .64 increment in student knowledge.
- The greater the need for utilization within the discipline,
the higher the level of usage by disciplinary faculty.
- The higher faculty utilization, the higher their expectations
were for student knowledge.
- A significant amount of variance in student knowledge is explained
by faculty utilization and expectation scores.
- Student learning outcomes in computer technology areas are directly
related to faculty inputs.
|
Noel Levitz Student Opinion Inventory Follow-Up
Consistent with the University’s assessment plan, a follow-up with
the Noel Levitz Student Opinion Inventory was undertaken Spring 2001.
Analysis of results show positive gains in student perceptions on all
of the measures, together with positive results when compared to peer
colleges and universities. This opinion inventory was supplemented with
a locally developed Student Opinion Survey (see sidebar) in the fall of
2001 that provided information about specific topics to service departments
and programs, all of whom participated in the development of the survey.
Actions Taken:
- The First-Year Experience, initiated fall 2002, was developed in part
on the basis of analysis of the Noel-Levitz follow-up survey (see FYE
feature, page __).
- Computer Services utilized results of the survey to assess their services.
- Analysis supported the new recruitment and retention model employed
by the University.
- Development of additional parking facilities.
SIDEBAR OPPOSITE NOEL-LEVITZ
Student Opinion Survey, Fall 2001
The Survey: Attachment VI
The Power of Integrated Messages
Data obtained through the Student Opinion Survey, Fall 2001, provides
statistical support for the effectiveness of Bemidji State University’s
integrated message approach to recruitment—The Lake/The Learning/The
Life. This integrated message approach, begun three years ago by
Kevin Drexel, focuses on the coordination of messages that are paired
with structured individualized student experiences with counselors
and other Bemidji State University personnel. Consequently, the
BSU Website, Recruitment Publications, Campus Activities, Student
Visits, and Personal Interactions with Campus Personnel all have
a common theme and ultimate objective of securing new student enrollment
at Bemidji State University.
When students were asked the degree to which each of these variables
affected their decision to attend Bemidji State University, a series
of strong correlations, ranging from .35 to .64 between variables,
showed the relative effects of the variables in the model. The strongest
relationships are between BSU Publications and the BSU Website (r=.64),
and between a Visit to BSU and Campus Activities (r=.52).

|
Assessment Practices Workshops & Related Activities
Faculty, departments, and administrators have participated in professional
workshops. There has been a substantial increase in participation in workshops,
professional meetings, and activities involving assessment experts.
- Faculty members have attended and presented at national assessment
meetings (see feature page __).
- The Mathematics Department is participating with MnSCU in an evaluation
of public school preparation for college math and reading.
- The University held an assessment practices day, March 9, 2001, which
included the following activities (Appendix E):
- Dr. Cecelia Lopez, Associate Director for the Higher Learning Commission
of the North Central Association, outlined the purpose of Shared Responsibility
and Values for assessment to faculty and staff, together with providing
a specialized seminar on best practices in departmental assessment activities;
- Dr. David Schuppt, Director of Institutional Effectiveness, Minnesota
State Colleges and University (MnSCU), presented a theoretical construct
for statewide assessment of outcomes;
- Craig Frocke, Transfer Curriculum, MnSCU, presented the legislated
evaluation protocols for all general education courses as they become
validated for inclusion in the Minnesota Transfer Agreement. This activity
has also triggered a re-evaluation by the faculty of the liberal education
program.
- Spring 2001, the President, Assessment Coordinator, and Director
of the Professional Development Center made an assessment presentation
at an NCA conference.
Student and University Services Assessment
In keeping with the University’s mission, and carrying on the work
of the Student Services Program Planning and Review (SSPPR) Committee
that preceded the current Student and University Services Committee (SUS),
the SUS in the year 2000 established a five-year review and annual report
schedule for the 30 student and university services departments. The five-year
review is modeled after the academic five-year review guidelines and was
approved through the University’s current planning process. See
Appendix B for the SUS review and annual report schedule.
SIDEBARS: FACULTY
(See Attachment for anecdotal reports from student &
university services)
OPENING STATEMENT FOR THESE, IF THEY ARE GROUPED, RATHER
THAN SPREAD THROUGHOUT THE DOCUMENT???
Anecdotal Reports of Faculty, Staff and Department Assessment
Activities
Departmental Assessment Committees
All academic departments have either assessment committees or designated
assessment coordinators.
Assessment Conferences & Sessions (selected)
Attended/Presented at by Bemidji State Faculty
- Council on Social Work Education APM March 2001 (Dr. M. A.
Reitmeir, Social Work Program)
- Council for the Administration of General and Liberal Studies
November 2000 (Presentation, Dr. Kay Robinson, Director of Liberal
Education)
Association of American College and Universities January 2001
(Presentation, Dr. Kay Robinson, Director of Liberal Education)
MnSCU CTL Workshop, April 2001 (Dr. Kay Robinson, Director of
Liberal Education) Association for Theatre in Higher Education,
August 2001 (Dr. Kay Robinson, Director of Liberal Education)
Association for Canadian Studies in the US (Dr. Kay Robinson,
Director of Liberal Education)
- National Communication Association Conference, November 2001
(Assessment session, Carol Richards, Theatre &Speech Communication
Department)
- Collaboration for the Advancement of College Teaching and Learning
conference, November 2001 (Presentation by Russ Lee, Laurie Desiderato,
B. Schudt, Psychology Department)
- Commission on Collegiate Nursing Education (CCNE), Washington,
DC, fall 2001 (Dr. Riki Scheela, Nursing Department)
- American Council on the Teaching of Foreign Languages (ACTFL),
Oral Proficiency Workshops (Dr. Kathleen Meyer. Dr. Maria Miranda,
Modern Languages Department)
- MACTE Congresses (Regularly attended by Professional Education
faculty; theme 2002 and 2003 is program and student assessment)
- NAIT Conference (Assessment sessions, Dr. Roy Blackwood, Mass
Communication Department)
External Community Assessment/Feedback Activities
- In 2001, Dr. Cheryl Beyers, Social Work Program, conducted a
survey of and held community meetings with social services practitioners
in northern Minnesota. The survey data were compiled and a report
was written.
Actions Taken: The feedback is being used in the ongoing revitalization
of the field component of the social work program.
- To receive feedback from practitioners, the Social Work program
meets four times a year with the Social Work Practitioners’
Advisory Council.
Intra-departmental Assessment Consultations
- Sociology faculty assisted other departments with the development
of surveys for their five-year reviews, including the academic
departments Criminal Justice and English, and the International
Student Services office.
Outside Accreditation
ADD NAMES OF AGENCIES
- Music: National Association of Schools of Music (NASM)
- Nursing: National League for Nursing
- Social Work: Council on Social Work Education
- Professional Education: Minnesota Board of Teaching
- Chemistry: American Chemical Society
Assessment Survey Development
Dr, Russ Lee, Psychology Department and Director of the Center for
Professional Development, developed the following forms. Distributed
by the Center, they are available for use by all faculty (Appendix
H).
- Summative faculty evaluation instrument
- Open-ended chair evaluation instrument
- Multiple-choice chair evaluation instrument
- Peer observation form
Research Assessment Study
- The Industrial Technology Department participated in a national
assessment study: The Certified Industrial Technology Exam, 2001-02,
was taken by 20 department students.
University-Wide Academic Integrity Survey: Sociology Department
- Academic Integrity Survey (U-WIDE, BUT NOT FROM R & A)
COMMENT & ACTION?
(Appendix I)
New Programs (Selected Examples)
- Idea – Survey – Action
In response to steady registration in writing courses, anecdotal
feedback from students, and a survey of students, the English
Department instituted a Bachelor of Fine Arts in Creative and
Professional Writing Fall 2001. A follow-up survey Spring 2002
indicates 28 self-declared majors in the new program.
- External Needs Assessment Outcome
In response to outcomes of assessment surveys of external constituencies
and student learning needs, the Mass Communication Department
has instituted a Bachelor of Science in Marketing Communication.
|
III.
Planning: Assessment to Planning – A Closed Loop
Planning Contents
Page - Introduction
Page - Overview of the Planning Committees
Page - Overview of the University and Executive Councils
Page - Initiatives Process
Page - Planning Committee Profiles
Sidebar Feature - The University Plan
Sidebar Feature - Program Contribution Indicators & Planning
SIDEBAR
Bemidji State University Planning
Academic Affairs Committee
Budget & Resource Allocation Committee
Computing, Technology & Learning Resources Committee
Recruitment & Retention Committee
Student & University Services Committee
University Council
Executive Council
FLOW CHART HERE from electronic file “M - Approval
Flow Chart.doc” (Is also appendix M) |
Introduction
At the time of the NCA April 2000 visit to Bemidji State, the University
had committed to its Experimental Planning Initiative, now called University
planning, and was in the initial stages of implementation. Issues under
discussion in the evaluators’ response included the investiture
of committee memberships, method of information flow and communication,
issue and policy development, involvement of relevant parties in the planning
process, and information feedback loops. The University is pleased to
report that the committees are fully functioning and, as evident from
the Planning Committee Profiles below, are engaged in planning based on
initiatives that arise from assessment and demonstrated need.
Overview of Planning Committees
- Membership in the committees has been established and includes representation
from relevant bargaining units and the student senate (see Planning
Committee Profiles, following).
- Co-chair positions are filled by a member elected from within each
committee, and a dean or director.
- Meetings are held on a regular basis. See minutes, http://info.bemidjistate.edu/News/planning/index.html.
- A common document format is used to forward recommendations, strategies
and activities to the next level, the University Council (Appendix L).
- Recommendations from the planning committees are forwarded to the
University Council. See Appendix M.
- Policy issues are forwarded to the University Council and then to
the Executive Council and the President. See Appendix M.
- Minutes and relevant documents are posted regularly on a University
web site, accessible from the University’s home page (http://info.bemidjistate.edu/News/planning/index.html).
Overview of the University Council and Executive Council
University Council
Recommendations received from planning committees and acted on: Appendix
N
Minutes: http://info.bemidjistate.edu/News/planning/index.html
The University Council is the primary planning committee for the University.
It responds to reports and recommendations from the six planning committees;
it may initiate activities and inquiries independently; it will provide
analysis and recommendations to the Executive Council concerning all phases
of University activities and functions, with emphasis on identifying recommended
University priorities in the context of both the institutional mission
and the budget. (Experimental Planning Initiative, 2000 Self-Study).
- Membership is comprised of co-chairs from the planning committees;
the Vice President for Academic and Student Affairs and the Vice President
for Administrative Affairs serve as co-conveners.
- Meetings are held weekly. See minutes, http://info.bemidjistate.edu/News/planning/index.html.
- Policy recommendations of the Council are forwarded to the Executive
Council (Appendix __)
Executive Council
Recommendations received from the University Council and acted on: Appendix
___
Minutes: http://info.bemidjistate.edu/News/planning/index.html
The President’s Executive Council advises the President on issues
of University policy. It receives recommendations and analysis from the
University and assists the President in making decisions about policies
and the implementation of policies (Experimental Planning Initiative,
2000 Self-Study).
- Membership is comprised of the President, Vice Presidents, Deans,
the Assistant to the President for Enrollment Management, and the Athletic
Director.
- Policy recommendations are forwarded to the President.
Initiatives Process
Flow Chart: Feature page ___, and Appendix M
Progress Chart: Appendix N
Initiatives may be brought to a planning committee by anyone in the University.
From the planning committee the initiatives proceed through a feedback
network that includes other planning committees, bargaining units, and
the student senate. As appropriate, initiatives pass from the planning
committee to the University Council, the Executive Council, and the President,
with feedback loops to Meet & Confer, the student senate, and other
stakeholders. Recommended and approved initiatives are implemented as
appropriate.
Planning Committee Profiles
including initiatives and actions taken
SEE ATTACHMENT
(The planning committee profiles are attached as a separate document –
but I think I’ll bring them back in here for the report.)
SIDEBAR Feature ?
The University Plan
The University Plan, which is in development, includes an overall strategic
plan for the University including academics and embracing additional plans
in four discrete areas: Finances, Facilities, Development and Athletics,
and Technology. The plan will identify directions for the University,
and will make recommendations on priorities and on the allocation of resources.
The University Plan is scheduled for completion by the end of fall semester
2002. It reflects the values and priorities of academic strategic plans
developed at the department and college levels (Appendix __), and of plans
developed in other areas including the University Foundation, the Development
Office, and the Alumni Office, Athletics, and Student Advising Services.
SIDEBAR
Bemidji State University
Draft of Five-Year Goal and Areas of Distinction
Proposed by the new president, and currently under discussion by
the University.
Five-Year Goal
To be the Midwest’s premier student-centered university integrating
liberal arts with career development to prepare students for life-long
learning and leadership in a global society.
Areas of Distinction that focus on:
- holistic approach to the development of he personal, social,
and intellectual self;
- high quality learning experiences enabling students, departments,
and programs to meet and exceed national standards of achievement;
- ethnic and cultural diversity in our campus community with particular
emphasis on recruiting, retaining, and graduating American Indian
students;
- international and multicultural values with commitment to global
partnerships and a significant international student presence;
- interdisciplinary learning experiences that address complex
societal issues and scientific problem-solving;
- implications of and uses for technology;
- leadership and engagement in community development partnerships;
- innovations in private/public partnership and educational alliances.
|
SIDEBAR
Program Contribution Indicators: Closure of the Assessment
Loop
Exemplar: Computer Sciences Position Vacancy
The Program Contribution Indicator is one component of a department’s
request to open a position vacancy. The Indicator provides material
in support of the request, including assessment and five-year review
results and comments from external evaluators, and is an example
of how assessment at the department level contributes to budgetary
decision-making at the university level.
Exemplar: Computer Science Position Vacancy
The recent position vacancy announcement in Computer Science is
an example of how assessment at the department level can affect
budgetary decisions at the university level. Student assessment
in Computer Science, and recommendations by an external evaluator,
pointed to the need for the department to offer classes in computer
networking. In part because of this assessment documentation, a
networking position vacancy has been announced. |
IV.
Conclusion: Focus and Commitment
Page - Accomplishment Highlights from the Report
Page - Challenges
Page - Overview Discussion of Challenges
Page - Closing Comments
Accomplishment Highlights from the Report
Culture of Assessment, as demonstrated in Section II
- All academic departments are on schedule with five-year reviews and
assessment, an increase from 37% to 100% from the year 2000.
- Academic assessment is ongoing, with activities occurring every year.
- Assessment results contribute to decision-making at the department,
college and university levels.
- Individual academic departments and faculty members engage in assessment
activities beyond program assessment.
- Student and university service departments now engage in ongoing
assessment activities and annual reports that contribute to decision-making.
Planning to Act, as demonstrated in Section III
- Initiatives addressed by the planning committees derive from assessment
or other demonstration of need.
- Planning committee recommendations contribute to planning at the
larger level of the institution, and result in actions taken.
Challenges
Assessment
- Plan and implement a campus-wide assessment day where results of
assessment are shared with the campus community.
- Departments: Making accommodations for time required to plan, implement,
analyze, and act on assessment activities; movement from indirect to
direct assessment.
- Liberal Education: Identification of comprehensive assessment instruments;
assessment of Liberal Education courses by departments.
- Graduate Studies: Evaluation and analysis of existing assessment
instruments (e.g., comprehensive and oral examinations) of graduate
courses and programs by departments, with action taken on feedback gained.
Planning
- Continued refinement of communication, especially with bargaining
units.
- Completion of the University Plan that is in development, and continue
to insure that the plan continues to express the University’s
mission and vision.
- Continue to recognize the distinction between day-to-day operations
and long-range planning (example see BRAC short-term planning item).
- Keep the planning process inclusionary.
- Broaden awareness and understanding of, and participation and confidence
in, the planning process, including awareness of planning successes.
- Prepare and distribute to the campus an annual report that identifies
actions taken and their relation to accomplishment of the University’s
mission and vision.
- Identifying resources for funding initiatives and mandates.
Overview Discussion of Challenges
The University exhibits a culture of assessment that includes ongoing
academic assessment activities that contribute to decision-making. Some
of the assessment activities have focused on indirect measures such as
alumni and student surveys. As departments evolve their assessment abilities,
feedback from these measures provide departments with a sense of who they
are and what their accomplishments are. They also provide positive and
meaningful assessment experience that can be applied to direct assessment.
As assessment plans are modified, they are be reviewed by the deans, the
Academic Affairs Committee, the College Assessment Coordinators, and the
Office of Research and Assessment to insure that modifications include
direct learning outcome measures. The latter can be gathered to produce
an assessment picture that can be added to assessment portfolios for Liberal
Education and Graduate Studies.
When asked the question “what challenges occurred related to assessment,”
a number of department chairs respond with a single word: “Time.”
Many assessment activities are labor intensive in planning, implementation,
and analysis. With assessment as a standard and a constant for academic
departments, the University continues to explore ways to provide time
or other compensation for faculty, such as the appointed college coordinators
who assist departments with assessment.
Planning Challenges
The Planning Process:
Two years into the new planning process, committee members have adjusted
to the committees’ structures, and have learned to listen to, respect,
and cooperate with representatives from academic, staff, bargaining, and
administrative units. The primary challenges remain communication, and
elegance of efficiency in a system that represents multiple constituents.
The challenges for planning have thus moved away from the learning curve
of the new structure and toward effective and efficient operation of the
committees and the University. With the leadership of President Quistgaard
and newly appointed Vice President for Student and Academic Affairs Joann
Fredrickson, and the continuing guidance of the planning committees, the
University is addressing those challenges.
The University Plan:
As part of their operational growth the planning committees and University
and Executive Councils, as well as the colleges, the Office of Distance
Learning, and other offices and departments have developed goals and strategic
plans. These combined with goals and strategic plans developed by academic
departments and the Colleges are the basis for the new University Plan.
Financial Resources:
Bemidji State addresses its financial challenges through a number of venues
including the Budget and Resource Allocation Committee, the Office of
Advancement; cooperation with MnSCU and with state legislators; and review
and reallocation of budget lines and items. Capital campaign initiatives
will be guided by the new University Plan scheduled for implementation
at the end of fall semester 2002.
Closing Comments
The unevenness of implementation of the University’s approved Assessment
Plan has been addressed and a campus culture of assessment has become
a way of life. Programs have defined appropriate and measurable objectives
for program educational goals and are collecting data that they use in
decision-making.
Procedures for integrating planning with other critical areas of the
institution (program assessment, five-year program reviews, resource allocation,
committee and council infrastructure) are in place and are operative.
An overall University Plan that incorporates an Information Technology
Plan, Campus Plan, Development and Athletics Plan, and a Financial Plan,
is in development and is based in part on completed strategic plans for
the colleges and other University offices. It is scheduled for completion,
including approval by all University constituents, at the end of fall
semester 2002.
SIDEBAR
Development of the Monitoring Report
Work on the Monitoring Report began fall semester 2001. The process
included the following:
Outlines & drafts -
- Beginning in November 2001, outlines and drafts of the report
were provided, at random intervals, to all bargaining units, with
requests for feedback.
- Drafts and updates of the Monitoring Report were posted on a
web site (http://info.bemidjistate.edu/News/planning/index.html,
under University Council) beginning March 25, 2002. The site was
advertised through the University’s faculty-staff e-mail
listserv and the President’s Newsletter.
Interviews and presentations –
- The writer interviewed 27 department chairs, planning committee
co-chairs, deans, and directors to learn the subtleties of assessment
and planning activities.
- The writer made presentations to the University Council, the
Executive Council, bargaining unit Meet and Confers sessions,
the Faculty Senate, and the Student Senate.
- The writer met frequently with the Director of Research &
Assessment, the deans, the Vice President for Academic and Student
Affairs, and the President.
Anecdotal reports -
- A call was issued to faculty and academic departments for descriptions
of assessment activities in addition to those in assessment plans
(see page __).
- A similar call was made to student and university staff and
departments (see Attachment __).
The document –
- The Monitoring Report document was produced by the Publications
Office.
|
|