Draft
7/23/02

Bemidji State University
NCA Monitoring Report

Please forward comments and corrections to
Dr.Joann Fredrickson
Vice President for Academic and Student Affairs
jfredrickson@bemidjistate.edu

Note: This is a draft, and some items are not completed.


CONTENTS

  1. Introduction: A Culture of Assessment and Planning
  2. Assessment: Balance and Consistency in Departmental Assessment
  3. Planning: Assessment to Planning - A Closed Loop
  4. Conclusion: Focus and Commitment
    (Attachments and Appendixes may be viewed in the Office of Academic & Student Affairs.)

Attachments

  1. University Assessment Plan: A Resource Manual for Department and Program Review Guidelines and Bemidji State University Assessment Plan
  2. Academic Program Review Guidelines
  3. Summary of Bemidji State University Assessment Plan: Dimensions of Student Learning, Outcomes and Student Learning Objectives
    • III.- A Academic Profile Data Results & Summary
  4. Bemidji State University Data Book 2000-2001
  5. Report on the Assessment of Computer Technology Utilization, Expectations and Student Knowledge
  6. Student Opinion Survey, Fall Semester, 2001 VII. "Telling Our Story: Outreach and Partnership Efforts at Bemdiji State University"

Appendixes

  1. Reviewers Questionnaire (External Consultant)
  2. Five-Year Review Schedule for Service Departments
  3. College Assessment Coordinator Reports C-1 Academic Department Profiles
  4. Liberal Education Course Forms D-1 Assessment of Liberal Education Area VIII: People and the Environment
  5. Dr. Cecelia Lopez, Assessment Conference
  6. IFO Liberal Education Committee Charge
  7. Director of Liberal Education Position Description
  8. Liberal Education Curriculum
  9. Thesis/Research Paper Committee Members Assessment Of Student Performance In Oral Defense
  10. Graduate Student's Assessment Of Graduate Program
  11. Master's Degree Exit Survey
  12. Planning Committee Document Format
  13. Initiative Generation and Approval Flow Chart
  14. Planning Committee Recommendations to the University Council
  15. Revenue Sharing Proposal to Support Development of Web-Based Courses at Bemidji State O-1 Online Course Self-Study & Agreement Draft
  16. Budget Information Sheets
  17. Budget Proposal Template
  18. Budget & Resources Committee Grant Writing Quality Improvement Process
  19. Budget & Resources Committee SWOT Analysis
  20. 2001-2002 BSU Student Employment Telephone Survey

I. Introduction: A Culture of Assessment and Planning

NCA Evaluation
April 9-12, 2000 the North Central Association of Schools and Colleges conducted a comprehensive evaluation for continued accreditation of Bemidji State University. The review was based on NCA's General Institutional Requirements and the five Criteria for Accreditation. The evaluation also included review of the University's Experimental Planning Initiative, which was recognized through a Memorandum of Agreement between North Central and the University.

Recommendation for Full Accreditation
In their "Report of a Visit to Bemidji State University," the evaluators recommended that the University be awarded full accreditation: "The current NCA team has concluded that the University has satisfied all the North Central General Institutional Requirements (p. 4). Furthermore, the team members were "unanimous in their view that Bemidji State University does meet" the Criteria for Accreditation (p. 5):

  • Continue the accreditation of Bemidji State University.
  • Schedule the next comprehensive evaluation for 2009-2010.

The team also required a monitoring report to NCA on assessment and operational planning by September 30, 2002 (p. 50).

Monitoring Report
The evaluators' 2000 Report identifies two areas to be addressed in the Monitoring Report, assessment and planning:

The written (monitoring) report must specifically address what the team found to be an unevenness of implementing the University's approved Assessment Plan. While the campus culture pertaining to assessment is still evolving, it has not yet become an institutional priority or a way of life. Some programs have defined appropriate and measurable objectives for program educational goals; others are currently functioning at a more elementary stage in the process; and still other areas have yet to use the data collected in a manner that helps them determine if the measures are adequate and appropriate. The report will need to also address the University's need to develop a well articulated plan, or set of procedures to meaningfully integrate the experimental planning initiatives with other critical areas of the institution, i.e., program assessment plans, five year program review schedules, resource allocation, and the new committee and council infrastructure. Institutional operational plans (i.e. Information Technology Plan, a Campus Master Plan or a Financial Plan for the University), are not yet fully developed and must also be integrated into the overall University Plan (p. 50).

University Progress
Bemidji State University has addressed the assessment and planning challenges raised by the self-study team in 2000. That the many accomplishments discussed in sections II and III were made concurrent with changes in a number of senior administrative positions is a testament to the overall strength of the University's commitments. The changes in leadership since the 2000 Self-Study visit include the naming of a new President and a new Vice President for Academic and Student Affairs, and the filling of all open deanships with permanent deans. The current Acting Vice President for Administrative Affairs will be replaced by a permanent vice president in 2002-2003.

Summary of Assessment Progress (See Section II for details.)

  • The five-year review process, including assessment, is integrated into the University Assessment Plan.
  • In 2000, 10 of 27 programs were on schedule with reviews and assessment. As of spring 2002, 27 of 27 programs are on schedule.
  • All department assessment plans include ongoing activities.
  • Academic assessment feedback is incorporated into planning at the department, college and university levels.
  • The University has increased its support for assessment planning and activities.

Summary of Planning Progress (See Section III for details.)

  • Membership on the planning committees has been established.
  • The planning committees, the University Council and the Executive Council meet on a regular basis.
  • Protocols for committee processes and action have been established.
  • Committee minutes and other documents are posted regularly on a planning committee campus web site (http://info.bemidjistate.edu/News/planning/index.html).
  • Processes for integrating planning "with other critical areas of the institution" (Report, p. 50) are evidenced in the planning committee profiles which include descriptions of initiatives, their origins, and actions taken..

II. Assessment: Balance and Consistency in Departmental Assessment

Contents
Page - Background, including progress, supportive measures, and culture of assessment.
Page - Quick-glance summary, by college, of department five-year review and assessment reporting status.
Page - Quick-glance summary, by college, comparison of five-year review and assessment status, Spring 2000 to Spring 2002. Page - Quick-glance summary, by college, areas assessed.
Page - Sample academic department profiles.
Page - Liberal Education Assessment.
Page - Graduate Studies Assessment.
Page - University-Wide Assessment.
Page - Student and University Services Assessment

SIDEBARS:Assessment as "an institutional way of life": faculty, staff, and service department activities additional to ongoing academic assessment.

Background
As summarized in Section I, and discussed below, the University has made considerable progress in its goal of nurturing a culture of assessment that is reflected in the University Assessment Plan (Attachment I) and is evenly implemented in academic areas and across the University at large.

Progress since the 2000 visit

  • Five-year review and assessment guidelines, approved by NCA in 1996, and now integrated with the University Assessment Plan (Attachment I) have been published by the Academic Affairs Committee and are used by departments for review and assessment planning and implementation (Academic Program Review Guidelines, Attachment II, and Summary of Bemidji State University Assessment Plan, Attachment III).
  • An external consultant report form developed by the Academic Affairs Committee is utilized by all external consultants (Reviewers Questionnaire, Appendix A).
  • Approved, viable plans: All departments have current, approved five-year plans, including assessment plans consistent with the University's published assessment guidelines (Attachment I). The plans are re-submitted for review as part of every five-year review cycle.
  • All departments are on time with implementation of their assessment plans, and all plans include ongoing activities.
  • Findings from assessment are utilized in departmental planning and, through the deans, are carried forward through the planning process at the college and university levels.
  • University service areas are engaged in five-year review cycles per plans developed by the Student and University Services committee (Appendix B).

A Commitment to Assessment
The University employed a number of measures, including allocation of resources, to accomplish, in the past two years, full compliance with its assessment plan:

  • Each of the three colleges has an Assessment Coordinator, at .33 FTE credits of release time per semester, per college. The coordinators report to their respective deans (College Assessment Coordinator Reports, Appendix C).
  • Additional resources have been allocated to the Office of Research and Assessment.
  • Through the office of the Vice President for Academic and Student Affairs, funding is available to departments for external consultant reviewers.
  • Departmental five-year review schedules, including assessment, are required; departments are on-time within their cycles.
  • The "Academic Program Review and Guidelines" and the "Summary of (the University's) Assessment Plan," noted above provide step-by-step guidance to departments in the development of assessment plans and preparation of five-year reviews (Attachments II and III).

Culture of Assessment: Ongoing Departmental Assessment Activities
At the time of the University's next scheduled comprehensive self-study accreditation review, 2009-2010, all departments will have completed at least two five-year review and assessment cycles. Departmental progress is tracked by the college assessment coordinators and the Office of Research and Assessment, which is developing an on-line departmental assessment portfolio web site. Departmental assessment activity is also tracked through a Mid-Cycle Report submitted to the dean (Attachment II, page 9), and the status of every department's five-year review is reported in the University's Data Book 2000-2001 (Attachment IV) which is published annually.

In their site visit report in 2000, the evaluators noted that Bemidji State "is making progress in implementing its assessment program and thus it appears to be at Level Two in the NCA continuum" (p. 32). Evaluators noted some "unevenness of implementing the University's approved Assessment Plan," particularly at the department level (p. 50). As indicated above, and evidenced in the following charts and profiles, assessment is now implemented evenly across the academic spectrum.

Quick-Glance Summary by College
Departmental Five-Year Review and Assessment Reporting Status

The following profiles were developed through review of records, new input from departments, and interviews with department chairs by the Monitoring Report editor.

Assessment reports are a required component of five-year reviews. For ongoing assessment activities and actions taken as a result of assessment see department profiles, following.

Mid-cycle progress reports are a required component in the five-year cycle (Attachment II, page 9).

Department/ Program Last Comprehensive Review Next Comprehensive Review
College of Arts & Letters
English Spring 2001-Fall 2002 2006-2007
History Spring 2001-Fall 2002 2006-2007
Mass Communication 2000-2001 2005-2006
Modern Languages Spring 2001-Fall 2002 2006-2007
Music Spring 2001-Fall 2002 2006-2007
Philosophy 1998-1999 2003-2004
Theatre and Speech Communication 2000-2001 2006-2007
Visual Arts 2000-2001 2006-2007
College of Professional Studies
Accounting 2000-2001 2005-2006
Business Administration 2000-2001 2005-2006
Physical Education, Health & Sport 2000-2001 2005-2006
Industrial Technology 2000-2001 2005-2006
Professional Education 2000-2001 2005-2006
College of Social & Natural Sciences
Biology 2000-2001 2005-2006
Chemistry 1998-1999 2003-2004
Computer Science 2000-2001 2005-2006
Criminal Justice 2001-2002 2006-2007
Economics 2001-2002 2006-2007
Environmental Studies 1998-1999 2003-2004
Geography 1998-1999 2003-2004
Mathematics 2000-2001 2005-2006
Nursing (8-year accreditation cycle) 1995-1996 2003-2004
Physics and Science 2000-2001 2005-2006
Political Science 2000-2001 2005-2006
Psychology 1999-2000 2004-2005
Social Work 1998-1999 2003-2004
Sociology 1997-1998 2002-2003

Quick-Glance Summary, by College
Comparison of Five-Year Review and Assessment Status
Spring 2000 to Spring 2002

On Schedule at Time of 2000 Self-Study On Schedule at Time of 2002 Monitoring Report
College of Arts & Letters
Philosophy English
  History
  Mass Communication
  Modern Languages
  Music
  Philosophy
  Theatre and Speech Communication
  Visual Arts
College of Professional Studies
Physical Education, Health & Sport Accounting
Professional Education Business Administration
  Professional Education
  Industrial Technology
  Physical Education, Health and Sport
College of Social & Natural Sciences
Chemistry Biology
Environmental Studies Chemistry
Geography Computer Science
Nursing Criminal Justice
Psychology Economics
Sociology Environmental Studies
Social Work Geography
  Mathematics
  Nursing
  Physics and Science
  Political Science
  Psychology
  Sociology
  Social Work

Quick-Glance Summary, by College
Areas Assessed (Selected)

 
Majors
Alumni
College of Arts & Letters
English
X
X
History
X
X
Mass Communication
X
X
Modern Languages
X
X
Music
X
X
Philosophy
X
Theatre and Speech Communication
X
X
Visual Arts
X
College of Professional Studies
Accounting
X
X
Business Administration
X
X
Industrial Technology
X
X
Physical Education, Health and Sport
X
X
Professional Education
X
X
College of Social & Natural Sciences
Biology
X
X
Chemistry
X
Computer Science
X
X
Criminal Justice
X
Economics
X
X
Environmental Studies
X
X
Geography
X
Mathematics
X
X
Nursing
X
Physics and Science
X
X
Political Science
X
Psychology
X
X
Social Work
X
X
Sociology
X
X

Academic Department Assessment Profiles, by College

An assessment profile has been created for all academic departments. A sample profile from each college follows. Profiles for all departments are in Appendix C-1.

Contents of Profiles

  • Department Statistics
  • Five-Year Review Dates
  • Ongoing Assessment Activities
  • Assessment Outcomes
  • Actions Taken
  • Challenges with Assessment
  • Modifications to Plan
  • Summary of Assessment Plan
    1. Questions asked
    2. Dimensions of student learning
    3. Student learning outcomes
    4. Assessment strategies
    5. Data sources

Departments

College of Arts & Letters College of Professional & Graduate Studies College of Social & Natural Sciences
English
History
Mass Communication
Modern Languages
Music
Philosophy
Theatre and Speech Communication
Visual Arts
Accounting
Business Administration
Professional Education
Industrial Technology
Physical Education, Health and Sport
Biology
Chemistry
Computer Science
Criminal Justice
Economics
Environmental Studies
Geography
Mathematics
Nursing
Physics and Science
Political Science
Psychology
Sociology
Social Work

NOTE for Draft 07/23/02
Profiles for CAL, English; COPS, Physical Education, Health & Sport; and SNS Geography will be included here. They are currently being proofread by the departments.

Liberal Education Assessment

Approved Liberal Education Assessment Plan
The University's approved Assessment Plan (Attachment I) calls for two primary Liberal Education assessment activities (page 19):

  • General student surveys conducted by the Office of Research & Assessment;
  • Assessment and review of Liberal Education course offerings.

The Liberal Education Director and Committee continue to pursue a comprehensive approach to Liberal Education assessment, and to cooperate with departments on direct assessment of Liberal Education courses.

General Student Surveys
ETS Academic Profile (Office of Research & Assessment)
As noted elsewhere in this document (in University-wide Assessment Activities, below), the first Academic Profile cycle of testing occurred in the fall of 2001 (FY 2002) with a sample of juniors and seniors who are scheduled to graduate with degrees in natural sciences and nursing. Preliminary analysis indicates that the results for Bemidji State University are comparable to other similar comprehensive institutions. As expected, upper classmen (seniors and juniors) score higher than freshmen on all the scales (Attachment III-A, Academic Profile Data Results & Summary).

Subsequent student learning outcomes testing will occur with samples of students in arts and letters, education, business and accounting, and social sciences. Preliminary data from the mathematics portion of the exam indicate that students perform slightly above the national average, as anticipated by the Mathematics Department.

SIDEBAR OPPOSITE ETS ACADEMIC PROFILE:
Preliminary analysis, Fall 2001 ETS Academic Profile
(Attachment III-A, Academic Profile Data Results & Summary)

  • Bemidji State University scores are higher than the published average scores for a national sample of comprehensive universities;
  • Mean scores for the upper class cohort are higher than for the freshmen cohort in reading, writing, mathematics and critical thinking.
  • Even though the "proficiency scores" are higher at upper class levels, the absolute percentages of students scoring at the proficient level is exceptionally low. This is also true at the higher proficiency levels of writing and mathematics.

The Liberal Education Committee and Director will address these results in 2002-2003.

Assessment and Review of Liberal Education Course Offerings
Courses are approved for the Liberal Education program by the Liberal Education Committee. Course proposals are submitted on forms (Appendix D), and include identification of learning outcomes and objectives, and methods of assessment (Attachment III). Course offerings in Liberal Education are additionally reviewed by area on a rotating basis (year one, Areas One and Two, etc.).

Additional Assessment Activities
In addition to the activities called for in the University's Assessment Plan, some departments carry out assessment of Liberal Education courses as part of their program assessment.
Mathematics Placement (Mathematics Department)
Incoming freshmen take the CAAP math placement test, which determines their placement in developmental or intermediate algebra classes.

Composition Assessment (English Department)
Two writing across the curriculum assessments were conducted in the last four years by Liberal Education. The results were inconclusive; a new approach is being developed by the English Department.

People and the Environment Assessment
The People and the Environment component of Liberal Education (which is in both the previous and the revised curricula) is assessed every semester (Appendix D-1).

Outcomes
Students have moved in the direction expected: They are more aware of, and more knowledgeable and concerned about environmental issues.

Changes to course based on assessment

  • Content has been modified to engender more hope in students (who can become depressed by the seriousness of what they learn), while still provoking awareness.
  • Active learning activities have been added (discussion, small groups, field study).
  • Dynamics of the interdisciplinary team have been modified.

Modification of assessment instrument

  • Increased number of questions about cognitive content.
  • Instrument modified to better identify attitude and behavioral changes in students.

SIDEBAR IN LIB ED SECTION
Departmental Liberal Education Assessment Exemplar: Geography.

Starting 2001, the Geography Department began a comprehensive evaluation of student learning outcomes in all of its liberal education courses. This evaluation required the development of a general knowledge instrument. After establishing their liberal education goals and learning objectives, the department identified specific knowledge items that were desired outcomes of their courses, and they identified in which courses these knowledge items were to be conveyed. These items became the basis for a general survey instrument of geography knowledge that was administered according to a pretest—posttest quasi-experimental design. The department's methodology and sampling of students allowed them to identify both geography majors and non-majors, together with the number of geography courses that the students had taken prior to the exam.

Assessment findings include the following:

  • Departmental faculty and the college dean's approval of assessment methodology, and perceived value of the results.
  • A plan for continued assessment (stage three) of selected courses using the same format.
  • Identification of goal areas that need modification (e.g. changes in course outlines, additional articulation among faculty teaching toward the same specific goals, inclusion of goals not currently reflected in courses, etc.).
  • Interest in presenting basic assessment plan at a regional or national conference.

Utilization of results:

  • Integration of more technology into Liberal Education courses.
  • Creation of a Geo Center room which includes additional computers for student use.
  • Renovation and redefinition of the map library including the addition of access to external map libraries.
  • Inclusion of laboratory time in class time to allow educational offerings that would otherwise require the hiring of additional personnel.

Copies of the Geography Department assessment are available on request.

Implementation
General Student Surveys

As noted above, the ETS Academic Profile has been implemented.

Assessment and Review of Liberal Education Course Offerings
The process for evaluating Liberal Education courses has been functioning since before 1996.

Institutional Culture
Dr. Cecelia Lopez visited the campus March 9, 2001 (Appendix E ) Session topics included student learning outcomes, pre and post testing, and assessment strategies. In addition to attendance by University faculty and staff, representatives were present from the region's technical and community colleges.

As noted above, proposals for courses to be considered for inclusion in Liberal Education must include assessment measures.

Assessment is included in the charge of the IFO Liberal Education Committee (Appendix F) and in the Director of Liberal Education position description(Appendix G).

Departments are including Liberal Education courses in their assessment plans. This year additions were made in English, History, Music, Visual Arts, and Geography.

The University's Liberal Education Director receives .33 FTE release time. The position description was revised Spring 2002 (Appendix E), in accordance with its charge (Appendix E) and a request from the Faculty Senate. As in the past, in addition to coordination of approval and ongoing review of Liberal Education courses, and development and implementation of assessment, the Director's duties include attendance and presentations at assessment conferences. A new Director will be announced fall semester 2002. Outgoing Director Dr. Kay Robinson attended the following conferences (asterisk indicates Dr. Robinson made a presentation):

  • Council for the Administration of General and Liberal Studies--Chicago, November 2000
  • Association of American Colleges and Universities
    • New Orleans, January 2001
    • MnSCU CTL Workshop--Duluth, April 2001
    • Theatre in Higher Education--Chicago, August 2001
    • Canadian Studies in the United States

Use of Information
General Student Surveys

As noted above, the ETS Profile has been implemented.

Assessment and Review of Liberal Education Course Offerings
New or renewing courses that meet the requirements for Liberal Education, including the use of Dimensions of Student Learning and assessment measures, are included in the Liberal Education curriculum.

Assessment Loop
Liberal Education at Bemidji State is included in departmental, college, and university decision-making, including action by planning committees. For example, the newly instituted First-Year Experience (FYE) seminar originated in part as the result of information received from the follow-up Noel-Levitz Inventory (see FYE sidebar, page __). FYE was approved through the planning committees, the University Council and the Executive Council. Freshman students are currently advised into FYE; a proposal to include the year-long FYE seminar in the University's graduation requirement is in progress. The proposal was not implemented with the program in part due to state mandated reconfiguration of the University's general education program (see "Challenges" below).

Challenges for Liberal Education Assessment
In the Fall of 2001 Bemidji State was notified by MnSCU that it must realign its Liberal Education offerings to conform with the Minnesota Transfer Curriculum. In order to meet this requirement, the Liberal Education Committee suspended review of new course proposals, and of courses within the existing Liberal Education categories. In response to a directive from the Faculty Senate, in 2002-2003 the Liberal Education Committee and Director will study the revised curriculum, and will develop new review and assessment strategies to match the Transfer Curriculum format (previous and new program categories: Appendix H).

SIDEBAR:
First-Year Experience
A major retention effort of the University for Fall 2002 is a two-semester comprehensive First-Year-Experience course. The one-credit per semester seminars will share enrollment with Freshman Writing sections in an effort to create the same sense of cohort at Bemidji State as is seen in private institutions.

Although the First-Year-Experience draws much from Towson State University in Maryland, which consistently achieves in excess of 80% freshman to sophomore retention, it also has features unique to Bemidji State. For instance, every FYE students will receive instruction directly from professionals in career services, counseling, records, informational technology, the library, and other student support offices.

The FYE seminars are taught by faculty from across the disciplines. The strong academic focus includes field excursions, student governance and club fairs, instruction in electronic portfolios and WebCT, and an outside speaker series. In addition, a common literary text (Mango Street by Sandra Cisneros) will be read and discussed by the approximately ___ students registered for FYE fall semester 2002.

Assessment
The First-Year Experience will be assessed (EACH SEMESTER? ANNUALLY? FOR SATISFACTION? FOR DIRECT LEARNING OUTCOMES? )

Graduate Studies Assessment

Evidence of Approved Plan
The Graduate Studies Assessment Plan is a component of the University’s NCA approved Assessment Plan (Attachment I, page 20-24).

Implementation of the Approved Plan
The plan has five components that are administered in all programs:

  1. Comprehensive examinations, in two parts, written and oral. Departments are currently being asked to review, analyze, and act on the results of these examinations.
  2. Demonstration of a competency within the discipline (such as statistics, or a second language). These competency requirements are listed in the department entries in the Graduate Catalog.
  3. A thesis reporting substantial work of research or creativity.
  4. An instrument evaluating student competence as a reflection of the graduate curriculum administered to orals committee evaluators at the time of the oral exam (Appendix I).
  5. An instrument evaluating the University’s graduate study strengths and weaknesses administered to students at the time of the oral exam (Appendix J).

Two additional assessment activities are currently in practice:

  1. A Graduate Studies Master’s Degree Exit Survey is mailed to students with their diplomas (Appendix K).
  2. Graduate Studies graduation rates are analyzed on a regular basis.

Assessment by Departments:
The comprehensive exams, the thesis, and the competency exam (items 1, 2, and 5) are administered by the departments. As noted above, a plan is in development for the review, analysis and act on these assessments.

Assessment by the Office of Graduate Studies:
The evaluating instruments and the exit survey (items 4, 6 and 7) are administered by the Graduate Office.

Institutional Culture
As noted above, the Graduate Studies assessment plan is included in the University’s Assessment Plan, and assessment activities are carried out by academic departments and by the Office of Graduate Studies.

Department assessment plans are approved by the Academic Affairs Committee which includes a member of the Graduate Studies Committee. The Graduate Studies Committee charge includes responsibility for graduate study policy.

Graduate Committee - one person to Academic Affairs, which approves all University assessment plans.

Use of Information
Assessment carried out by the Graduate Studies Office (items 4, 6 and 7 above) has resulted in the following changes:

  • To encourage timely completion of their degrees, and to insure that student work is current in the field, graduate students are held to the seven year window for application of graduate courses toward a graduate degree;
  • Revision of candidacy forms;
  • Development of additional forms to assist students and advisors in planning and tracking their progress through established procedures;
  • Development (in progress) of continuous enrollment for students actively working on their degrees;
  • Presentation of workshops to the Graduate Student Association on candidacy and other graduate study processes.

Assessment Loop
The numerous changes in Graduate Studies forms and procedures made in the last two years, which were identified through the assessment efforts noted above, are expected to assist students through enrollment, study and graduation. All of the new processes were approved by the Graduate Studies Committee which has representation from all departments offering graduate courses.

University-Wide Assessment

Office of Research & Assessment

Academic Affairs Committee
As reflected in its charge. the Academic Affairs Committee if continually addresses assessment activities university-wide:

The Academic Affairs Committee will oversee both the five-year academic program review cycle as well as all the University's student outcomes assessment efforts. It will identify academic objectives and priorities within the context of the University's mission, and assist in the planning, development, and refinement of both programmatic assessment and student outcomes assessment for the University. This includes all academic areas, advancement programs, service areas, and Liberal Education. The Academic Affairs Committee may also be called upon to review and make recommendations about academic policies.

See the Academic Affairs Committee profile, page ___, for details.

ETS Academic Profile
As indicated in the 2000 Self-Study, COMP was discontinued by ACT, but the University has begun a program utilizing the Academic Profile published by ETS for the general education assessment of mathematics, reading/critical thinking and writing. The first cycle of testing occurred in the fall of 2001; analysis indicates that the results are comparable to those for other similar comprehensive institutions. A sample of juniors and seniors in natural sciences and nursing is scheduled. As noted in the Liberal Education section above, subsequent outcomes testing will occur with samples of students in arts and letters, education, business and accounting, and social sciences (Attachment III-A, Academic Profile Data Results & Summary).

Technology Utilization Survey
The Office of Research and Assessment, working with the Computer Technology and Learning Resources Committee and other planning committees, the offices of Computer Services and Department of Computer Science, conducted a study of faculty and student computer technology preparedness. The results are documented in the “Report on the Assessment of Computer Technology Utilization, Expectations and Student Knowledge” (Office of Research and Assessment, January, 2002). Indications include “that there is a strong statistical relationship between faculty and staff computer technology utilizations and their expectations for student utilization...”; this could mean “that the higher the commitment to instructional computer literacy, the higher the return in student computer literacy” (Attachment, page 4). The results provide a baseline for further study of technology utilization at Bemidji State.

SIDEBAR OPPOSITE TECHNOLOGY UTILIZATION SURVEY

The Computer Technology Knowledge Examination was administered to a cross-section of freshmen through senior students and the related Computer Technology Utilization Inventory was administered to all faculty and staff in the fall of 2001. The purpose of these tests was to determine the extent of student general education computer knowledge and the degree of faculty utilization of computer technology at four levels of behavior and within 17 areas of technology (Attachment V). Analyses of the data yielded the following findings:

  • A direct, statistically significant relationship between student knowledge and faculty/staff utilization was found (r=.64 and .61); that is, for every increment in faculty utilization there was a .64 increment in student knowledge.
  • The greater the need for utilization within the discipline, the higher the level of usage by disciplinary faculty.
  • The higher faculty utilization, the higher their expectations were for student knowledge.
  • A significant amount of variance in student knowledge is explained by faculty utilization and expectation scores.
  • Student learning outcomes in computer technology areas are directly related to faculty inputs.

Noel Levitz Student Opinion Inventory Follow-Up
Consistent with the University’s assessment plan, a follow-up with the Noel Levitz Student Opinion Inventory was undertaken Spring 2001. Analysis of results show positive gains in student perceptions on all of the measures, together with positive results when compared to peer colleges and universities. This opinion inventory was supplemented with a locally developed Student Opinion Survey (see sidebar) in the fall of 2001 that provided information about specific topics to service departments and programs, all of whom participated in the development of the survey.
Actions Taken:

  • The First-Year Experience, initiated fall 2002, was developed in part on the basis of analysis of the Noel-Levitz follow-up survey (see FYE feature, page __).
  • Computer Services utilized results of the survey to assess their services.
  • Analysis supported the new recruitment and retention model employed by the University.
  • Development of additional parking facilities.

SIDEBAR OPPOSITE NOEL-LEVITZ
Student Opinion Survey, Fall 2001
The Survey: Attachment VI

The Power of Integrated Messages
Data obtained through the Student Opinion Survey, Fall 2001, provides statistical support for the effectiveness of Bemidji State University’s integrated message approach to recruitment—The Lake/The Learning/The Life. This integrated message approach, begun three years ago by Kevin Drexel, focuses on the coordination of messages that are paired with structured individualized student experiences with counselors and other Bemidji State University personnel. Consequently, the BSU Website, Recruitment Publications, Campus Activities, Student Visits, and Personal Interactions with Campus Personnel all have a common theme and ultimate objective of securing new student enrollment at Bemidji State University.
When students were asked the degree to which each of these variables affected their decision to attend Bemidji State University, a series of strong correlations, ranging from .35 to .64 between variables, showed the relative effects of the variables in the model. The strongest relationships are between BSU Publications and the BSU Website (r=.64), and between a Visit to BSU and Campus Activities (r=.52).

Assessment Practices Workshops & Related Activities
Faculty, departments, and administrators have participated in professional workshops. There has been a substantial increase in participation in workshops, professional meetings, and activities involving assessment experts.

  • Faculty members have attended and presented at national assessment meetings (see feature page __).
  • The Mathematics Department is participating with MnSCU in an evaluation of public school preparation for college math and reading.
  • The University held an assessment practices day, March 9, 2001, which included the following activities (Appendix E):
    - Dr. Cecelia Lopez, Associate Director for the Higher Learning Commission of the North Central Association, outlined the purpose of Shared Responsibility and Values for assessment to faculty and staff, together with providing a specialized seminar on best practices in departmental assessment activities;
    - Dr. David Schuppt, Director of Institutional Effectiveness, Minnesota State Colleges and University (MnSCU), presented a theoretical construct for statewide assessment of outcomes;
    - Craig Frocke, Transfer Curriculum, MnSCU, presented the legislated evaluation protocols for all general education courses as they become validated for inclusion in the Minnesota Transfer Agreement. This activity has also triggered a re-evaluation by the faculty of the liberal education program.
  • Spring 2001, the President, Assessment Coordinator, and Director of the Professional Development Center made an assessment presentation at an NCA conference.

Student and University Services Assessment
In keeping with the University’s mission, and carrying on the work of the Student Services Program Planning and Review (SSPPR) Committee that preceded the current Student and University Services Committee (SUS), the SUS in the year 2000 established a five-year review and annual report schedule for the 30 student and university services departments. The five-year review is modeled after the academic five-year review guidelines and was approved through the University’s current planning process. See Appendix B for the SUS review and annual report schedule.

SIDEBARS: FACULTY
(See Attachment for anecdotal reports from student & university services)

OPENING STATEMENT FOR THESE, IF THEY ARE GROUPED, RATHER THAN SPREAD THROUGHOUT THE DOCUMENT???

Anecdotal Reports of Faculty, Staff and Department Assessment Activities

Departmental Assessment Committees
All academic departments have either assessment committees or designated assessment coordinators.

Assessment Conferences & Sessions (selected)

Attended/Presented at by Bemidji State Faculty

  • Council on Social Work Education APM March 2001 (Dr. M. A. Reitmeir, Social Work Program)
  • Council for the Administration of General and Liberal Studies November 2000 (Presentation, Dr. Kay Robinson, Director of Liberal Education)
    Association of American College and Universities January 2001 (Presentation, Dr. Kay Robinson, Director of Liberal Education)
    MnSCU CTL Workshop, April 2001 (Dr. Kay Robinson, Director of Liberal Education) Association for Theatre in Higher Education, August 2001 (Dr. Kay Robinson, Director of Liberal Education)
    Association for Canadian Studies in the US (Dr. Kay Robinson, Director of Liberal Education)
  • National Communication Association Conference, November 2001 (Assessment session, Carol Richards, Theatre &Speech Communication Department)
  • Collaboration for the Advancement of College Teaching and Learning conference, November 2001 (Presentation by Russ Lee, Laurie Desiderato, B. Schudt, Psychology Department)
  • Commission on Collegiate Nursing Education (CCNE), Washington, DC, fall 2001 (Dr. Riki Scheela, Nursing Department)
  • American Council on the Teaching of Foreign Languages (ACTFL), Oral Proficiency Workshops (Dr. Kathleen Meyer. Dr. Maria Miranda, Modern Languages Department)
  • MACTE Congresses (Regularly attended by Professional Education faculty; theme 2002 and 2003 is program and student assessment)
  • NAIT Conference (Assessment sessions, Dr. Roy Blackwood, Mass Communication Department)

External Community Assessment/Feedback Activities

  • In 2001, Dr. Cheryl Beyers, Social Work Program, conducted a survey of and held community meetings with social services practitioners in northern Minnesota. The survey data were compiled and a report was written.
    Actions Taken: The feedback is being used in the ongoing revitalization of the field component of the social work program.
  • To receive feedback from practitioners, the Social Work program meets four times a year with the Social Work Practitioners’ Advisory Council.

Intra-departmental Assessment Consultations

  • Sociology faculty assisted other departments with the development of surveys for their five-year reviews, including the academic departments Criminal Justice and English, and the International Student Services office.

Outside Accreditation
ADD NAMES OF AGENCIES

  • Music: National Association of Schools of Music (NASM)
  • Nursing: National League for Nursing
  • Social Work: Council on Social Work Education
  • Professional Education: Minnesota Board of Teaching
  • Chemistry: American Chemical Society

Assessment Survey Development
Dr, Russ Lee, Psychology Department and Director of the Center for Professional Development, developed the following forms. Distributed by the Center, they are available for use by all faculty (Appendix H).

  • Summative faculty evaluation instrument
  • Open-ended chair evaluation instrument
  • Multiple-choice chair evaluation instrument
  • Peer observation form

Research Assessment Study

  • The Industrial Technology Department participated in a national assessment study: The Certified Industrial Technology Exam, 2001-02, was taken by 20 department students.

University-Wide Academic Integrity Survey: Sociology Department

  • Academic Integrity Survey (U-WIDE, BUT NOT FROM R & A)
    COMMENT & ACTION?
    (Appendix I)

New Programs (Selected Examples)

  • Idea – Survey – Action
    In response to steady registration in writing courses, anecdotal feedback from students, and a survey of students, the English Department instituted a Bachelor of Fine Arts in Creative and Professional Writing Fall 2001. A follow-up survey Spring 2002 indicates 28 self-declared majors in the new program.
  • External Needs Assessment Outcome
    In response to outcomes of assessment surveys of external constituencies and student learning needs, the Mass Communication Department has instituted a Bachelor of Science in Marketing Communication.

III. Planning: Assessment to Planning – A Closed Loop

Planning Contents
Page - Introduction
Page - Overview of the Planning Committees
Page - Overview of the University and Executive Councils
Page - Initiatives Process
Page - Planning Committee Profiles

Sidebar Feature - The University Plan
Sidebar Feature - Program Contribution Indicators & Planning

SIDEBAR
Bemidji State University Planning

Academic Affairs Committee
Budget & Resource Allocation Committee
Computing, Technology & Learning Resources Committee
Recruitment & Retention Committee
Student & University Services Committee
University Council
Executive Council

FLOW CHART HERE from electronic file “M - Approval Flow Chart.doc” (Is also appendix M)

Introduction

At the time of the NCA April 2000 visit to Bemidji State, the University had committed to its Experimental Planning Initiative, now called University planning, and was in the initial stages of implementation. Issues under discussion in the evaluators’ response included the investiture of committee memberships, method of information flow and communication, issue and policy development, involvement of relevant parties in the planning process, and information feedback loops. The University is pleased to report that the committees are fully functioning and, as evident from the Planning Committee Profiles below, are engaged in planning based on initiatives that arise from assessment and demonstrated need.

Overview of Planning Committees

  • Membership in the committees has been established and includes representation from relevant bargaining units and the student senate (see Planning Committee Profiles, following).
  • Co-chair positions are filled by a member elected from within each committee, and a dean or director.
  • Meetings are held on a regular basis. See minutes, http://info.bemidjistate.edu/News/planning/index.html.
  • A common document format is used to forward recommendations, strategies and activities to the next level, the University Council (Appendix L).
  • Recommendations from the planning committees are forwarded to the University Council. See Appendix M.
  • Policy issues are forwarded to the University Council and then to the Executive Council and the President. See Appendix M.
  • Minutes and relevant documents are posted regularly on a University web site, accessible from the University’s home page (http://info.bemidjistate.edu/News/planning/index.html).

Overview of the University Council and Executive Council

University Council
Recommendations received from planning committees and acted on: Appendix N
Minutes: http://info.bemidjistate.edu/News/planning/index.html

The University Council is the primary planning committee for the University. It responds to reports and recommendations from the six planning committees; it may initiate activities and inquiries independently; it will provide analysis and recommendations to the Executive Council concerning all phases of University activities and functions, with emphasis on identifying recommended University priorities in the context of both the institutional mission and the budget. (Experimental Planning Initiative, 2000 Self-Study).

  • Membership is comprised of co-chairs from the planning committees; the Vice President for Academic and Student Affairs and the Vice President for Administrative Affairs serve as co-conveners.
  • Meetings are held weekly. See minutes, http://info.bemidjistate.edu/News/planning/index.html.
  • Policy recommendations of the Council are forwarded to the Executive Council (Appendix __)

Executive Council
Recommendations received from the University Council and acted on: Appendix ___
Minutes: http://info.bemidjistate.edu/News/planning/index.html

The President’s Executive Council advises the President on issues of University policy. It receives recommendations and analysis from the University and assists the President in making decisions about policies and the implementation of policies (Experimental Planning Initiative, 2000 Self-Study).

  • Membership is comprised of the President, Vice Presidents, Deans, the Assistant to the President for Enrollment Management, and the Athletic Director.
  • Policy recommendations are forwarded to the President.

Initiatives Process
Flow Chart: Feature page ___, and Appendix M
Progress Chart: Appendix N
Initiatives may be brought to a planning committee by anyone in the University. From the planning committee the initiatives proceed through a feedback network that includes other planning committees, bargaining units, and the student senate. As appropriate, initiatives pass from the planning committee to the University Council, the Executive Council, and the President, with feedback loops to Meet & Confer, the student senate, and other stakeholders. Recommended and approved initiatives are implemented as appropriate.

Planning Committee Profiles
including initiatives and actions taken

SEE ATTACHMENT
(The planning committee profiles are attached as a separate document – but I think I’ll bring them back in here for the report.)

SIDEBAR Feature ?

The University Plan
The University Plan, which is in development, includes an overall strategic plan for the University including academics and embracing additional plans in four discrete areas: Finances, Facilities, Development and Athletics, and Technology. The plan will identify directions for the University, and will make recommendations on priorities and on the allocation of resources.

The University Plan is scheduled for completion by the end of fall semester 2002. It reflects the values and priorities of academic strategic plans developed at the department and college levels (Appendix __), and of plans developed in other areas including the University Foundation, the Development Office, and the Alumni Office, Athletics, and Student Advising Services.

SIDEBAR
Bemidji State University
Draft of Five-Year Goal and Areas of Distinction
Proposed by the new president, and currently under discussion by the University.

Five-Year Goal
To be the Midwest’s premier student-centered university integrating liberal arts with career development to prepare students for life-long learning and leadership in a global society.

Areas of Distinction that focus on:

  • holistic approach to the development of he personal, social, and intellectual self;
  • high quality learning experiences enabling students, departments, and programs to meet and exceed national standards of achievement;
  • ethnic and cultural diversity in our campus community with particular emphasis on recruiting, retaining, and graduating American Indian students;
  • international and multicultural values with commitment to global partnerships and a significant international student presence;
  • interdisciplinary learning experiences that address complex societal issues and scientific problem-solving;
  • implications of and uses for technology;
  • leadership and engagement in community development partnerships;
  • innovations in private/public partnership and educational alliances.

SIDEBAR

Program Contribution Indicators: Closure of the Assessment Loop
Exemplar: Computer Sciences Position Vacancy

The Program Contribution Indicator is one component of a department’s request to open a position vacancy. The Indicator provides material in support of the request, including assessment and five-year review results and comments from external evaluators, and is an example of how assessment at the department level contributes to budgetary decision-making at the university level.

Exemplar: Computer Science Position Vacancy
The recent position vacancy announcement in Computer Science is an example of how assessment at the department level can affect budgetary decisions at the university level. Student assessment in Computer Science, and recommendations by an external evaluator, pointed to the need for the department to offer classes in computer networking. In part because of this assessment documentation, a networking position vacancy has been announced.

IV. Conclusion: Focus and Commitment

Page - Accomplishment Highlights from the Report
Page - Challenges
Page - Overview Discussion of Challenges
Page - Closing Comments

Accomplishment Highlights from the Report

Culture of Assessment, as demonstrated in Section II

  • All academic departments are on schedule with five-year reviews and assessment, an increase from 37% to 100% from the year 2000.
  • Academic assessment is ongoing, with activities occurring every year.
  • Assessment results contribute to decision-making at the department, college and university levels.
  • Individual academic departments and faculty members engage in assessment activities beyond program assessment.
  • Student and university service departments now engage in ongoing assessment activities and annual reports that contribute to decision-making.

Planning to Act, as demonstrated in Section III

  • Initiatives addressed by the planning committees derive from assessment or other demonstration of need.
  • Planning committee recommendations contribute to planning at the larger level of the institution, and result in actions taken.

Challenges

Assessment

  • Plan and implement a campus-wide assessment day where results of assessment are shared with the campus community.
  • Departments: Making accommodations for time required to plan, implement, analyze, and act on assessment activities; movement from indirect to direct assessment.
  • Liberal Education: Identification of comprehensive assessment instruments; assessment of Liberal Education courses by departments.
  • Graduate Studies: Evaluation and analysis of existing assessment instruments (e.g., comprehensive and oral examinations) of graduate courses and programs by departments, with action taken on feedback gained.

Planning

  • Continued refinement of communication, especially with bargaining units.
  • Completion of the University Plan that is in development, and continue to insure that the plan continues to express the University’s mission and vision.
  • Continue to recognize the distinction between day-to-day operations and long-range planning (example see BRAC short-term planning item).
  • Keep the planning process inclusionary.
  • Broaden awareness and understanding of, and participation and confidence in, the planning process, including awareness of planning successes.
  • Prepare and distribute to the campus an annual report that identifies actions taken and their relation to accomplishment of the University’s mission and vision.
  • Identifying resources for funding initiatives and mandates.

Overview Discussion of Challenges

The University exhibits a culture of assessment that includes ongoing academic assessment activities that contribute to decision-making. Some of the assessment activities have focused on indirect measures such as alumni and student surveys. As departments evolve their assessment abilities, feedback from these measures provide departments with a sense of who they are and what their accomplishments are. They also provide positive and meaningful assessment experience that can be applied to direct assessment. As assessment plans are modified, they are be reviewed by the deans, the Academic Affairs Committee, the College Assessment Coordinators, and the Office of Research and Assessment to insure that modifications include direct learning outcome measures. The latter can be gathered to produce an assessment picture that can be added to assessment portfolios for Liberal Education and Graduate Studies.

When asked the question “what challenges occurred related to assessment,” a number of department chairs respond with a single word: “Time.” Many assessment activities are labor intensive in planning, implementation, and analysis. With assessment as a standard and a constant for academic departments, the University continues to explore ways to provide time or other compensation for faculty, such as the appointed college coordinators who assist departments with assessment.

Planning Challenges

The Planning Process:
Two years into the new planning process, committee members have adjusted to the committees’ structures, and have learned to listen to, respect, and cooperate with representatives from academic, staff, bargaining, and administrative units. The primary challenges remain communication, and elegance of efficiency in a system that represents multiple constituents.
The challenges for planning have thus moved away from the learning curve of the new structure and toward effective and efficient operation of the committees and the University. With the leadership of President Quistgaard and newly appointed Vice President for Student and Academic Affairs Joann Fredrickson, and the continuing guidance of the planning committees, the University is addressing those challenges.

The University Plan:
As part of their operational growth the planning committees and University and Executive Councils, as well as the colleges, the Office of Distance Learning, and other offices and departments have developed goals and strategic plans. These combined with goals and strategic plans developed by academic departments and the Colleges are the basis for the new University Plan.

Financial Resources:
Bemidji State addresses its financial challenges through a number of venues including the Budget and Resource Allocation Committee, the Office of Advancement; cooperation with MnSCU and with state legislators; and review and reallocation of budget lines and items. Capital campaign initiatives will be guided by the new University Plan scheduled for implementation at the end of fall semester 2002.

Closing Comments
The unevenness of implementation of the University’s approved Assessment Plan has been addressed and a campus culture of assessment has become a way of life. Programs have defined appropriate and measurable objectives for program educational goals and are collecting data that they use in decision-making.

Procedures for integrating planning with other critical areas of the institution (program assessment, five-year program reviews, resource allocation, committee and council infrastructure) are in place and are operative. An overall University Plan that incorporates an Information Technology Plan, Campus Plan, Development and Athletics Plan, and a Financial Plan, is in development and is based in part on completed strategic plans for the colleges and other University offices. It is scheduled for completion, including approval by all University constituents, at the end of fall semester 2002.

SIDEBAR
Development of the Monitoring Report

Work on the Monitoring Report began fall semester 2001. The process included the following:

Outlines & drafts -

  • Beginning in November 2001, outlines and drafts of the report were provided, at random intervals, to all bargaining units, with requests for feedback.
  • Drafts and updates of the Monitoring Report were posted on a web site (http://info.bemidjistate.edu/News/planning/index.html, under University Council) beginning March 25, 2002. The site was advertised through the University’s faculty-staff e-mail listserv and the President’s Newsletter.

Interviews and presentations –

  • The writer interviewed 27 department chairs, planning committee co-chairs, deans, and directors to learn the subtleties of assessment and planning activities.
  • The writer made presentations to the University Council, the Executive Council, bargaining unit Meet and Confers sessions, the Faculty Senate, and the Student Senate.
  • The writer met frequently with the Director of Research & Assessment, the deans, the Vice President for Academic and Student Affairs, and the President.

Anecdotal reports -

  • A call was issued to faculty and academic departments for descriptions of assessment activities in addition to those in assessment plans (see page __).
  • A similar call was made to student and university staff and departments (see Attachment __).

The document –

  • The Monitoring Report document was produced by the Publications Office.

BSU Today
Comments regarding this page may be e-mailed to jswartz.