Working on my Masters in Library and Information Science granted some wonderful opportunities when I was working at the King Library at San Jose State University. A culminating MLIS student, Bridget Kowalczyk, was developing an online tutorial for the Reference and Academic Services Department that would teach and train staff to recognize different service points in the Library. As I worked through the tutorial, I learned which positions at the King Library were responsible to provide certain services. At the end of the tutorial, I was given a quiz to determine if I could identify which library staff was responsible for service in multiple scenarios. I made a few mistakes on the quiz, but felt that I had gained some knowledge of service points at the King library. Afterwards I was able to sit down with Bridget and Lorene Sisson, head of the Reference and Academic Services Department, and discuss the tutorial. We discussed what I thought the purpose and goal of the tutorial was against Bridget’s proposed goal. They questioned me about the format of tutorial and the language used to convey the information. They then reviewed the results of my quiz and shared my results with me in comparison to others who took the tutorial. Most of my responses were correct, but the ones I got wrong the majority of others answered incorrectly as well. Bridget and Lorene explained that those questions were either not worded correctly or the answers were not clear. They asked me what I would do to fix these issues. After the evaluation was over, we talked about how important it is for library leaders to plan, create and evaluate library programs and services.

Libraries are in business to provide service. Without effective means of evaluating all of our services, were are operating in a dark vacuum. The only effective way to know whether we are providing the services our communities want or need is to have our communities evaluate us. Through the course of time, every library’s service community will change, and evaluation programs are the only effective way to determine what changes need to be made in order to continue providing appropriate services and resources. Libraries can ill afford to provide continued resources of items that are not utilized. In order to provide our services library administrators need to evaluate vendors, jobbers and library staff to ensure the library’s return of investment. Library budgets must stretch as far as possible. Vendors do not all provide the same services, and administrators should constantly be comparing services. Library administrators, technical services and acquisitions staff must take into consideration the cost it takes to stay with their loyal vendor when another vendor is able to provide more services at a lower cost. The same is unfortunately true of library staff. Staff performance affects the budget as well, and administrators should use evaluations to take appropriate actions to ensure quality of service.

Evaluations should not be considered dreaded events. They have a tendency to be complex and a hassle when it comes down to conducting them and tabulating their results. Yet to those organizations which understand their value, evaluations are viewed as tools used to improve and build up programs and services. As librarians we have many tools made available to evaluate our programs and services. We have staff observations, utilization and circulation statistics, acquisition requests, budget reports, best practices, community and patron surveys, and customer service feedback. Many of these tools are effectively integrated into the daily operation of the library and provide constant results, while others need to be manually conducted. Through the use of the integrated library system, library staff can create and conduct simple surveys and polls that users can use while searching items in the catalog or checking items out on the automated service machines. Planning the questions to be addressed by the surveys and polls is difficult. Librarians must think through their questions, estimating what type of results they expect and choose the appropriate language in which the survey will be conveyed.

Goals, purposes and desired outcomes should be apparent and measurable as part of the evaluation process. It would be practically impossible to measure whether a program was successful or not if there were no goals associated with its completion. Evaluations are most effective when there are specified criteria to generate quantifiable results. Technical services provide a prime example of how an examination of quantifiable results can determine cost effectiveness of vendor solutions compared to in-house services. Administrators can calculate the cost and time it takes a cataloger to process a new acquisition and prepare it for circulation. This result can be compared to what a vendor quotes the cost to be, allowing the administrator to make an appropriate and informed decision. Periodic evaluations should be integrated into any program or service. Weekly and monthly programs can evaluated at the end of each session by staff and patrons. This will assist library staff to determine the program’s effectiveness and modify it to meet the goals of the program and the patrons.

One of my projects as an intern at The California Maritime Academy Historical Archives was to plan and implement an Oral History Project. The program’s inception began with a meeting of the Vice President of Public Affairs, the Library Director, the Archivist and me. In this meeting, we established the goal of the project to be the documentation of the influence of The California Maritime Academy and its alumni in modern maritime history. With that goal in mind, I began to research other oral history projects on the Internet conducted by universities and government organizations. I had to analyze and evaluate each of their web sites and programs with a focus on layout, procedures and forms. Each organization had vastly different layouts to their sites, depending on whether their oral histories were made available over the Internet or could only be accessed at the actual archive. My evaluation of all of the programs’ procedures showed that the Oral History Association’s Guidelines are the practiced standard. My foray into each projects’ forms yielded a large variation in wording and purpose. The Library Director instructed me to make our forms as succinct as possible. I evaluated over 40 various forms, ranging from biographical information to copyright release to item donation. I was able to define criteria that allowed me to compile the procedures and forms needed for our oral history project.

The Oral History Packet demonstrates my ability to analyze and evaluate oral history projects based on the goals of my institution and the criteria given me by the Library Director. Through this packet, we are sharing the goals of the program, providing participants with all of the necessary forms needed to submit an oral history and the procedures and suggestions to conduct their own oral history interviews. This project shows how I was able to synthesize others’ work and create a unique product specific to the needs of The California Maritime Academy Historical Archives.

Information literacy programs are an integral part of instructional services at academic libraries. While an intern at The California Maritime Academy Library, I was able to work with the Information Fluency Librarian, Mindy Drake, to teach and assess information literacy skills in the EGL 100 course. Multiple assessment methods were used to determine the information literacy of each student.

We began by devising our own survey to assess students early in the semester. We referred to the ACRL Information Literacy Competency Standards for Higher Education to provide the criteria upon which we needed to base our assessment of the students. The survey we created was comprised of multiple choice questions, rating and situational scenarios. I was given the task to create a spreadsheet to individually record each student’s responses. The survey allowed us to determine that there exists a correlation between a student’s major and his or her information literacy skills as suggested by Carol Kalthau. The spreadsheet provided a means to quantify the results. Ms. Drake and I analyzed and evaluated the results to determine which information literacy skills students already possessed and those that we would need to focus on during our instruction time.

The semester ended with our administration of the Information and Computing Technology Literacy Assessment (ICT) to the freshmen enrolled in COMS 100. The ICT is a national standardized test developed by ETS, based on the ACRL Information Literacy Competency Standards for Higher Education. Results compare our students to that of the national average. Test results are confidential to the institution at this time, but we are able to compare and evaluate the results of our survey to those of the ICT. Improvements were made in all six skill areas, but that did not change the fact that the students tested were below the national average according to the ICT test. Despite the computer proficiency that each student demonstrated, incoming freshmen at The Maritime Academy lack the information fluency skills needed in higher education.

My participation in the information literacy testing gave me insight to the course evaluation procedures at The California Maritime Academy. It was determined that the minimal involvement granted to the librarians in freshman level computer and information classes was insufficient. Evaluations conducted by the Library and Curriculum Committee resulted in the creation of a new course, LIB 100. The focus of this new course is to develop the basic information literacy skills at the campus to the national average. My work in accessing and evaluating information literacy demonstrates that I am able to take national assessment criteria and compare it to a localized service area. This experience attests to me the significance of evaluating established programs. While the adage “Leave well enough alone” is widely practiced, those professions where service is the main goal cannot afford this passive approach. As my career develops I will use my ability to evaluate programs and services to bring positive change to the communities I serve.

I am confident in my ability to evaluate programs and services on specified criteria. My participation in conducting and tabulating the results of various evaluations has shown me how important it is to create quality evaluative material. This skill is universal, and I can use it benefit any library I work for. I plan on attending seminars sponsored by the ALA and independent business organizations to learn how to effectively develop surveys and questionnaires so that I might be better prepared to serve my institution. I know that the key to performing a good evaluation is to be consistent in judging items according to specified criteria. Failure to do so on my part will result in wasted funds and resources that could jeopardize my library. I will use this skill to effectively lead my library and staff in providing the services my community needs.