To finish up year two of our IMLS-funded grant, the CC-PLUS team pulled together a pilot project to test out our software at a larger scale. Pilot participants included six consortia across the US and Canada representing a diverse group of library sizes and types. The overall goal of the pilot was to investigate the following questions:

• Does CC-PLUS work at scale?
• What issues still exist with the interface and tools?
• What supporting documentation or tools are needed to make this effective?

The six pilot consortia met for the first time in January 2021 to discuss pilot goals and logistics. Instead of working on a shared research or comparison project, participants chose to set individual goals for their participation. Pilot participants agreed to meet weekly through mid-March to compare experiences and share information. After that, they would continue to have access to the pilot instance of CC-PLUS until the end of the grant period (June 30, 2021) but would meet only twice, at the mid-point in April and again at the end in June.

Individual goals among the pilot consortia varied. Some planned to compare CC-PLUS to their current workflows for gathering statistics to determine what advantages might be gained. Others developed collaborative workflows with their members institutions, and in effect ran a pilot-within-the-pilot. The ultimate goal for several was to develop a service for consortia members involving self-service for obtaining usage data. Still others planned to use the data gathered through CC-PLUS with other data on costs to create analyses and visualizations using other tools.

All of the pilot participants were able to make some progress on their goals within the initial six-week period, and some findings are already apparent. Most importantly, the pilot participants worked together to create documentation of the credentials needed to access specific vendors. In the world of COUNTER and SUSHI, there are different combinations of URLs, requestors, customers, and API, along with vendor-defined extension fields in some cases, that need to be combined for requests. Keeping a shared list of which vendors require which credentials, and which reports they produce was helpful. A community that supports this kind of information sharing would be immensely helpful to users going forward. As a result of the pilot a full suite of user documentation is being created and several templates and lists will be made available.

In addition to support outside of CC-PLUS itself, the pilot participants highlighted several areas where we can still do work to support the management of credentials. The process for uploading and updating credentials for various institutions and providers of data at present is complex and relies on very precisely formatted spreadsheets. But the act of even gathering the credentials in the first place can be time-consuming and require a lot of cooperation with consortium members. While we can’t do anything about the latter, a redesign of the processing of adding and managing the credentials within CC-PLUS could help to smooth the process. A participant from the Canadian Research Knowledge Network (CRKN) noted: “We recently completed a similar exercise for 20% of our members in a different context and it was a lengthy and challenging process. However, that both CRKN and members would then be able to access their reports in a single point of access would certainly facilitate usage data analysis for both the consortium and members long-term.”

Another area for improvement identified was the need for more nuance around user permissions and roles. CC-PLUS was designed for four different types of users: a consortial admin who can manage all institutions and statistics providers in the system, library managers who can manage their own institutions and credentials, universal viewers who can view the statistics gathered for all institutions and institutional users who can view the statistics for their own institution only.

The pilot presented us with our first very large consortium user: the Kentucky Virtual Library (KYVL), “ a consortium of nearly 300 Kentucky libraries and institutions, including colleges and universities, public libraries, K12 schools, hospitals, Department of Defense libraries, the Kentucky Department for Libraries and Archives (KDLA), and more.” The existing 4 user roles did not meet KYVL’s needs to have administrators and managers of various groups of libraries within their consortia, for example, a user who could manage all the high school libraries, or all the public libraries. Access could only be granted to manage all institutions and providers, or only one. This pilot feedback has been extremely valuable to our future design plans.

In March, informed by the pilot feedback and other project goals, the CC-PLUS Steering Committee submitted a proposal to the Institute for Museum and Library Services for an additional two years of funding. The new grant project will have three overarching goals:

1. Improve CC-PLUS based on feedback: enhance user interface, administrative controls, and improve database design and speed.
2. Fund a demonstration project that uses CC-PLUS as part of a larger data analysis project, combining usage data in a seamless pipeline with other library data for analysis.
3. Build a strong CC-PLUS community.

The last goal is inspired by the pilot in several ways. Documentation and supporting information gathered by and for the pilot group will be used to create a hub for users. The listserv created to support the pilot participants communication will be repurposed into a general CC-PLUS users list. Finally, the model created by the pilot, wherein a consortium can have access to a hosted instance of CC-PLUS for several months to try it out, and then can export data to be ingested and used in a self-hosted version, will be extended to others.

The CC-PLUS Steering Committee would like to thank the pilot participants for their dedication and important contribution to the project:

• CRKN – Émilie Laveleé-Funston and Jaclyn McLean
• GALILEO – John Stephens and Ken Henslee
• KYVL – Ilona Burdette and Enid Wohlstein
• OhioLINK – Joanna Voss
• SCELC – Jason Price
• TRAILS – Hannah McKelvey