Involve and collaborate with a broad range of users during software design, development, testing, and deployment to increase the return on investment.

New Mexico Department of Transportation's experience in designing and deploying a web-based software application to simplify the increasing complexity of coordinating rural transit funding.

Date Posted
04/05/2006
TwitterLinkedInFacebook
Identifier
2006-L00201

National Evaluation of the New Mexico Client Referral, Ridership, and Financial Tracking (CRRAFT) System (Final Evaluation Report)

Summary Information

Beginning in 2000, the Alliance for Transportation Research Institute (ATRI) at the University of New Mexico and the Public Transportation Programs Bureau (PTPB) of the New Mexico Department of Transportation began developing the Client Referral, Ridership, and Financial Tracking (CRRAFT) system. CRRAFT is a Web-based software application that is intended to simplify the increasing complexity of coordinating rural transit funding, invoicing, and reporting in New Mexico. CRAFT is developed as a single application to help manage transit services for all transportation providers that receive funding through the Federal Transit Administration's (FTA) rural transit (FTA Section 5311), Job Access Reverse Commute (JARC, FTA Section 3037), and/or Temporary Assistance for Needy Families (TANF) programs.

In December 2001 and September 2003, the U.S. Department of Transportation ITS Joint Program Office (JPO) provided funding to ATRI, via FTA, to develop CRRAFT. Subsequently, the JPO selected CRRAFT to be the subject of a national evaluation to measure or confirm the expected outcomes of the system. This lesson is based on the national evaluation Final Report, released in 2005.

Lessons Learned

The implementation of ITS software should actively involve a broad range of users during design, development, testing, and deployment to ensure the software investment results in a system that meets user needs and is accepted by the users. Although overall CRRAFT resulted in more accurate invoices, less time researching and collecting invoice information, and better communication and coordination between New Mexico Department of Transportation (NMDOT) and transit agencies, the CRRAFT system had a more positive impact on the NMDOT's PTPB than on the transit agencies. Transit agencies that provided a large number of demand responsive trips tended to be dissatisfied with CRRAFT's overall performance because of the time required to manually enter trips into the scheduler, and then to reconcile scheduled and actual trips. Further, because CRRAFT did not support all transit agency needs, many transit agencies were forced to re-enter ridership data to support their own agency reporting requirements. The duplication of work led to additional labor and dissatisfaction with CRRAFT.



In general, the following examples from the implementation of the CRRAFT software application provide suggestions on improving the overall return on investment:

  • Develop requirements based on user needs to improve user acceptance. Although CRRAFT supported the needs of PTPB, the software did not support several important transit agency needs. Two of these features were: access to data for internal transit agency reporting and the need to quickly enter scheduled trip data. Consequently, many transit agencies were forced to enter trip/ridership data twice to support their internal agency reporting requirements and receive funding from PTPB. Since the CRRAFT scheduler module was not designed to provide transit agencies with the ability to quickly enter trips into the scheduler module, many transit agencies found it easier and quicker to continue using their pre-CRRAFT scheduling methods to support their day-to-day operations. However, to receive funding from PTPB, transit agencies had to enter the trip/ridership data into CRRAFT to generate the required invoice. As would be expected, many transit agencies that provided a large number of trips were dissatisfied with CRRAFT.
  • Involve users of the system in the design, development, testing, and deployment of the software to reduce post-implementation changes and reduce costs. Had the transit agencies been more involved in the design, development, testing, and deployment of the CRRAFT software, needed features would have been identified sooner and included before the software was deployed. The CRAFT development team tried to address the numerous requests for changes to accommodate the various transit agency needs until funding became constrained and additional changes were not possible.



This lesson points out that good requirements and end user participation in the development process are critical to ensure that software is designed which support user needs and reduce the costs of post-development software changes. Software designed and developed without collaboration of representatives of the various users (e.g., PTPB, large and small transit agencies providing demand responsive and fixed route services) can result in reduced user acceptance, duplication of work, and additional costs for software refinements.