Lesson

Clearly communicate requirements and testing procedures to connected vehicle device developers, and allow for industry input and iteration for less mature devices.

Experience from fostering the development of connected vehicle devices for the Safety Pilot Model Deployment in Ann Arbor, Michigan.


September 2015
Michigan; Ann Arbor; Michigan; United States


Background (Show)

Lesson Learned

The USDOT developed and implemented a collaborative process through which the DSRC device development community would eventually be capable of supplying devices for the Safety Pilot Model Deployment (SPMD). The USDOT identified two key activities in an effort to engage and develop this relatively immature community of developers.

First, the USDOT funded a subset of the research and development costs through a set of procurements that resulted in multiple awards to device developers. The developers were tasked with producing a small number of prototype devices, roughly 2 – 5. These devices were then subjected to functional testing by the USDOT. The USDOT returned the devices to the developers with the test results for refinement and retesting. This activity was important for engaging the device development community because the majority of device developers were small organizations, and they did not have the research and development funding to develop these devices without USDOT support. Also the business model for developing some of these devices was not well established as a relatively small number of devices were needed for SPMD, and the profit made from this activity likely would not cover the development costs. These USDOT funded contracts attracted a relatively large number of device developers to the process.

For the second activity, the USDOT encouraged an open exchange of information across the development community through the use of weekly technical coordination meetings, sample functional testing events, plugfests, and other activities aimed at rapidly increasing the developers’ experience. The primary purpose of this open exchange was to ensure that the devices produced by the developers were interoperable with each other. This required all developers to interpret and implement the requirements in the same manner. The weekly open meetings shared lessons learned and other vital information across the development community, as well as documentation, e.g., meeting notes, example code, for the community to utilize in their development processes.

The USDOT developed the initial version of the device requirements for Vehicle Awareness Devices (VADs), Aftermarket Safety Devices (ASDs), and Roadside Units (RSUs) and provided them as input into the Product Development Phase, as illustrated in Figure 4. During the Product Development Phase, the device requirements were iteratively revised based on the results from the functional testing of the prototype devices and on input from the device developer community. At the completion of this iterative process, a final set of the requirements was generated for each of the devices deployed in the SPMD. This final set of requirements was used to qualify devices for the research Qualified Product List (rQPL). The collective set of requirements for each type of device was identified in the published specification document.

The USDOT sought to mature the requirements to a point that the final set would be able generate devices that would operate throughout the SPMD with only minimal manual intervention from the Test Conductor or device supplier. As it was, the majority of devices deployed in the SPMD were still operational after one year of data collection, however, there were a number of gaps in the device requirements that impacted the Test Conductor’s ability to monitor and maintain the devices throughout the Model Deployment. For example, the final requirements lacked test functionality typically needed for less mature or prototype devices, including diagnostic functions, reset capabilities, detailed status messages and status indicator lights. As a result, the Test Conductor had to utilize a variety of manual methods to monitor both the in-vehicle and infrastructure devices deployed in the field to identify device failures. In some cases, when a failed device was discovered, without having reset capabilities the Test Conductor had to physically remove the device and ship it back to the vendor to perform the device reset. This resulted in additional downtime for the affected devices which reduced the volume of data collected and increased the amount of Test Conductor support required.

Related recommendations made in the source report include:
  • Determine the most efficient and effective activities, i.e., conference calls, workshops, working meetings, “plugfests?, to engage the device community and increase industry knowledge to ensure a common understanding and interoperability of devices, applications, and infrastructure to support project objectives.
  • Tailor device requirements on a project by project basis depending on the intended use of the devices and current device maturity level. Clearly communicate requirements to device developers and incorporate the appropriate functionality for less mature devices, such as test capabilities and functions to monitor device up-time.
  • Utilize an iterative process for developing device requirements that incorporates cycles of industry input and device testing. Complete the requirement generation and Qualified Product Lists (QPLs) prior to all device and system use. If it is anticipated that the requirements will be updated after testing based on field deployment, incorporate enough time in the schedule to procure updated components and retest.
  • Institutionalize and formalize all testing procedures for the QPL. Provide clear and unambiguous qualification test procedures, with pass/fail criteria, as part of the test plan.
  • Provide testing specifications covering the performance of all components, to allow common understanding and expectation, and allow sufficient time for iteration.
  • Develop application tests that are specific to the type of technology being deployed, and place additional emphasis on false positive scenarios.
  • Evaluate impacts of procuring devices from various vendors included on the QPL. Balance the cost of increased procurement complexity of having more device types with the cost of possible schedule impacts due to possible device production delays. Consider the impacts of having to conduct interoperability testing between large numbers of devices and schedule implications for any needed hardware or software updates.


Lesson Comments

No comments posted to date

Comment on this Lesson

To comment on this lesson, fill in the information below and click on submit. An asterisk (*) indicates a required field. Your name and email address, if provided, will not be posted, but are to contact you, if needed to clarify your comments.



Source

Safety Pilot Model Deployment: Lessons Learned and Recommendations for Future Connected Vehicle Activities

Author: Kevin Gay, Valarie Kniss (Volpe)

Published By: U.S. Department of Transportation

Source Date: September 2015

EDL Number: FHWA-JPO-16-363

URL: http://ntl.bts.gov/lib/59000/59300/59361/FHWA-JPO-16-363.pdf

Other Lessons From this Source

Lesson Contacts

Lesson Contact(s):

Kevin Gay
US DOT
kevin.gay@dot.gov

Lesson Analyst:

Gregory Hatcher
Noblis
202-488-5704
ghatcher@noblis.org


Rating

Average User Rating

0 ( ratings)

Rate this Lesson

(click stars to rate)


Lesson ID: 2016-00756