Objective:
To build test development environments that produce a set of integrated testing tools, including self-adapting test suites that are dynamically created and factor in unique characteristics for defined subsets of a given specification or sets of specifications.

Background:
Many enterprises have become driven by digital data, including health care, manufacturing, e-commerce, and homeland security. Each of these enterprises defines standards for the exchange and manipulation of industry-based information. These standards are often complex, with many optional features and include references to other standards or are comprised of many base standards that must work seamlessly together. As the plethora of standards is promulgated throughout the industry, the demand for metrics to measure conformance is essential. Old methods of systematically hand crafting tests over the course of several years must give way to newer methods of generating self-adapting tests.

It has become well known that software is not adequately tested and is rushed to market full of bugs. A recent NIST study states that the annual cost to the country for inadequate testing is as high as 59 billion dollars. A primary reason for inadequate testing is the time and expense of developing comprehensive test suites. Today’s standards are often large and complex, and have gained widespread industry support through universal and all-inclusive designs with many optional features. However, this approach results in standards that cannot be sufficiently constrained to provide a single and consistent interpretation – prohibiting plug-and-play installations. As a consequence, systems are difficult to implement and debug, resulting in undue costs. Many standards organizations have addressed this problem by developing profiles. A profile is a proper subset of the standard that specifically states the optional constructs and processing rules. Profiles are key to standardization at the implementation level and promote plug and play systems. However, if profile rules are not followed then interoperability problems will still persist. Conformance testing of these profiles is essential.

Profiles provide a path to conformance and interoperability testing. However, since each installation is free to define its own set of constraints to meet individual requirements, a single profile or “standard” is not obtainable. Profile creation is essentially unbounded, which proves problematic for testing. Legacy and the dynamic nature of today’s industries limit the consolidation that is hoped for with profile creation. The state space typically increases as opposed to decreasing. In this environment, testing becomes a challenging and difficult problem. Current practice involves meticulous by hand debugging of implementations as problems appear. Test suites and tools are not available; consequently advanced testing is limited. The need for adaptable and automated test tools is apparent. Our innovative approach is centered upon the automatic and dynamic creation of tests based on profiles represented in XML. Building upon our prior experience in XML-based testing, we propose to develop a data driven approach that consists of encapsulating the structure and semantics of a profiled specification or set of specifications. A core set of reusable APIs are built as the test engine, providing the ability to configure test services and applications as needed, such as message validation and generation. Extending this initial approach, we address the need for different types of tools to enable testing of a composite set of standards (such as the HITSP Interoperability Specifications) by developing a configurable, customizable toolkit. This toolkit will tie together disparate testing tools through a common interface, providing a user access to the set to tools needed to ensure conformance and interoperability.

This approach will be applied to HISTP Interoperability Specifications, focusing on Health Level 7 (HL7) version 2 specifications with the expectation that the methodologies developed will be applicable in other domains. The HL7 version 2 messaging standard is large and complex; it is constrained with message profiles and thus implementation requirements are unknown until design time. For testing such systems this requires an automated and self-adapting approach. There is strong consensus that a standards-based implementation is the key for the US to reap the benefits of healthcare information technology infrastructure. However, achieving true interoperability in major enterprises is difficult due to the complex interaction of the standards involved. Having a NIST-developed method for developing test suites that can be tailored to a particular community promises to have a positive impact on many strategic focus areas, significantly reducing the software development cost, and promoting interoperability among many critical applications.


 

Page created October 2008

Web site point of contact