Manufacturing Engineering Laboratory NIST logo
HOME About MEL Research products and services what's new search

Return to
Programs page

Projects

Metrology and Standards for Advanced Perception Systems in Intelligent Manufacturing

Performance Simulation (PerfSim): A Hybrid, Real-Virtual Testing Environment

Mobility and Manipulation Performance Metrics and Standards

Safety for Next Generation Manufacturing

Control Systems and Positioning for Nanoscale Measurements and Standards

 

 

 

 

 

 

 

 

 

 

melwebmaster

Date created:
March 28, 2008
Last updated:

Return to top

Return to top

Return to top

Return to top

Return to top

Return to top

Return to top

Return to top

Return to top

The Thermo robot with the SmartTRACK
sensor attached
The Thermo robot with the SmartTRACK 
sensor attached

Intelligent Systems Program

Measurement Science for Intelligent Manufacturing Robotics and Automation

Program Manager: Michael Shneier

Annual FTEs: 10 NIST staff

5.5 guest researchers

15.5 total FTEs

Challenge:

Manufacturers anticipate a future in which humans will work side by side on the shop floor with versatile and adaptable robots and other automation systems. To realize this vision, equipment will need to perceive the geometry, location, color, features, and movements of items and people in its vicinity and have an intrinsic ability to adjust for variations in parts, environment, and other conditions, rather than relying on fixed automation and costly infrastructure.  This must be achieved in both macro-scale manufacturing and in the emerging micro- and nano-scale manufacturing industries. The long-term challenges include developing the performance measurement science to characterize constituent components of intelligent manufacturing systems, defining the target performance goals, and measuring how well a component or overall system meets the goals.

Overview

The program addresses the needs of US industry for more flexible and efficient manufacturing processes that will accelerate innovation and advance U.S. leadership in high value, knowledge-intensive manufacturing, which is seen as the future of U.S. manufacturing. It does so by providing measurements, standards, and performance evaluation methods for the key infrastructure and control components of the manufacturing environment. These include sensors for perceiving and measuring the environment, methods of controlling manipulators and vehicles for part handling, safety systems to ensure that humans and robots can work together, and interfaces between the different systems to assure that information is passed correctly. While the program is divided into several focused projects, many tasks span multiple projects, and most of the projects take advantage of a test bed currently being constructed in the NIST Shops. Human-robot collaboration and safety are addressed both through continued contributions to standards organizations and through components of each project. The availability of standards and measurement methods will drive product and process innovation in manufacturing and enable quick adoption of new technologies to  enable US manufacturers to retain leadership in high speed, integrated manufacturing and assembly.

Why NIST?

The work in the program fits within the measurements and standards objectives of the MEL mission, and addresses problems that are of industry-wide concern to manufacturing equipment vendors, integrators, and users.  It is unlikely that any of these institutions will undertake the work on their own because of the need for broad consensus in areas that relate to critical business decisions such as opening robot control systems to enable safety sensors to be integrated or components from multiple manufacturers to be “plug and play” compatible. Most activities are also too risky to warrant substantial investment by industry due to lack of standards, safety concerns, and the long-term nature of the required research.

The program supports several goals identified during the NIST Smart Assembly workshop in 2006, such as monitoring shop floors to maintain virtual models, improving sensing capabilities, providing additional capability and adaptability in assembly systems, providing intelligent, safe devices, and providing reusable assembly systems. Performance measures and standards are central to effective in-process measurement and have broad applications for real-time, continuous metrology in manufacturing processes. This will enable significant advances in automation, such as real-time visual servoing by robots, human/robot interaction, flexible feeders, and measurement and inspection of unfixtured parts in motion.  Real-virtual hybrid modeling and control will enable manufacturers to safely measure, reconfigure and understand the motion of robots, conveyances, parts and assemblies in a moving assembly line involving interacting humans.

Substantial, immediate benefits will accrue in industry segments including medical, energy, robotics, aerospace, automotive, and electronics.. The technology is fundamental and will be widely applied across a whole range of industries, from small manufacturers who currently do not use automation to large, highly automated factories. The capabilities developed under this program will enable substantial productivity and quality advances in sectors such as manufacturing, construction, and security, and provide a basis for the US to improve global competitiveness in these and numerous other industrial and service areas. This program will enhance not only manufacturing but also other applications of robots, including the emerging industry of home and industry service robots.

 

Projects

Measurement Science for Intelligent Manufacturing Robotics and Automation

Program Objectives
Objective 1:  Develop robust dimensional metrology methods to evaluate the performance of sensors for manufacturing in dynamic unstructured environments where people and machines interact.

Metrology and Standards for Advanced Perception Systems in Intelligent Manufacturing
Anticipated Completion Date: Q4/2011

Project Overview

The project will develop quantitative, reproducible test methods to evaluate the robustness, accuracy and performance characteristics of advanced perception systems for general assembly and other macro scale manufacturing operations. For advanced use on the manufacturing shop floor, perception systems need to sense three-dimensional structure in a dynamic, uncontrolled environment. In particular, they must be able to accurately determine the six-degree of freedom (6DOF) position and orientation of parts, people, robots and other objects in motion. This will enable new applications of automation, including continuous assembly where a robot interacts with a moving assembly line. Recent, on-going, innovations in sensor technology, computer speed, and perception algorithms promise to bring perception technology to a mature state, but standards and metrology are required to enable manufacturers to define and evaluate the achievement of satisfactory performance. This is called out as an important area for innovation in the NIST assessment of  the United States Measurement System (USMS)1

Deliverables and Intermediate Milestones:

Q3/FY09

Develop a roadmap for selecting sensors and developing evaluation metrics in collaboration with industry and academia. This will establish a strategy for reviewing and evaluating sensors that use diverse technologies to make their measurements.

ongoing through Q4/FY11

Establish dynamic 6DOF measurement protocols and metrics for new and existing sensor technologies.

Q4/FY10

Develop effective and robust sensor calibration protocols to support measurement protocols. This will enable accurate comparisons between test and reference systems

bi-annual, starting Q1/FY09)

Organize workshops for defining metrics, measurements, and protocol for selected manufacturing shop floor applications. This will assist researchers working on manufacturing applications.

Q4/FY09

Construct an ontology2 of manufacturing sensors and their characteristics in collaboration with manufacturing and sensor industries.  This will assist in standardizing terminology and defining sensor interfaces.

Q4/FY11

Develop models of sensor system performance to understand the error contributions of components. This will help researchers and sensor vendors understand how to optimize sensors to meet higher standards of performance

Q1/FY10 -
Q4/FY11

Establish metrics for evaluating perception-based safety sensors and algorithms. This will enable greater safety and establish new forms of human-robot collaboration in manufacturing applications

Customers:

  • USCAR, GM, Ford, Chrysler.
  • Robot manufacturers
  • Sensor manufacturers
  • AGV industry.

Collaborators: 

  • GM, Ford
  • Purdue University, Loyola University, NASA, Army Research Laboratory
  • Automated Precision, Inc.
  • General Dynamics
  • BFRL, MEL/PED

Measurement Science for Intelligent Manufacturing Robotics and Automation

Objective 2: Provide industries with standards, performance metrics, and infrastructure technology to enable the use of semi-autonomous and autonomous manipulators and vehicles in static and dynamic assembly operations and material handling for manufacturing.

Performance Simulation (PerfSim): A Hybrid, Real-Virtual Testing Environment
Anticipated Completion Date: Q4/2011

Project Overview:

PerfSim will provide industry with an adaptable, safe, and repeatable environment where manufacturing automation and robot performance standards can be developed and rapidly tested. The use of a high-fidelity simulation system with standardized interfaces will allow researchers and developers to test against, and participate in the development of, evolving performance standards without costly infrastructure requirements. In this project we will use a hybrid real/virtual system and modular architecture that allows for the seamless integration of real hardware components and algorithms with virtual reality. Simulated avatars are able to test robot/human safety standards, repeatable simulated system faults test reliability and fault tolerance, and simulated sensor outputs allow for the analysis of potential system improvements through the incorporation of currently unattainable levels of sensor performance.

We will develop forward-looking performance metrics using a scenario-driven model that incorporates end-user constraints, in accordance with the System and Component evaluation in Operationally Relevant Environments (SCORE) framework. The SCORE framework enables evaluation of a system at the component level, the system level, and in operationally-relevant environments. The framework is end-user focused and produces system requirements based on perceived future scenarios. This leads to test methods to measure how well systems perform against the requirements in categories including communications, mobility, logistics, human-robot interaction, navigation solutions, and sensor processing.  ISD will continue its very successful model of international competitions to drive innovation and foster researcher/developer collaboration and metric validation.

Deliverables and Intermediate Milestones:

Q4/FY08

Develop and implement an architecture and open source framework for integration of virtual and real sensing components onto robotic platforms. This milestone provides the foundation for the rest of the project.

The panel will be created by Q4/08 and will continue to exist for the remainder of the project

To generate user pull, create an industry panel of small and medium sized manufactures focused on developing end-user requirements for robotic manufacturing systems. The industry panel will help to define the critical areas of performance measurement that we will address.

A roadmap for the project metrics will be delivered by Q2/09. The roadmap is anticipated to run until Q4/11.

Develop real/virtual performance measures derived from industry requirements. These measures will allow end-users to have confidence in the products that they purchase, and will provide developers with concrete performance standards for their systems to meet.

Q4/08. It is anticipated that tool development will continue through Q4/11.

Develop open source tools to aid in the objective evaluation of the performance of system components against the above-defined performance measures. We will provide tools that are easy to use and understand and that will allow both developers and end-users to measure the performance of their systems utilizing our reproducible test methods and test fixtures.

Q2/FY09

Integrate the Mobility Open Architecture Simulation and Tools (MOAST) framework into the robotic arm test-bed. This will leverage previous work performed under MOAST and will immediately provide a Cartesian space controller for our test-bed. It will enable the evaluation and comparison of joint space vs. Cartesian space metrics

The first workshop will take place in Q3/2008. A follow-on workshop will occur in Q4/2008 and a mapping camp will occur in Q1/2009.

Hold navigation solution workshops that bring together industry and researchers to develop performance measures for robot-generated maps. There are currently no widely accepted methods for evaluating the quality of automatically generated maps of facilities. This workshop is the first step towards developing such methods. These methods will allow end-users and developers to measure their performance against other products and their peers.

It is anticipated that additional bi-annual workshops will continue for the duration of the project.

Q4/2010

Conduct an evaluation of next generation simulation engines and port our Unified System for Automation and Robot Simulation (USARSim) code to the most capable engine. The current version of the simulatoin engine that USARSim is based upon is nearing the end of its life cycle. A study will be performed to determine which of the over 30 commercial game engines is best suited for our next generation and the code base will be ported to that engine. The study will be completed Q2/2009 and the port of the code will be completed


Develop forward-looking real-world manufacturing scenarios and virtual implementations with evaluation metrics for distribution to researchers. This is a continuous effort to distribute the products of this project to our customers and partners.

Customers:

  • General Motors
  • Dixon Valve Company (A world leader in hose fittings and accessories)
  • Army Research Laboratory

Collaborators:

  • General Motors
  • Dixon Valve Company
  • University of California, Merced
  • George Mason University
  • Hood College
  • University of Maryland, Eastern Shore
  • IEEE Robotics and Automation Society
  • University of Freiburg
  • University of Sydney
  • University of Koblenz-Landau

Measurement Science for Intelligent Manufacturing Robotics and Automation

Mobility and Manipulation Performance Metrics and Standards
Anticipated Completion Date: Q4/2011

The project will provide industries with the necessary standards, performance metrics, and infrastructure technology to support the use of semi-autonomous and autonomous manipulators and vehicles, control system architectures, safety systems, and advanced measurement tools and methods, that enable their use in static and dynamic assembly operations and material handling for manufacturing. We will collaborate with key robot arm and AGV developers and user partners to advance the state of autonomous robotic technology through mobile-base robot arms, mobile robots, non-contact safety sensors and open, modular control system architectures that enable broader use of advanced perception, autonomous navigation, manipulation and parts/material handling techniques in the automotive, aerospace, and other industries.

The Mobility and Manipulation Project will develop performance measurement methods for autonomous assembly. The work will be grounded in a manufacturing test bed shown in the Figure below, which will provide an integration environment for the other projects. The project will also provide advanced, 3D measurement capabilities (currently non-existent for the AGV industry) through industry-requested measurements and direct collaboration with AGV manufacturers and users.  We are working with an industry consortium that includes Egemin, FMC and Danaher.  We are also working on potential collaborations with the forklift industry in applying non-contact measurement techniques to manned forklifts.

We will continue our participation in the development of automatic guided vehicle (AGV) safety standards (ASME B56.5) through membership on the standards committee.  NIST receives frequent requests from this committee to present new advances in AGV measurements and controls.

Deliverables and Intermediate Milestones:

Q2/FY09

Design and implement a performance measurement test bed taking into account input from industry. 

Q3/09

Procure, lease, or borrow an AGV for use in the test bed.

Q1/FY09 through Q4/FY11

Hold annual workshops with customers to develop performance metrics and standards for sensing and control in dynamic and autonomous assembly operations.

Q4/2009

Incorporate the NIST real-time control reference architecture (4D/RCS) by means of the MOAST environment into the test bed.

Q2/FY10

Measure the performance of a robot arm as it manipulates parts, and of a robot vehicle moving within the test bed using laser scanners and trackers, SkyTrax barcoding, perception and other measurement tools.. Document findings.

Q4/FY08 through Q4/FY10.

Carry out Phases 2 and 3 of the existing consortium agreement with the AGV industry

Q4/2009

Evaluate the use of a 3D range sensor on vehicles as a safety and measurement sensor.

Q4/11

Continue participation in the B56.5 safety standard committee to improve the safety of autonomous vehicles through

Q4/FY08

Develop the B56.5 safety standard language to address the use of non-contact safety sensor on AGV’s.

Q4/FY10

In order to enhance safety, use 3D sensors to measure locations and sizes of overhanging objects (cords, ladders, truck beds, fork lift tines), pallet openings and oncoming vehicles from a moving vehicle. This will further advance the B56.5 standard. Q4/FY10.

Q4/FY10

Organize a workshop with the Forklift and AGV industries to identify measurement issues and develop a roadmap for collaboration across the two industries.

Q4/FY11

Develop human/robot interaction, multi-robot, vehicle and arm/vehicle standards in complex and unstructured environments. Document findings.

Customers:

Concept drawing of a performance measurement test bed for an autonomous assembly environment, including a robot arm on a rail, an AGV, and mannequins working together safely without fences while being measured using advanced tools.
Concept drawing of a performance measurement test bed for an autonomous assembly environment, including a robot arm on a rail, an AGV, and mannequins working together safely without fences while being measured using advanced tools.
  • GM
  • Ford
  • Chrysler
  • Boeing
  • Northrop Grumman
  • US Army, Navy, Air Force
  • AGV manufacturers and users
  • Forklift manufacturers and users

Collaborators: 

  • USCar, SAE
  • RIA
  • MHIA
  • AGV manufacturers and users
  • Forklift manufacturers and users

 

 

Measurement Science for Intelligent Manufacturing Robotics and Automation

Objective 3: Enhance the safety of next generation manufacturing by participating in standards organizations and developing performance evaluation methods for sensor-based products that enable increased human interaction with machines without compromising safety.

Safety for Next Generation Manufacturing
Anticipated Completion Date: Q4/2011

Project Overview

The Next Generation Robot (NGR) is envisioned as a machine incorporating inherent safety design that enables collaborative human-robot interaction and promotes lean manufacturing by reducing handling time and removing safety barriers that slow down transportation and increase cost. Safe robots will require a variety of sensors to ensure that the environment around the robot remains safe even as humans and robots work together. Performance metrics and evaluation methods for these sensors will be required, as well as for the control systems of safe robots. This project will address these issues in conjunction with the specific work conducted in the other projects in the program. According to the USMS, “The absence of adequate sensors and detectors in applications related to… human health and safety… poses a significant measurement barrier to technological innovation for the United States.”

The project objectives are to promote inherently safe design and operating features of next generation robots, to develop the metrology technology and sensors that will facilitate the development of the NGR, and to participate in the development of industrial robots safety standards.

Deliverables and Intermediate Milestones (not already included in other projects):

 

Contribute key technical content and facil­itate collaborations to develop the indus­trial robot safety standard-ISO 10218-2. This standard specifies requirements and guidelines for the installation and safe operation of industrial robots.

 

Contribute key technical content and facil­itate collaborations to develop a Technical Report on Guidelines for Implementing ANSI/RIA/ISO 10218-1-2007.

 

Contribute key technical content and facil­itate collaborations to develop a Technical Report on Guidelines for Implementing ANSI/RIA/ISO 10218-2.


Organize workshops for defining metrics, measurements, and protocol for selected manufacturing shop floor applications. This will assist researchers working on manufacturing applications.

 

Develop and test prototypes of a MEMS-based robot safety microsensor.

Customers:

  • GM Automobile Assembly Plant, Arlington, TX.
  • Automotive manufacturers
  • Robot manufacturers
  • Safety sensor manufacturers
  • Aerospace manufacturers

Collaborators: 

  • Robotic Industries Association (RIA).
  • Occupational Safety and Health Administration (OSHA).
  • University of Texas Automation and Robotics Research Institute, Arlington, TX.

Measurement Science for Intelligent Manufacturing Robotics and Automation

Objective 4: Develop control and positioning systems for nanoscale measurements, manipulation, and standards, and enable scale-up interfaces and standards between the micro/nano manufacturing tools and the macro scale world.

Control Systems and Positioning for Nanoscale Measurements and Standards
Anticipated Completion Date: Q4/2011

Project Overview

Next generation manufacturing will extend to smaller and smaller components, including micro- and nano-structures. The measurement of properties and dimensions, imaging of structures, and modeling of behavior are all difficult at the nano-scale. Nano–manufacturing is an area called out in the USMS as needing solutions to measurement problems and requiring novel actuators.   A challenge of micro/nano manufacturing is the development of methods to build complex three-dimensional (3-D) micro/nano scale structures and devices using techniques that allow them to interface with the macro scale world (scale-up), at economic production rates.  New capabilities at the micro-nano scale must be combined to meet these challenges, such as, dexterous manipulation and assembly, precision motion control, and real-time metrology and sensing. The metrology of micro/nano structures, such as nanowires, nanoparticles, proteins, cells, etc., is a key enabling technology for advances in micro/nano technology and manufacturing, presenting many opportunities for novel micro/nano electromechanical and biomedical systems.

This is an enabling technology, which can affect the development of markets in many new classes of products that require accurate nano component 3-D position and orientation. For example this technology can accelerate the production of nano components, electronics, composite materials, sensors, fuel cells, etc. According to a Lux Research report, the nano technology products market could reach $2.6 trillion in approximately 10 years. Assuming the proper infrastructure gets put in place, an estimated 10 million manufacturing jobs worldwide − or about 11% of the total manufacturing jobs − may involve nano technology in that time frame. According to a SusChem report the nano technology machinery market is expected to grow by 30% per annum.

Project Objectives

  • Develop control and positioning systems for nanoscale measurements and standards.
  • Develop control and positioning systems for nanoscale manipulation and standards.
  • Develop interfaces and standards that bridge the gap between the micro/nano manufacturing tools and the macro scale world.
  • Stimulate the academic research community through administration of competitions such as the nanogram league of RoboCup (in conjunction with EEEL). This will drive universities to address research important to NIST and industry.

Deliverables and Intermediate Milestones 

Q4/FY09

Develop embedded force metrology sen­sors for the MEL/ISD MEMS nano posi­tioners with expected target accuracy of 50 mN and resolution of 10 mN.

Q4FY10

Develop embedded force metrology sen­sors for the MEL/ISD MEMS nano posi­tioners with expected target accuracy of 50 mN and resolution of 10 mN.

Q4/FY11

Advance and optimize the performance of the MEL/ISD MEMS nano positioners. (

Q4/FY11

Demonstrate the use of the MEL/ISD MEMS nano positioners for one or more nano metrology needs. These could be a next generation micro AFM or a micro nano material rheometer.

Q4/FY11

Develop advanced metrology, control, and positioning systems, for nanoscale manip­ulation, in order to support our nano metrology and standards effort.

Q1/FY09

Establish goals for 2009 nanogram RoboCup competition.

Q3/FY09
and successive years

Help administer RoboCup competition

Customers:

  • RPI Center for Automation Technologies and Systems
  • APNanotech.
  • American Society for Testing of Materials.
  • American National Standards Institute.
  • RoboCup for Nanomanufacturing Material Delivery and Removal.

Collaborators: 

  • RPI Center for Automation Technologies and Systems

1 An Assessment of the United States Measurement System: Addressing Measurement Barriers to Accelerate Innovation. (NIST Special Publication 1048), February, 2007.

2 An ontology describes what entities exist in a domain and what are the basic categories of and relationships between entities in the domain.

 

Manufacturing Engineering Laboratory Skip navigation