Development of multimodal collaborative robot system using hybrid programming methods

The development of collaborative robots (cobots) has enabled automation of several processes such as assembly, logistics, packaging etc. which were historically impossible to automate. To enable their use for these tasks, the requirements for performing a given task using a cobot system are looked into. Based on the study of state of the art methods, three hybrid programming approaches „instructive programming“, „programming by demonstration“ and „end-to-end learning“ are identified. Based on these approaches, three methods are developed in this thesis, each of which involves integrating vision systems into cobots. The methods were implemented in three different scenarios and compared on the basis of the identified requirements for their use in manufacturing environments. The first method „Instructive programming for autonomous task performance“ was implemented into a demonstrator capable of performing assembly tasks of fixing and screwing. The second method „Programming by demonstration through hand-eye coordination“ was implemented for a task of part handling where the cobot was programmed using a combination of a 2D camera and human demonstrations. Finally the third method „Task performance through deep reinforcement learning“ was implemented in a simulation environment for performing simple assembly tasks: peg in hole and ring in peg. It was concluded that the second method fulfilled most of the requirements of movable cobots as the method combined the human capabilities of adapting to new scenarios with the cobot capabilities of performing tasks with high accuracy and repeatability.