Developing Deep Learning-Based Real-time Museum’s Object Detection and Recognition Model

Abstract

The real-time based object recognition is very important and challenging tasks in computer vision, as there is an increasing interest in building an intelligent system. A smart tour guide system for the museum enables an exhibition more attractive by improving the interaction between the visitors and the interest of objects by means of a guide. The National Museum of Ethiopia is still present its exhibits a rather passive and non-engaging way. The prior works were used a NVIDIA shielded Tablet that has NVIDIA GPU to run the developed the context-based object recognizer that could create an interactive museum exhibition. The proposed model, mobile-based smart tour guide, can understand the context of the visit and visitor behavior and thereby filtering out which the visitor is interested in. A YOLOv3-tiny model was trained on the Darknet framework and the .weightsfile was converted to an intermediate weight (TensorFlow) called .pb file. And then, .pb file was converted to its TensorFlow Lite equivalent, .tflite file which enables on-device object detection with low latency and small binary size. The context of the visits and visitor’s behavior is analyzed by applying video processing pipelines on an online (real-time) video data and thereby the recognizer is stabled and hence it recognizes an interested object. First, the problem of multiple detections of the same object is cured by Non maximum Suppression (NMS). Second, those objects far from a visitor are ignored from being recognized. Third, the prediction of object persists for a C number of frames only is taken as interested object and is done by greedy data association tracking-by-detection techniques. Finally, the interested object id is filtered out and the object within that id is displayed on the mobile screen for further description about it to the visitor. So, the developed deep learning model able to detect objects with better accuracy by 97.47% and 69% on object detection and recognition respectively which averages an overall accuracy of 83.24%