Abstract:
An image of a rack in a supermarket displays a number of retail products. The identification and localiza-
tion of these individual products from the images of racks is a challenge for any machine vision system.
In this thesis, we aim to address this problem and suggest a set of computer vision based solutions for
automatic identification of these retail products. We design a novel classifier that differentiates the sim-
ilarly looking yet non-identical (fine-grained) products for improving the performance of our machine
vision system. The proposed fine-grained classifier simultaneously captures both object-level and part-
level (image of an object consists of multiple parts or image patches) cues of the products for accurately
distinguishing the fine-grained products. A graph-based non-maximal suppression strategy is proposed
that selects a winner region proposal among a group of proposals representing a product. This solves an
important bottleneck of conventional greedy non-maximal suppression algorithm for disambiguation of
overlapping region proposals generated in an intermediate step of our proposed system. We initiate the
solution of the problem of automatic product identification by developing an end-to-end annotation-free
machine vision system for recognition and localization of products on the rack. The proposed system in-
troduces a novel exemplar-driven region proposal strategy that overcomes the shortcomings of traditional
exemplar-independent region proposal schemes like selective window search. Finally, we find the empty
spaces (or gaps between products) in each shelf of any rack by creating a graph of superpixels for the
rack. We extract the visual features of superpixels from our graph convolutional and Siamese networks.
Subsequently, we send the graph along with the features of superpixels to a structural support vector
machine for discovering the empty spaces of the shelves. The efficacy of the proposed approaches are
established through various experiments on our In-house dataset and three publicly available benchmark
datasets: Grozi-120 [Merler et al., IEEE CVPR 2007, 1-8], Grocery Products [George et al., Springer
ECCV 2014, 440-455], and WebMarket [Zhang et al., Springer ACCV 2007, 800-810].
