We introduce a new object retrieval approach where besides cameras, Inertial Measurement Unit (IMU) sensors are used for the retrieval of 3D objects. Contrary to computationally intensive deep learning recognition and retrieval solutions we focus on lightweight methods which could be utilized in handheld devices and autonomous systems equipped with moderate computing power and memory. We use fast and robust compact image descriptors and the relative orientation of the camera to build multi-view centered retrieval object models. As for retrieval the Hough transformation paradigm is used to evaluate the results of queries applied on several frames of a video. We analyze the performance of our lightweight approach on several test datasets and with different comparisons, including automatic tracking for the generation of queries. These experiments show the advantages of our proposed techniques since retrieval rate could be significantly increased.