A new approach for lightweight video-based object recognition is introduced where a user moves a camera around a target object of interest. The extraction of image features and the retrieval algorithm is running in a lightweight mobile computer (tablet or phone). We apply a view based model of the objects and the matching of the query and candidate images is based on compact image descriptors coupled with relative orientation.