SIMCO: SIMilarity-based object COunting

We present SIMCO, the first agnostic multi-class object counting approach.SIMCO starts by detecting foreground objects through a novel Mask RCNN-basedarchitecture trained beforehand (just once) on a brand-new synthetic 2D shapedataset, InShape; the idea is to highlight every object resembling a primitive2D shape (circle, square, rectangle, etc.). Each object detected is describedby a low-dimensional embedding, obtained from a novel similarity-based headbranch; this latter implements a triplet loss, encouraging similar objects(same 2D shape + color and scale) to map close. Subsequently, SIMCO uses thisembedding for clustering, so that different types of objects can emerge and becounted, making SIMCO the very first multi-class unsupervised counter.Experiments show that SIMCO provides state-of-the-art scores on countingbenchmarks and that it can also help in many challenging image understandingtasks.