Topological singular set of vector-valued maps, I: applications to manifold-constrained Sobolev and BV spaces