It was asked in [J. Math. Log. , Vol. 22, No. 01, (2022)] if equality on the reals is sharp as a lower bound for the complexity of topological isomorphism between oligomorphic groups.We prove that under the assumption of weak elimination of imaginaries this is indeed the case. Our methods are model theoretic and they also have applications on the classical problem of reconstruction of isomorphisms of permutation groups from (topological) isomorphisms of automorphisms groups. As a concrete application, we give an explicit description of Aut(GL(V)) for any vector space of dimension $\aleph_0$ over a finite field, in affinity with the classical description for finite dimensional spaces due to Schreier and van der Waerden.