Giant elliptical galaxies, believed to be built from the merger of lesser galaxies, are known to house a massive black hole (MBH) at their center rather than a compact star cluster. If low-and intermediate-mass galaxies do indeed partake in the hierarchical merger scenario, then one needs to explain why their dense nuclear star clusters are not preserved in merger events. A valuable clue may be the recent revelation that nuclear star clusters and MBHs frequently co-exist in intermediate-mass bulges and elliptical galaxies. In an effort to understand the physical mechanism responsible for the disappearance of nuclear star clusters, we have numerically investigated the evolution of merging star clusters with seed BHs. Using BHs that are 1%-5% of their host nuclear cluster mass, we reveal how their binary coalescence during a merger dynamically heats the newly wed star cluster, expanding it, significantly lowering its central stellar density, and thus making it susceptible to tidal destruction during galaxy merging. Moreover, this mechanism provides a pathway to explain the observed reduction in the nucleus-to-galaxy stellar mass ratio as one proceeds from dwarf to giant elliptical galaxies.