But they didn’t leave! They took over. Back to top How the Anglo-Saxons took over Pevensey Castle One Roman fort in Sussex is Pevensey Castle. After the Romans left, the Britons used the fort at ...