Located c.500m away from Oystermouth Castle and 200m beyond the shoreline at mumbles, Silverwood castle is mentioned in two records dating to 1856 and 1899. A stone building, reputedly 'Silverwood Castle', had been laid bare after unusually strong tides according to Archaeology in Wales (which continues on to say that the structures observed were probably vestiges of oyster cultivation tanks or fish traps). (Arch in Wales 1999)

Sources :
Spurgeon, C.J., Roberts, D.J., Thomas, H.J. , 1999 , Supposed Castles in Glamorgan: A Review