This event is described at length by a section of the Brutiau derived from an earlier chronicle written at Llanbadarn Fawr covering the period 1100-26 (Stephenson 187-8). Gruffudd ap Rhys the prince of Deheubarth came to Aberystywth and made camp at:

Ystrat Antarron, a oed gyfarwyneb a'r castel. A'r castell oed ossodedic ar benn mynyd a oed yn llithraw hyt afon Ystwyuth. Ac ar yr afon yd oed pont (Jones, 94).

Translation: Ystrad Antaron, which was opposite the castle. And the castle was situated on the top of a hill which sloped down to the river Ystwyth. And over the river there was a bridge (Jones, 95).

Then `the keepers of the castle sent archers to the bridge to skirmish with them? (Jones, 94-5). This is where the main conflict took place and although the Welsh headed towards the castle, they were forced back before reaching it and defeated (Jones, 94-5).

Ystrat Antarron likely refers to the small valley running down to the River Ystwyth, from the house called Antaron, and although the name is quite likely a later antiquarian identification, it does fit the evidence (Wmffre, 974). The Cottonian Chronicle reference `battle took place at Aberystwyth castle?(Gough-Cooper, c437.2), suggests that the castle (SN 5850 7899) itself was attacked but the section of the Brutiau quoted above originates earlier and clearly shows that the castle itself was did not suffer any direct attack. Although no remains of a bridge over the Ystwyth are known the conflict likely took place on the river below the castle.

RCAHMW (Battlefields Inventory), Dec 2016

Bibliography
Gough-Cooper, Henry (ed.) The Cottonian Chronicle: Annales Cambriae, The C Text from London, British Library, Cotton MS Domitian A. i, ff. 138r?155r, online edition.
Jones, Thomas (ed. and trans), Brut y Tywysogyon or the Chronicle of the Princes: Red Book of Hergest Version (Cardiff, University of Wales Press, 1955).
Stephenson, David, `The `Resurgence? of Powys in the Late Eleventh and Early Twelfth Centuries?, in Lewis, C. P., (ed), Anglo-Norman Studies XXX. Proceedings of the Battle Conference 2007 (The Boydell Press, 2008) 182-195.