The name Hindsgavl is first mentioned in King Valdemar's Land Register from 1231, where it is believed that a medieval castle once stood. In 1295, the site is mentioned directly when King Erik Menved at Hindsgavl made a truce with the Norwegian king Erik Magnusson Præstehader.

Northwest of present-day Hindsgavl are the remains of the old castle close to the coast. The site consists of an approximately 15-meter-high, oval castle mound with a plateau of around 85 by 70 meters. Part of the western slope has been eroded by the Little Belt, while the other sides are surrounded by a five to eight meter wide, water-filled moat.

The surrounding meadow, which used to be swampy, provided natural protection against attacks. Access was via a causeway from the east leading to the castle’s farmyard on a low rise south of the mound. From there, a drawbridge and a path led up to the top of the mound, where the foundations of several medieval buildings can still be seen.