It has been established that in 1326 Niels Jensen was the owner of Søgård and of a series of farms in the area. He was buried in the local church. Some 200 m (660 ft) south of today's manor, there is a defensive site consisting of a hill and a moat which was apparently the first Søgard. The Crown had extensive interests in the area as Tvevad Castle, documented by Valdemar the Great, lay just 2 km to the south. The medieval castle can still be seen in Tvevadskov (Tvevad Woods). In 1860, at the end of Vomme Sø, an octagonal burial chapel was built for the de Mylius's by Vilhelm Tvede.