A view of Castle Ilpenstein, Joris van der Haagen from the old master collection

History of Ilpenstein
Volckert Overlander, mayor and alderman of the city of Amsterdam, obtained the seigneury of Purmerland and Ilpendam in 1618. In 1622 he had the castle Ilpenstein built on an old entrenchment erected at the beginning of the Eighty Years' War. The towers and moat made it look like a fortress, but it was actually more of a mansion than a defendable castle. One of the lords of the castle is still known: Frans Banninck Cocq, lord of Purmerland and Ilpendam, who as captain of the Amsterdam militia was immortalised by Rembrandt in his Night Watch. After the death of Banning Cocq in 1655 (and that of his widow in 1678), the castle passed to the Amsterdam regent family De Graeff.

In 1662 at the wedding of Pieter de Graeff and Jacoba Bicker at Ilpenstein the poet Gerard Brandt and pensionary Johan de Witt were present,[1] Joost van den Vondel sang of this marriage with the poem Ter bruiloft van den weledelen heer Peter de Graef, Jongkheer van Zuitpolsbroek en de weledele mejoffer Jakoba Bikker,[2] while his colleague Jan Vos sang of this marriage with another verse, Huwelyk van den Eed. Lord Pieter de Graaf, Iongheer van Zuidt-Polsbroek, and Mejuffer Jakoba Bikker,[3] and his colleague Jan Vos sang this marriage with another verse, Huwelijkk van den Eed.

The last residents of Ilpenstein Castle were Christina Elisabeth de Graeff (†1872) and her husband Jacob Gerrit van Garderen (†1856). Some residents were buried in the crypts of the Reformed Church of Ilpen