The earliest known record of Castlecary comes from 1304 when a writ was sent from St Andrews to the sheriff of Stirling by King Edward I. It contained orders for the sheriff to bring all of the forces under his command to 'Chastel Kary' where they were to obey the orders of Sir Thomas de Morham, the eldest son of the laird of Dunipace, and Alwyn de Kalentir, laird of Callendar. There is place-name evidence for the existence of a motte and baillie there which would have been, in all likelihood, the fortalice of these lands at that time. The lordship of the estate lay in the hands of Lothian families from the fourteenth century but from the middle of the fifteenth we find a branch of the Livingstons resident there. First mention of these comes from 1450 when Robert, son of Henry Livingston of Manerstoun (near Blackness) was described as 'the late'. This record mentions that Robert had suffered forfeiture of the lands of Castlecary and Walton for 'high treason and the crime of les majestie'. Nevertheless, it is evident that a Livingston connection was re-established as Henry Livingston of Middilbyning (?Binns) had a dispute with several people regarding the lands in 1473. When Henry and his wife granted Castlecary and Walton to their son Patrick in 1491 the confirmation charter from the Crown contains the phrase: 'and also with a tower and fortalice and mansion to be newly constructed by the said Henry in the lands of Weltoun (Walton). This is the present castle and Patrick is the first encountered to be overtly titled 'of Castlecary'. Archibald Levyngstoun of Castlecary, on record in 1530 must have been the son or grandson of Patrick. Sometime soon after he invested Archibald Fawup as chaplain of the Chapel of the Blessed Virgin Mary in Linlithgow, he was 'slaughtered' by John Kerr who, subsequently, was beheaded for the crime. The family remained as the lairds there for several generations; the last to be designed 'of Castlecary' was Harry Livingston who died sometime in or around 1657. He was survived by his son John but he doesn't appear to have retained the lands. They came into the possession of the Baillie family; Patrick Baillie of Shirhill was there in 1685. This family remained until the early eighteenth century when the castle and lands were acquired by James Dundas when he married Bethia Baillie. James was then designed 'of Castlecary'. He belonged to the family of Dundas who held the estate of Kerse and were to become the Earls of Zetland.
John Reid 2005