The Knights Templars owned Swanton Manor in Harrold in the 13th century. They claimed in 1278-79 to hold it of John de Grey of the gift of a certain “Flandina Maudyt” in the reign of Henry III. This refers to a grant made in 1240 by Ralph de Carun, his wife, and her heirs, of one-sixth of a mill and 151 acres of land in Harrold to Robert de Sanford, Master of the Templars, which grant was ratified in 1242.
In 1244 the Master made complaint that Ralph Morin had trespassed on his common pasture, and Ralph recognized his right to common pasture in Swanton for 460 sheep (of which pasture for 80 was of the gift of the said Ralph), 35 cattle and 40 pigs. In 1253 the Knights Templars obtained a grant of free warren in their possessed lands of Swanton and Harrold.
Major landowners in Harrold and holders of the Manorial Rights, were the de Greys. The fourth lord, Edward, changed sides and betrayed the Lancastrian King Henry VI at Northampton in 1460 and was rewarded by Edward IV, becoming Treasurer of England from 1463-64 and Earl of Kent in 1465.
It was fortunate that when the Lancastrian Henry VII triumphed in the wars of the Roses, he consolidated the warring factions by marrying Edward’s daughter, Elizabeth, and the second Earl redeemed himself by playing his part in supporting Henry VII against an uprising in Cornwall. The third Earl, however, was a gambler. He alleged that the King’s advisers tricked him out of substantial estates. Nevertheless, he continued to play a prominent role under Henry VIII, but his attempts to rehabilitate his fortunes merely made matters worse. When he died, his impoverished son merely styled himself Sir Henry Grey.