The mansion was built by Pierre Surget (1731-1796), a French planter, in 1788, over 2,500 acres of Spanish land grant. As such, it is one of the earliest private residence in Natchez. After his death, his widow Catherine (d'Hubert) Surget expanded the grounds of the property. By 1850, the house belonged to their son Jacob Surget, though their other son James Surget was the one living there and had sixteen house servants in residence. It remains in the Surget family.
In September 1861, a group of planters rounded up slaves after hearing rumors they schemed to “kill their masters”, and “ravish”, “ride” and “take the ladies for wives”. Ten were hanged at Cherry Grove Plantation.
It has been listed on the National Register of Historic Places since March 31, 1983.