The name Hanbury Manor is a modern one. The first house on this site, called Poles, was built around 1730 and purchased by Sampson Hanbury in the 1820's. In 1890, the house was demolished and replaced by a new Jacobean-style house that still stands today.