Epaphras Hoyt was a son of David Hoyt and brother to Elihu, Persis Hoyt Sheldon, Mercy Hoyt Hitchcock, and Mary Hoyt Williams, and five other siblings. He grew up in the Old Indian House on Deerfield's Common in a house built in 1699 by John Sheldon. His father was a tavern keeper and a wigmaker. In 1815, when he was 45, he purchased the early-eighteenth-century house on Lot 18, which he extensively remodeled. He was postmaster and register of deeds with an office near the center of town. When the Brick Meeting House was built on that lot in the 1820s, Mr. Hoyt moved his office building to his homelot at the south end of town.
Arthur Wellesley Hoyt (1811-1899) Arthur W. Hoyt was the youngest child and only son of Epaphras Hoyt. He grew up on Lot 18 at the south end of Deerfield's main street. In 1857, when he was 46 years old, he bought Lot 29 Iii in the center of town, facing the Common. He took down a house and a shop and moved two others to make room for an elaborate Italianate house. Valued at $4000 it was the most expensive house on the street. He took out a $10,000 mortgage on the house in 1861 and sold it seven months after his wife died in 1863.