Yarmouth has been a settlement for over a thousand years, and is one of the very earliest on the Isle of Wight. The first record of a settlement here was in King Ethelred the Unready's record of the Danegeld tax of 991. It was originally called Eremue, meaning "muddy estuary". The Normans laid out the streets of Yarmouth on the grid system, a plan which can still be seen in the layout today. It grew rapidly, being given its first Charter as a town in 1135.
Until the building of the Castle regular raids on the Island by the French continued and in 1544 the town of Yarmouth was reputed to have been burned down. Legend has it that the church bells were carried off to Cherbourg or Boulogne.