The oldest account was from Lucy Mack Smith, who remembered Joseph referring to Cumorah in early 1827 when he was late coming home from Manchester.
Currently in the United States there is a controversy about federal officers in cities such as Portland, Oregon, who are protecting federal property. This is reminiscent of the challenges faced by Stephen S. Harding, the fourth territorial Governor of Utah.
In 1862, U.S. President Abraham Lincoln appointed Harding as Governor of the Utah Territory. Harding had conflicts with Brigham Young and was unable to enforce federal law. In 1863, President Lincoln appointed Harding as Chief Justice of the Colorado Territory's Supreme Court.
Harding was born in Palmyra, NY, on February 24, 1808. His family moved to Indiana in 1820, but he remembered Joseph Smith, Jr., from seeing him fishing often.
In 1829, Harding returned to Palmyra to visit friends, including Pomeroy Tucker. While visiting the printing shop, he met Martin Harris, Oliver Cowdery, Joseph Smith, and Joseph's father. When he expressed interest in the Book of Mormon that was being prepared for printing, he was invited to spend the night at the Smith home. Oliver Cowdery read from the manuscript.
Harding left to visit his brother, but two weeks later he ran in to Martin Harris again in Palmyra. Here's the account Harding wrote in a published letter.
"About two weeks after this I met Martin Harris. He was glad to see me; inquired how I felt since my dream. He told me that since he saw me at Mr. Smith's he had seen fearful signs in the heavens. That he was standing alone one night, and saw a fiery sword let down out of heaven, and pointing to the east, west, north, and south, then to the hill of Cumorah, where the plates of Nephi were found."
Harding wrote this letter in 1882, but he relates several details that attest to his solid memory. Of course, this is not proof that the hill in New York is the same as the Cumorah of Mormon 6:6, but it does show that at least Martin Harris believed it was, even before the Book of Mormon was printed.
That Harris would refer to Cumorah this way is another corroboration of other accounts that Joseph Smith's contemporaries understood that the hill where Joseph found the plates was the Cumorah of the Book of Mormon.