As an adult, he revisited his home town of Shirley, Maine, where he had lived for less than three years. In his typical humorous fashion, he wrote about visiting the home where he was born:
A man ought not to criticise his birthplace, I presume, and yet, if I were to do it all over again, I do not know whether I would select that particular spot or not. Sometimes I think I would not. And yet, what memories cluster about that old house! There was the place where I first met my parents. It was at that time that an acquaintance sprang up which has ripened in later years into mutual respect and esteem.
Nye describes observing the "bric-a-brac" in the yard of that home and helps himself to a small stone as a memento. He hopes his theft goes unnoticed: "There was another stone in the yard, so it may be weeks before any one finds out that I took one of them."
He notes that here, "amid the barren and inhospitable waste of rocks and cold," is a place a "great man" would never select to be born. He proudly notes that he has by his own efforts risen from such humble beginnings to a lofty rank as a great man (tongue in cheek, of course).
Still, my birthplace is all right as a birthplace. It was a good quiet place in which to be born. All the old neighbors said that Shirley was a very quiet place up to the time I was born there, and when I took my parents by the hand and gently led them away in the spring of '53, saying, 'Parents, this is no place for us,' it again became quiet.
It is the only birthplace that I have, however, and I hope that all the readers of this sketch will feel perfectly free to go there any time and visit it and carry their dinner as I did. Extravagant cordiality and overflowing hospitality have always kept my birthplace back.