1560s, from chesten nut (1510s), with superfluous nut (n.) + Middle English chasteine, from Old French chastain (12c., Modern French châtaigne), from Latin castanea "chestnut, chestnut tree," from Greek kastaneia, which the Greeks thought meant either "nut from Castanea" in Pontus, or "nut from Castana" in Thessaly, but probably both places are named for the trees, not the other way around, and the word is borrowed from a language of Asia Minor (cf. Armenian kask "chestnut," kaskeni "chestnut tree"). In reference to the dark reddish-brown color, 1650s. Applied to the horse-chestnut by 1832.
Slang sense of "venerable joke or story" is from 1885, explained 1888 by Joseph Jefferson (see "Lippincott's Monthly Magazine," January 1888) as probably abstracted from the 1816 melodrama "The Broken Sword" by William Dimond where an oft-repeated story involving a chestnut tree figures in an exchange between the characters "Captain Zavior" and "Pablo":
Jefferson traced the connection through William Warren, "the veteran comedian of Boston" who often played Pablo in the melodrama.