early 13c. "millstone," from grind (v.) in sense of "sharpen" + stone (n.); meaning "revolving stone disc used for sharpening, etc." is from c.1400. Phrase nose to the grindstone in use by 1530s; originally to get control of another and treat him harshly:
The main modern (reflective) sense of "work hard" is from 1828.