"small piece," c.1200; related Old English bite "act of biting," and bita "piece bitten off," probably are the source of the modern words meaning "boring-piece of a drill" (1590s), "mouthpiece of a horse's bridle" (mid-14c.), and "a piece bitten off, morsel" (c.1000). All from Proto-Germanic *biton (cf. Old Saxon biti, Old Norse bit, Old Frisian bite, Middle Dutch bete, Old High German bizzo "biting," German Bissen "a bite, morsel"), from PIE root *bheid- "to split" (see fissure).
Meaning "small piece, fragment" is from c.1600. Sense of "short space of time" is 1650s. Theatrical bit part is from 1909. Money sense in two bits, etc. is originally from Southern U.S. and West Indies, in reference to silver wedges cut or stamped from Spanish dollars (later Mexican reals); transferred to "eighth of a dollar."