Old English gat "she-goat," from Proto-Germanic *gaitaz (cf. Old Saxon get, Old Norse geit, Danish gjed, Middle Dutch gheet, Dutch geit, Old High German geiz, German Geiß, Gothic gaits "goat"), from PIE *ghaidos "young goat," also "play" (cf. Latin hædus "kid").
The word for "male goat" in Old English was bucca (see buck (n.)) until late 1300s shift to he-goat, she-goat (Nanny goat is 18c., billy goat 19c.). Meaning "licentious man" is attested from 1670s. To get (someone's) goat is from 1910, perhaps with notion of "to steal a goat mascot from a racehorse," or from French prendre sa chèvre "take one's source of milk."
