early 15c., "servants of a household," from Latin familia "family servants, domestics collectively, the servants in a household," thus also "members of a household, the estate, property; the household, including relatives and servants," from famulus "servant," of unknown origin. The Latin word rarely appears in the sense "parents with their children," for which domus (see domestic) was used.
In English, sense of "collective body of persons who form one household under one head and one domestic government, including parents, children, and servants, and as sometimes used even lodgers or boarders" [Century Dictionary] is from 1540s. From 1660s as "parents with their children, whether they dwell together or not," also in a more general sense, "persons closely related by blood, including aunts, uncles, cousins;" and in the most general sense "those who descend from a common progenitor" (1580s). Meaning "those claiming descent from a common ancestor, a house, a lineage" is early 15c. Hence, "any group of things classed as kindred based on common distinguishing characteristics" (1620s); as a scientific classification, between genus and order, from 1753.
Replaced Old English hiwscipe. As an adjective from c.1600; with the meaning "suitable for a family," by 1807. Family values first recorded 1966. Phrase in a family way "pregnant" is from 1796. Family circle is 1809; family man "man devoted to wife and children, man inclined to lead a domestic life" is 1856 (earlier it meant "thief," 1788, from family in a slang sense of "the fraternity of thieves").
The phrase is attested from 1844.