A differential group is a differential space which is also a group and in which the group operations are smooth. The paper begins with a presentation of the basic notions in this topic. Next, the authors investigate the tangent space and the Lie algebra of all left invariant vector fields on a differential group. In the last section it is shown that for any differential group there exists a unique covariant derivative with respect to which all left invariant vector fields are parallel.