The dumbbell satellite is a simple structure consisting of two point masses connected by a massless rod. We assume that it moves around the planet whose gravity field is approximated by the field of the attracting center. The distance between the point masses is assumed to be much smaller than the distance between the satellite's center of mass and the attracting center, so that we can neglect the influence of the attitude dynamics on the motion of the center of mass and treat it as an unperturbed Keplerian one. Our aim is to study the satellite's attitude dynamics. When the center of mass moves on a circular orbit, one can find a stable relative equilibrium in which the satellite is permanently elongated along the line joining the center of mass with the attracting center (the so called local vertical). In case of elliptic orbits, there are no stable equilibrium positions even for small values of the eccentricity. However, planar periodic motions are determined, where the satellite oscillates around the local vertical in such a way that the point masses do not leave the orbital plane. We prove analytically that these planar periodic motions are unstable with respect to out-of-plane perturbations (a result known from numerical investigations cf. Beletsky and Levin Adv Astronaut Sci 83, 1993). We provide also both analytical and numerical evidences of the existence of stable spatial periodic motions.