We study the dynamic stability of low Reynolds number swimming near a plane wall from a control-theoretic viewpoint. We consider a special class of swimmers having a constant shape, focus on steady motion parallel to the wall, and derive conditions under which it is passively stable without sensing or feedback. We study the geometric structure of the swimming equation and highlight the relation between stability and reversing symmetry of the dynamical system. Finally, our numerical simulations reveal the existence of stable periodic motion. The results have implications for design of miniature robotic swimmers, as well as for explaining the attraction of micro-organisms to surfaces.