The introduction of small motion techniques such as small angle rotation approximation has enabled the three-dimensional reconstruction from a small motion of a camera, so-called structure from small motion (SfSM). In this letter, we propose a closed-form solution dedicated to the rotation estimation problem in SfSM. We show that our method works with a minimal set of two points, and has mild conditions to produce a unique optimal solution in practice. Also, we introduce a three-step SfSM pipeline with better convergence and faster speed compared to the state-of-the-art SfSM approaches. The key to this improvement is the separated estimation of the rotation with the proposed two-point method in order to handle the bas-relief ambiguity that affects the convergence of the bundle adjustment. We demonstrate the effectiveness of our two-point minimal solution and the three-step SfSM approach in synthetic and real-world experiments under the small motion regime.