Swarm robotics (SR) is a research field about how to design a large number of robots so that they can generate meaningful collective behaviors. One of the promising approaches in designing a control policy is reinforcement learning (RL). However, it is well known that the sparse reward problem may arise, especially in cases of solving highly complex problems. Curriculum learning (CL) can be one of the effective approaches to overcoming this difficulty. In this paper, we propose a novel method called Self-Teaching Automatic Curriculum Learning (STACL). The training progress of different lessons is compared by agents to determine which lesson should be trained in the next episode. The collective wall-jumping task, in which the robots have to generate collective wall-jumping behavior to jump over the high wall and reach the goal as soon as possible, is employed to illustrate the effects. Simulation results show that the proposed approach has the fastest convergence speed and the most stable performance. In addition, we also conducted experiments to examine the flexibility of the developed controllers.