In the context of classical mechanics, we study the conditions under which higher-order derivative theories can evade the so-called Ostrogradsky instability. More precisely, we consider general Lagrangians with second order time derivatives, of the form L(̈phia, dot phia, phia, q(i), q(i)) with a = 1,⋯,n and i = 1,⋯,m. For n = 1, assuming that the qi's form a nondegenerate subsystem, we confirm that the degeneracy of the kinetic matrix eliminates the Ostrogradsky instability. The degeneracy implies, in the Hamiltonian formulation of the theory, the existence of a primary constraint, which generates a secondary constraint, thus eliminating the Ostrogradsky ghost. For n > 1, we show that, in addition to the degeneracy of the kinetic matrix, one needs to impose extra conditions to ensure the presence of a sufficient number of secondary constraints that can eliminate all the Ostrogradsky ghosts. When these conditions that ensure the disappearance of the Ostrogradsky instability are satisfied, we show that the Euler-Lagrange equations, which involve a priori higher order derivatives, can be reduced to a second order system.