Stability

Rockets can be stabilized in a number of ways:

LENGTH

As we discussed in the Early Rockets section, the stability of a rocket can be controlled by increasing its length.  In olden times, they used rocket sticks to make the rocket longer and more accurate to launch at enemies.

The same rule applies in normal rocketry.

The length is usually 10 to 20 times the rocket's diameter.
Of course, the longer the rocket is, the heavier it becomes and so more thrust is needed.

FINS

Fins ensure stability and keep the rocket moving in a straight line.

They increase drag, but the stability makes an improvement in the rocket's flight.

The Saturn V had fins at its lower end to improve stability of flight.

Newer rockets such as the Atlas, Delta or Titan rockets do not require fins as their rocket engines can move and thrust in the correct direction to stabilize the rocket.  This is known as thrust vectoring.

Water bottle rockets need fins because the rocket motor cannot be thrust vectored.

The size of the fins also matter.
They should be 2x the diameter lengthwise and 1x the diameter wide.

Three fins are the optimal number but four fins will also work.

Atlas V rockets like most modern rockets, do not require fins as the rocket engines can be turned to face in different directions to keep the rocket stable.

Shuttle launch showing the liquid-fuel rocket engines moving at launch stabilizing the Shuttle