Considering the Center of Gravity
Every object has a center of gravity (CoG). It is the location on the object where gravity pulling on any one side is balanced by gravity pulling on the opposite side. Another way to think of it is the average location of the weight of an object. If the CoG stays above the base of the object (the part resting on the ground), the object will not fall over.
The picture shows the famous Leaning Tower of Pisa. This tower is known for having a very visible tilt, about 4 degrees from being perfectly vertical (straight up and down). You can compare the arrow in the picture to the columns on the tower to see how tilted it is. Although the tower is so tilted, the CoG is still above the base of the tower and so, the tower stays standing! If you further pushed the Leaning Tower in the above picture so it tilted more and more until the arrow passed the base of the tower, then it would fall over.
You can also think about a seesaw or a scale. The fulcrum, the point on the seesaw where the board rests, is directly under the seesaw's CoG. If it weren't, the seesaw would fall to one side instead of teetering up and down.