Oberth effect
If you are in a non circular orbit around a body, (including parabolic and hyperbolic orbits) you get more velocity change by burning fuel when close to the body, then when further away. This seems counter intuitive, but is true, and it is called the Oberth effect after the scientist who discovered it.
This article will give 3 explanations to explain this effect.
Contents
Reverse gravity-drag
Gravity-drag is when you are launching a rocket in a gravity well. Not only do you have to lift the rocket & payload, you have to lift the fuel which will be used higher up in the launch. You have fuel low in a gravity field, and are burning it higher. Effectively this acts as an energy 'tax' when you are lifting fuel higher in a gravity field.
Look at it the other way, let us say that you have fuel high in a gravity well, and burn it when you are lower. That fuel has potential energy, which turns into kinetic energy as you fall closer to the central body. When it is burnt lower, you have already converted some of that potential energy into kinetic energy, so the burn is more efficient. This is the reverse gravity-drag 'tax refund'.
Proportion of KE used to accelerate exhaust
Another way of looking at this, is what percentage of the energy of burning the fuel / oxidizer (just called 'fuel' for the rest of this article), is used to accelerate the ship and what percentage is used to accelerate the exhaust gas?
Imagine a space ship in deep space which has enough fuel to make a velocity change of 1,000 m/s. We will assume the ship has the same mass as the exhaust gas. When you burn this fuel, the ship gains 1,000 m/s of velocity, but the exhaust gas ALSO gains 1,000 m/s of velocity. So 50% of the energy of burning the fuel goes to the ship and 50% goes to the exhaust.
Now imagine this ship in an elliptical orbit so it will be travelling 1,000 m/s when it is close to the planet. If you burn that fuel, when it is close to the planet, the exhaust gases are at rest with respect to the planet. The ship was moving 1,000 m/s before the burn, and after the burn it is travelling 2,000 m/s with respect to the planet.
There is no change when looking at this burn when looking at it with a reference frame travelling with the ship / fuel system. But when looking at the burn from the reference frame of the planet, more velocity is gained by burning the fuel lower in the gravity well. With respect to the planet's reference frame, 100% of the energy is used to accelerate the ship, and 0% accelerates the exhaust gases.
Since the reference frame we care about is the planet's (we want the ship to move away from the planet), the Oberth effect gives us a bonus.
Math of kinetic energy and momentum
Energy can be stored in several forms. There is chemical energy, potential energy (the energy of being high in a gravity field) and Kinetic energy (the energy of motion). The energy of a system does not change, but we can change the form of this energy takes.
Kinetic Energy (KE) is the energy of motion. Its value is calculated with this formula: KE = 1/2 m * v^2. (Where m = mass, and v = velocity.)
Let us say you fire a rocket with mass 1 (after the fuel has been burnt), which will give your space craft 1 m/s velocity change.
Let us fire this rocket when you are high in an orbit moving at 1 m/s (with respect to the planet). You get a 1 m/s change so the new velocity is 2 m/s. What is the KE of the ship when you fire its engines high in the gravity well?
KE = 1/2 * 1 * 2^2 = 1/2 * 1 * 4 = 2 energy.
Now let us fire the rocket when it has fallen much closer to the planet. When we fire the engines, we are moving 10 meters / second (with respect to the planet). After adding 1 m/s onto the current speed of the ship, you are moving 11 m/s.
KE = 1/2 * 1 * 11^2 = 1/2 * 1 * 121 = 60.5 energy.
Now this is an extreme case where the ships is moving 10 times faster near the planet than far from it, but it illustrates the point. The energy of motion of the this rocket is much higher when the fuel is burnt after you have fallen from a point of high potential energy to a point of lower potential energy. You are taking the potential energy of the fuel high in the gravity well, and converting it more efficiently to kinetic energy.
Summary
When a ship in an non-circular orbit burns fuel as close as possible to the body it is orbiting, it converts the potential energy held in the fuel, into additional kinetic energy (the energy of motion). Thus it will gain more speed from a burn when the burn is close to the primary.
For a more mathematical analysis see: [1]