What makes the Moon orbit the Earth?

Years ago I bought Richard Feynmann's book Six Easy Pieces. It's only short, but contains fairly dense introductions to fundamental matters of physics. I'm enjoying the chapter on gravitation, because it goes into some detail on simple things I hadn't thought too deeply about.

For instance, we know the Moon is kept in orbit by the Earth's gravity, but sometimes I've casually wondered: if the moon is attracted to the Earth, why does it stay a constant distance away? Why isn't it visibly pulled towards the Earth?

Feynmann offers an enjoyable thought experiment concerning the firing of a bullet, which I'll try to relate. In it, we assume a steady Earth and no air resistance, so a bullet, once fired, maintains a steady path and doesn't slow down.

If you drop a bullet, it will fall 5m in one second (allowing 10ms^-2 for gravity). If you fire a bullet at, say, 1000ms^-1, then in one second it will travel 1000m horizontally but of course it will still fall 5m vertically. At some point it will hit the ground.

But the Earth is not flat, it is round. Without gravity, the path of a bullet would be flat. If you fired it horizontally, it would gradually get further from the ground and eventually fly off into space. The round Earth would curve away beneath it. With gravity, of course, the bullet "falls". So the straight path causes it to get further away from the Earth, but gravity causes it to get closer to the Earth. What if these two phenomena cancelled each other out? What if you fired the bullet so fast that in one second it gained 5m in height by virtue of its flat path, but also fell 5m in height because of gravity? In that case, after one second in the air, it would still be at the same height at which it started. During the next second, the same thing would happen: it would travel 1000m, but its lift would be cancelled by its fall. The bullet would, in effect, be in orbit.

That's how the Moon orbits the Earth. In a sense, the Moon wants to travel in a straight line at a high speed. That straight line would cause it to get further away from the Earth, but the Earth's gravity causes it to "fall" back into place. Thankfully, it's obviously going at just the right speed to stay in orbit. Too fast, and we wouldn't have tides. Too slow, and we'd all be dead.