OK, might be a stupid question, but do you have the "Auto" button activated? If the Auto button is activated (the little green light should be lit, but yours could be burned out) AND the fan control is set to Auto, the climate control will try to keep the cabin at whatever temperature the slide is set to, and it will turn the blower fan on and off to achieve this, as well as kick the compressor on/off as necessary.
Usually when you activate the Auto button the A/C button will light up with it, but it is possible to press the A/C button to turn the A/C off, but still leave the Auto button active.
"The major difference between a thing that might go wrong and a thing that cannot possibly go wrong is that when a thing that cannot possibly go wrong goes wrong, it usually turns out to be impossible to get at or repair."
-Douglas Adams, Mostly Harmless