I'd suspect the thermostat is stuck open, but could also be a mixing door problem.
The thermostat in an engine opens, when the temperature exceeds a certain temperature. If the temperature drops beneath that, it will start to close, to prevent the water pump from circulating the coolant out to the radiator to cool down. This keeps your engine operating at optimal temperature, and hot water in the engine is available to the heater core.
If the thermostat is stuck open, it will continually cool the engine and can drop the temperature below where it should run, and you would get cool air flowing into the cabin.
It might also be a mixing door problem however. You should check to see if you can hear the door opening/closing when this problem occurs. If you don't hear it moving, then probabably thermostat.