I then will need to be able to set up a powered fan setup to pull air from outside in the winter. That would give me my source of cool air for those months and I would just close the HVAC vents and put small covers over them to seal them better.
Sounds like a great idea. Learn from my mistakes and make sure you have a *very* good screen to keep rodents & bugs out though...