Although it is very cold here I was skiing in a long sleeved top and a jacket with a zip-in fleece this week and was absolutely fine - I'd suggest getting a "buff" or neck warmer so you can pull it up over your face when you're on the lifts. A couple of extra layers shouldn't take up much room in your case so better to have them with you and be prepared.