For starter's, bring/buy an umbrella. And light gloves... since you can't hide your hands in your pocket and hold an umbrella and coffee at the same time. Maybe you'll buy a toque while you're at it.
Taking a stroll around the seawall (Stanley Park) is kind of a nice introduction to the city whatever the weather. But I'd walk out the park and up Robson if you're looking for a bite, as the selection and prices are better.
If you're looking for a heavy meal, the Chicken and Waffles at Tuc Craft Kitchen in Gastown is bbbphhhhhhhhhh. Too good. Also it gets you into Gastown which is kind of an interesting area to explore. I recommend hitting it during the daytime, as it can be a little sketchy later.
If you need a liquor store, Legacy Liquor on Manitoba st is decidely huge and their selection is terrific. However if you're looking to simply sit down and drink then the CRAFT Beer Market on 1st ave is not far away. It's a unique repurposed barn-looking place, but it's large and they have 100+ beers/ciders on tap. It's quite impressive. Of course there's lots of microbrews around too if you aren't looking for food to go with your drinks.
From there you could stroll through the Olympic Village, and eventually get to Granville Island. Granville Island is your typical public market-cum-tourist trap, with lots of everything (except parking). Street performances, shops, etc. Probably worth a wander if you've never been there and it looks interesting. If you do make it there, and you've yet to eat a poutine, I highly recommend stopping at Edible Canada (during brunch) and getting the Mt. Lehman Duck Poutine. You're not going to come to Canada and not eat a poutine are you?
If you keep south of False Creek and continue walking west you'll get to kits beach. It's all nice walking. Then you'll be close to Rain or Shine ice cream. I like the London Fog ice cream but you do you.
If you're looking for a bit of exercise with a rewarding view that doesn't eat a whole day's time, try the Grouse Grind. You'll get a good view of things. Just call ahead before you go. I can't remember when they shut the trail down for the season. You'll be in the neighbourhood of the Capilino Suspension bridge but personally I'd pass on it. It's just a walking bridge over a gorge... and it ain't free either. But it's kind of a touristy thing so you may hear about it.
As for Victoria, are you flying to the island or taking the Ferry via bus/car?