I also recommend the Rocks (markets are nice, when they are on), and you can walk across the Harbour Bridge from that area. Taronga Zoo which is world class, Opera House and surrounding area for the view, and the Botanic Gardens, which are wonderful for running around on real grass and picnicking on the grass.
Darling Harbour is ok too though very touristy, and has a couple of good playgrounds. If it's unpleasant weather look into all the museums to see if they have any kids' exhibitions on. The Powerhouse Museum could be a good one to start with:
http://play.powerhousemuseum.com/
Have fun.