mages can have pets.
but mages cant tame pets.
you can have: non combatant LIVING pets; parriots from booty bay, snakes from valley of spirit or can buy special pets off from auction house with $$ depending on ur server.
or you can have combatant pets: mechanical dragonkin, combat chicken, mechanical yeti and so on UNLESS you are enginneer. while some can get gained from doing quest.
or u can have non-combatant MECHANICAL pets, life-like mech toad, lil' smoky, pet bombling, again, only if you are enginneer.
or like above said, you can have a water elemental to fight for you once you have reached required level.
horse? yes you can have horse, but not as pets, but as ride, unless you on alliance side. or undead horse if your horde side.
hope it helps!