As described in the Prestige, "Every great magic trick consists of three acts. The first act is called 'The Pledge'; The magician shows you something ordinary, but of course... it probably isn't. The second act is called 'The Turn'; The magician makes his ordinary some thing do something extraordinary. Now if you're looking for the secret... you won't find it, that's why there's a third act called, 'The Prestige'; this is the part with the twists and turns, where lives hang in the balance, and you see something shocking you've never seen before." Each magician works from this basic system.
I think Magician rivalries are intrinsic to their profession for a few reasons:
1) They compete for the same audience and revenue (i.e their livelihood)
2) They are performers and love the applause...when someone is getting the applause they think they should be getting, there is envy and jealousy.
3) Magicians/Illusionists all pull from the same bag'o'tricks....they add their own spin on the trick, but essentially the priciples are the same. So the only variation is the showmanship and presentation of each magician.
4) If a new trick is invented by one magician, the others will want to figure it out...who better to figure out a magic trick than another magician. There is value on a new trick, so when someone figures out how you do your new invention, the value diminishes. This is a primary cause of Magician Rivalry--stealing each others material and adding their own spin to claim it as their own.