There are various reasons for this. Firstly, we ask you to tell people where the information came from. Secondly, including a link helps to prevent Google from removing our pages from search results. If Google sees the same material over and over again it begins to treat that material as a type of spam and will stop showing any pages that have that material. If however, the material links back to the original page then Google will identify our page as the original source.
For savy web makers you can even use the categorical link identifier. But don’t worry about that if you don’t know what that is. 🙂