Cite means either “to quote” or “to commend” and is usually a verb. As a noun, cite means “a short citation or reference.”
Site means “a place or setting” and is normally a noun or sometimes a verb associated with the act of “placing” something.
Sight might be either a noun or a verb involving vision or the act of seeing.
These days, many people incorrectly spell site when referring to a webcite (website). And website, by the way, has developed into one word beginning with a lowercase letter.
Here’s a usage note from Dictionary.com.
The transition from World Wide Web site to Web site to website as a single uncapitalized word mirrors the development of other technological expressions which have tended to take unhyphenated forms as they become more familiar. Thus email is gaining ground over the forms E-mail and e-mail, especially in texts that are more technologically oriented. Similarly, there is an increasing preference for closed forms like homepage, online, and printout.
Example: He could cite many famous quotations about the importance of this historic site, but the sight of the world leader actually standing there in person left the biggest impression.