WebCrawler - Cookie Manegment

Question

Long time reader, first time I actually had to register and post: Simply put, I’m a college student that wants to create a quick web crawler for a research paper for a 400 level class. The issue is even with my experience in C# and the .NET architecture, I cannot work myself around a problem: There seems to be a lot of sites now that as soon as you “grab” (stream it or download or any other way) a web page, they reply by giving you the dreadful “Your are using an incompatible browser.” Any way I do it using a WebClient class or HttpWebResponse class, it never works. I’m not looking for any code here, just a general suggestion or idea for a solution to make my crawler look as if it’s a normal browser such as IE. Any help at all would be greatly appreciated.