**DONOTDELETE**
(Unregistered)
11/01/04 07:14 AM
Re: Create intelligent Web spiders
I can think of an improvment by checking the stream content type instead of the file extention. It should be "text/html" and then you can use HttpURLConnection or HttpsURLConnection for getting the html content.