|
|
Create intelligent Web spiders
|
|
Create intelligent Web spiders
|
|
**DONOTDELETE**
|
|
(Unregistered)
|
|
11/01/04 07:14 AM
|
|
|
|
|
|
|
|
|
Re: Create intelligent Web spiders
|
|
I can think of an improvment by checking the stream content type instead of the file extention. It should be "text/html" and then you can use HttpURLConnection or HttpsURLConnection for getting the html content.
|
|
**DONOTDELETE**
|
|
(Unregistered)
|
|
11/01/04 09:58 AM
|
|
|
|
|
|
|
|
|
Re: Create intelligent Web spiders
|
|
Thanks for the great code! However, I miss the separation between Model, View and Controller. If somebody wants to reuse this code in a slightly different context, he will have to do lots of copy paste!
|
|
**DONOTDELETE**
|
|
(Unregistered)
|
|
11/11/04 08:25 AM
|
|
|
|
|
|
|
|
|
Re: Create intelligent Web spiders
|
|
Nice article. Code is interesting to play with. Thank you.
|
|
**DONOTDELETE**
|
|
(Unregistered)
|
|
11/30/04 02:16 PM
|
|
|
|
|
|
|
|
|
Re: Create intelligent Web spiders
|
|
Thanks. That's a great suggestion. Takes some of the guess work out.
|
|
**DONOTDELETE**
|
|
(Unregistered)
|
|
01/09/06 09:00 PM
|
|
|
|
|
|
|
|
|
Re: Create intelligent Web spiders
|
|
Hi
Problem is that what if you just needed .html files and not say jsp files. Both return text/html as the contenttype
Nick
|