Tutorial :Getting the text of a webpage with HTTPClient



Question:

I'm just getting started with HTTPClient, and I want to take a webpage and extract out the raw text from it minus all the html markup.

Can HTTPClient accomplish that? If so, how? Or is there another library I should be looking at?

for example if the page contains

<body><p>para1 test info</p><div><p>more stuff here</p></div>  

I'd like it to output

para1 test info more stuff here  


Solution:1

I'd suggest using HttpComponents Client (HTTPClient 4) (instead of version 3 you've linked to).

This being said, it's independent of the HTTP client library (there are others). What you need is to convert the HTML into plain text. This could be of interest: http://www.rgagnon.com/javadetails/java-0424.html


Solution:2

No. HttpClient handles network protocol - sending requests and receiving responses. It's up to you to figure out what to do with the response once you receive it. That said, you can use other libraries to parse HTML as others suggested.


Solution:3

The HTML Parser library might be what you are looking for. It allows for extraction of content from a HTML document.


Solution:4

As others have mentioned, you need an HTML parsing library. Here is a relevant question.


Note:If u also have question or solution just comment us below or mail us on toontricks1994@gmail.com
Previous
Next Post »