Python Requests Extract HTTP Response Body

This article will teach you how to extract the HTML body from an HTTP response using the requests library.

Installing Requests

Before proceeding with this tutorial, ensuring you have the requests package installed in your environment is good.

You can run the command if you are using pip:

$ sudo pip install requests
$ sudo pip3 install requests

For Anaconda users, you can use the command:

$ sudo conda install -c anaconda requests

Making HTTP Request

The following example shows how to use the requests method to make a request to a given resource on the web.

import requests
response = requests.get('http://google.com')

The example above starts by importing the requests package. We then make a GET request to the specified URL. This should return a response object which we save into a response object.

Extract HTML Body From the Response Object

We can use the content method to extract the HTML body from the response object, which returns the response’s content.

An example code is as shown:

import requests
response = requests.get('https://google.com')
print(response.content)

The query above should return the HTTP body as shown:

You can then write the resulting HTML content to a HTML file.

Conclusion

In this brief article, you learned how to use the response.content() method to extract the HTML body from a response object.

Python Requests Extract HTTP Response Body

Installing Requests

Making HTTP Request

Extract HTML Body From the Response Object

Conclusion

About the author

John Otieno