Encoding and Decoding

Beautiful Soup tracks the document's encoding and offers methods that move between the tree, strings, and bytes: .encode() returns bytes in the detected encoding, and .decode_contents() and .decode() return the subtree as a string.

.encode() - Back to Bytes

from bs4 import BeautifulSoup

html = "<p>caf\u00e9 & r\u00e9sum\u00e9</p>"
soup = BeautifulSoup(html, "html.parser")

data = soup.p.encode("utf-8")
print(type(data))   # <class 'bytes'>
print(data)         # b'<p>caf\u00e9 & r\u00e9sum\u00e9</p>'

This is how you write a parsed document back to disk in a specific encoding:

with open("out.html", "wb") as f:
    f.write(soup.encode("utf-8"))

.decode_contents() - Children as a String

.decode_contents() serializes only the tag's children (not the tag itself) into a string. Useful for extracting the inside of a wrapper:

html = "<div class="box"><p>inner</p></div>"
soup = BeautifulSoup(html, "html.parser")
print(soup.div.decode_contents())  # <p>inner</p>

.decode() - The Whole Subtree as a String

.decode() is the same idea but includes the tag itself:

print(soup.div.decode())  # <div class="box"><p>inner</p></div>

When to Use What

• .encode() when writing a (modified) document to a file or HTTP response body.
• .decode_contents() when you need the inner HTML of a container as a string.
• .decode() when you need the full element markup as a string.
• .get_text() when you want text only, no tags at all.

Summary

Encoding methods let you move between tree and bytes/strings: encode() returns the document as bytes in a chosen encoding, decode_contents() returns a tag's inner HTML as a string, and decode() returns the full element markup as a string. Use them when writing output files or extracting markup, and get_text() when you want plain text.