Specifying the Parser
Beautiful Soup does not parse HTML itself; it delegates to a parser backend you choose. The second argument to the constructor selects it, and the choice affects speed, tolerance of broken markup, and which features are available.
The Three Parsers
| Parser | Install | Character |
|---|---|---|
| html.parser | built-in | Fast, forgiving; limited HTML5 behavior |
| lxml | pip install lxml | Fastest, very forgiving, good for XML too |
| html5lib | pip install html5lib | Slowest; parses like a browser DOM |
Choosing a Parser
from bs4 import BeautifulSoup
html = "<div><p>unclosed <span>text"
soup_html5 = BeautifulSoup(html, "html.parser")
soup_lxml = BeautifulSoup(html, "lxml")
soup_html5lib = BeautifulSoup(html, "html5lib")Rules of thumb:
- lxml for most work: fast and forgiving, and the only one that handles XML well at speed.
- html.parser when you cannot install extra packages.
- html5lib when you need to match exactly what a browser does with broken markup (it is much slower).
Parsing XML
For XML documents pass the lxml parser (or xml if lxml is missing). You can keep the XML declaration in the output:
xml = "<?xml version="1.0"?><feed><entry>e1</entry></feed>"
soup = BeautifulSoup(xml, "lxml")
print(soup.xmldecl if hasattr(soup, "xmldecl") else "parsed")
for e in soup.find_all("entry"):
print(e.get_text())Summary
The parser argument decides how your document is turned into a tree. Use lxml by default for speed and XML support, fall back to the built-in html.parser when you cannot install extras, and choose html5lib only when you need browser-faithful parsing of badly broken HTML.