Skillversitas

Parsing Tables and XML

Two document shapes deserve special treatment: HTML tables, which are the most common structured data in old-school web pages, and XML, which needs a parser that respects nesting and (optionally) the declaration.

HTML Tables

from bs4 import BeautifulSoup

html = '''
<table>
  <tr><th>Name</th><th>Qty</th></tr>
  <tr><td>Widget</td><td>3</td></tr>
  <tr><td>Gadget</td><td>5</td></tr>
</table>
'''
soup = BeautifulSoup(html, "html.parser")

for row in soup.find_all("tr"):
    cells = [c.get_text(strip=True) for c in row.find_all(["th", "td"])]
    print(cells)
# ['Name', 'Qty'], ['Widget', '3'], ['Gadget', '5']

Notes that save pain:

  • Real tables use colspan/rowspan; a naive row-by-row extraction will misalign. Check for the colspan attribute and repeat or skip cells accordingly.
  • th cells may appear in header rows; decide whether they are data or headers based on the table structure.

XML Parsing

Use the lxml parser for XML. The declaration survives the round trip, and case is preserved (unlike HTML, where tag names are lowercased):

xml = "<?xml version="1.0" encoding="utf-8"?><catalog><Book id="1">XML</Book></catalog>"
soup = BeautifulSoup(xml, "lxml")
print(soup.catalog.Book["id"])  # 1
print(soup.prettify())          # XML with declaration

A Note on Namespaces

XML namespaces complicate tag names. For simple feeds (RSS, Atom) plain tag names work; for deeply namespaced documents, consider a dedicated XML library (ElementTree, lxml.etree) where namespaces are first-class.

Summary

For tables, iterate tr elements and extract th/td text, but handle colspan and rowspan when the table is not a clean grid. For XML, parse with lxml to keep the declaration and case, and reach for a dedicated XML library when namespaces get involved.