Instead of Beautiful Soup: selectolax

Why selectolax is better

• Extracting the title, links, scripts and a meta tag from the home pages of the top 754 domains took 2.39 seconds with selectolax, 9.09 seconds with Beautiful Soup on lxml, and 61.02 seconds with Beautiful Soup's default html.parser, per selectolax's README.
• Its author reports a 5-30x speedup from swapping Beautiful Soup for selectolax in a text-extraction job over web archive files.
• In an independent test, Peter Bengtsson found selectolax about 7 times faster than PyQuery at extracting text from 10,000 HTML snippets.
• It is written in Cython over the Lexbor HTML5 engine and supports modern CSS selectors such as :has() and :nth-child(), plus a pseudo-class for matching text.

When to keep Beautiful Soup

Keep Beautiful Soup for small scripts where speed does not matter, for XML, and for code that edits documents through its rich tree-navigation API. It has been maintained since 2004, has thorough documentation, and was used by 34% of respondents in JetBrains' 2024 Python survey.

Switching

A rewrite of the parsing code, usually small. BeautifulSoup(html) becomes LexborHTMLParser(html), select() becomes css(), select_one() becomes css_first(), and get_text() becomes text(). Searches written with find_all() and attribute filters must be expressed as CSS selectors, and code that walks siblings or modifies the tree needs checking against selectolax's node API.

Watch out

selectolax is mainly the work of one developer and has a far smaller community, so tutorials and answers are scarcer. It parses HTML only, not XML, and offers CSS selectors rather than a find_all-style search API. Installing an older release on a new Python version can require Cython to compile it.

Also worth a look

• lxml A fast C-based HTML and XML parser with XPath support; pick it when you need XPath or XML.
• parsel Scrapy's selector library on top of lxml, with CSS, XPath and regex selectors; pick it for scraping pipelines that mix query styles.

More newer, better libraries, runtimes and developer tools

• Moment.js →, use date-fns
• Lodash →, use es-toolkit
• Axios →, use Ky
• Redux →, use Zustand
• styled-components →, use Tailwind CSS
• Prisma ORM →, use Drizzle ORM
• ImageMagick →, use libvips