Extract links from HTML with Python: a safe local practice project
SkillNexis · 2026-06-25 · Updated 2026-10-21
HTML extraction turns markup into records you can inspect and test. Start with a document you control: that separates learning to parse HTML from network failures, access permissions and changes on someone else’s website. This project extracts links from a local HTML string with Python’s standard library. It sends no requests and needs no third-party package.
1. Define the records you want
Our output is a list of href values from anchor tags. We preserve their order and keep relative links unchanged. An anchor without an href is skipped. This version does not collect link labels, resolve relative URLs or filter protocols. Those are separate requirements; writing them down prevents an apparently successful parser from silently doing the wrong job.
2. Run a complete example
Save this as extract_links.py and run python extract_links.py in a terminal. HTMLParser calls handle_starttag when it encounters an opening tag. Attributes arrive as name/value pairs, so a dictionary makes the href lookup easy to read.
from html.parser import HTMLParser
class Links(HTMLParser):
def __init__(self):
super().__init__()
self.links = []
def handle_starttag(self, tag, attrs):
if tag == "a":
href = dict(attrs).get("href")
if href is not None:
self.links.append(href)
def extract_links(document):
parser = Links()
parser.feed(document)
parser.close()
return parser.links
sample = '<a href="/learn">Learn</a><a href="https://example.com/?a=1&b=2">Example</a><a>Missing</a>'
print(extract_links(sample))
assert extract_links(sample) == ["/learn", "https://example.com/?a=1&b=2"]
assert extract_links("") == []The expected printed output is ['/learn', 'https://example.com/?a=1&b=2']. The parser converts the HTML character reference in the second URL. The third anchor has no href and contributes nothing. An empty document gives an empty list. Notice that a fresh parser is created for each document: reusing one object without clearing links would accidentally combine results from multiple inputs.
3. Test changes one at a time
Add a fourth anchor pointing to /learn. It should produce a duplicate because deduplication is not part of our contract. Then try a link containing only #section; it remains a fragment reference. If your project needs unique absolute URLs, first define the base URL, resolve relative references, allow only the required protocols, and deduplicate while preserving order. Keep a saved sample document in your project so a change in parsing code can be tested without relying on a live website.
4. Know the boundary of this exercise
This extractor reads attributes, not a browser’s final page after JavaScript runs. A link generated by client-side code will not exist in the original HTML string. HTMLParser also is not a sanitizer: do not render untrusted extracted markup as trusted HTML. Display extracted values as text. For unusually nested or broken documents, inspect the output and choose a parser suitable for the document rather than adding fragile string replacements.
5. Before collecting remote pages
Prefer an official API or permitted export when available. Confirm that the intended collection is permitted, respect published access rules and rate limits, and stop when access is denied. Robots instructions do not by themselves grant permission to reuse content. Avoid collecting personal information for a practice project. Keep fetching and parsing separate, handle failed responses explicitly, and do not bypass login or access controls.
Next: practise processing text with Python or read about handling an HTTP response. Reference: Python HTMLParser documentation.