Extract Content as Markdown
Convert any webpage to clean Markdown text using the CaptureKit content API — strip ads and boilerplate to get the pure article or documentation content.
Overview
The CaptureKit content endpoint fetches a webpage, strips navigation, ads, and layout chrome, then returns the main body as clean Markdown. Use it for building RAG datasets, indexing documentation, or archiving articles.
Prerequisites
- A CaptureKit API key — get one at app.capturekit.dev
- Install dependencies for your language:
pip install requestsSteps
Fetch the page content
Call GET /v1/content with the url parameter.
import requests
API_KEY = "YOUR_API_KEY"
response = requests.get(
"https://api.capturekit.dev/v1/content",
headers={"x-api-key": API_KEY},
params={"url": "https://stripe.com/docs/payments"},
)
data = response.json()
print(data)Access the Markdown content
The response includes markdown, title, description, author, published_at, and word_count.
print(f"Title : {data.get('title')}")
print(f"Author : {data.get('author', 'N/A')}")
print(f"Word count : {data.get('word_count')} words")
print()
print("--- Markdown preview ---")
markdown = data.get("markdown", "")
print(markdown[:1000])Crawl and archive a list of documentation pages
Fetch and save multiple pages as Markdown files for offline search or RAG ingestion.
import os, time, requests
API_KEY = "YOUR_API_KEY"
os.makedirs("docs_archive", exist_ok=True)
PAGES = [
"https://stripe.com/docs/payments",
"https://stripe.com/docs/billing",
"https://stripe.com/docs/connect",
]
for page_url in PAGES:
r = requests.get("https://api.capturekit.dev/v1/content",
headers={"x-api-key": API_KEY}, params={"url": page_url})
data = r.json()
slug = page_url.rstrip("/").split("/")[-1]
path = f"docs_archive/{slug}.md"
with open(path, "w") as f:
f.write(f"# {data.get('title', slug)}\n\n")
f.write(data.get("markdown", ""))
print(f"Saved: {path} ({data.get('word_count', 0)} words)")
time.sleep(1)
print("Done!")The extracted Markdown is ideal as context chunks for a RAG (Retrieval-Augmented Generation) pipeline. Chunk by headings and embed with your preferred vector store for semantic search over any website's documentation.
Screenshot a Webpage
Capture a full-page or viewport screenshot of any URL as a PNG or JPEG file using the CaptureKit API — no headless browser to maintain.
Analyze a Page with AI
Send a URL to the CaptureKit AI analysis endpoint and receive structured insights about the page's content, design, technology stack, and SEO metadata.