Engineered For Every Major Documentation Platform
DocHarvest uses dynamic heuristics to detect documentation frameworks, bypassing DOM noise to extract author-original markdown directly.
GitBook Harvesting Heuristics
Deep GitBook spaces integration. Traverses multi-version dropdowns, parses space manifest JSON, and probes raw .md endpoints to fetch author-original markdown.
- Direct .md raw endpoint probing bypassing HTML conversion
- Multi-version space selector traversal (/v/v2.0/, /v/latest)
- Preserves embedded code block tabs and parameter tables
- Automatic TOC generation from summary tree structure
<!-- Raw Scraper Output (42.8 KB HTML Soup) -->
<div class="gitbook-root-container" data-space="sp_987x">
<nav class="sidebar-nav-sticky-top"><div class="cookie-banner-wrap">...</div>
<ul class="nav-tree-level-1"><li class="active"><a href="/v/v2.0/auth">OAuth</a></li>...</ul>
<main class="page-content-wrapper">
<div class="header-anchor-wrap"><h1 id="oauth2">OAuth 2.0 Auth<a class="anchor" href="#oauth2">¶</a></h1></div>
<div class="alert alert-warning"><svg class="icon">...</svg><span>Token expires in 3600s</span></div>
<div class="code-block-container" data-lang="python">
<div class="code-header"><span class="lang-label">Python</span><button class="copy-btn">Copy</button></div>
<pre><code><span class="token-keyword">import</span> <span class="token-variable">requests</span>...</code></pre>
</div>
</main>
</div>---
source_url: https://docs.openalgo.in/v/v2.0/api-reference/oauth
title: "OAuth 2.0 Authentication"
crawl_date: "2026-08-23T16:42:19Z"
content_hash: "sha256-e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
site_version: "v2.0"
---
# OAuth 2.0 Authentication
> ⚠️ **Warning:** Access tokens expire after 3600 seconds. Refresh via `/oauth/v2/token`.
## Request Signature
```python
import requests
response = requests.post(
"https://api.openalgo.in/oauth/token",
json={"client_id": "pk_live_...", "grant_type": "client_credentials"}
)
```One Crawl. Four Production-Ready Formats.
Every crawl emits a predictable four-part output matrix ready for Claude, Cursor, vector databases, and offline reading.
1. Consolidated Markdown — Unified book.md for LLM System Prompts
Merges the entire documentation portal into a single coherent, top-to-bottom Markdown file with an automatic table of contents and internal anchor links.
- Automated hierarchical Table of Contents
- Relative link remapping to internal document anchors
- Zero redundant headers, sidebars, or footers
- Perfect for pasting directly into Claude Project Knowledge or ChatGPT Custom GPTs
✓ Injected SHA-256 cryptographic hashes.
✓ Output written to disk in 0.4s.
Inspect the Generated Compilation Artifacts
Switch between raw generated artifacts to see how DocHarvest formats Markdown, vector JSONL, llms.txt, and SQLite FTS5 indices.
---
title: "OpenAlgo v2.0 Complete Developer Handbook"
source_domain: "docs.openalgo.in"
harvest_timestamp: "2026-08-23T16:42:19Z"
total_pages: 364
generator: "DocHarvest v10.0.1"
---
# Table of Contents
1. [Architecture Overview](#1-architecture-overview)
2. [OAuth 2.0 Authentication](#2-oauth-20-authentication)
3. [Order Execution API](#3-order-execution-api)
4. [WebSocket Market Data Feed](#4-websocket-market-data-feed)
---
# 1. Architecture Overview
OpenAlgo operates as an ultra-low latency execution broker wrapper...Why DocHarvest Outperforms Raw Scrapers & Cloud APIs
Compare DocHarvest against raw scrapers and proprietary cloud services.
| Engine Capability | DocHarvest v10.0.1 | Cloud Reader APIs |
|---|---|---|
Native AST Heuristic Framework Detection (GitBook, Mintlify, Docusaurus) Automatically isolates article DOMs and probes raw markdown endpoints directly. | Partial | |
Zero HTML/JSX Soup in Markdown Output Strips cookie banners, navbars, sidebars, and interactive widget code. | ||
Standard llms.txt & Vector RAG JSONL Compilation Builds unified RAG chunk files with token counts and SHA-256 content hashes. | ||
Pure-Python PDF Handbook Generation with TOC (fpdf2) Zero external C-library dependencies (no WeasyPrint or wkhtmltopdf). | ||
Built-in FastMCP Server for Cursor, Claude Desktop & Windsurf 8 native MCP tools running over stdio directly inside your IDE agent. | API Key Req | |
Embedded SQLite FTS5 BM25 Full-Text Search Database Instant sub-15ms keyword search queries across thousands of harvested pages. | ||
100% Free, Open Source (MIT) & Local Privacy No subscription fees, no credit limits, and zero data leaves your local machine. |
Give Cursor & Claude Native Documentation Powers
Connect DocHarvest to your AI agent via FastMCP stdio in 30 seconds. Your agent gains tools to crawl, index, and query external documentation on demand.
{
"mcpServers": {
"docharvest": {
"command": "python",
"args": ["-m", "gitbook_downloader.mcp_server"]
}
}
}Three Specialized Workflows. Zero Friction.
Whether you are training embeddings, coding on a plane, or managing documentation backups in CI/CD.
AI & RAG Engineers
- Zero noise (89% prompt token reduction)
- Direct drop-in for LangChain & ChromaDB
- FastMCP server for Cursor & Claude Desktop
Offline Developers & Researchers
- Pure-Python PDF generation (zero C-deps)
- Embedded SQLite FTS5 search (<15ms queries)
- Standalone desktop GUI (docharvest.exe)
DevOps & Archival Teams
- Semver snapshot diffs across crawls
- Auto-recovering PID domain locks
- Headless CLI automation in CI/CD
Latest Release & Live Build Telemetry
Standalone binaries, release notes, and real-time commit logs from the active repository pipeline.
# DocHarvest v10.0.1 Version 10.0.1 resolves key UI state discrepancies, completes the standalone executable renaming (`docharvest.exe`), adds project renaming inside the Document Library, and introduces an interactive About section with full creator attribution. ### ✨ Features & Capabilities - ✏️ **Project Rename Feature in Document Library**: - Added `StorageManager.rename_domain` and `SearchIndex.rename_domain` to safely update directory paths, metadata JSON, and SQLite FTS5 search index entries. - Added a **Rename (✏️)** action button on library cards and an interactive rename modal dialog in `frontend/src/views/LibraryView.tsx`. - ❤️ **Interactive About Modal & Creator Attribution**: - Added `frontend/src/components/AboutModal.tsx` displaying engine specifications (AST + FastMCP + fpdf2), runtime telemetry, GitHub/Showcase links, and **"Made with ❤️ by Rohan Shetty"**. - Accessible via the sidebar footer heart trigger and the `Ctrl+K` command menu. ### 🐛 Bug Fixes & Hardening - 🏷️ **Universal Brand & Binary Propagation**: - Standalone build pipeline (`build_exe.py`) updated to produce `docharvest.exe` (primary) alongside `gitbook-dl.exe` for backwards compatibility. - Added `docharvest` CLI command entry point in `pyproject.toml` (`[project.scripts]`). - Desktop GUI window title, sidebar header, and Capture Studio badges updated to `DocHarvest v10.0.1` and `v10.0 Engine`. - `.github/workflows/build-release.yml` now stages and publishes both `docharvest-*` and `gitbook-dl-*` cross-platform binaries. ### ⚡ Performance & Architecture - ⚡ **Document Library Badge Race Condition Fixed**: - Eliminated the tick-0 mock fallback in `frontend/src/lib/bridge.ts` that caused the count badge to flash `(1)` before the WebView2 Python bridge attached. - Cleaned the sidebar navigation item badge for a distraction-free library view. ### 📦 Downloadable Binaries & Verification Checksums | Asset / Executable | Target Platform | File Size | SHA-256 Checksum | | :--- | :--- | :--- | :--- | | `docharvest-macos-latest` | macOS (Apple Silicon / Intel) | 29.4 MB | `9744f68b951cfe8b6d4e68e6038d7c1464c7b8eb97754b8ba73f0f5d38071c26` | | `docharvest-ubuntu-latest` | Linux x64 | 47.4 MB | `f9467b446b5528370bae581a41d1b170a731eccbbaffd4e3a1bf21c406b2bb63` | | `docharvest-windows-latest.exe` | Windows x64 | 33.5 MB | `03c6b6e06a6476000e15edc0ffc421c599b99bd8cdb1ff2517245edfbdf48135` | | `gitbook-dl-macos-latest` | macOS (Apple Silicon / Intel) | 29.4 MB | `9744f68b951cfe8b6d4e68e6038d7c1464c7b8eb97754b8ba73f0f5d38071c26` | | `gitbook-dl-ubuntu-latest` | Linux x64 | 47.4 MB | `f9467b446b5528370bae581a41d1b170a731eccbbaffd4e3a1bf21c406b2bb63` | | `gitbook-dl-windows-latest.exe` | Windows x64 | 33.5 MB | `03c6b6e06a6476000e15edc0ffc421c599b99bd8cdb1ff2517245edfbdf48135` | | `SHA256SUMS.txt` | All Platforms | < 1 KB | *(Verification Manifest)* | #### Verification Instructions ```bash # Linux / macOS integrity verification: sha256sum -c SHA256SUMS.txt # Windows PowerShell integrity verification: Get-FileHash -Algorithm SHA256 gitbook-dl-windows-latest.exe ``` --- **Full Changelog**: https://github.com/RohannShetty/gitbook-downloader/compare/v10.0.0...v10.0.1
docs: overhaul README with screenshots, add GitHub Profile README, and standardize social links
feat: embed StartupBar widget loader script
feat(showcase): revamp DocHarvest website to Next.js 16 with Terminal-Engine theme, FastMCP showcase & live telemetry
fix(pages): enable trailingSlash for static GitHub Pages subpage routing
feat(pages): deploy Next.js portfolio website to GitHub Pages
feat: add premium personal portfolio Next.js website under portfolio/
chore(release): v10.0.1 - DocHarvest hotfix release, binary naming, library rename & about section
fix(docs): track docs/src/data and fix root data ignore rule
Frequently Answered Questions
Everything you need to know about AST extraction, SPA documentation support, and FastMCP integration.