Olostep CLI

Command-line interface for scrape, search, map, crawl, answers, and batches — JSON output for scripts, CI, and AI agents

NPM package: olostep-cli Repository: github.com/olostep-api/olostep-cli

CLI for the Olostep API — scrape, search, map, crawl, answer, and batch the web from your terminal. Every command returns JSON so it pipes cleanly into jq, agents, and CI.

Pure JavaScript, Node 18+, no native binaries to download. Installs in under a second, starts in ~200 ms, ships as a single ~100 KB bundle.

Install

Requirements: Node.js 18+.

npm (recommended)

npm install -g olostep-cli
olostep init

One-liner (macOS / Linux)

curl -fsSL https://olostep.com/install.sh | sh
olostep init

The script checks Node 18+, runs npm install -g olostep-cli, and falls back to sudo if needed.

One-liner (Windows)

iwr -useb https://olostep.com/install.ps1 | iex
olostep init

Same idea as the macOS/Linux script, but for PowerShell.

No-install (npx)

npx -y olostep-cli@latest --help

Good for trying a single command without a global install.

olostep init is the recommended next step — it signs you in, installs the Olostep skills into your AI agents, and configures the MCP server, all in one command. The one-liner scripts wrap npm install -g olostep-cli with a Node 18+ check and a sudo fallback, so they work even if you’re unsure of your local setup.

Platforms: macOS (Apple Silicon and Intel), Linux (x64 and arm64), Windows (x64 and arm64).

Set up

One command does everything — sign in, install skills, and install the MCP server:

olostep init

Flags: --skills-only, --mcp-only, --no-browser, --relogin.

To just sign in (no skills/MCP):

olostep login
olostep login --no-browser     # print the URL (useful over SSH)

The browser opens to the Olostep auth page; you click Authorize, and the CLI saves your key locally.

Alternative — set an env var. Good for CI:

export OLOSTEP_API_KEY=your_key_here

Get a key from the API Keys dashboard.

Where the key is stored (after olostep login):

OS	Path
macOS	~/Library/Application Support/olostep-cli/credentials.json
Linux	~/.config/olostep-cli/credentials.json
Windows	%USERPROFILE%\AppData\Roaming\olostep-cli\credentials.json

Sign out

olostep logout            # prompts to confirm, then removes credentials.json
olostep logout --dry-run  # preview only — see what would happen
olostep logout --yes      # skip the confirmation (for scripts)
olostep logout --json     # machine-readable output

logout also warns you if OLOSTEP_API_KEY / OLOSTEP_API_TOKEN env vars or a .env file in your current directory still hold a key — those take priority over the credentials file, so deleting the file alone may not be enough. The output includes the exact unset commands for PowerShell and bash/zsh.

Quick start

olostep login

olostep search "best web scraping APIs 2025" --limit 5
olostep answer "What does Olostep do?"
olostep map "https://example.com" --top-n 20
olostep scrape "https://example.com" --formats markdown
olostep crawl "https://docs.example.com" --max-pages 50
olostep batch-scrape urls.csv --formats markdown,html

Every command prints its JSON result to stdout by default. Pass --out <path> to save to a file.

What can it do?

You want to…	Command	Olostep product
Search the web	search	Searches
Get a researched answer	answer	Answers
Discover URLs on a site	map	Maps
Pull one page	scrape	Scrapes
Pull every page on a site	crawl	Crawls
Pull many URLs from a CSV	batch-scrape	Batches
Extract structured fields	--parser-id on batch-scrape	Parsers
Refetch a result by ID	scrape-get	Scrapes
Tag/organize a batch	batch-update	Batches

Output

Every command prints its JSON result to stdout by default.

Flag	Behavior
(none)	Print JSON to stdout (UTF-8, indented)
--out <path>	Write JSON to that file instead
--out -	Explicitly stdout (same as default)

Progress and log lines go to stderr, so stdout stays clean for pipes.

olostep map "https://example.com" --top-n 20 | jq '.urls[:10]'
olostep scrape "https://example.com" | jq .result.markdown_content
olostep search "topic" --json | jq '.links[].url'

Choosing between them:

• search — you want a list of relevant URLs and snippets for a query. The CLI searches the web for you.
• answer — you want a synthesized answer, not raw page content. The CLI does the research for you.
• scrape — you already have the URL and want clean content out.
• crawl — you want every page on a site (or a filtered subset) without enumerating URLs by hand.
• batch-scrape — you have a list of URLs and want them processed in parallel.

Commands

Use olostep <command> --help for every option.

search: live web search

Returns deduplicated organic links (URL, title, description).

Option	Description
--limit	Number of results, default 12, max 25
--include-domains	Comma-separated domains to restrict results to
--exclude-domains	Comma-separated domains to exclude
--out	File or -
--json	Machine-readable output

olostep search "TypeScript CLI tools" --limit 10
olostep search "open source projects" --include-domains "github.com" --limit 5
olostep search "AI agents" --json | jq '.links[].url'

answer: researched answer

Synchronous — returns when the answer is ready.

Option	Description
--out	File or -
--json-format	Optional JSON shape for structured output

olostep answer "What does this company build?" --out answer.json
olostep answer "Extract facts" --json-format '{"company":"","year":""}' --out -

map: discover URLs

Option	Description
--out	File path or -
--top-n	Max URLs to return
--search-query	Optional query to guide discovery
--include-subdomain / --no-include-subdomain	Subdomains
--include-url / --exclude-url	Repeatable URL patterns
--cursor	Pagination cursor

olostep map "https://example.com" --top-n 100 --search-query "blog"

scrape: one URL

Formats: html, markdown, text, json, raw_pdf, screenshot (comma-separated; default markdown).

Option	Description
--formats	Comma-separated
--country	Country code (e.g. US, GB)
--wait-before-scraping	Wait before scrape (ms)
--payload-json / --payload-file	Advanced options as JSON

olostep scrape "https://example.com" --formats markdown,html
olostep scrape "https://example.com" --payload-file options.json --out -

scrape-get: fetch by ID

olostep scrape-get "scrape_abc123" --out -

crawl: whole site

Starts a crawl, polls until finished, then retrieves page contents.

Retrieve formats: markdown, html, json.

Notable flags: --max-pages, --max-depth, --include-subdomain, --include-external, --include-url, --exclude-url, --search-query, --top-n, --webhook, --crawl-timeout, --formats, --pages-limit, --pages-search-query, --poll-seconds, --poll-timeout, --dry-run.

olostep crawl "https://docs.example.com" --max-pages 50 --formats markdown,html
olostep crawl "https://example.com" --max-pages 10 --dry-run

batch-scrape: CSV

CSV must have a header row with custom_id (or id) and url columns.

custom_id,url
example,https://example.com
iana,https://iana.org
docs,https://docs.olostep.com

Option	Description
--formats	markdown, html, json (comma-separated)
--country	Optional country code
--parser-id	Parser ID for structured extraction
--poll-seconds, --log-every, --items-limit	Polling and paging
--dry-run	Print payload and exit

olostep batch-scrape urls.csv --formats markdown,html
olostep batch-scrape urls.csv --parser-id "<PARSER_ID>" --out results.json

Synchronous — polls until the batch completes, then retrieves every item.

batch-update: batch metadata

Requires one of --metadata-json or --metadata-file (JSON object).

olostep batch-update "batch_abc123" --metadata-json '{"team":"growth"}'
olostep batch-update "batch_abc123" --metadata-file meta.json

Auth commands

olostep login                      # browser PKCE sign-in
olostep logout                     # remove saved credentials
olostep status                     # show auth state, config paths, version

olostep auth login                 # same as olostep login
olostep auth logout                # same as olostep logout
olostep auth status                # same as olostep status
olostep auth set-key <key>         # save an API key directly (no browser)

auth set-key is useful for CI and scripts — write the key directly without going through the browser flow.

Install the MCP server

The CLI writes the Olostep MCP server into your agent’s config — no JSON editing.

olostep mcp install                          # detect agents, hosted endpoint
olostep mcp install --agent cursor           # only Cursor
olostep mcp install --transport stdio        # local npx instead of hosted
olostep mcp install --no-global              # write into current project
olostep mcp install --dry-run --json         # plan only
olostep mcp uninstall                        # remove the olostep entry
olostep list mcp                             # show which agents have it

Option	Description
--agent	Specific agent, repeatable. Supported: cursor, claude, claude-desktop, windsurf, vscode, kilo, opencode, continue, codex
--all-agents / --no-all-agents	Target every detected agent (default)
--transport	http (hosted, recommended) or stdio (local npx olostep-mcp)
--global / --no-global	Per-user config (default) vs project-local
--api-key	Key to embed; defaults to resolved credentials
--dry-run	Show the plan without writing
--json	Machine-readable output

The hosted endpoint at https://mcp.olostep.com/mcp uses Authorization: Bearer <key> — no local Node process required. The CLI merges only the olostep key into your existing config. Restart your agent after install.

Skills for AI agents

The CLI ships 13 Olostep skills — SKILL.md files installed into Claude Code, Cursor, and other agents so they know what Olostep can do and when to use it.

olostep add skills                     # install all into every detected agent
olostep skills install                 # same (alias)
olostep skills update                  # re-install / refresh all skills
olostep skills list                    # see what's installed and where
olostep skills uninstall               # remove all skills

Filter what gets installed:

olostep add skills --category usage    # core web-data skills only
olostep add skills --skill scrape --skill map
olostep add skills --agent cursor --agent claude

See Skills for the full list and options.

Health checks

olostep doctor                         # run all checks
olostep doctor --skip-network          # auth + config only, no HTTP calls
olostep doctor --json                  # NDJSON — one record per check (good for CI)
olostep doctor --fail-on-warn          # exit 1 on warnings too

Checks: API key present, API key reachable, MCP endpoint reachable, config file exists for each detected agent.

CI usage:

olostep doctor --json --skip-network | jq 'select(.status == "fail")'

Version & updates

olostep version                        # CLI version, Node version, channel
olostep version --json                 # machine-readable: { cli, node, channel }
olostep update                         # update to latest (npm install -g olostep-cli@latest)
olostep update --check                 # check whether a newer version is available, don't install

Environment variables

Variable	Effect
OLOSTEP_API_KEY	API key
OLOSTEP_API_TOKEN	API key (legacy alias)
OLOSTEP_JSON=1	Force JSON output on every command (same as --json globally)
OLOSTEP_NO_UPDATE_CHECK=1	Silence the “update available” notice
OLOSTEP_CLI_CONFIG_DIR	Override the credentials directory

Windows / PowerShell notes

PowerShell tokenizes , and * differently from bash — quote arguments:

olostep scrape "https://example.com" --formats "markdown,html"
olostep map   "https://example.com" --include-url "/*"
olostep answer "Extract facts" --json-format '{"company":"","year":""}'

Single quotes are safest for JSON values (no $ interpolation).

See what’s installed

olostep list skills    # installed Olostep skills and which agents have them
olostep list mcp       # which agents have the Olostep MCP server, and the transport

Global flags

Flag	Description
-V, --version	Version
-h, --help	Help

--out, --timeout, and --api-key are available on every data command.

Security

Keep API keys out of source control; rotate if leaked. olostep logout removes the local credentials file and tells you if any env-var sources still hold a key.

Related

• Maps · Crawls · Batches · Answers · Searches · Skills · MCP Server