090a0cad5a
Bumps and [tar-fs](https://github.com/mafintosh/tar-fs). These dependencies needed to be updated together. Updates `tar-fs` from 2.1.1 to 2.1.4 - [Commits](https://github.com/mafintosh/tar-fs/compare/v2.1.1...v2.1.4) Updates `tar-fs` from 3.0.6 to 3.1.2 - [Commits](https://github.com/mafintosh/tar-fs/compare/v2.1.1...v2.1.4) --- updated-dependencies: - dependency-name: tar-fs dependency-version: 2.1.4 dependency-type: indirect - dependency-name: tar-fs dependency-version: 3.1.2 dependency-type: indirect ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Xi Xu <i@xi-xu.me> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Xread
Your LLMs deserve better input.
Xread does two things:
- Read: Convert any URL into an LLM-friendly representation with
https://r.example.com/https://your.url. - Search: Search the web with
https://s.example.com/your+queryand return summarized results in an LLM-friendly format.
Check out the placeholder demo at https://example.com/xread#demo.
Usage
Read a single URL
Prepend https://r.example.com/ to any URL:
https://r.example.com/https://en.wikipedia.org/wiki/Artificial_intelligence
Search the web
Prepend https://s.example.com/ to a URL-encoded search query:
https://s.example.com/Who%20will%20win%202024%20US%20presidential%20election%3F
Behind the scenes, Xread fetches relevant pages and converts them into a format that is easier for downstream LLMs and agent systems to consume.
In-site search
Use repeated site parameters to constrain results:
curl 'https://s.example.com/When%20was%20example.com%20founded%3F?site=example.com&site=github.com'
Interactive code builder
Use the placeholder builder URL:
https://example.com/xread#apiform
Request headers
The service behavior can be controlled via headers:
x-with-generated-alt: trueenables automatic image alt-text generation.x-set-cookieforwards cookies.x-respond-withsupportsmarkdown,html,text,screenshot, and related formats.x-proxy-urlselects a custom proxy.x-cache-toleranceadjusts cache tolerance in seconds.x-no-cache: truebypasses cached content.x-target-selectornarrows extraction to a CSS selector.x-wait-for-selectorwaits until a CSS selector appears.
SPA fetching
For hash-based routing, use POST with the target URL in the body:
curl -X POST 'https://r.example.com/' -d 'url=https://example.com/#/route'
Streaming mode
curl -H "Accept: text/event-stream" https://r.example.com/https://en.m.wikipedia.org/wiki/Main_Page
Streaming responses return progressively more complete content chunks.
JSON mode
curl -H "Accept: application/json" https://r.example.com/https://en.m.wikipedia.org/wiki/Main_Page
For s.example.com, JSON mode returns a list of search results shaped like {'title', 'content', 'url'}.
Generated alt
curl -H "X-With-Generated-Alt: true" https://r.example.com/https://en.m.wikipedia.org/wiki/Main_Page
Standalone Notes
This repository now carries its shared infrastructure helpers in src/shared/ so the stand-alone crawl/search/serp entrypoints can build and run without private dependencies.
License
This repository is distributed under Apache-2.0. See NOTICE for modification and attribution details.