Define the question
Give OpenScience a question, date range, sources, and inclusion criteria. Start in plan mode if those choices are not settled.
/plan Review protein language models for predicting mutation effects.
Cover work published since 2022. Include experimental validation and
separate preprints from peer-reviewed papers. Propose the search terms
and inclusion criteria before searching.
A broad request such as "find everything about cancer" is hard to verify. Narrow it to an organism, method, outcome, or comparison.
Search and collect evidence
After reviewing the plan, ask OpenScience to search and save a source table:
Run the agreed search. Save results/sources.csv with title, authors,
year, DOI or source URL, publication status, method, dataset, and the
reason each paper was included. Record the search date and queries.
Available literature databases include PubMed, Europe PMC, arXiv, bioRxiv/medRxiv, OpenAlex, Crossref, and Semantic Scholar. The database directory lists the built-in sources.
Search access and full-text access are different. A title or abstract result does not mean the full paper was read. Ask OpenScience to identify abstract-only evidence and inaccessible sources explicitly.
How papers are found and read
The literature tool is the agent's default path to papers.
- Search runs one query against OpenAlex and arXiv together, merges records that describe the same paper (by DOI, arXiv id, or title), and ranks papers both sources agree on first. Each candidate carries the DOI or arXiv id, venue, citation count, abstract, landing page, and whether an open full text exists. The result says which source answered, which answered through a fallback, and which failed. Other connectors (
pubmed,europepmc,biorxiv,crossref,semantic-scholar) can be named withsources. - Read takes a DOI, arXiv id, URL, local PDF, or exact title, resolves the open full text (arXiv PDFs directly, publisher and repository copies through OpenAlex's open-access locations), downloads it once into the session's paper cache, and extracts page-addressed text with
pdftotext(poppler) or PyMuPDF. The agent reads the opening pages, a page range, or the passages matching a phrase, each tagged with its page number. When only the abstract is open, the result says so instead of pretending to have read the paper.
The cache lives in the session's scratch workspace under papers/, so re-reading a paper costs nothing and survives across turns.
When arXiv rate limits
arXiv's API answers bursts of requests with 429 for a while, often after a long stall. OpenScience now treats one such answer as a signal: the API is held for a cooldown (a minute, doubling on repeats) and arXiv records are served from OpenAlex, which indexes every arXiv paper under 10.48550/arXiv.<id>, or from the paper's own arxiv.org/abs page. Results obtained this way are marked via. If every route fails, the error reports the HTTP status, the endpoint, the number of attempts, and any requested wait, together with the concrete alternatives, so the agent can choose a different source instead of retrying the same call.
Text extraction needs pdftotext (macOS: brew install poppler; Debian and Ubuntu: apt-get install poppler-utils) or the pymupdf Python package. Without either, the PDF is still downloaded and the agent can open it with the read tool.
Compare findings
Group the included papers by evaluation method. Compare datasets,
baselines, reported metrics, and failure cases. Cite a source for every
numerical claim and distinguish author claims from your interpretation.
OpenScience can synthesize the material it retrieves, but a polished summary can still contain errors. Check the most important claims against the linked source.
Save a review packet
Ask for a report, a source table, and a bibliography. Keep excluded papers and their reasons when the workflow requires a reproducible screening record.
Suggested outputs:
| File | Contents |
|---|---|
results/review.md | Findings, disagreements, limitations, and open questions. |
results/sources.csv | Included sources and extracted evidence. |
results/references.bib | Bibliographic records you have checked. |
results/search-methods.md | Queries, sources, dates, and inclusion criteria. |
These are requested deliverables, not automatic outputs of every search.
Revisit the review
Reopen the session, refer to the saved source table, and state the new cutoff date. Ask it to report additions and changes instead of silently replacing earlier conclusions.
For a formal systematic review, agree on the applicable screening and reporting process with your team. OpenScience can assist with the work; a search alone does not establish completeness.