News Site Article Archive
Last updated November 10, 2026
Name news sites and a past date range and get full article text, dates, and authors as rows. Walks sitemaps and the Internet Archive index across years. No code, no AI, no login. Paywalled pages are left out.
Features
- Walks a site's sitemaps and the Internet Archive index, even years back.
- Full text, published date, title, author, and section per article.
- Pages gone from the site can be read from an archive copy.
- Paywalled and refused pages are free rows with no text.
- Pay per event: Article row at $0.003.
Input
sitesarrayoptionalNews sites by domain, for example techcrunch.com, up to 20.
date_fromstringoptionalThe earliest published date, as YYYY-MM-DD. Default: 30 days before Date to.
date_tostringoptionalThe latest published date, as YYYY-MM-DD. Default: today.
max_articles_per_sitestringoptionalArticles to return from each site, up to 5,000. Default 100.
url_containsarrayoptionalOptional. Keep only addresses with any of these texts, for example /politics/.
min_wordsstringoptionalPages with fewer words are not counted as articles and are free. Default 80.
use_archive_indexstringoptionalyes (default): when the sitemaps run out, also find addresses in the Internet Archive capture index, which reaches years that sitemaps no longer list. no: sitemaps only.
use_archive_copystringoptionalyes (default): a page the site answers with 404, 410, 500, 502, or 504 is read from the Internet Archive, at the archived row price. Refused and paywalled pages are never read from an archive. no: gone pages return a free row.
obey_robotsstringoptionalyes (default): pages robots.txt disallows are skipped and counted. no: read them anyway.
max_charge_usdstringoptionalOptional. The run stops before a charge that would pass this amount. Summary rows are always returned.
Output schema
{
"row_type": "string",
"site": "string",
"url": "string",
"final_url": "string",
"found_from": "string",
"read_from": "string",
"archive_url": "string",
"archive_captured_at": "string",
"http_status": "integer",
"title": "string",
"author": "string",
"section": "string",
"published_at": "string",
"date_source": "string",
"modified_at": "string",
"description": "string",
"language": "string",
"image_url": "string",
"word_count": "integer",
"text": "string",
"date_from": "string",
"date_to": "string",
"candidates_found": "integer",
"articles_returned": "integer",
"articles_from_archive": "integer",
"pages_paywalled": "integer",
"pages_blocked": "integer",
"pages_gone": "integer",
"pages_out_of_range": "integer",
"pages_no_date": "integer",
"pages_not_article": "integer",
"pages_skipped_robots": "integer",
"pages_failed": "integer",
"sitemap_files_read": "integer",
"archive_months_read": "integer",
"archive_months_total": "integer",
"oldest_article": "string",
"newest_article": "string",
"row_status": "string",
"error_reason": "string",
"billable_events_json": "string",
"scraped_at": "string"
}Pricing
| Event | Free plan price |
|---|---|
| Article row | $0.003 |
| Archived article row | $0.003 |
| Actor Start | $0.021 |
Pay per event on the Apify platform. Paid Apify plans pay less per event. No subscription, no seat license.
On the Apify free plan the actor stops at $1.50 of charges a month, about 500 articles. Any paid Apify plan removes the cap.
Limits
- Coverage depends on the site. Sites behind an anti bot wall (for example a Cloudflare challenge) refuse the proxy pool and return free rows. In testing, apnews.com and nytimes.com article reads were refused, so those sites return little. The actor does not solve challenges, and it does not use residential addresses or a browser to get past one.
- Paywalled text is not returned, and no archive copy is read in its place. Pages that read as paywalled by a marker or a phrase are judged by fixed rules, so a rare one can be missed or wrongly flagged.
- Article addresses are only as complete as the sitemaps and the Internet Archive's captures. An article with neither is not found, and one first captured after the range ends is found only if a sitemap lists it. Use the summary row counts to judge a run.
- The capture index is a public service that can be slow or busy. A month takes 10 to 30 seconds, and 10 years is 120 months per site, so a long range needs a long run timeout. Up to 12 sitemap files are read per site, and up to 40 month pages per month.
- With several sites, each takes a fair share of the run's time and the summary row says when a site used its share. Raise the run timeout for long ranges.
- Oldest months are read first, so a low maximum returns the oldest articles in the range, not a spread across it.
- No browser. A site that builds its articles with JavaScript in the visitor's browser returns little or nothing, and those pages count as not articles.
- A page with no published date on it or in its address cannot be placed in the range, so it is counted and not returned. A date taken from the address gives the day only.
- A run reads up to 5 times the maximum articles plus 100 pages per site, then stops that site and says so.
- Publishers' terms differ, and several state limits on automated collection or reuse, including for AI training. How you use the text is yours to check.