URL to RSS - Custom Curated RSS Feeds, RSS From Any Site

The site you care about most is the one that publishes no feed

Following a supplier's blog, an agency's news section or a competitor's listing usually runs into the same wall: the page offers no RSS, or offers a trimmed one, or changed it without warning. Whoever curates content depends on what others decide to publish as a feed. URL to RSS flips that dependency: you pick the page, you decide which part counts as an article, and the feed lives on your own site.

From link to feed: a walkthrough with URL to RSS

Imagine you want to follow a supplier's blog that has no RSS. This is how the process would go.

Step 1: you give the seed URL

You paste the address of the page that lists the articles. The plugin crawls it, finds the links it contains and visits each one to extract its content. You can limit crawl depth, crawl rate and the maximum number of articles, or keep only links with a specific class or ID.

Step 2: you point at the blocks in the visual builder

Instead of writing selectors by hand, you use the visual selector to mark what is the title, what is the content and what is noise. Here is the first limit: if the site generates its HTML with JavaScript, that code has to be executed. The plugin supports this with PhantomJS installed on your server or with the HeadlessBrowserAPI service, which avoids installing anything else.

Step 3: you set filters to keep junk out

• Discard articles without an image, or with a title or content that is too short or too long.
• Publish only if certain keywords appear, or skip items when others appear.
• Generate categories from the crawled pages.

Step 4: you schedule it and forget it

Each feed gets its own URL, like the feeds your WordPress already has. You can paste it into an RSS reader, into any software that consumes feeds or into your site's code. The content refreshes at the frequency you set, with no manual regeneration, and you can create as many feeds as you need rules.

Common questions before building your first feed

Does it work with any website?

With almost any site you can open in a browser. Those that depend on JavaScript require PhantomJS or HeadlessBrowserAPI, and those that require a login can be handled by sending custom cookies.

What does my server need?

The PHP DOM extension is a requirement. If you plan to run JavaScript with PhantomJS, it must also be installable on your hosting; if it can't be, the external service remains an option.

Can I crawl without getting blocked?

You can set several proxies: the plugin picks one at random on each access. Even so, whether a site accepts your visits is up to that site, not the plugin.

Is it useful for stores?

It supports creating feeds of WooCommerce products.

What you can do to the text before it reaches the feed

Extracted content goes through optional processing: replacing words, translating with Google Translate or rewriting with synonyms using the built-in spinner or other services such as The Best Spinner, SpinRewriter or WordAI. Those external services have their own accounts and terms. You can also customize title and content with shortcodes, add your own tags to each item, generate a featured image or screenshots of the crawled pages, and shorten outgoing links with Shorte.st.

A sensible warning: reproducing other people's content, especially when automatically rewritten, may clash with the copyright of the source site. Use it to curate, quote and link, and check what you are allowed to reuse. Before committing to a particular site, test the rule on its real structure and check in the preview that the chosen blocks read well.

When you set the pace

The plugin keeps a detailed activity log and lets you save and restore the rule list from a file. It is a fine-tuning tool: it rewards whoever spends an afternoon refining each rule.