Interlinking

Have Byword link new articles through to the rest of your site

This is an advanced feature within Byword. It's highly recommended to read through this page, and in particular Usage requirements prior to use.

One area of content writing where AI approaches typically fall down is in integrating content naturally with the rest of your site. Where human writers will typically interlink their articles through to the rest of your site, AI tools often have difficulty doing so.

This is where Byword's interlinking feature comes in. It allows you to save sitemaps, and have Byword insert links to those sitemaps when writing new articles.

Setting up interlinking

To get started, head to the interlinking page. There, you'll be asked to provide a URL for a sitemap which you'd like to work with, along with any URL rules that you'd like to apply.

URL rules

If you'd like Byword to use the entirety of your sitemap, you can ignore the two URL fields below the sitemap input. If you want Byword to just look at specific URLs inside your sitemap however, you can use these fields to specify these URLs.

If you provide one string to 'URL must contain', then Byword will filter for URLs that contain that string. For example, if you just want Byword to look at your blog posts, you could insert /blog/ (or the appropriate URL string) in this field. If you want Byword to look at multiple URL strings, you can put these in a comma separated list (like in the image above), and Byword will scan any URLs that match either of the strings provided.

'URL must not contain' works similarly, but in reverse. Byword will exclude any URLs which contain a string provided in 'URL must not contain'. This is useful if you don't want Byword to link your articles through to any irrelevant pages, for example contact or support pages.

Sitemap indexes

Byword also works fine for sitemap indexes. That is, if your sitemap is actually a collection of sitemaps, Byword will scan and index all of your sub-sitemaps.

Scan your sitemap

Once you're ready, hit 'Scan Sitemap', and Byword will start looking through the sitemap you provided above. This should take less than a minute.

Once finished, you'll see a summary of Byword's scan results. It'll tell you how many URLs it found, and let you see an example of those URLs in a Google Sheet (just to check that any rules you applied have worked as intended).

You might notice Byword telling you that it's sampled your URLs. This will happen if there are a large number of URLs that meet your criteria; Byword will randomly sample from these URLs to save on processing and compute costs. This will happen for sitemaps with more than 2,500 URLS for regular users, and for sitemaps with more than 5,000 URLs for Unlimited users.

Once you're happy with your scan results, hit 'Index Sitemap'. Byword will then add your sitemap to the indexing queue, and you'll be given an ETA. What Byword will do at this point is to visit each URL within your sitemap, read and understand the content, and save a condensed version of that page to its database ('indexing'). This can take some time, especially when the servers are busy, and will also send crawler traffic to your sitemap's URLs.

Once your sitemap has been indexed

Once your sitemap has finished being crawled and indexed, you'll see it appear at the top of your interlinking page as follows:

You can rename your sitemap to something more legible by clicking on the URL field. This is particularly useful if you have two sitemaps in Byword from the same sitemap URL, which have different URL rules applied.

You can also validate the URLs in the sitemap by clicking on the number of URLs that appear.

Sitemap settings

Below your list of sitemaps, you'll notice some settings that you can configure.

The first option, maximum links per article, sets an upper limit on the number of links that Byword will insert into each article. This isn't a guarantee, and the number of links in each article may be lower than that set here, but Byword will do its best to hit that number.

The second option, currently active sitemap, affects which sitemap Byword associates with your articles. You'll also have the option to change this pre-generation on the single and batch generator pages.

Note that what matters for article generation is the sitemap selected at the time the article was sent to the generator. If you request a batch of articles with Sitemap A selected, then immediately change your active sitemap to Sitemap B, the articles will still generate with links to Sitemap A.

Interlinking rules

Byword inserts links into your articles based on similarity. This is good for pointing readers to related content, but can cause complications if you want to send readers to specific pages. To achieve this, you can use custom rules.

Custom rules let you tell Byword: whenever {keyword} appears, link it to {url}.

You can add custom rules on the interlinking page, so long as you have one or more sitemaps in your account.

Once you add a custom rule, you'll see it appear below the input field. You can add multiple keywords to a rule by clicking on a rule's keyword, and adding additional keywords on new lines:

When multiple keywords are added to a rule, Byword will replace occurrences of any of those keywords with links to the URL on the right. If one keyword is triggered (e.g. 'financial analysis', in the example above), no other keywords will be triggered in that rule (even if 'financial forecast' is found in the article, it won't be replaced with a link).

Some important notes about how keywords work:

  • Byword will look for naive pluralisations. For example, the keyword 'financial forecast' in the example above will also trigger if 'financial forecasts' is found in the article (meaning you don't have to add the latter as a separate keyword).

  • Similarly, Byword will also look for other semantically similar words. For example, the keyword 'financial forecast' will also be triggered if 'financial forecasting' is found in the article.

  • Byword will never insert more than the specified number of links per article (which by default is 5). Byword will prioritise inserting links via custom rules, over links which it has found based on similarity:

    • Example 1: If you have 5 custom rules set, and all are triggered in an article, then (assuming your max links per article is set to 5), Byword will insert all five URLs in the triggered custom rules (and none based on article similarity).

    • Example 2: If you had 6 rules triggered instead of 5, but your max links per article is still 5, Byword will randomly choose 5 of your 6 rules to use for link insertion.

Usage requirements

There are a few important things to bear in mind before using interlinking:

  • Your sitemap should have a sufficiently high number of links in it that are relevant to the articles you're generating. You should exercise caution when using this feature with sitemaps that contain fewer than 100 URLs, and abstain where your sitemaps don't contain any URLs that are relevant to your topic area (Byword will still try to insert links).

  • The URLs in your sitemap need to be descriptive.

    • Do use interlinking if your URLs look like https://example.com/how-to-bake-bread

    • Don't use interlinking if your URLs look like https://example.com/post/13412

  • You should be careful when including programmatic article URLs in your sitemaps. This is because Byword's interlinking feature works by finding articles which are most similar to the one that its currently writing. Including programmatic article URLs can result in situations where Byword links through to multiple programmatic pages of the same structure, which can lead to a poor reader experience.

The perfect sitemap therefore is one which:

  • Contains a large number of URLs, ideally a few hundred to a few thousand.

  • Has URLs which are descriptive (e.g. https://example.com/how-to-bake-bread).

  • Doesn't contain programmatic articles (or has them filtered out, via URL rules).

Unlimited users

Interlinking is available to all users, except for those on Unlimited accounts that do not currently have GPT4 API keys. Unlimited users whose accounts don't support GPT4 will be able to create sitemaps, but Byword won't be able to insert links into their articles.

It should also be noted that interlinking can significantly increase costs per article, by between $0.10-$0.20 per article. This can vary between accounts however, and users are advised to test the impact on their OpenAI costs before generating high volumes of interlinked articles.

Frequently asked questions

Why is it taking so long to index my sitemap?

Byword uses a very computationally expensive approach to index sitemaps, which involves scanning each page and translating it to a numerical representation (an embedding vector) of its content. The trade-off for using what is a fairly advanced approach is that it takes time.

The interlinking page will give you live updates on its ETA, as well as a percentage completion figure when your sitemap starts processing.

Why has the number of URLs decreased between scanning and indexing?

Scanning returns the total number of URLs that Byword can find in your sitemap. Byword might end up indexing fewer URLs, because Byword may skip some. Byword will skip indexing of a URL if:

  • It's too short, and so doesn't make for a good interlinking candidate.

  • Your server refuses the request, or it times out.

I asked Byword to insert {number} of links per article, but sometimes it inserts fewer

The number of links you specify per article is an upper limit, and Byword will always try to hit that. It may come under that limit in some cases, where Byword isn't able to appropriately insert all links.

Last updated