HAPPILY IMPERFECT

AI is reading my content

At least that’s my guess from looking at all the single page views all across my blog, in a somewhat random manner.
Is this a good thing, or a bad thing?
Given I willingly publish my content to the public World Wide Web, can I really complain?
If I chose to, can I opt-out? If so, how?
It’s been on my mind for a while now but I’m still just bumbling along.

Similar musings from the archive

Category:

One response to “AI is reading my content”

  1. Tom Brandis avatar
    Tom Brandis

    Well you could use your robots.txt file (https://www.gordonmclean.co.uk/robots.txt) to tell scrapers not to read your pages, however in my experience this is ignored by many of the worst offenders anyway.

    More reliable ways of blocking bots include proof-of-work (e.g. Anubis https://anubis.techaro.lol/) or checking if the browser can set a cookie or run javascript (most bots can’t). Do bear in mind that this also blocks browsers with js/cookies disabled including some reader modes.

What are your thoughts?

This site uses Akismet to reduce spam. Learn how your comment data is processed.