AI (Artificial Intelligence) technology has made tremendous progress in recent years. Its ability to perform specific tasks, such as generating text, images and sounds, can now be assessed.
Now, if we go further to more complex tests like evaluating jobs, or more specifically evaluating AI systems Ability to do SEO?
Below, we will test the Generative Pre-trained Transformer 3 (GPT-3) created by OpenAI.
Let’s remember that an AI system will mimic the data it was trained on.
SEO was built with the advancement of search engines and everything is well documented in blogs, books and interviews.
So AI might learn from all the SEO content available.
Therefore, the test will measure the following metrics:
- Continuing SEO theme consistency.
- Relevance of SEO Modifications.
- Ability to answer SEO questions.
Continuing SEO theme consistency
The following tests will focus on the AI system’s ability to understand the SEO situation and its ability to coherently address the topic in an understandable and consistent manner.
The first test will take examples that talk about SEO horror stories—that is, short stories that solve SEO disasters—and see how well the AI performs.
I took two examples from Aleyda Solis’s website, which are AI generated:
Two examples:
- When all internal pages of a website are canonicalized as the home page. #seohorrorstories
- When someone forgets to block the site test environment…starts getting indexed. #seohorrorstories
Results generated by OpenAI’s GPT-3 AI:
- When someone says “I have a great idea! Let’s put our site in the footer of an unrelated site!” #seohorrorstories
- When someone forgets to check the staging environment’s robots.txt before going live. #seohorrorstories
- When someone makes changes to .htaccess without telling anyone #seohorrorstories
- When we switch domains and have not forwarded the old page for over 60 days. #seohorrorstories
- When customers create thousands Thin content pages no value #seohorrorstories
- When client insists on using relational database table as URL (?tbl=table&id=2) #seohorrorstories
Analysis: The generated text is varied and coherent.
Also, after some research, we can conclude that they are unique.
So the AI understood that it had to mimic a short piece of text talking about SEO and added the hashtag #seohorrorstories each time.
Here’s a screenshot of the full example; we can generate an endless stream of #seohorrorstories.
Regarding creativity, we can go a step further in the test and try to measure the consistency of creating a conversation between two SEO experts.
For the next test, we’ll give the AI a context and instructions, and let it generate a discussion.
In this case, the AI will have to create a dialogue between two experts – one experienced, one beginner: when someone forgets to block the site test environment…starts to be indexed.
Results generated by OpenAI’s GPT-3 AI:
Aleda: Oh no! When did you realize this happened?
Paul: I just checked Google Analytics this morning and found that we have some traffic coming from google.com.
Aleda: That’s not good. Have you checked that the site test environment is still being indexed?
Paul: Yes, I did. it is.
Aleyda: Okay, we need to fix this right away. First, let’s check if any pages have been indexed.
Paul: There are several.
Aleda: Great. Now we need to block the test environment so that Google stops indexing it.
Paul: How do we do it?
Aleyda: We can add a robots.txt file to the test environment. This will tell Google not to index any pages on the site.
Paul: Okay, that makes sense. Let’s do it.
Screenshot of OpenAI, April 2022Analysis: Conversations still very coherent
The AI system even explained how the problem was detected and suggested a solution to the problem of indexing pre-production sites with robots.txt.
Next, let’s take one of the #seohorrorstories invented by AI and generate relevant conversations.
This is the new situation: when clients create thousands of pages of worthless stripped-down content.
Results generated by OpenAI’s GPT-3 AI:
Aleyda: Well, that’s a bit of a problem. You create a lot of pages with thin content that don’t really provide any value to the user.
Paul: Yes, I know. I just want to cram as many keywords as possible into the site.
Aleyda: Unfortunately, this won’t help your SEO. In fact, it can actually hurt your rankings.
Paul: So what should I do?
Aleyda: You need to create higher quality content that provides value to users. This will help you rank higher in search engines.
Screenshot of OpenAI, April 2022Analysis: AI again manages to create real conversations by explaining why creating thin content doesn’t work for SEO or even recommending solutions.
We can conclude that the AI system can talk strictly and accurately about SEO, but can it also make relevant SEO modifications to the text?
As it happens, GPT-3 offers a text editing mode that will be available since March 2022.
Let’s test it out.
Relevance of SEO Modifications
The first test will ask the AI to replace the link anchor with something more marketable to better highlight the product.
Results generated by OpenAI’s GPT-3 AI:
Screenshot of OpenAI, April 2022Analysis: The test was relatively successful.
Here, with context in mind, AI replaces “Redwood Steakhouse” with “Best Steak in New York.”
This solution works regardless of the number of links.
Next, the second test consisted of doing two tasks at the same time: asking the AI to change the link anchor, but add an H1 tag with a title to the beginning of the document.
Results generated by OpenAI’s GPT-3 AI:
Screenshot of OpenAI, April 2022Analysis: The test was successful again and the results can be applied to thousands of documents. The title of the summary document is added to the correct location.
Finally, we will ask the AI to use
- The label adds a list of three related links.
Screenshot of OpenAI, April 2022Analysis: Once again, the format is well respected and the anchors remain consistent.
These links are fictitious, but GPT-3 can easily be selected from the link database to use real links.
In short, GPT-3 is good at editing text according to the context given the correct instructions. This can be an important time-saver in terms of large-scale revisions.
Ability to answer SEO questions
Now we are going to test the SEO knowledge of an artificial intelligence system.
I choose to use BrightEdge’s SEO Testbut any test can be used:
As with every previous test, we used GPT-3 OpenAI technology to answer the questions in the SEO MCQ. The number of correct answers measures test quality.
Screenshot of OpenAI, April 2022Below, we’ll see how each question is presented to the AI; the text highlighted in green represents its chosen answer.
BrightEdge provides a table of questions with correct answers in one column and AI-selected answers in adjacent columns:
Screenshot by author, April 2022The GPT -3 results were encouraging: the AI scored 16/20, and even then it lacked context about what went wrong.
Now, let’s ask the AI what Google has recently said about Publish Google MUM May 18, 2021.
Screenshot of OpenAI, April 2022What is the primary purpose of Google MUM?
Again, as you can see, the AI answers the question perfectly.
In short, we have a truly responsive SEO engine if asked very specific questions.
limitation
Now, we can look at the limitations of AI-driven SEO.
Here is a non-exhaustive list:
The ability of artificial intelligence to fix unknown or poorly documented problems.
Often, in the SEO business, understanding what’s actually going on when a problem occurs is time-consuming, tedious, and sometimes never explained. AI cannot find solutions for data that is not part of its analysis.
The ability to take into account everything that exists.
There are often many restrictions on what has been done or tested on a site or group of sites. Therefore, it is currently impossible to provide an AI system with all source code and project documentation to find an accurate answer that takes into account all existing information.
Ability to measure the true quality of a content or idea by its originality or its representation of the context of the current situation.
AI systems do not yet work in real time and lack ingenuity and creativity. So the real measure of content quality is minimal at the moment. Humans are often the ones evaluating quality content. AI often detects spam and poor quality content.
All in all, AI systems can outperform novice SEOs or people with very little SEO knowledge, but they still have a long way to go to replace seasoned SEO experts.
Overall, the AI and human duo will always remain the best combination, with humans focusing on tasks with the highest added value.
In conclusion, all seasoned SEO experts are far superior to AI SEO.
More resources:
Featured Image: Shutterstock/metamorworks
