An web optimization competitor evaluation helps you perceive the place and why your rivals are successful in search.
On this article, I’ll present you easy methods to analyze your rivals quicker by combining Semrush knowledge with the facility of AI instruments.
Tips on how to arrange the Semrush MCP
The Semrush MCP (Mannequin Context Protocol) connects your AI device to your Semrush account to supply entry to useful aggressive evaluation knowledge. This allows you to analyze competitor domains, key phrases, and backlinks.
Join the Semrush MCP to Claude by clicking the “+” icon in any chat. Then click on “Connectors” > “Add connector” > “Browse connectors.”
Seek for Semrush and click on the “Join” button subsequent to the Semrush connector. Then, observe the on-screen directions to attach your Semrush account.
The Semrush connection makes use of API models while you immediate Claude to entry Semrush knowledge through the MCP. The prompts under are designed to reduce API unit utilization whereas nonetheless offering helpful aggressive evaluation knowledge.
You may see your API models in your Semrush account underneath “Subscription data” > “API Items.” I ran the prompts under and used round 16K API models, however it will range relying on how a lot knowledge you request per run, the way you phrase the prompts, and what number of you ship.
I’ve instructed numbers to make use of in every immediate as a place to begin. I like to recommend protecting these numbers to start with and solely scaling up when it is advisable to, as it will assist maintain your utilization low.
Subsequent, click on the “Initiatives” choice on the left after which click on the “New mission” button on the high right-hand nook.
Making a mission will allow you to add context information (see under) and maintain all of your competitor evaluation duties in a single place.
Give your mission a reputation and an outline, then click on “Create mission.”
Subsequent, create a enterprise context file to offer Claude details about your corporation to assist enhance its outputs. Embody details about your corporation’s trade, any particular merchandise you promote, and the subjects you already cowl in your web site.
Use the immediate under to let Claude construct your context file for you, with the assistance of Semrush’s MCP so as to add your present natural search efficiency knowledge:
Create a enterprise context file for [yourdomain.com] that I can add to this mission. Goal nation: [country].
Essential details about my enterprise: [What you sell, who you sell to, competitors you know of, topics you want to rank for, topics to avoid, and your content resources (team size, budget, whether you can do original research or build tools)]
Use Semrush knowledge so as to add details about my enterprise’s natural visitors, natural key phrases, Authority Rating, referring domains, high 10 pages by visitors, and the principle subjects we already rank for.
Write it in Markdown. Add “Final up to date: [today’s date]” on the high. Label something you inferred with “(inferred)” so I can examine it, and ask me about something you possibly can’t fill in.
Right here’s an instance output of this context file:
Obtain this file, then navigate to your mission and click on the “Library” icon on the highest right-hand aspect of your mission window, choose “+ Add” within the Library, click on “Add information,” and add the context file you simply downloaded.
Your Claude mission will use this enterprise context for every of the 4 use instances under to return extra related competitor, key phrase, matter, and backlink outreach steering.
4 methods to make use of the Semrush MCP for web optimization competitor evaluation
Observe the 4 steps so as under to make use of the Semrush MCP for a complete web optimization competitor evaluation. Or, use any of the strategies individually at any time to investigate your rivals.
The prompts assume you’ve created a mission and added a enterprise context file from the earlier part. In the event you haven’t, modify the prompts accordingly.
I like to recommend operating every step in the identical chat, or pasting in outcomes from earlier chats. This provides Claude the context from every step to supply higher outcomes.
1. Determine and prioritize rivals
Step one of any web optimization competitor evaluation is to establish and prioritize which rivals to evaluate. You probably have a few rivals in thoughts, nevertheless it’s necessary to focus in your closest rivals to make sure you benefit from your sources.
Use the immediate under to establish and prioritize your rivals to investigate in steps two, three, and 4, based mostly on their search efficiency, key phrase overlap together with your area, and extra:
Utilizing Semrush knowledge for [country], discover the highest 10 natural rivals for [yourdomain.com]. Add any enterprise rivals from my enterprise context file that are not already on the record.
Solely embody actual rivals. Exclude boards, social platforms, video platforms, and reference websites (e.g., Reddit, YouTube, Wikipedia), even when they share numerous key phrases with us.
For [yourdomain.com] and every competitor, pull:
- Natural visitors
- Natural key phrases
- Widespread key phrases with [yourdomain.com]
- Key phrase overlap % (frequent key phrases divided by [yourdomain.com]’s complete natural key phrases)
- Authority Rating
- Referring domains
- Natural visitors development during the last 12 months (rising, flat, or declining)
If my enterprise context file already has these figures for [yourdomain.com], use these.
Return one desk with [yourdomain.com] as the primary row. Embody these columns: Competitor, Kind (Direct competitor or Content material competitor), What they do (one line), and the metrics above.
Beneath the desk, advocate 3-4 rivals to investigate within the subsequent steps, with a one-sentence purpose for every.
Right here’s an instance of the desk output for the beard care model Mad Viking Beard Co:
Claude then supplied some context, together with the highest 4 rivals based mostly on enterprise relevance, key phrase overlap, and web site authority:
Use the record of prioritized rivals for the following three prompts.
2. Analyze competitor key phrases
Analyze your rivals’ key phrases utilizing AI to discover content material gaps you’re lacking, and to shortly examine whether or not you must create a brand new piece of content material to fill that hole, or optimize an current web page.
Use the immediate under to search out key phrases your rivals rank for that you just don’t, and phrases they rank for larger than you, and to arrange them into matter clusters:
Utilizing Semrush Key phrase Hole knowledge for [country], examine [yourdomain.com] with the rivals recognized in Step 1.
Half 1:
Discover key phrases that a number of rivals rank for within the high 20 however [yourdomain.com] does not rank for in any respect. Restrict to the highest 10 by search quantity.
Half 2:
Discover key phrases the place [yourdomain.com] ranks 11-50 and no less than one competitor ranks within the high 10. Restrict to the highest 10 by search quantity.
Guidelines:
- Run every half as a single Key phrase Hole name masking all rivals without delay. Do not run rivals separately.
- Type by search quantity and request 10 rows at a time. Solely request the following 10 if fewer than 10 key phrases qualify after making use of the principles under.
- “High 10” means the highest 10 throughout all rivals mixed, not per competitor.
- To examine for current pages, pull [yourdomain.com]’s record of rating pages as soon as and use it for each cluster.
- Omit branded key phrases.
- Do not embody key phrases that are not related to my enterprise, based mostly on my enterprise context file.
Group the key phrases into matter clusters. Every cluster ought to share a subject and search intent, so one web page might goal it.
For every cluster, examine whether or not [yourdomain.com] already has a web page focusing on it.
Return one desk with these columns: Cluster, Major key phrase, Secondary key phrases, Whole search quantity, Common key phrase problem, Intent, Opponents rating, Our current web page (or “None”), Motion (Create new or Replace current), Precedence (Excessive, Medium, or Low).
Set precedence based mostly on relevance to my enterprise and potential to drive conversions. Inside every precedence stage, kind by search quantity and key phrase problem.
This step is often the heaviest when it comes to API utilization, so I like to recommend limiting the variety of key phrases that the device analyzes throughout all rivals to 10 as a place to begin.
As soon as you’ve got examined the immediate and made certain you get first rate outcomes, you possibly can then improve this quantity.
Beneath is an instance output desk with the subject clusters, key phrases, and related metrics.
You may see within the current web page column that even the place there isn’t any current web page, Claude highlighted the closest various. For instance, within the first row, Claude flagged the collections web page for beard oil, though there isn’t any web page particularly for “beard progress oil.”
Additional down, Claude supplied notes on the key phrase choice. The place a number of key phrases had the identical quantity, Claude seemed on the enterprise context file and opted for “beard oil vs balm” because it was most related to one of many merchandise laid out in that file.
That is an instance of how including context to your mission offers the AI device extra to work with than a easy seed key phrase. In flip, the AI device is ready to use its reasoning capabilities to make choices tailor-made to your model, quite than simply return a listing of generic phrases.
3. Discover and analyze competitor content material that performs effectively
Analyze your rivals’ top-performing content material to grasp how one can enhance your personal content material’s search efficiency.
Use the immediate under, together with the key phrases recognized in step two, to search out the pages that drive rankings in your rivals:
For every of the highest 3 precedence clusters from Step 2:
1. Use Semrush to search out the highest-ranking competitor URL for the first key phrase in [country]. If two of my rivals rank within the high 10, analyze each pages. Ignore discussion board outcomes.
2. Go to every web page utilizing your internet fetch device. Word:
- Content material kind (e.g., how-to information, listicle, comparability, template, free device, product or class web page)
- Angle and goal reader
- Fundamental sections (H2s)
- Approximate size
- Standout parts, equivalent to authentic knowledge, examples, visuals, templates, instruments, skilled quotes, or video
- Final up to date date, if proven
3. Pull the highest 10 natural key phrases for every URL from Semrush. Flag any that weren’t within the Step 2 cluster.
4. If [yourdomain.com] already has a web page for this cluster, go to it and pull its high 10 key phrases too. Examine it aspect by aspect with the competitor web page(s) and establish: subtopics they cowl that we do not, key phrases they rank for that we do not, and variations in format, depth, and freshness.
For every cluster, give me:
- A abstract desk of the pages you analyzed
- What the top-ranking pages have in frequent
- If we’re creating a brand new web page: a brief temporary with goal key phrase, secondary key phrases, really helpful content material kind, instructed H2s, and 1-2 methods we might make our web page extra helpful than theirs, based mostly on my enterprise context file
- If we’re updating an current web page: a prioritized record of modifications
This immediate will yield a number of outputs. The primary of those is a listing of pages inside a particular cluster, together with details about the content material itself and why the AI device thinks it performs effectively.
For instance, for madvikingbeard.com, the primary cluster was for the time period “finest beard oil.” Claude really helpful updating an current web page based mostly on its evaluation of the competitor web page that ranks effectively for this time period.
Claude’s evaluation included details about the angle of the piece of content material, what the principle sections seemed like, how lengthy it was, and even particular design choices on the web page. This supplies a superb start line for you or your group to create or replace your personal piece of content material.
Additional down, Claude outlined the highest 10 key phrases for the competitor web page, together with place, quantity, and key phrase problem metrics. It then summarized the rankings and defined why focusing on the key phrase with a class web page is sensible over a weblog submit.
Whereas the ultimate resolution nonetheless rests with you (the human), this evaluation, carried out robotically and at scale, makes it a lot simpler to shortly categorize clusters into new content material and updates to current pages.
One factor I seen on this run is that Claude checked out two totally different pages on madvikingbeard.com’s web site: a weblog submit (which can at first look like a superb candidate for the key phrase) and the collections web page.
On this case, because the competitor’s rating web page is a group web page, that is the one Claude really helpful updating. Claude then carried out a side-by-side evaluation of each assortment pages, outlining subtopics that the competitor targets that Mad Viking doesn’t, and even particular details about the product playing cards on the web page and which FAQs to make use of.
One of many particular suggestions was so as to add scent notes to each product card. I went to the collections web page on the competitor’s web site to confirm that the rival was truly doing this.
I counsel visiting competitor pages your self to confirm any particular suggestions the AI device makes. Keep in mind that that is only a device designed to flag potential actions you might wish to take. Don’t skip the human evaluation step earlier than making any choices.
4. Have a look at your rivals’ backlinks
Have a look at your rivals’ backlinks to grasp which of their pages earn hyperlinks from probably the most referring domains. These are the pages that could be contributing probably the most to your rivals’ authority and serving to them present up in search.
Use AI to take your backlink evaluation additional by automating the analysis and prioritization stage utilizing the immediate under:
Utilizing Semrush backlink knowledge, discover the highest 20 pages on [competitor.com] by variety of referring domains. Solely embody URLs in [e.g., /blog/], and skip the homepage.
For every web page, be aware the variety of referring domains and backlinks. Then go to the web page utilizing your internet fetch device and be aware the content material kind (e.g., authentic analysis, statistics roundup, free device, template, information, definition, case research) and what it covers in a single sentence.
Return one desk with these columns: Web page, Referring domains, Backlinks, Content material kind, What it covers.
Then discover 5 pages we might realistically create or enhance a model of, based mostly on my enterprise context file. Work via the desk, beginning with the most-linked, till you have got 5. For each, examine whether or not [yourdomain.com] has a web page on the identical matter and advocate:
- Create: we do not have a web page on this. Say what we might have to make a greater one (e.g., authentic knowledge, design assist).
- Enhance: we now have a web page on this. Say what it might have to compete.
In the event you run out of pages earlier than discovering 5, inform me what number of you discovered.
Here is an instance output for considered one of madvikingbeard.com’s rivals, Stay Bearded. You may see it organizes the highest 20 pages by referring domains and notes the content material kind, together with a short clarification of what it covers.
The output is helpful for getting at-a-glance details about your rivals’ pages that get probably the most hyperlinks, past simply numbers. When performing hyperlink evaluation at scale, that is a lot quicker than visiting each web page manually to be taught what they cowl.
Additional down, Claude supplies suggestions on pages to create or enhance, checking your web site’s rating pages and drawing on your corporation context file. For instance, it recommends making a web page on beard errors, which earned hyperlinks from probably the most referring domains on the competitor’s web site.
Claude additionally supplies steering to create a greater model of the web page by backing it with authentic knowledge, together with before-and-after illustrations, and through the use of the tone it recommends.
Madvikingbeard.com truly already had a submit on this matter, however Claude didn’t decide it up because it doesn’t have any rankings within the Semrush knowledge. To account for this, I like to recommend manually importing a listing of all of your web site’s pages to your Claude mission.
Because the web page doesn’t rank, the suggestions would nonetheless apply to enhancing the present web page, quite than creating a brand new one.
I would not make any modifications to your web site simply based mostly on the steering of the AI device. However this can be a very fast option to flag pages you could wish to create or enhance, with out having to manually search via your rivals’ pages and match them up with your personal pages.
You may carry out this evaluation frequently and maintain your corporation context file updated with your personal new pages as you add them.
Flip these steps into an interactive dashboard
A ultimate optionally available step is to show the competitor analyses above right into a single interactive dashboard. That is useful in your personal group, nevertheless it additionally makes it straightforward to share your plans and why you are going to take these actions with key stakeholders and different members of your group.
Use the immediate under to show the evaluation into an interactive dashboard inside Claude:
Flip the outcomes from this [conversation/project] into an interactive dashboard with 5 tabs: Opponents, Key phrase clusters, Content material evaluation, Linkable belongings, and Motion plan.
Make each desk sortable. Within the Motion plan tab, mix the suggestions from every step into one record with these columns: motion, web page or matter, effort (Low, Medium, or Excessive), affect (Low, Medium, or Excessive), and the step it got here from. Type by affect, then effort.
Present the date the information was pulled within the header. Solely use numbers from this dialog.
Beneath is an instance output for madvikingbeard.com. You may see a abstract of metrics like natural visitors and the way that modified yr over yr, together with details about the highest 10 rivals under that. Holding the dashboard updated is so simple as prompting Claude every month to replace the numbers, because it makes use of Semrush knowledge.
The “Key phrase clusters,” “Content material evaluation,” and “Linkable belongings” tabs all present useful summaries of the findings of every related step. Nevertheless, the “Motion plan” tab is crucial, because it supplies a prioritized record of actions that you just or your group might take based mostly in your AI web optimization competitor evaluation.
It estimates the trouble and affect stage that can assist you prioritize duties. And since it is interactive, you possibly can kind by whichever column is most necessary to you.
Carry out web optimization competitor evaluation at scale with the Semrush MCP
Utilizing AI for aggressive evaluation is all about performing analyses extra effectively. You continue to want a human to make the choices based mostly on the information you discover, however gathering insights is a lot quicker while you join your favourite AI device to a dependable knowledge supply like Semrush.
Begin performing your personal AI aggressive evaluation utilizing the Semrush MCP right now.
