Robots.txt is not an appropriate or effective way of blocking sensitive or confidential material. It only instructs well-behaved crawlers that the pages are not for them, but it does not prevent your server from delivering those pages to a browser that requests them. One reason is that search engines could still reference the URLs you block (showing just the URL, no title or snippet) if there happen to be links to those URLs somewhere on the Internet (like referrer logs). Also, non-compliant or rogue search engines that don't acknowledge the Robots Exclusion Standard could disobey the instructions of your robots.txt. Finally, a curious user could examine the directories or subdirectories in your robots.txt file and guess the URL of the content that you don't want seen.
For the purposes of our testing, we standardized keyword queries across the five tools. To test the primary ad hoc keyword search capability with each tool, we ran queries on an identical set of keywords. From there we tested not only the kinds of data and metrics the tool gave, but how it handled keyword management and organization, and what kind of optimization recommendations and suggestions the tool provided.
Place strategic search phrases on pages. Integrate selected keywords into your website source code and existing content on designated pages. Make sure to apply a sug­gested guideline of one to three keywords/phrases per content page and add more pages to complete the list. Ensure that related words are used as a natural inclu­sion of your keywords. It helps the search engines quickly determine what the page is about. A natural approach to this works best. In the past, 100 to 300 words on a page was recommended. Many tests show that pages with 800 to 2,000 words can outperform shorter ones. In the end, the users, the marketplace, content and links will determine the popularity and ranking numbers.
Capturing and keeping attention is one of the hardest parts of our job today. Fact: It's just going to get harder with the advent of new technology and conversational interfaces. In the brave new world we're stepping into, the key questions are: How do we get discovered? How can we delight our audiences? And how can we grow revenue for our clients? Watch this session to learn how to make your marketing and advertising efforts something people are going to want to consume.
Crawlers are largely a separate product category. There is some overlap with the self-service keyword tools (Ahrefs, for instance, does both), but crawling is another important piece of the puzzle. We tested several tools with these capabilities either as their express purpose or as features within a larger platform. Ahrefs, DeepCrawl, Majestic, and LinkResearchTools are all primarily focused on crawling and backlink tracking, the inbound links coming to your site from another website. Moz Pro, SpyFu, SEMrush, and AWR Cloud all include domain crawling or backlink tracking features as part of their SEO arsenals.
Once you have your keyword list, the next step is actually implementing your targeted keywords into your site’s content. Each page on your site should be targeting a core term, as well as a “basket” of related terms. In his overview of the perfectly optimized page, Rand Fishkin offers a nice visual of what a well (or perfectly) optimized page looks like:
SEO is not an appropriate strategy for every website, and other Internet marketing strategies can be more effective, such as paid advertising through pay per click (PPC) campaigns, depending on the site operator's goals. Search engine marketing (SEM) is the practice of designing, running and optimizing search engine ad campaigns.[56] Its difference from SEO is most simply depicted as the difference between paid and unpaid priority ranking in search results. Its purpose regards prominence more so than relevance; website developers should regard SEM with the utmost importance with consideration to visibility as most navigate to the primary listings of their search.[57] A successful Internet marketing campaign may also depend upon building high quality web pages to engage and persuade, setting up analytics programs to enable site owners to measure results, and improving a site's conversion rate.[58] In November 2015, Google released a full 160 page version of its Search Quality Rating Guidelines to the public,[59] which revealed a shift in their focus towards "usefulness" and mobile search. In recent years the mobile market has exploded, overtaking the use of desktops, as shown in by StatCounter in October 2016 where they analyzed 2.5 million websites and found that 51.3% of the pages were loaded by a mobile device [60]. Google has been one of the companies that are utilizing the popularity of mobile usage by encouraging websites to use their Google Search Console, the Mobile-Friendly Test, which allows companies to measure up their website to the search engine results and how user-friendly it is.
A variety of methods can increase the prominence of a webpage within the search results. Cross linking between pages of the same website to provide more links to important pages may improve its visibility.[48] Writing content that includes frequently searched keyword phrase, so as to be relevant to a wide variety of search queries will tend to increase traffic.[48] Updating content so as to keep search engines crawling back frequently can give additional weight to a site. Adding relevant keywords to a web page's metadata, including the title tag and meta description, will tend to improve the relevancy of a site's search listings, thus increasing traffic. URL canonicalization of web pages accessible via multiple URLs, using the canonical link element[49] or via 301 redirects can help make sure links to different versions of the URL all count towards the page's link popularity score.
You can confer some of your site's reputation to another site when your site links to it. Sometimes users can take advantage of this by adding links to their own site in your comment sections or message boards. Or sometimes you might mention a site in a negative way and don't want to confer any of your reputation upon it. For example, imagine that you're writing a blog post on the topic of comment spamming and you want to call out a site that recently comment spammed your blog. You want to warn others of the site, so you include the link to it in your content; however, you certainly don't want to give the site some of your reputation from your link. This would be a good time to use nofollow.
When referring to the homepage, a trailing slash after the hostname is optional since it leads to the same content ("https://example.com/" is the same as "https://example.com"). For the path and filename, a trailing slash would be seen as a different URL (signaling either a file or a directory), for example, "https://example.com/fish" is not the same as "https://example.com/fish/".
This can be broken down into three primary categories: ad hoc keyword research, ongoing search position monitoring, and crawling, which is when Google bots search through sites to determine which pages to index. In this roundup, we'll explain what each of those categories means for your business, the types of platforms and tools you can use to cover all of your SEO bases, and what to look for when investing in those tools.
All of this plays into a new way businesses and SEO professionals need to think when approaching what keywords to target and what SERP positions to chase. The enterprise SEO platforms are beginning to do this, but the next step in SEO is full-blown content recommendation engines and predictive analytics. By using all of the data you pull from your various SEO tools, Google Search Console, and keyword and trend data from social listening platforms, you can optimize for a given keyword or query before Google does it first. If your keyword research uncovers a high-value keyword or SERP for which Google has not yet monetized the page with a Quick Answer or a Featured Snippet, then pounce on that opportunity.
The Java program is fairly intuitive, with easy-to-navigate tabs. Additionally, you can export any or all of the data into Excel for further analysis. So say you're using Optify, Moz, or RavenSEO to monitor your links or rankings for specific keywords -- you could simply create a .csv file from your spreadsheet, make a few adjustments for the proper formatting, and upload it to those tools.

LinkResearchTools makes backlink tracking its core mission and provides a wide swath of backlink analysis tools. LinkResearchTools and Majestic provide the best backlink crawling of this bunch. Aside from these two backlink powerhouses, many of the other tools we tested, such as Ahrefs, Moz Pro, Searchmetrics, SEMrush, and SpyFu, also include solid backlink tracking capabilities.
The terms SEO experts often start with are page authority (PA) and domain authority (DA). DA, a concept in fact coined by Moz, is a 100-point scale that predicts how well a website will rank on search engines. PA is the modern umbrella term for what started as Google's original PageRank algorithm, developed by co-founders Larry Page and Sergey Brin. Google still uses PageRank internally but has gradually stopped supporting the increasingly irrelevant metric, which it now rarely updates. PA is the custom metric each SEO vendor now calculates independently to gauge and rate (again, on a scale of 100) the link structure and authoritative strength of an individual page on any given domain. There is an SEO industry debate as to the validity of PA and DA, and how much influence the PageRank algorithm still holds in Google results (more on that in a bit), but outside of Google's own analytics, they're the most widely accepted metrics out there.

In 1998, two graduate students at Stanford University, Larry Page and Sergey Brin, developed "Backrub", a search engine that relied on a mathematical algorithm to rate the prominence of web pages. The number calculated by the algorithm, PageRank, is a function of the quantity and strength of inbound links.[22] PageRank estimates the likelihood that a given page will be reached by a web user who randomly surfs the web, and follows links from one page to another. In effect, this means that some links are stronger than others, as a higher PageRank page is more likely to be reached by the random web surfer.
A website or URL’s ranking for keywords or keyword combinations varies from search engine to search engine. A domain may rank for a certain keyword in the top 3 on Bing, but not even be on the first page of the Google search results for the same keyword. Of course, the same is true of all search engines – Bing, Google, Yahoo and every other search engine uses its own method for calculating rankings and therefore ranks websites differently.
Everyone knows intent behind the search matters. In e-commerce, intent is somewhat easy to see. B2B or, better yet, healthcare, isn't quite as easy. Matching persona intent to keywords requires a bit more thought. In this video, we'll cover how to find intent modifiers during keyword research, how to organize those modifiers into the search funnel, and how to quickly find unique universal results at different levels of the search funnel to utilize.
×