To avoid undesirable content in the search indexes, webmasters can instruct spiders not to crawl certain files or directories through the standard robots.txt file in the root directory of the domain. Additionally, a page can be explicitly excluded from a search engine's database by using a meta tag specific to robots (usually ). When a search engine visits a site, the robots.txt located in the root directory is the first file crawled. The robots.txt file is then parsed and will instruct the robot as to which pages are not to be crawled. As a search engine crawler may keep a cached copy of this file, it may on occasion crawl pages a webmaster does not wish crawled. Pages typically prevented from being crawled include login specific pages such as shopping carts and user-specific content such as search results from internal searches. In March 2007, Google warned webmasters that they should prevent indexing of internal search results because those pages are considered search spam.
The length of a title tag that Google will show will vary (it’s based on pixels, not character counts) but in general 55-60 characters is a good rule of thumb here. If possible you want to work in your core keyword, and if you can do it in a natural and compelling way, add some related modifiers around that term as well. Keep in mind though: the title tag will frequently be what a searcher sees in search results for your page. It’s the “headline” in organic search results, so you also want to take how clickable your title tag is into account.
It is important you spread all that real ‘PageRank’ – or link equity – to your sales keyword / phrase rich sales pages, and as much remains to the rest of the site pages, so Google does not ‘demote’ pages into oblivion – or ‘supplemental results’ as we old timers knew them back in the day. Again – this is slightly old school – but it gets me by, even today.
At the moment, I don’t know you, your business, your website, your resources, your competition or your product. Even with all that knowledge, calculating ROI is extremely difficult because ultimately Google decides on who ranks where in its results – sometimes that’s ranking better sites, and sometimes (often) it is ranking sites breaking the rules above yours.
An SEO technique is considered white hat if it conforms to the search engines' guidelines and involves no deception. As the search engine guidelines are not written as a series of rules or commandments, this is an important distinction to note. White hat SEO is not just about following guidelines but is about ensuring that the content a search engine indexes and subsequently ranks is the same content a user will see. White hat advice is generally summed up as creating content for users, not for search engines, and then making that content easily accessible to the online "spider" algorithms, rather than attempting to trick the algorithm from its intended purpose. White hat SEO is in many ways similar to web development that promotes accessibility, although the two are not identical.
The more important concept to keep in mind is that you want to choose keywords that best relate to the content present on a web page and on a website; if you don't have a dog food comparison matrix, then don't bother including comparison-related keywords; you are misleading your users, and certainly not fooling Google. So in an ideal world, you do have a comparison section, a reviews section, and a rankings section, housed on different pages or sections of your site, with each one tagged with the appropriate keywords. Correspondingly, your SEO and PPC search engine marketing efforts should that content by driving review keywords to the review pages and so on.
QUOTE: Each piece of duplication in your on-page SEO strategy is ***at best*** wasted opportunity. Worse yet, if you are aggressive with aligning your on page heading, your page title, and your internal + external link anchor text the page becomes more likely to get filtered out of the search results (which is quite common in some aggressive spaces). Aaron Wall, 2009
to avoid throwing link equity away, you might create HIGH-LEVEL IN-DEPTH TOPIC PAGES on your site and redirect (or use canonical redirects) any related expired content that HAVE INCOMING BACKLINKS, to this topic page (and keep it updated, folding content from old pages, where relevant and there is traffic opportunity, to create TOPIC pages that are focused on the customer e.g. information pages)
Description meta tags are important because Google might use them as snippets for your pages. Note that we say "might" because Google may choose to use a relevant section of your page's visible text if it does a good job of matching up with a user's query. Adding description meta tags to each of your pages is always a good practice in case Google cannot find a good selection of text to use in the snippet. The Webmaster Central Blog has informative posts on improving snippets with better description meta tags18 and better snippets for your users19. We also have a handy Help Center article on how to create good titles and snippets20.
Google knows who links to you, the “quality” of those links, and whom you link to. These – and other factors – help ultimately determine where a page on your site ranks. To make it more confusing – the page that ranks on your site might not be the page you want to rank, or even the page that determines your rankings for this term. Once Google has worked out your domain authority – sometimes it seems that the most relevant page on your site Google HAS NO ISSUE with will rank.
Length of site domain registration; (I don’t see much benefit ON IT”S OWN even knowing “Valuable (legitimate) domains are often paid for several years in advance, while doorway (illegitimate) domains rarely are used for more than a year.”) – paying for a domain in advance just tells others you don’t want anyone else using this domain name, it is not much of an indication that you’re going to do something Google cares about).
But Google isn’t the only reason why keywords are important. Actually, it’s less important, because you should always focus on the user: on your visitors and potential clients. With SEO you want people to land on your website when using a certain search term or keyword. You need to get into the heads of your audience and use the words they use when they are searching.
OBSERVATION – You can have the content and the links – but if your site falls short on even a single user satisfaction signal (even if it is picked up by the algorithm, and not a human reviewer) then your rankings for particular terms could collapse – OR – rankings can be held back – IF Google thinks your organisation, with its resources, or ‘reputation, should be delivering a better user experience to users.
Be sure to re-evaluate these keywords every few months -- once a quarter is a good benchmark, but some businesses like to do it even more often than that. As you gain even more authority in the SERPs, you'll find that you can add more and more keywords to your lists to tackle as you work on maintaining your current presence, and then growing in new areas on top of that.