SEO is not an appropriate strategy for every website, and other Internet marketing strategies can be more effective, such as paid advertising through pay per click (PPC) campaigns, depending on the site operator's goals. Search engine marketing (SEM) is the practice of designing, running and optimizing search engine ad campaigns. Its difference from SEO is most simply depicted as the difference between paid and unpaid priority ranking in search results. Its purpose regards prominence more so than relevance; website developers should regard SEM with the utmost importance with consideration to visibility as most navigate to the primary listings of their search. A successful Internet marketing campaign may also depend upon building high quality web pages to engage and persuade, setting up analytics programs to enable site owners to measure results, and improving a site's conversion rate. In November 2015, Google released a full 160 page version of its Search Quality Rating Guidelines to the public, which revealed a shift in their focus towards "usefulness" and mobile search. In recent years the mobile market has exploded, overtaking the use of desktops, as shown in by StatCounter in October 2016 where they analyzed 2.5 million websites and found that 51.3% of the pages were loaded by a mobile device . Google has been one of the companies that are utilizing the popularity of mobile usage by encouraging websites to use their Google Search Console, the Mobile-Friendly Test, which allows companies to measure up their website to the search engine results and how user-friendly it is.
Today, however, SEM is used to refer exclusively to paid search. According to Search Engine Land, Search Engine Marketing is “the process of gaining website traffic by purchasing ads on search engines.” Search Engine Optimization, on the other hand, is defined as “the process of getting traffic from free, organic, editorial or natural search results.”
What’s your audience searching for? – Just a few years ago, the average user didn’t trust search engines to understand conversational questions. They were searching with clunky phrases like “flower delivery new york.” Now people feel comfortable typing in things like “who delivers roses near me?” Changes in searcher habits are usually subtle, but will affect which keywords will be most valuable for your site. Instead of focusing on keywords that get you more traffic, focus on those that translate into conversions, revenue and profits.
At Yoast, we practice what we call ‘holistic SEO‘. This means that your primary goal should be to build and maintain the best possible website. Don’t try to fool Google, but use a sustainable long-term strategy. Ranking will come automatically if your website is of extremely high quality. Google wants to get its users to the right place, as its mission is to index all the world’s online information and make it universally accessible and useful.
So if you think about it, SEO is really just a process of proving to search engines that you are the best site, the most authoritative, the most trusted, the most unique and interesting site that they can offer to their customer - the searcher. Get people to talk about you, produce good quality content, get people to link to you, and Google will be more confident that you are the best result that they can offer to their searchers, and that’s when you will start ranking on the first page of Google.
You may not want certain pages of your site crawled because they might not be useful to users if found in a search engine's search results. If you do want to prevent search engines from crawling your pages, Google Search Console has a friendly robots.txt generator to help you create this file. Note that if your site uses subdomains and you wish to have certain pages not crawled on a particular subdomain, you'll have to create a separate robots.txt file for that subdomain. For more information on robots.txt, we suggest this Webmaster Help Center guide on using robots.txt files13.