Introduction
If you publish a web site, whether it be for commercial, information, or hobby purposes, then you’ll likely want to gain as many visitors as you can. One invaluable method of attracting visitors is through search engines. In order to get your pages ranking highly on search engine results pages (or SERPS), you can use a technique called “Search Engine Optimisation” (or SEO). There are many things you can do to push your page up in the results pages, some of which are frowned upon (and which could get your site banned from search results), and others which are simply considered to be good web design practice. This article focuses on the latter, and explains how you can make the search engine’s life as easy as possible when it comes to matching search terms with your page content.
Bear in mind that search engines all have different criteria for ranking sites, and that these criteria are kept very secret. The tips described here are known to work with most search engines, but it doesn’t necessarily mean that they will work with all search engines. For example, one search engine may consider the “alt” attribute of an image to be more important than another search engine does (this may also vary over time, because search engines often make changes to their search algorithms). Other search engines may choose to ignore it altogether.
SEO is something which comes with experience, and the tips described here are what I have learned from many years of web development as a hobby.
What Not To Do
Before considering how to optimise pages, it is probably a good idea to look at what not to do. In general, anything involving deception, such as serving one page to search engines, and another to human visitors, is a bad idea. Not only is it dishonest, but it could also get you banned from the search engine results. Stuffing pages full of keywords is also a bad idea. Of course you should use popular keywords and key phrases on your pages, and also use variations of them (i.e. mention alternate English and American spellings such as “optimisation” and “optimization”… You see what I did there?), but only use them when they are needed.
Another important point to remember, is that you should never perform any search engine optimisation which has detrimental effects for human visitors. A carefully laid out page which is well written, is a lot better than a badly laid out page, stuffed so full of keywords that it is unreadable. Remember, attracting users is only half of the job. Once you have got a visitor to your site, you want to keep them, and you want them to return again and again!
Choose An Appropriate URL
Choosing a URL for a page is extremely important. Ideally, the location of a page should never change. If it does, it will break all incoming links, and it could take some time to restore your position in the search engine results pages. This isn’t always possible to do, as you can’t predict the future, but you should think carefully about the URL before you publish a page. A URL should:
- Provide a hint to the purpose of the page.
- Be long enough to be meaningful, but be short enough to be memorable (for the user’s sake).
- Not include file extensions for HTML content, i.e. don’t use “mypage.html” or “flowers.php”.
- Not contain special characters whenever possible.
- Not contain long query strings for static pages, i.e. “http://www.example.com/article.php?article=5226&format=text&bananas=yellow&strawberries=red”.
Whilst you should use descriptive filenames with appropriate file extensions for files on your site (i.e. Annual-Report.pdf), you should avoid this for HTML pages which form the main user interface of your site. Instead, use filenames such as “index.html”, and “index.php”, and have your server serve these by default when a directory is requested. By doing this, you save the headaches of changing every link if, for example, you switch from using static HTML pages to those generated by PHP. Links should be made to point to directories, rather than the “index” file inside the directory. For example, link to “http://www.example.com/flowers/” (always include the trailing slash!) instead of “http://www.example.com/flowers/index.html”. This also prevents duplicate content from appearing in the SERPS—see the section on using one URL per page.
Query strings are often necessary when serving dynamic content, and although most seach engines now support their use, they may refuse to index pages with excessively long query strings. To avoid this, you could use a rewrite engine to convert these URLs into more user friendly URLs.
If you must change a URL at a later date, then use a “301” redirect, rather than a page containing some kind of redirection code. Those “We have moved to [http://somewhere/]. Please update your bookmarks. You will be redirected in 15 seconds.” pages which may be found all over the internet, are not terribly search engine friendly. A “301” redirect on the other hand, is, because search engines have very good understanding of HTTP response codes. From my personal experience, search engines often download the target page within seconds of receiving a “301”!
If you are running the Apache web server, you can arrange a “301” redirect by entering into your main “.htaccess” or “httpd.conf” file:
# Permanently redirect "/old/file/" to "/new/file/"
RedirectMatch 301 ^/old/file/$ http://www.example.com/new/file/
(If you are unfamiliar with the syntax, please read the Apache mod_alias documentation before trying this out!)
Use One URL Per Page
Sometimes, the same page is accessible via several URLs. This might be because a company has a domain name for their full name (i.e. Silicon Graphics Inc.), and one for an initialism (i.e. SGI). It can also be because a site is accessible via more than one hostname, for example, http://example.com/ and http://www.example.com/ (a “www” and “non‑www” version). These will likely be treated by search engines as different pages, despite the fact that they are actually one and the same. This is undesirable, because you can end up with different people linking to your site with different URLs, which obviously lowers your incoming link count for a specific page.
To avoid these problems, you should choose which name you prefer (known as a canonical name), and stick to using it. All other hostnames and domain names should be redirected to this canonical name via a 301 permanent redirect. If you are using the Apache web server (with mod_rewrite enabled), you can place something similar to the following in a .htaccess file or a Directory section of your httpd.conf.
# Make sure site is only accessible via one hostname.
RewriteEngine on
RewriteCond %{HTTP_HOST} !^www\.example\.com$
RewriteRule .* http://www.example.com%{REQUEST_URI} [R=301,L]
(If you are unfamiliar with the syntax, please read the Apache mod_rewrite documentation before trying this out!)
If someone requests http://example.com/some/file/, they will be redirected to http://www.example.com/some/file/.
Choose A Sensible Page Title
A well thought out title placed inside the “<title>My Page Title</title>” tags of your page is a must. Many search engines use this in the search results page (as does the “Favourites” folder of popular web browsers), and is one of the main criteria people will use to decide whether to pay a visit to your site, or simply move on to the next result. A good title should:
- Make the purpose of the page obvious in as few words as possible.
- Include one or two keywords.
- Not include redundant information.
- Not use excessive capital letters or punctuations marks.
When choosing a title, think about the purpose of your page, and then put it into a few words. It doesn’t have to be grammatically correct, as long as it makes sense. For example, a suitable title for a page explaining how to do flower arranging, would be “Flower Arranging Tutorial”. This contains important keywords, and is straight to the point.
Bad choices of title would be “Susan’s flower arranging web site: a tutorial on how to arrange flowers”, or “~~@@ SUSAN'S FLOWER ARRANGING TUTORIAL!!!! @@~~”. The first would likely be truncated on a search engine results page (i.e. “Susan's flower arranging web site: a...”), so the meaning would be unclear. Even if it wasn’t truncated, the purpose of this page won’t be obvious when you have a page of ten search results sitting in front of you.
As for the second title, a page which uses excessive punctuation doesn’t look professional. Rather it looks like the content is so bad, that the page has to use other techniques to attract visitors (how may times have you seen an eBay auction with something like “BARGAIN L@@K!!!!” in the title?). Besides which, a title written in all capitals is actually harder to read on a computer screen.
Use Of Meta Data
Use of the HTML “<meta>” tag is an excellent way to provide additional information about your pages, and some search engines will use them to provide more accurate results. The main use of a “meta” tag in search engine optimisation is to provide a description of your page. Many search engines will use this text (or at least part of it) in the search engines results pages. A description should:
- Expand on the text used in the title.
- Include one or two keywords, and if appropriate, an alternate name for the topic of the page.
- Be more prose than the title, but still be concise.
For example: (I’m sure someone who does flower arranging for a living could think of something much better!)
<meta name="description"
content="A guide written by Susan Smith, about creating floral
arrangements for special occasions.">
You can also use meta “keywords”, although due to abuses in the past, very few search engines pay attention to these any more. I never bother with them, as they take time to create, use bandwidth, and incorrect usage of them (i.e. repeating words) can actually result in your site being penalised in search results!
Use Of The “alt” Attribute On Images
Even when search engines do know how to handle image formats, they are still unable to understand the intentions of images in your documents. Good use of the “alt” attribute on the HTML <img> tag will remedy this. Not only is this good for search engine optimisation, but it is required for site accessibility, for example with screen readers or non‑graphical web browsers.
<img src="./susan-steve-brighton.jpg"
alt="[Image of Susan and Steve eating chips on Brighton pier.]"
title="Susan & Steve in Brighton">
If there is important text in the image, then this should be included in the “alt” text. Note that the use of square brackets around the text is my personal style. I like to use them to show that the “alt” text is not part of the main page text. Also note the use of the “title” attribute on the image. I always include these on images, because some web browsers will display the “alt” text when you pass your mouse over the image, which is not what we want. If you include the “title” attribute, then the “title” text should be displayed instead.
Note that certain search engines provide dedicated image searches, and may use the “alt” text in the search results. Therefore, it is important that the “alt” attributes describe your images well. You should also give your images a filename which is related to the image (i.e. don’t leave it as “DCIM93287.JPG”, which your digital camera decided to name it). This can have dramatic improvements on your image search results!
Placing Of Important Content
Some search engines place more weight on text which appears near the beginning of the HTML source, than that which appears in the rest. A common tip is to place important text near the top, and place navigation links at the bottom.
If this is convenient for you, then by all means do so, but I certainly wouldn’t sacrifice the layout of your page just to do this. A tutorial, for example, will probably contain an introduction, which would presumably appear at the very beginning of the article. If you make sure your introduction is written well, and includes a few keywords, then you should be able to easily achieve this.
Use External Scripts & Styles
Rather than placing style and script content directly into “<style>” and “<script>” tags, you should should place them in external files, and reference these files from your page. This is good practice anyway, because it keeps the styling and scripting separate from the content, and can save bandwidth when the same code is used on several pages.
<!-- Document Head -->
<script type="text/javascript" src="./script.js"></script>
<link rel="stylesheet" type="text/css" media="screen" href="./style.css"
title="My Style">
(Don’t forget that the “<script>” tag must be closed, even though it doesn’t contain any content!)
Use Semantic Markup
Using HTML tags by their meaning (semantic markup), rather than how they are displayed (presentational markup) will help a search engine get a better idea of the structure of your page. Particularly, structuring your pages with headings “<h1>, <h2> … <h6>” should help your search engine optimisation a lot, particularly if they contain one or two carefully chosen keywords. For example:
<h1>Creating A Flower Arrangement</h1>
<h2>Introduction</h2>
<p>(an introduction)</p>
<h2>Requirements</h2>
<p>You will need:</p>
<ul>
<li>Flowers.</li>
<li>…</li>
</ul>
Just remember that a page must only have one “<h1>”. Using more than one “<h1>” on a page could cause search engines to think you are trying to trick them.
Use Valid Markup
When writing your pages, you should make sure that your HTML code is free of syntax errors. This is good practice in web design, because you should never rely on the error handling capabilities of web browsers. However, it is even more important for search engine optimisation, because there is no guarantee that the search engines have any error handling ability. A syntax error could potentially result in your whole page being unreadable by the search engine, and a large percentage of potential visitors will never even be able to find your site!
It’s a good idea to validate your documents against one of the W3C’s HTML 4.01 DTDs. You can perform the validation using standard SGML tools, but if you don’t have access to these, there are plenty of online validation services available. I particularly recommend the WDG HTML Validator, because it warns against poorly supported SGML constructs, can validate in batch mode, and can even validate an entire site at once. In fact the WDG also provide their own HTML 4.01 compatible DTD, which enforces some additional requirements for good design practice.
It’s also possible to validate against DTDs which extend the HTML standard. Although extended attributes and tags (i.e. the infamous <marquee> tag) might not be understood by search engines, they shouldn’t cause problems as long as their use is syntactically correct.
Internal Links
Ensure that your site has good internal link structure. Whilst it is overkill to link every single page to every single other page, you should make sure that the main sections of your site link to each other. You should also try to place links between pages covering similar topics. This makes your site more accessible to search engine crawlers, and may help to return more relevant results when searches are performed.
External Links
Google uses an algorithm called PageRank. I won’t go into detail, but it assigns a numerical weighting on pages, depending on the reputation of the pages linking to them. The higher this page rank is, the higher up in the search results you page should appear for certain keywords.
To get a better PageRank, you could speak to other site owners, and ask them if they would like to exchange links. Swapping links with respected sites covering similar topics as yours, will help a lot more than links from unrelated sites.
HTML Anchors
When placing HTML anchors on the page, the text in the link is helpful for describing what the target page is about. The worst thing you could ever put in the link text is “click here”! Not only does it fail to describe the target page, but it doesn’t flow nicely in a selection of text. There is also no guarantee that your visitor is using a mouse—if they are using, for example, a keyboard for navigation, then “click here” makes little sense!
A good anchor tag looks like:
<p>When using <a href="http://www.opengl.org/"
title="Official OpenGL Web Site">OpenGL</a> … </p>
Note the use of the “title” attribute. There has been much debate about whether it helps significantly in SEO or not, but it certainly won’t hurt to use it—graphical web browsers may display the title text as a mouse “tooltip”, aiding users in knowing what they are about to click on.
Incidentally, if you don’t believe how important the link text is, just do a Google Search of “click here”!
Don’t lock search engines out!
This may seem rather obvious, but the causes behind an un‑crawl‑able web page can sometimes be a lot more subtle. For example, if you would like a page to be crawled by a search engine, you should:
- Never use browser detection scripts which only grant entry to certain user agents (this is bad web design practice anyway, but many people still do it!).
- Never require requests methods other than “GET” (i.e. “POST”) in order to view the page (this generally isn’t a problem, because “POST” requests are intended for requests which modify the state of the server, which you probably wouldn’t want a search engine robot doing!).
- Never require a cookie to be set in order to view the page.
- Never use JavaScript to perform tasks which could be performed using standard HTML elements. For example, don’t try to recreate the functionality of an “
<a href="">” tag in JavaScript. (If you do need to use behaviour like this, then use a standard HTML tag, and use JavaScript DOM methods to delete it, replacing it with your code.)
If you use login cookies for a page, for example to show additional information to registered users, then you should make sure that the page still works with cookies disabled.
Server / Site Maintenance
If you ever find it necessary to replace all of your pages with a “This site is currently down for maintenance.” notice, then you must serve it with a “503 Service Unavailable” HTTP status code. If you send a “200 OK” code, and a search engine crawler downloads your page, then your content could be replaced in the SERPS by your “site unavailable” notice. You may wish to send a “Retry-After” header with the response, to give a suggestion to the search engine crawlers of when your site will be back up and running. Of course there is no guarantee that they will obey this.
Design for humans, not robots!
Take a look through your web site logs, and see which search terms cause people to find your pages. Type these search terms into some search engines, see how near to the top of the results your pages come. By looking at these results, you can work out if you’re getting things right, and if not, think about how to improve them.
Just remember that you are writing your pages for people, and not for search engines. Most of what has been described here is nothing more than good web design technique—but that’s all that is required!