Article printed from SiteProNews: http://www.sitepronews.com
  HTML version available at: http://www.sitepronews.com/archives.html

Yahoo's Spider Slurp 
By Jim Hedger

How To Write For Slurp The Spider

As the world's second most popular search tool, Yahoo moves a 
tremendous amount of traffic and is a very credible alternative 
to Google. Yahoo receives over 2.76 billion page views per day 
from hundreds of millions of unique users. It boasts over 157 
million registered users enjoying mail, shopping and discussion 
groups and increasingly personalized search and news services. 
For the past two years, Yahoo, Google and MSN have been embroiled 
in a hard-fought battle for the loyalty of search engine users 
forcing all three firms into the hyper-evolution we are 
witnessing today. Over the next three Wednesdays we are going 
to examine how the Big-3 spiders work, what they look for and how 
to best prepare your sites for multiple visits from the bots that 
rank them. Today, we are starting with Yahoo's bot, SLURP.

Getting Found By Slurp
The first thing to know about Slurp is that like its better 
known cousin, Google-bot, Slurp "discovers" sites by following 
links from one site to another, reading and recording nearly 
everything it finds in its path. The majority of websites 
referenced by Yahoo were originally included in its database 
because they were accessed by Slurp following links from another 
site. 

Yahoo suggests adding an inbound link to all pages in your site 
to guarantee those pages will be discovered by Slurp. They also 
recommend an internal sitemap linked to from the Index (or home) 
page of the site. To encourage Slurp to spend more time 
deep-crawling your content, Yahoo recommends the addition of 
"good authoritative links pointing into your site", from highly 
reputable sources such as news sites, established business 
partners and other sites relevant to your business or service. 

Manual submission of the site is only recommended if for some 
reason or another Slurp does not find the site on its own. This 
is increasingly rare however as server-logs show Slurp is one of 
the most active spiders out there. In other words, if a site 
Slurp has already indexed links to your site, Slurp will almost 
certainly be visiting very soon. Webmasters should never have to 
pay submission fees to get into Yahoo's index since according to 
Yahoo's Tim Mayer, 99% of Yahoo's index is crawled by Slurp for 
free.

It is still important to make sure your site is ready to receive 
a visit from Slurp. To ensure Slurp is able to travel across 
your entire site, provide standard HREF text links as opposed to 
forms, Flash or java script navigation tools. Webmasters are 
encouraged to avoid tracking and communication methods that rely 
on using cookies across every page of the site. If you have a 
database driven site or a site that creates unique sessions for 
each user, avoid embedding session IDs in URL's. Lastly, use 404 
pages to redirect users (and spiders) to the root (index) page 
if a page or site URL becomes invalid. Yahoo also asks webmasters 
of sites with shopping carts to use robot.txt exclusions in the 
source of the shopping carts.

Where Your Site Has Been Included. Results May Vary...
Yahoo has seen enormous change over the past seven years. What 
started as a paid-inclusion, human edited search directory, has 
grown into the second largest database of indexed content. Yahoo 
is on the cutting edge of integrating several forms of media 
into their search offerings and will likely soon produce its own 
entertainment content like an online HBO. Yahoo is flirting with 
the concept of becoming an infotainment portal again but the 
core of its offerings remains firmly rooted in search. 

Yahoo search results come in multiple formats including: 
Yahoo-Local, Yahoo-products, MY-Yahoo (personalized results), 
specific nation-based Yahoo's, and the standard Yahoo.com One of 
Yahoo's goals appears to be presenting individual search-users 
with results that best match their personal needs. For instance, 
Yahoo would like to present constantly updated geographic-specific 
references when a user searches for daily-use items such as 
groceries, repair-workers, real estate and other services one 
would normally use a telephone directory to find. Similarly, 
Yahoo wants to present the entire global database of references 
when a user searches for international news, trans-national 
products or vacation plans. Being certain your website gets 
served up for all levels of search, local, regional and global, 
will be important if you wish to serve a market larger than 
your general region or community.

Getting Rankings
Yahoo's search engine ranks sites based on a formula that is 
very similar to the algorithms used by rivals Google and MSN. 
Yahoo values many of the same elements other search engines do 
including keyword enriched domain names, titles, meta tags, and 
content. Yahoo also values keywords found in the anchor text of 
internal links, though the effect at Yahoo is not as powerful 
as it is on Google. 

According to Yahoo, well optimized pages and sites will continue 
to get good results across all versions of their search engine. 
By opening your site to Yahoo Slurp and performing well-planned 
optimization services across every page, a good SEO can nearly 
always achieve Top placements on Yahoo. The trick is in offering 
Slurp the information it needs to read, record and rank your 
site. If that information is included on each page, a set of 
text-based links is woven through the site to provide easy 
passage for Slurp, and Yahoo is told what your business is, 
where your business is located, and who your business serves, 
your site should achieve strong rankings. 

Due to the advent of personal, local, regional and global 
search results, it is highly recommended to add full contact and 
address information on every page of a site. This information 
should be as precise as possible and should include street 
address, unit or suite number, zip or postal code, state or 
province, county and country, full telephone information 
(including area codes), and if possible, the approximate 
longitude and latitude of your business location. (look up your 
longitude and latitude here: http://www.astro.com/atlas)

When writing for Slurp here are a few basic fundamentals:

- Have a descriptive URL. 
- Use keyword enriched titles on each page of the site.
- Place keyword enriched description and keywords meta tags on 
  each page of the site. 
- Use robot.txt files to keep Slurp out of your shopping cart 
  or log in pages. 
- Place keyword enriched text in the first paragraphs of your 
  site-copy. 
- Use HREF links to direct Slurp through each page of the site. 
- Add a sitemap page and be certain there is a link from the 
  index page to the sitemap. 
- Be certain that geographic specific information is mentioned 
  on each page of the site. Always have a contact page that 
  also lists geographic specific information. 
- Write a press release and send it to as many blogs, news-wires 
  and press release sites as possible. 
- Acquire strong, relevant incoming links from sites with topics 
  similar to yours. 
- Update your site frequently. 
- Enjoy and value your placements.

================================================================
Jim Hedger is a writer, speaker and search engine marketing 
expert based in Victoria BC. Jim works with a limited group of 
clients and provides consultancy services to StepForth Search 
Engine Placement (http://www.stepforth.com). He has worked as an 
SEO for over 5 years and welcomes the opportunity to share his 
experience through interviews, articles and speaking engagements. 
Hedger can be reached at jim.hedger@gmail.com
================================================================

Copyright © 2004 Jayde Online, Inc.  All Rights Reserved.

SiteProNews is a registered service mark of Jayde Online, Inc.