Article printed from SiteProNews: http://www.sitepronews.com
  HTML version available at: http://www.sitepronews.com/archives.html

How To Get Indexed By Search Engines...Fast
by David Bell ©Copyright 2004

Why Can't I Get Indexed by the Search Engines? Unfortunately, 
this is an all too common question. If it makes you feel any 
better, you're not the only one frustrated about the length of 
time it takes to be indexed, or the many pitfalls involved. It 
often takes anywhere from two days to as much as six months to be 
listed on a search engine. For example, last month Excite finally 
updated its index for the first time since last August! Luckily, 
Excite is the most extreme case lately, but waiting several weeks 
to a month can also be extremely frustrating.  

The WebPosition Submitter report will give you current time 
estimates for each engine so you'll know what to expect. However, 
an engine at any time could choose to delay their indexing beyond 
the "norm" for maintenance or other reasons. On the flip side, 
you could get lucky and submit just a couple days before an 
engine does a complete refresh of their database. Therefore, 
submission times can never be an exact science since we're all 
ultimately at the mercy of the engine. If you've submitted your 
site and have waited the estimated time to be indexed and there's 
still no listing, what do you do now?  

Here are 16 tips that should help you solve this problem: 

1. First, be sure you're not already indexed but just don't know 
it. Unfortunately, none of the major engines are kind enough to 
e-mail or notify you as to if and when you've been indexed. The 
method to determine if a page or domain has been indexed varies 
from one engine to another, and in many cases, it's difficult to 
tell for sure. Never assume that you're not indexed just because 
you searched for a bunch of keywords and you never came up in the 
first few pages of results. You could be in there but buried near 
the bottom. In addition, it's not very practical to check the 
status of a number of pages on each major engine each week. 
Fortunately, WebPosition has a URL verification feature in the 
Reporter that makes this process much easier. Each time you run a 
mission, it will report which URLs exist and do not exist in each 
engine.  

2. Make sure you have uploaded the pages to your site before 
submitting them. This one seems obvious, but submitting a page 
that does not exist or submitting with a subtle typo in the URL 
is a goof we might all make at one time or another. If you're 
using WebPosition's Submitter, there's a checkbox on tab 2 that 
forces WebPosition to verify that all your URLs are valid before 
submitting them.  

3. If you have information inside frames, that can cause problems 
with submissions. It's best if you can create non-framed versions 
of your pages. You should then submit the non-frames versions of 
your pages which can of course point to your framed Web site. 
Alternatively, you can enter your relevant text within the 
NOFRAMES area of a framed page which most search engine spiders 
will read.  

4. Search engine spiders cannot index sites that require any kind 
of registration or password. A spider cannot fill out a form of 
any kind. The same rule applies regarding indexing of content 
from a searchable database, because the spider cannot fill out a 
form to query that database. The solution is to create static 
pages that the engines will be able to find.  

5. Dynamic pages often block spiders. In fact, any URL containing 
special symbols like a question mark (?) or an ampersand (&) will 
be ignored by many engines.  

6. Most engines cannot index text that is embedded in graphics. 
Text that appears in multimedia files (audio and video) cannot be 
indexed by most engines. Information that is generated by Java 
applets or in XML coding cannot be indexed by most engines.  

7. If your site has a slow connection or the pages are very 
complex and take a long time to load, it might time out before 
the spider can index all the text. For the benefit of your 
visitors and the search engines, limit your page size to less 
than 60K. In fact, most Webmasters recommend that your page size 
plus the size of all your graphics should not exceed 50K-70K. If 
it does, many people on dial up connections will leave before the 
page fully loads. 

8. If you submit just your home page, don't expect a search 
engine to travel more than one or two links away from the home 
page or the page that you submitted. Over time they may venture 
deeper into your site, but don't count on it. You'll often need 
to submit pages individually that appear further down into your 
site or have no link from the home page. 

9. If your Web site fails to respond when the search engine 
spider pays a visit, you will not be indexed. Even worse, if you 
are indexed and they pay a visit when your site is down, you'll 
often be removed from their database! Therefore, it pays to have 
a reliable hosting service that is up 99.5% of the time. However, 
at some point a spider is going to hit that other 0.5% and end up 
yanking your pages by mistake. Therefore, it pays to keep a close 
eye on your listings. 

10. If you have ever used any questionable techniques that might 
be considered an overt attempt at spamming (i.e., excessive 
repetition of keywords, same color text as background, or other 
things that the WebPosition Page Critic warns you about), an 
engine may ignore or reject your submissions. If you're having 
trouble getting indexed in the expected amount of time, make sure 
your site is spam-free. 

11. If your site contains redirects or meta refresh tags these 
things can sometimes cause the engines to have trouble indexing 
your site. Generally they will index the page that it is 
redirecting TO, but if it thinks you are trying to "trick" the 
engine by using "cloaking" or IP redirection technology, there's 
a chance that it may not index the site at all. 

12. If you're submitting to a directory site like Yahoo, Open 
Directory, NBCi, Looksmart, or others, then a human being will 
review your site. They must decide the site is of sufficient 
"quality" before they will list it. I recommend you read the 
submission guide on the directory tab of the WebPosition 
Submitter. It contains tips to improve your chances of obtaining 
a good listing on these directories. 

13. A number of engines no longer index pages residing on many 
common free web hosting services. The common complaint from the 
engines is that they get too many "junk" or low-quality 
submissions from free web site domains. Therefore, they often 
choose not to index anyone from those domains or they limit 
submissions from them. It's always best to buy your own domain 
name (very important) and place it on a respected, paid hosting 
service to avoid being discriminated against. 

14. Some engines have been known to drop pages that cannot be 
traveled to from the home page. HotBot has been rumored to do 
this. You may want to consider submitting your home page that 
links either directly or indirectly to your doorway pages. 

15. Make sure you're submitting within the recommended limits. 
Some engines do not like more than a certain number of 
submissions per day for the same domain. If you exceed the limit, 
you may find that all your submissions are ignored. Fortunately, 
WebPosition's submitter will warn you regarding current limits 
and recommend you stay within them. Some submission consultants 
feel it is dangerous to submit more than ONE page a day to a 
engine for a given Web site. For those who wish to be 
ultra-conservative in their approach, the WebPosition Submitter 
includes a checkbox to limit submissions to one URL per day per 
engine. 

16. Last but not least, sometimes the engines just lose 
submissions at random through technical errors and bugs. 
Therefore, some people like to resubmit once or twice a month for 
good merit in case they do lose a submission. Certainly if you've 
followed all the "rules" and are still not listed, re-submit! 
Sometimes a little persistence is all that's needed. 

If any of the above scenarios apply to your submission, you 
should make the necessary adjustments and re-submit. If that 
still does not work, you should consider e-mailing or calling the 
search engine and asking them politely why you have not been 
indexed yet. Sometimes they will reply back with "Sorry, there 
was a problem with our system and I've now made sure you'll be 
indexed within the next couple days." Or, sometimes they'll tell 
you why you were not indexed. In other cases, they will ignore 
your e-mail and you'll have to keep e-mailing or calling them 
until they respond. Still, it's definitely worth the effort to 
get your site listed with the major engines assuming you also 
take the time to optimize your pages so you'll achieve top 
rankings. I hope this helps in your future marketing decisions.  

================================================================
David Bell is Manager, Online Marketing, at 
http://www.wspromotion.com/ , a leading Search Engine Optimization 
services firm and Advertising Agency.  
================================================================

Copyright © 2004 Jayde Online, Inc.  All Rights Reserved.

SiteProNews is a registered service mark of Jayde Online, Inc.