Site-maps, robots, cloaking, spiders...

ed-u.com - Traffic Plus!

Choose a page below...  Webmasters' Education: Start here - Why me? What you have to know 2 keywords = 1000 phrases Bad titles, low traffic Free meta tags Speaking in tongues Slow pages equal more traffic Free and Easy Yahoo for free Master of your own domain? Spiders cannot fill in forms Good and bad doorways Order the engines around Do you use frames? Are you a magnet? Visitors by default Lasso your lost visitors $100 = 10,000 visitors Others work - you earn Ads - Don't click here? Invite the competitors over E-mails and warnings Infectious or immunized? All the best submission tools 100s of search engines The "must have" web tools Free for all sites Pressing the editor's button Legal or e-legal? Where do I start? A to Z index   Web Guest Writers: Web kiosks for the poor Online Price Mistakes Thesaurus Ranking HTML or text e-mails? ALTered Images Renaming your Images Free tools to create your site Invasion of the Amazons Advertising v. PR Will content be profitable? Responding to spam What is spam? Pay for position Web host tips Webmaster beauty Bad web design: Animation CGI: What the heck is that? Long live banners Great press releases Pop-up-ads E-mail Sigs - 7 Key Items Ad tips for affiliate marketing I want to keep my frames Creating Doorway Pages Measuring Link Popularity Domains and hyphens Why have a domain name  General Education: Homepage | Web Search Education Resources Active Online Pupil Forums Art | Film | Dance | Bands Countries of the World Digs UK | US | Canada Finance | Economics/Biz Homework Helpers | Exams Kiddies Korner | Tots | Pets Jobs UK/EU | US | Canada Medicine and Health Museums and Galleries Mystery Shopping Club PE | Sports | Sporting Events Power & Politics | Civil Rights Print Media, TV and Radio Problems and Advice Shops | Fashion | Books Subjects | Religion | Language Technology Education & ICT Teens | Just for Fun Travel | Destinations Schools UK | US | Canada Unis/Colls UK | US | Canada  Special Features: Essays - Full Writing Course What is Bullying? Stress in Teaching Drugs  Guest Contributors: The best Dad? You're an Idiot! Slave Caster of Freedom Out of the mouths of babes The Right to Life? The Nostradamus Hoaxes Explaining terrorism to a child Internet 2 a scam? Break a Rule, Bad Girl Britannica near extinction? The 1st time I really lied Nigerian Scam Letters Singular turns plural English Writing In debt? Too busy at work? Bullying... Our Stories Start to live your dreams Recognize your potential Stop worrying, please! Public speaking Elegant resumes In praise of black sheep Ritalin - Straight-jacketing? ed-u.com's full list of pages

Increase Traffic to your Web-site - Traffic Tips and Secrets... We invite you to add ed-u.com to your list of favorites/bookmarks. Internet Explorer users please click here, and others, right click here -> ed-u.com. Also, you can learn how to make any ed-u.com page your start page by clicking here.   Web ed-u.com Importance of site maps A site-map can be a search engine crawler friendly page. Create a HTML page and fill it with all the links on your site (see ed-u.com's site map for an example). Linking to a site map page from your homepage helps the search engine spiders find and index all the pages of your site easier. It has the added benefit of getting a lost visitor back on track. If you have hundreds of pages, you could split up the site map pages to include only 50 URLs (Uniform Resource Locators / web addresses) each and make sure all these pages are linked to each other. See also how Wired.com have utilised crawl pages at www.wired.com/news/archive - click on any one of their links to understand the structure. Notes: If your links are embedded in javascript, search engines have problems travelling through them. Avoid using image maps, most search engines cannot crawl them. The Robots.txt File The robots.txt file is a file that can tell search engine robots which areas of your site not to index. It is particularly useful if you want to keep some areas of your web-site away from public view, such as password protected pages or "office use only" pages. A robot is a small software program that visits sites and automatically requests documents from them for indexing. They use the hypertext links found within documents to travel to their next destination. A search engine is a database of HTML pages gathered by a robot. Robots are also known as webbots, bots, web crawlers or spiders. More on the robots.txt file: Robots Exclusion Database of Robots Searchtools.com Robots Internet Tips Robots Robots Check The Web Robots Database Come into my parlour said the spider... Cloaking and Spiders Below are some web-sites that contain information about the risky subject of page cloaking and hiding your HTML from the search engines (not for the novice). Spider Food Spiderhunter Search Engine Base Search Engine Forums Fantomaster Cloaking FAQ Beware, even though you may think you are cloaking ethically, you might make a mistake and be flagged as a spammer. The need to keep up to date with changing spider IP addresses is also a problem. MSN say that they will ban sites that use cloaking technology and will share information about companies they've caught with Inktomi (the search provider for many major portals) and with other search engines. Google.com calls cloaking "egregious" and says "it goes against our ideal of us seeing the same thing a search engine sees". Do you use frames on your site? Click here to find out why frames are maybe not such a great idea... Please click here to report inappropriate or "broken" links. Save this site Please don't forget to add ed-u.com to your list of favorites/bookmarks. Internet Explorer users please click here and others, right click here - - ed-u.com - - Also, you can learn how to make any ed-u.com page your start page by clicking here. Share this site Click here to tell your friends, family or colleagues about ed-u.com. Feedback If you know of any advice that could be usefully included on this site, please click here to e-mail. I would also like as much feedback as possible, good and bad. Thank you - Robert Brady.