Google Hell is the worst fear of the untold numbers of companies that depend on search results to keep their business visible online. Getting stuck there means most users will never see the site, or at least many of the site's pages, when they enter certain keywords. And getting out can be next to impossible--because site operators often don't know what they did to get placed there.
Google's programmers appear to have created the supplemental index with the best intentions. It's designed to lighten the workload of Google's "spider," the algorithm that constantly combs and categorizes the Web's pages. Google uses the index as a holding pen for pages it deems to be of low quality or designed to appear artificially high in search results.
Those pages are scanned far less frequently than those in the main index, meaning that once a page is marked for Google Hell, it can languish there for as long as a year before Google even deigns it worthy of a reappraisal. And as Google tries to manage an explosively growing Web, more and more sites are finding themselves thrown into the search engine's digital dungeon.
If that makes the world's leading Web-crawler sound judgmental, consider Google's difficult position. The search juggernaut is faced with the endless task of reading and ranking the ever-expanding Web's billions of pages, the equivalent of putting the Earth's population in order from tallest to shortest every few minutes. Meanwhile there are growing numbers of pages filled only with junk text and advertising, designed solely to fool the engine. It's Google's task to sort out the trash from the worthwhile, and to do it better and faster than competitors like Yahoo! (nasdaq: YHOO - news - people ), Microsoft (nasdaq: MSFT - news - people ), or InterActiveCorp's (nasdaq: IACI - news - people ) Ask.com.
In retrospect, Sanar thinks he can trace his problem to a search marketing consultant he had paid $35,000 to improve Skyfacet's Google rankings. He now believes the consultant mistakenly replicated content on many of the site's pages, making them look like duplicate--that is, spam--content. But even after he reversed the consultant's changes, he couldn't get Skyfacet's pages out of Google Hell, where they remain today.
Other online businesses have similar stories. MySolitaire.com, another online diamond business, spent January to June of 2006 in the supplemental index. Amit Jhalani, the site's vice president of search marketing, says he figures that cost his business $250,000 in sales, and he says he still doesn't know why the site's pages got Google's thumbs-down.
"So many of the rules are vague," Jhalani says. But he admits that he tried gray-area tactics like buying links from more established sites to juice his traffic. "For a small site like ours, you have to stay right on the edge to compete with sites with bigger budgets," he confesses.
Jhalani says he removed the links that may have offended Google, but the site remained in Google's gulag. Jhalani wrote Google asking the search engine to reappraise MySolitaire; nothing happened. Since Google ranks sites partially by the quality of sites that link to them, he painstakingly contacted every site that seemed to be of low quality and linked to MySolitaire, asking them to remove their links, sometimes even sending cease-and-desist letters. Finally the site returned to Google's main index last June, though Jhalani has no way of knowing just what finally caused Google's algorithm to forgive him.
The criteria for which pages are targeted for the supplemental index remains a subject of guesswork. But Web designers have found that pages with duplicate content, few words or pictures, and a lack of links to other quality sites are the most likely to be pulled in. Most agree that newly created sites are especially vulnerable.
As for Google's own take on its supplemental index, the company is typically tight-lipped. Google's official page for Webmasters cryptically notes that Google is "able to place fewer restraints on sites that we crawl for this supplemental index than we do on sites that are crawled for our main index," a phrase that puzzles most search marketers.
In an e-mail, Google product manager Prashanth Koppula offers little more in the way of an explanation. Asked if the supplemental index is getting bigger, he responds that "new pages are constantly being added," but that the "algorithmic nature" of Google's spider makes it hard to measure the index's size or how fast it's growing. That's not a problem, Koppula says, because supplemental results are no less legitimate than normal results, and pages in the supplemental index aren't checked any less frequently by Google's spider.
But Jim Boykin, another search marketing consultant and blogger, doesn't buy it. "If your page is in the supplementals, it won't rank for any competitive search, and it can be really hard to get it out," he says. "That's why we call it Google Hell."
