In the mid-1990s, web search had a quality problem. The early engines mostly counted words: a page that repeated your query often enough ranked well, whether or not it was any good. Pages stuffed with keywords could outrank the pages people actually wanted. The fix did not come from better word counting. It came from reading the web's links — and it arrived, almost simultaneously, from three different directions.
The idea all three shared was simple: a link is a vote. If many pages link to a page, that page is probably worth something. The three algorithms differed in how they counted those votes, and those differences decided their fates.
PageRank: votes weighted by the voter
At Stanford, Larry Page and Sergey Brin started from the observation that the web is a citation network, much like academic literature. Their project, first called BackRub, treated every link as an endorsement — but with a twist. Not all votes count equally. A link from an important page is worth more than a link from an obscure one, and importance is itself defined by the votes a page receives. The score is therefore recursive: a page's rank depends on the ranks of the pages linking to it, divided among their outgoing links, with a damping factor (usually given as 0.85) modeling a surfer who sometimes jumps to a random page instead of following a link.
Brin and Page described the system in a Stanford technical report in January 1998 and in the paper "The Anatomy of a Large-Scale Hypertextual Web Search Engine" at the Seventh World Wide Web Conference in April 1998. The crucial engineering property was that PageRank is query-independent: it can be computed once for the whole web, ahead of any query, and then combined with ordinary text matching at search time. That made it cheap to serve at scale, and it became the foundation of Google, founded later that year.
HITS: hubs and authorities, computed per question
At almost the same time, Jon Kleinberg at IBM's Almaden Research Center was working on a different question. Instead of asking which pages are important in general, he asked which pages are authoritative on a particular topic. His answer, HITS (Hyperlink-Induced Topic Search), gives every page two scores that reinforce each other: a good authority is a page linked to by many good hubs, and a good hub is a page that links to many good authorities. Starting from a small set of pages matching a query, the algorithm expands to their neighbors and iterates the two scores until they settle.
Kleinberg circulated the idea as an IBM research report in May 1997, presented it at the ACM-SIAM Symposium on Discrete Algorithms in January 1998, and published the full version in the Journal of the ACM in 1999. HITS was elegant and influential — it clarified the hub-authority structure of the web and fed later work on topic distillation and community detection — but it had a practical weakness for a public search engine: the scores are query-dependent, computed fresh for each question over a focused subgraph, which is expensive to do for every query at web scale. It became a celebrated algorithm rather than the ranking core of a major engine.
RankDex: the same insight, a different fate
The third thread is the least known outside China. Robin Li, then a software engineer at IDD Information Services, a Dow Jones subsidiary in New Jersey, developed a link-analysis ranking method he called RankDex in 1996. Like PageRank, it ranked pages by the hyperlinks pointing to them rather than by word frequency alone. Li received a US patent for the technology in 1999, and later reporting notes that Larry Page cited Li's work in some of his own PageRank patents.
What happened next is the interesting part. RankDex was built inside a financial-information company, where it stayed a component of internal systems. Li moved to the search company Infoseek in 1997, and in January 2000 he co-founded Baidu with Eric Xu, where RankDex technology went into the new engine. The same core insight thus reached production twice: once as the ranking foundation of a company built around it, and once as a technology carried into a company founded years later.
Why the comparison matters
Put side by side, the three algorithms map out the design space of link-based ranking. PageRank is global and precomputed; HITS is topical and per-query; RankDex sits with PageRank on the global side but arrived through the patent system and corporate R&D rather than a university prototype. All three treat links as endorsements, and all three were developed within about two years of each other — a genuine case of parallel invention, not a single eureka moment.
None of the three survives in pure form today. Once link scores became valuable, an entire industry grew up to manipulate them, and search engines responded with defenses such as the nofollow attribute and with hundreds of additional ranking signals. But the underlying shift survived: relevance stopped being only about what a page says about itself, and started being about how the rest of the web treats it. That shift — from counting words to reading votes — is still the foundation under modern search, however many layers have been built on top of it.
Terry writes about independent building and the tools behind it at airgrix.com.
Top comments (0)