Web Science & Human Dynamics

How people actually browse the Web, and why the PageRank random surfer gets it wrong.

Overview

Between 2008 and 2010, working with complete HTTP logs from a university population, Bruno Gonçalves studied how people actually navigate the Web — not how link-graph models assume they do. The recurring finding across this work is that the random surfer at the heart of PageRank is a poor description of real browsing, and that reproducing genuine traffic requires agents with topical interests and memory.

Research Areas

  • Beyond PageRank

    Empirical Web traffic turns out to be a poor match for PageRank and other Markovian surfer models — real navigation carries structure those models cannot reproduce.

  • Session Structure

    Recovering what a browsing session actually looks like from complete HTTP request logs, with user identity preserved across requests.

  • Agent-Based Models

    Building generative models of browsing in which agents carry topical interests and memory of pages they liked, reproducing traffic patterns that random-surfer models miss.

  • Individual Dynamics

    Characterizing single users through their return times to a page and the intervals between consecutive visits, rather than through aggregate statistics.

PageRank Is Not How People Browse

Analysis of aggregate and individual Web requests showed that PageRank is a poor predictor of traffic. Using empirical patterns generated by a thousand users over two months, this work characterized properties of Web traffic that Markovian models cannot reproduce — in both aggregate statistics such as page traffic and in the structure of individual browsing. The proposed alternative gives each agent topical interests and a memory of pages it has liked, which recovers the observed behavior far better.

Preserving user identity across the request log — unusual for the period — is what made the individual-level results possible, including the distribution of return times to a page and the intervals between consecutive visits.

Key Publications

A complete list is available on the Publications page.