Chapter 5 Practical Web Crawling


    Chapters ?? and ?? presented the model and architecture implemented in the WIRE crawler. When we tested our implementation, we found that there were several problems of Web crawling which did not become evident until a large crawl was executed. We are interested in documenting these problems for two reasons: • To help other crawler designers, because most… (More)

    2 Figures and Tables


    • Presentations referencing similar topics