The backup has finished, but not without errors. As I have never tried to mirror a site as large as this my bot tripped what was supposed to be only a code sanity check of "Quit on 100000 links crawled". Given the top-down method of the bot, this means that it only got down deep enough to save the first and last few pages of threads. This is is only a problem for the big boards with more than about 100 topics (when you get more than 6 pages of threads).
I'm going to try and crawl those boards individually and see if I can get a little deeper in this time. I'll start with one of the smaller ones to see if this gets any better results, and then I'll be back on things.
---
The Master said: "It is all in vain! I have never yet seen a man who can perceive his own faults and bring the charge home against himself."
>Analects: Book V, Chaper XXVI
I'm going to try and crawl those boards individually and see if I can get a little deeper in this time. I'll start with one of the smaller ones to see if this gets any better results, and then I'll be back on things.
---
The Master said: "It is all in vain! I have never yet seen a man who can perceive his own faults and bring the charge home against himself."
>Analects: Book V, Chaper XXVI