get the contents of whole site like some wiki or wikia

2014-02-24 1 min read Learning Uncategorized

For wikis and wikia, generally if you are trying to get some url mirror, then websucker.py is an excellent option. This script is in the python sources so, to get this tool,

yumdownloader --source python

Install the rpm downloaded in current directory and then go to ~/rpmbuild/SOUURCES.  You should find a Python-*.tar.xz file here, just extract with

tar xvf Python*.tar.xz

and there you go, you should find the tool in Tools/webchecker/websucker.py.

Enhanced by Zemanta
comments powered by Disqus