Checking the links to your site (from affiliates) using cron and bash script.

If you have some affiliate links and are concerned about their presence then you can setup cron to check the links for you regularly. For this you need a very simple script (as below) and a cron entry (example further down).

#!/bin/bash –
#          FILE:
#         USAGE:  ./
#   DESCRIPTION:  Check if the links exists on affiliate sites
#       OPTIONS:  —
#          BUGS:  —
#         NOTES:  —
#        AUTHOR:  Amit Agarwal (AKA), [email protected]
#       COMPANY:  Individual
#       VERSION:  1.0
#       CREATED:  04/07/2010 08:19:29 AM IST
#      REVISION:  —

websites=\”space seperated website address like\”
echo \”Checking the sites for Links\”
echo $websites|tr \’ \’ \’\\n\’
for i in $websites
curl -s $i |grep > /dev/null
if [ $? != 0 ]
echo \”Link not found on $i\”

Then make a crontab entry like this:

40 08 * *  * (echo \”Subject: Link verification on affiliate sites @ `date`\”;<fullpath>/ |/usr/sbin/sendmail <email>


Mirror or download a website with a tool much better than wget

If you want to mirror a website, the simplest tool I know if is wget. Once you have used wget, you know the troubles associated with it. You also get added couple of files as search html files, which are useless and wastage of bandwidth for you as well as the webserver.

lftp -e \”mirror -c\”  <URL>

This will mirror the website for you and will not leave with all those un-necessary files.