Month: November 2013

Apache server stats with small and useful bash script.

Just copy this script to your web-server cgi-bin directory and enjoy.

The script with show the common errors like 404 Error, Internal Server Error and others. It will show the User agent distribution using simple commands like grep, uniq, awk and so on.

You would need to change the tfile – which is temporary file and also the access.log path in the next line.

Just re-direct the output to some file with html extenstion. You could even put this in the cron which re-directs the output to some html in server document root.

sudo grep $(date '+%d/%b/%Y') /var/log/apache2/access.log >$tfile
thits=$(grep -c . $tfile)
echo "Content-type: text/html"
echo ""
echo "Report for $(date '+%d/%b/%Y')

Report for $(hostname) on $(date ‘+%d/%b/%Y’)


echo "Total hits :: $thits"
echo ""
echo "User agent distribution :: "
#awk -F" '{print $6}' $tfile  | sort | uniq -c | sort -fr

awk -F" '{print $6}' $tfile | sed 's/(([^;]+; [^;]+)[^)]*)/(1)/' |sort |uniq -c|sort -fr

echo "
User response code: "
awk 'BEGIN{
a[206]="Partial Content";
a[301]="Moved Permanently";
a[304]="Not Modified";
a[401]="Unauthorised (password required)";
a[404]="Not Found";
a[500]="Internal Server Error";
{print $9 " => "a[$9]""}' $tfile | sort | uniq -c | sort -nr

echo "
404 Error Summary::"
awk '($9 ~ /404/)' $tfile | awk '{print $9,$7}' | sort | uniq -c|sort -nr|head -5

echo "
IP Visit counts :: "
awk '{print $1}' $tfile | grep -o "[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}" | sort -n | uniq -c | sort -nr|while read count ip
	name=$(echo $ip|/usr/bin/logresolve)
	printf "%5st%-15st%sn" $count $ip $name

# cat $tfile |grep -o "[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}" | sort -nr | uniq -c | sort -n

echo "
Top Agents :: "
cat $tfile | awk -F" '{print $6}'| sort -n | uniq -c | sort -nr |head -5 

echo "
Top urls ::"
cat $tfile  | awk -F" '{print $2}'| sort -n | uniq -c | sort -nr |head -5

echo -n "
Total Bytes ::: "
cat $tfile | awk '{ sum += $10 } END { if ( sum > 1024*1024) {print sum/(1024*1024)"Mb"}else if ( sum > 1024) {print sum/1024"Kb";}else print sum }'

echo -n "
Total Seconds :: "
cat $tfile  | awk '{ sum += $13 } END { print sum }'
# sed 's/.*GET (.*) HTTP.*/1/g' $tfile|awk -F/ '{if ( NF > 3 ) print $2"/"$3"/"$4; else print $2;}'|sort|uniq -c

Apache Server Stats – Download the script in zip file format.

And here is a sample ::

Report for <server> on <date>

Total hits :: 

User agent distribution :: 

User response code: 

404 Error Summary::

IP Visit counts :: 

Top Agents :: 

Top urls ::

Total Bytes ::: 

Total Seconds ::

Hope that it will be useful.

Enhanced by Zemanta

bash – using the vi mode more effectively.

Vi blogging bundle
Vi blogging bundle (Photo credit: pedro mg)

You can first set the bash mode to vi. This will enable some vim like features to bash. So, add this to .bashrc :

set -o vi

Once, you have done that then its time to get more out of the vi mode. First, check some things and set some interesting stuff.

#Display all the bindings :
bind -P

#Copy the current bindings and use it in inputrc:
bind -p |grep -v self-insert>~/.inputrc

In the inputrc, so generated, all the functions and macros not bound are commented with “#” and you can set them to your choice as in other examples.

And here is a link that can get you started.


Enhanced by Zemanta

zswap – compcache, compressed swap for better performance

First, here is a link to article on compcache.

zswap is already in the kernel and you can see the documentation in the kernel documentation. Here is the name of the file if you need:

/usr/share/doc/kernel-doc-$(uname -r)/Documentation/vm/zswap.txt

Here is the overview, in case you do not want to install kernel-doc


Zswap is a lightweight compressed cache for swap pages. It takes pages that are
in the process of being swapped out and attempts to compress them into a
dynamically allocated RAM-based memory pool.  zswap basically trades CPU cycles
for potentially reduced swap I/O.  This trade-off can also result in a
significant performance improvement if reads from the compressed cache are
faster than reads from a swap device.

NOTE: Zswap is a new feature as of v3.11 and interacts heavily with memory
reclaim.  This interaction has not be fully explored on the large set of
potential configurations and workloads that exist.  For this reason, zswap
is a work in progress and should be considered experimental.

Some potential benefits:
* Desktop/laptop users with limited RAM capacities can mitigate the
    performance impact of swapping.
* Overcommitted guests that share a common I/O resource can
    dramatically reduce their swap I/O pressure, avoiding heavy handed I/O
    throttling by the hypervisor. This allows more work to get done with less
    impact to the guest workload and guests sharing the I/O subsystem
* Users with SSDs as swap devices can extend the life of the device by
    drastically reducing life-shortening writes.

Zswap evicts pages from compressed cache on an LRU basis to the backing swap
device when the compressed pool reaches it size limit.  This requirement had
been identified in prior community discussions.

To enabled zswap, the “enabled” attribute must be set to 1 at boot time.  e.g.


Enhanced by Zemanta