pigz -parallel gzip

2012-03-26 1 min read Fedora Linux
Here is a short description of pigz: pigz, which stands for parallel implementation of gzip, is a fully functional replacement for gzip that exploits multiple processors and multiple cores to the hilt when compressing data. And for the installation: sudo yum install pigz With pigz, if you don’t have many things running on your multi processor machine then you will see a significant improvement when you are gzipping the files. Continue reading

Quick tip on zipping logs in real time.

2010-10-13 1 min read bash Learning Linux
Sometimes, some small things that we don’t actually think can be useful are such useful. I faced this couple of days back when I was working on something and the amount of logs getting generated and the files getting rotated was too fast. If I had to use this for sometime, I needed some script, application or something to make sure that the logs are zipped every few seconds. Finding an application for this would take time and what good is bash if we need to find applications for this. Continue reading

concatenate compressed and uncompressed logs

2010-07-15 1 min read bash Fedora Linux
<span style="font-size: x-large;"><a href="http://feedproxy.google.com/~r/Command-line-fu/~3/iwFUyltYgjM/concatenate-compressed-and-uncompressed-logs">concatenate compressed and uncompressed logs $ find /var/log/apache2 -name 'access.loggz' -exec <a class="zem_slink freebase/guid/9202a8c04000641f800000000001abd9" title="Gzip" rel="homepage" href="http://www.gzip.org/">zcat {} ; -or -name 'access.log' -exec cat {} ; This command allows you to stream your log files, including gziped files, into one stream which can be piped to <a class="zem_slink freebase/guid/9202a8c04000641f800000000000584b" title="AWK" rel="homepage" href="http://cm.bell-labs.com/cm/cs/awkbook/index.html">awk or some other command for analysis. Note: if your version of &#8217;find&#8217; supports it, use: Continue reading