Unix shell script for removing duplicate files

2011-05-16 1 min read Bash Linux

The following shell script finds duplicate (2 or more identical) files and outputs a new shell script containing commented-out rm statements for deleting them (copy-paste from here):

::: updated on 02 May 20121, seems like wordpress did not like it so well so reformatting the code :::::::

#!/bin/bash -
#===============================================================================
#
#          FILE:  a.sh
#
#         USAGE:  ./a.sh
#
#   DESCRIPTION:
#
#       OPTIONS:  ---
#  REQUIREMENTS:  ---
#          BUGS:  ---
#         NOTES:  ---
#        AUTHOR: Amit Agarwal (aka), amit.agarwal@roamware.com
#       COMPANY: blog.amit-agarwal.co.in
#       CREATED: 02/05/12 06:52:08 IST
# Last modified: Wed May 02, 2012  07:03AM
#      REVISION:  ---
#===============================================================================

OUTF=rem-duplicates.sh;
echo "#!/bin/sh" >$OUTF;
find "$@" -type f -exec md5sum {} \; 2>/dev/null | sort --key=1,32 | uniq -w 32 -d |cut -b 1-32 --complement |sed 's/^/rm -f/' >>$OUTF

Pretty good one line, I must say 🙂

Continue reading

mussh - Multihost SSH wrapper

2010-05-25 1 min read Bash Fedora Linux

Description:

Mussh is a <a class="zem_slink freebase/en/shell_script" title="Shell script" rel="wikipedia" href="http://en.wikipedia.org/wiki/Shell_script">shell script that allows you to execute a command or script over ssh on multiple hosts with one command. When possible mussh will use <a class="zem_slink freebase/en/ssh-agent" title="Ssh-agent" rel="wikipedia" href="http://en.wikipedia.org/wiki/Ssh-agent">ssh-agent and RSA/DSA keys to minimize the need to enter your password more than once.

It is a utility for <a class="zem_slink freebase/en/system_administrator" title="System administrator" rel="wikipedia" href="http://en.wikipedia.org/wiki/System_administrator">system administrators maintaining more than one hosts. The utility will help the admin to execute the same command on multiple hosts with a single command.

Continue reading

Awk Introduction Tutorial – 7 Awk Print Examples

2010-05-19 0 min read Bash Fedora Linux
\"LAS
Image by Getty Images via Daylife

awk examples

Awk Introduction Tutorial – 7 Awk Print Examples This is the first article on the new awk tutorial series. We’ll be posting several articles on awk in the upcoming weeks that will cover all features of awk with practical examples. In this article, let us review the fundamental awk working methodology along with 7 practical awk print examples. Note: Make sure you review our earlier […]

Continue reading

Linux Shell Scripting Tutorial – A Beginner\’s handbook

2010-04-11 0 min read Bash Learning Linux
\"Tux,
Image via Wikipedia

<a class="zem_slink freebase/en/unix_shell" title="Unix shell" rel="wikipedia" href="http://en.wikipedia.org/wiki/Unix_shell">Linux Shell Scripting Tutorial – A Beginner&#8217;s handbook —

This book is for students and Linux System Administrators. It provides the skills to read, write, and debug <a title="Linux" href="http://bash.cyberciti.biz/guide/Linux">Linux <a class="zem_slink freebase/en/shell_script" title="Shell script" rel="wikipedia" href="http://en.wikipedia.org/wiki/Shell_script">shell scripts using <a title="Bash" href="http://bash.cyberciti.biz/guide/Bash">bash <a title="Shell" href="http://bash.cyberciti.biz/guide/Shell">shell. The book begins by describing <a title="Linux" href="http://bash.cyberciti.biz/guide/Linux">Linux and simple scripts to automate frequently executed commands and continues by describing conditional logic, <a class="zem_slink freebase/en/interactivity" title="Interactivity" rel="wikipedia" href="http://en.wikipedia.org/wiki/Interactivity">user interaction, loops, menus, traps, and functions. Finally, book covers various sys admin related scripts such as making a backup, using <a class="zem_slink freebase/guid/9202a8c04000641f80000000045c9c5b" title="Cron" rel="wikipedia" href="http://en.wikipedia.org/wiki/Cron">cron jobs, writing interactive tools, web based tools, remote login, ftp and database backup related scripts. This book is intended for Linux <a class="zem_slink freebase/en/system_administrator" title="System administrator" rel="wikipedia" href="http://en.wikipedia.org/wiki/System_administrator">system administrators or students who have mastered the basics of a Linux Operating System. You should be able to:

Continue reading

prepend to a file with sponge from moreutils

2010-03-16 1 min read Linux

<a href="http://bashcurescancer.com/prepend-to-a-file-with-sponge-from-moreutils.html">A few weeks I wrote about a tool, which helps you easily prepend to a file. I submitted prepend to moreutils and Joey was kind enough to point out this could be done with `sponge&#8217;.? sponge reads standard input and when done, writes it to a file:
Probably the most general …<h6 class="zemanta-related-title">Related articles by Zemanta <ul class="zemanta-article-ul"> <li class="zemanta-article-ul-li"><a href="http://helpdeskgeek.com/linux-tips/crontab-howto-tutorial-syntax/">Automating Tasks in Linux Using Crontab (helpdeskgeek.com) <li class="zemanta-article-ul-li"><a href="http://www.slideshare.net/progrium/sxsw-evented-web">How WebHooks Will Make Us All Programmers (slideshare.net) <li class="zemanta-article-ul-li"><a href="http://almirkaric.com/2010/01/07/debugging-python-multiprocessing/">Debugging python (multi)processing (almirkaric.com) <li class="zemanta-article-ul-li"><a href="http://www.slideshare.net/royzimmer/you-can-do-it-start-using-perl-to-handle-your-voyager-needs">You Can Do It! Start Using Perl to Handle Your Voyager Needs (slideshare.net) <div class="zemanta-pixie"><a class="zemanta-pixie-a" title="Reblog this post [with Zemanta]" href="http://reblog.zemanta.com/zemified/b2b2afad-1679-46ab-831c-90ae0cc4d30f/"><img class="zemanta-pixie-img" src="http://blog.amit-agarwal.co.in/wp-content/uploads/2010/08/reblog_b14.png" alt="Reblog this post [with Zemanta]" /><span class="zem-script more-related more-info pretty-attribution paragraph-reblog">

Continue reading
Newer posts