find duplicate entry in a list in bash with sed

2010-06-25 1 min read Bash Fedora Learning

Here I will take an example of rss2email list, but I guess I will be able to pass on the concept.

Here is example of the output of the r2e list command:

1: http://blog.amit-agarwal.co.in/feed (default: amitag@localhost)
2: http://feeds2.feedburner.com/AllAboutLinux (default: amitag@localhost)
3: http://feeds2.feedburner.com/Command-line-fu (default: amitag@localhost)
4: http://blogs.members.freewebs.com/Members/Blogs/viewBlogRSS.jsp?userid=29731143 (default: amitag@localhost)

Target here is to get the list of all duplicate entries if any. So, first we need to remove the numbers from the begining and the email ID from the end.

Continue reading

remove/replace text/path in config file.

2010-06-25 1 min read Bash

Lot of times I end up changing a particular text in config file to some other text. And when I have to do it for multiple files, all files having the text in multiple places, I end up opening the files in vim and then doing a globlal replace. But this is not efficient so I cam up with some one liners to do this for me. For removing any occurance of text in all files in the directory :

Continue reading

bash script to change the vim colorscheme from the list of locally available schemes

2010-06-24 0 min read Bash Fedora Learning Linux Vim Tips
\"MacVim
Image via Wikipedia

Today I was just wondering how good it would be to have a script to change the <a class="zem_slink freebase/en/vim" title="Vim (text editor)" rel="homepage" href="http://www.vim.org/">vim colorscheme from the <a class="zem_slink freebase/en/cmd_exe" title="Command Prompt" rel="wikipedia" href="http://en.wikipedia.org/wiki/Command_Prompt">command prompt. The plan for the script was simply to print the list of available schemes and then let the user select the actions from there on. So, here is the result:

Continue reading

Script to add all the partitions to the fstab.

2010-06-24 5 min read Bash Fedora

I wrote a simple script today to get all the partitions on all the disks and then create <a class="zem_slink freebase/en/fstab" title="Fstab" rel="wikipedia" href="http://en.wikipedia.org/wiki/Fstab">fstab entry with them. The script also gets the <a class="zem_slink freebase/en/universally_unique_identifier" title="Universally Unique Identifier" rel="wikipedia" href="http://en.wikipedia.org/wiki/Universally_Unique_Identifier">UUID and uses the same to make the entry 🙂

  <td>
    <div class="bash codecolorer">
      <span class="co0">#!/bin/bash -</span><br /> <span class="co0">#===============================================================================</span><br /> <span class="co0">#</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;FILE: &nbsp;mount_add_all.sh</span><br /> <span class="co0">#</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; &nbsp; USAGE: &nbsp;./mount_add_all.sh</span><br /> <span class="co0">#</span><br /> <span class="co0"># &nbsp; DESCRIPTION: &nbsp;Find all the <a class="zem_slink freebase/en/disk_partitioning" title="Disk partitioning" rel="wikipedia" href="http://en.wikipedia.org/wiki/Disk_partitioning">disk partitions</a> and all those that are not present</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; &nbsp; in the fstab.</span><br /> <span class="co0">#</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; OPTIONS: &nbsp;---</span><br /> <span class="co0"># &nbsp;REQUIREMENTS: &nbsp;---</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; &nbsp; &nbsp;BUGS: &nbsp;---</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; &nbsp; NOTES: &nbsp;---</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; &nbsp;AUTHOR: &nbsp;<a class="zem_slink" title="Digital Inspiration" rel="homepage" href="http://amit-agarwal.co.in">Amit Agarwal</a> (AKA), amit.agarwal@amit-agarwal.co.in</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; COMPANY: &nbsp;Individual</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; VERSION: &nbsp;1.0</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp; CREATED: &nbsp;06/19/2010 10:44:34 PM IST</span><br /> <span class="co0"># &nbsp; &nbsp; &nbsp;REVISION: &nbsp;0.1</span><br /> <span class="co0">#===============================================================================</span><br /> <span class="re2">file</span>=<span class="st0">"/tmp/fstab"</span><br /> <span class="re2">disk_types</span>=<span class="br0">&#40;</span>hd sd<span class="br0">&#41;</span><br /> <span class="re2">count</span>=<span class="co1">${#disk_types}</span><br /> <br /> find_disks <span class="br0">&#40;</span><span class="br0">&#41;</span><br /> <span class="br0">&#123;</span><br /> <span class="kw3">echo</span> <span class="co1">${#disk_types}</span> <span class="re1">$count</span><br /> <span class="kw1">for</span> i <span class="kw1">in</span> $<span class="br0">&#40;</span><span class="kw2">seq</span> <span class="nu0">1</span> <span class="re1">$count</span><span class="br0">&#41;</span><br /> <span class="kw1">do</span><br /> <span class="br0">&#40;</span><span class="br0">&#40;</span> <span class="re2">j</span>=i-<span class="nu0">1</span> <span class="br0">&#41;</span><span class="br0">&#41;</span><br /> <span class="kw3">echo</span> <span class="re1">$i</span> <span class="re1">$j</span><br /> <span class="kw3">echo</span> <span class="re1">$j</span> <span class="co1">${disk_types[0]}</span> <span class="co1">${disk_types[1]}</span><br /> disk_array<span class="br0">&#91;</span><span class="re1">$j</span><span class="br0">&#93;</span>=$<span class="br0">&#40;</span><span class="kw2">ls</span> <span class="sy0"><</span>a <span class="re2">class</span>=<span class="st0">"zem_slink freebase/en/device_file_system"</span> <span class="re2">title</span>=<span class="st0">"Device file"</span> <span class="re2">rel</span>=<span class="st0">"wikipedia"</span> <span class="re2">href</span>=<span class="st0">"http://en.wikipedia.org/wiki/Device_file"</span><span class="sy0">>/</span>dev<span class="sy0"></</span>a<span class="sy0">>/</span><span class="co1">${disk_types[$j]}</span><span class="br0">&#91;</span>a-z<span class="br0">&#93;</span><span class="br0">&#41;</span><br /> <span class="kw3">echo</span> <span class="co1">${disk_array[$j]}</span><br /> <span class="kw1">done</span><br /> <span class="br0">&#125;</span> &nbsp; &nbsp;<span class="co0"># ---------- &nbsp;end of function find_disks &nbsp;----------</span><br /> <br /> find_partitions <span class="br0">&#40;</span><span class="br0">&#41;</span><br /> <span class="br0">&#123;</span><br /> <span class="kw3">echo</span> <span class="st0">"In find_partitions"</span><br /> <span class="re2">l</span>=<span class="nu0"></span><br /> <span class="re2">count</span>=<span class="co1">${#disk_types}</span><br /> <span class="kw1">for</span> i <span class="kw1">in</span> $<span class="br0">&#40;</span><span class="kw2">seq</span> <span class="nu0">1</span> <span class="re1">$count</span><span class="br0">&#41;</span><br /> <span class="kw1">do</span><br /> <span class="br0">&#40;</span><span class="br0">&#40;</span><span class="re2">j</span>=i-<span class="nu0">1</span><span class="br0">&#41;</span><span class="br0">&#41;</span><br /> <span class="kw1">for</span> k <span class="kw1">in</span> <span class="co1">${disk_array[$i]}</span><br /> <span class="kw1">do</span><br /> <span class="kw3">echo</span> <span class="re1">$k</span><br /> <span class="sy0"><!</span>--loginview start--<span class="sy0">></span>disk_partitions<span class="br0">&#91;</span><span class="re1">$l</span><span class="br0">&#93;</span>=<span class="sy0">`</span><span class="kw2">fdisk</span> <span class="re5">-l</span> <span class="re1">$k</span> <span class="sy0">|<</span>a <span class="re2">class</span>=<span class="st0">"zem_slink freebase/en/grep"</span> <span class="re2">title</span>=<span class="st0">"Grep"</span> <span class="re2">rel</span>=<span class="st0">"wikipedia"</span> <span class="re2">href</span>=<span class="st0">"http://en.wikipedia.org/wiki/Grep"</span><span class="sy0">></span><span class="kw2">grep</span><span class="sy0"></</span>a<span class="sy0">></span> <span class="st0">"^/dev"</span> <span class="sy0">|</span><span class="kw2">grep</span> <span class="re5">-v</span> <span class="st0">"Extended$"</span><span class="sy0">|</span><span class="kw2">cut</span> <span class="re5">-d</span><span class="st0">" "</span> -f1<span class="sy0">`</span><br /> <span class="kw3">echo</span> <span class="co1">${disk_partitions[$l]}</span> <span class="re1">$l</span><span class="sy0"><!</span>--loginview end--<span class="sy0">></span><br /> <span class="br0">&#40;</span><span class="br0">&#40;</span>l++<span class="br0">&#41;</span><span class="br0">&#41;</span><br /> <span class="kw1">done</span><br /> <span class="kw1">done</span><br /> <span class="br0">&#125;</span> &nbsp; &nbsp;<span class="co0"># ---------- &nbsp;end of function find_partitions &nbsp;----------</span><br /> <br /> add_fat <span class="br0">&#40;</span><span class="br0">&#41;</span><br /> <span class="br0">&#123;</span><br /> <span class="kw3">echo</span> <span class="st0">"# Added with script for <span class="es2">$label</span> and <span class="es2">$k</span>"</span> <span class="sy0">>></span> <span class="re1">$file</span><br /> <span class="kw3">echo</span> <span class="st0">"<span class="es2">$uuid</span> /mnt/<span class="es2">$label</span> vfat&nbsp; nosuid,nodev,uhelper=udisks,uid=500,gid=500,shortname=mixed,dmask=0077&nbsp; 0 0"</span> &nbsp;<span class="sy0">>></span> <span class="re1">$file</span><br /> <span class="br0">&#125;</span> &nbsp; &nbsp;<span class="co0"># ---------- &nbsp;end of function add_fat &nbsp;----------</span><br /> <br /> add_<span class="sy0"><</span>a <span class="re2">class</span>=<span class="st0">"zem_slink freebase/en/ext3"</span> <span class="re2">title</span>=<span class="st0">"Ext3"</span> <span class="re2">rel</span>=<span class="st0">"wikipedia"</span> <span class="re2">href</span>=<span class="st0">"http://en.wikipedia.org/wiki/Ext3"</span><span class="sy0">></span>ext3<span class="sy0"></</span>a<span class="sy0">></span> <span class="br0">&#40;</span><span class="br0">&#41;</span><br /> <span class="br0">&#123;</span><br /> <span class="kw3">echo</span> <span class="st0">"# Added with script for <span class="es2">$label</span> and <span class="es2">$k</span>"</span> <span class="sy0">>></span> <span class="re1">$file</span><br /> <span class="kw3">echo</span> <span class="st0">"<span class="es2">$uuid</span> /mnt/<span class="es2">$label</span> ext3     defaults     0 0"</span> <span class="sy0">>></span> <span class="re1">$file</span><br /> <br /> <span class="br0">&#125;</span> &nbsp; &nbsp;<span class="co0"># ---------- &nbsp;end of function add_ext3 &nbsp;----------</span><br /> <br /> find_UUID <span class="br0">&#40;</span><span class="br0">&#41;</span><br /> <span class="br0">&#123;</span><br /> <span class="kw3">echo</span> <span class="st0">"In find_UUID with <span class="es2">$l</span>"</span><br /> <span class="br0">&#40;</span><span class="br0">&#40;</span>l--<span class="br0">&#41;</span><span class="br0">&#41;</span><br /> <span class="kw1">for</span> i <span class="kw1">in</span> $<span class="br0">&#40;</span><span class="kw2">seq</span> <span class="nu0"></span> <span class="re1">$l</span><span class="br0">&#41;</span><br /> <span class="kw1">do</span><br /> <span class="kw3">echo</span> <span class="co1">${disk_partitions[$i]}</span><br /> <span class="kw1">for</span> k <span class="kw1">in</span> <span class="co1">${disk_partitions[$i]}</span><br /> <span class="kw1">do</span><br /> <span class="kw3">echo</span> <span class="re1">$k</span><br /> <span class="sy0"><!</span>--loginview start--<span class="sy0">></span><span class="re2">temp</span>=$<span class="br0">&#40;</span>blkid <span class="re1">$k</span><span class="br0">&#41;</span><br /> <span class="re2">label</span>=$<span class="br0">&#40;</span><span class="kw3">echo</span> <span class="re1">$temp</span><span class="sy0">|</span><span class="kw2">grep</span> <span class="st0">"LABEL="</span><span class="sy0">|</span><span class="kw2">sed</span> <span class="co3">\'</span>s<span class="sy0">/</span>.<span class="sy0">*</span><span class="re2">LABEL</span>=<span class="st0">"//\'|sed \'s/"</span>.<span class="sy0">*//</span><span class="co3">\'</span><span class="br0">&#41;</span><br /> <span class="re2">uuid</span>=$<span class="br0">&#40;</span><span class="kw3">echo</span> <span class="re1">$temp</span><span class="sy0">|</span><span class="kw2">grep</span> <span class="st0">"UUID="</span><span class="sy0">|</span><span class="kw2">sed</span> <span class="co3">\'</span>s<span class="sy0">/</span>.<span class="sy0">*</span><span class="re2">UUID</span>=<span class="st0">"//\'|sed \'s/"</span>.<span class="sy0">*//</span><span class="co3">\'</span><span class="br0">&#41;</span><br /> <span class="re2">atype</span>=$<span class="br0">&#40;</span><span class="kw3">echo</span> <span class="re1">$temp</span><span class="sy0">|</span><span class="kw2">grep</span> <span class="st0">"TYPE="</span><span class="sy0">|</span><span class="kw2">sed</span> <span class="co3">\'</span>s<span class="sy0">/</span>.<span class="sy0">*</span><span class="re2">TYPE</span>=<span class="st0">"//\'|sed \'s/"</span>.<span class="sy0">*//</span><span class="co3">\'</span><span class="br0">&#41;</span><span class="sy0"><!</span>--loginview end--<span class="sy0">></span><br /> <span class="kw1">if</span> <span class="br0">&#91;</span> <span class="re1">$atype</span> = <span class="st0">"vfat"</span> <span class="br0">&#93;</span><br /> <span class="kw1">then</span><br /> add_fat <span class="re1">$label</span> <span class="re1">$uuid</span> <span class="re1">$k</span><br /> <span class="kw1">fi</span><br /> <span class="kw1">if</span> <span class="br0">&#91;</span> <span class="re1">$atype</span> = <span class="st0">"ext3"</span> <span class="br0">&#93;</span><br /> <span class="kw1">then</span><br /> add_ext3 <span class="re1">$label</span> <span class="re1">$uuid</span> <span class="re1">$k</span><br /> <span class="kw1">fi</span><br /> <span class="kw3">echo</span> <span class="re1">$label</span> <span class="re1">$uuid</span> <span class="re1">$atype</span><br /> <span class="kw1">done</span><br /> <span class="kw1">done</span><br /> <span class="br0">&#125;</span> &nbsp; &nbsp;<span class="co0"># ---------- &nbsp;end of function find_UUID &nbsp;----------</span><br /> <br /> find_disks<br /> find_partitions<br /> find_UUID
    </div>
  </td>
</tr>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
\"Enhanced

Filename completion in Linux with mixed case and ignored case.

2010-06-23 1 min read Bash Linux

This is one of my favourites, alwahs keep it handy. Linux is case sensitive by nature but you will always find filenames jumbled up in the case. So what is the solution:

bind &#8221;set completion-ignore-case on&#8221;

Just execute the above in your bash command and file/dir name will behave like Windows. Remember that this is per session setting and thus the names do not get changed internally, it only changes the way to access it (specifically completion only). So if you want this behaviour in all your sessions put this in the appropriate profile file.

Continue reading

cdargs – bash cd command with bookmarks and browser.

2010-06-22 0 min read Bash Learning Linux
\"Screenshot
Image via Wikipedia

cdargs is a <a class="zem_slink freebase/en/bash" title="Bash" rel="homepage" href="http://tiswww.case.edu/php/chet/bash/bashtop.html">bash shell cd with bookmarks and browser. Quite useful if you are developer or someone who use&#8217;s the bash shell very frequently

Name       : cdargs

Arch : i686 Version : 1.35 Release : 5.fc12 Size : 114 k Repo : installed From repo : fedora Summary : The shell cd with bookmarks and browser URL : http://www.skamphausen.de/software/cdargs/ License : <a class="zem_slink freebase/en/gnu_general_public_license" title="GNU General Public License" rel="wikipedia" href="http://en.wikipedia.org/wiki/GNU_General_Public_License">GPLv2</a>+ Description: Enables the user to quickly change <a class="zem_slink freebase/en/working_directory" title="Working directory" rel="wikipedia" href="http://en.wikipedia.org/wiki/Working_directory">working directory</a> by navigating cd arguments : and expanding the shell built-in cd with bookmarks and browser.

Continue reading
Older posts Newer posts