find duplicate entry in a list in bash with sed

2010-06-25 1 min read Bash Fedora Learning

Here I will take an example of rss2email list, but I guess I will be able to pass on the concept.

Here is example of the output of the r2e list command:

1: http://blog.amit-agarwal.co.in/feed (default: amitag@localhost)
2: http://feeds2.feedburner.com/AllAboutLinux (default: amitag@localhost)
3: http://feeds2.feedburner.com/Command-line-fu (default: amitag@localhost)
4: http://blogs.members.freewebs.com/Members/Blogs/viewBlogRSS.jsp?userid=29731143 (default: amitag@localhost)

Target here is to get the list of all duplicate entries if any. So, first we need to remove the numbers from the begining and the email ID from the end.

remove/replace text/path in config file.

2010-06-25 1 min read Bash

Lot of times I end up changing a particular text in config file to some other text. And when I have to do it for multiple files, all files having the text in multiple places, I end up opening the files in vim and then doing a globlal replace. But this is not efficient so I cam up with some one liners to do this for me. For removing any occurance of text in all files in the directory :

regular expression, sed and grep tuturials and tricks.

2010-06-17 1 min read Linux

Simply a great one page article on sed and use of regular expressions.

<a href="http://almirkaric.com/2009/07/15/sed-and-grep-tips/" target="_blank">http://almirkaric.com/2009/07/15/sed-and-grep-tips/

Deleting lines from the text file

2010-06-01 1 min read Linux

Really useful for pretty big <a class="zem_slink freebase/en/computer_file" title="Computer file" rel="wikipedia" href="http://en.wikipedia.org/wiki/Computer_file">files.

<a class="zem_slink freebase/en/sed" title="Sed" rel="wikipedia" href="http://en.wikipedia.org/wiki/Sed">sed –silent ’1,100 d’ <a class="zem_slink freebase/en/filename" title="Filename" rel="wikipedia" href="http://en.wikipedia.org/wiki/Filename">filename

here the command will delete the first 100 lines. You can specify whatever range you want. This will not delete the lines from the file but only display the contents on the console after deleting the 100 lines from begining.

If you want to store it in different file then you can use ”>” or ”»”<h6 class="zemanta-related-title">Related articles by Zemanta <ul class="zemanta-article-ul"> <li class="zemanta-article-ul-li"><a href="http://www.centernetworks.com/guhmshoo-quit-facebook-day-cartoon">Guhmshoo Looks at Quit Facebook Day (cartoon) (centernetworks.com) <li class="zemanta-article-ul-li"><a href="http://blogs.securiteam.com/index.php/archives/1374">Sometimes it’s just Windows … (blogs.securiteam.com) <li class="zemanta-article-ul-li"><a href="http://www.makeuseof.com/tag/2-ingenious-apps-to-help-organize-files-and-folders-on-your-mac/">2 Ingenious Apps To Help Organize Files and Folders On Your Mac (makeuseof.com) <li class="zemanta-article-ul-li"><a href="http://www.labnol.org/software/windows-problems-and-solutions/13766/">Simple Solutions to Common Windows Problems (labnol.org) <li class="zemanta-article-ul-li"><a href="http://techblissonline.com/bluray-dvd-cd-data-file-recovery-software-free-download/">Blu Ray/DVD/CD Data file recovery software (free) to recover damaged files (techblissonline.com) <li class="zemanta-article-ul-li"><a href="http://www.ghacks.net/2010/05/23/get-rid-of-gnome-keyring-password-reminder/">Get rid of GNOME keyring password reminder (ghacks.net) <div class="zemanta-pixie"><a class="zemanta-pixie-a" title="Reblog this post [with Zemanta]" href="http://reblog.zemanta.com/zemified/a2e0d6ba-15f8-4a53-9f52-a28012b689cb/"><img class="zemanta-pixie-img" src="http://blog.amit-agarwal.co.in/wp-content/uploads/2010/08/reblog_b93.png" alt="Reblog this post [with Zemanta]" /><span class="zem-script more-related more-info pretty-attribution paragraph-reblog">

vim – get the list of all the scripts available.

2010-05-27 1 min read Learning Linux Vim Tips

To get a list of all the scripts in the vim.sf.net project, you can run the following command. This will list all the scripts with the script id.

wget -O – ’http://www.vim.org/scripts/script_search_results.php?&show_me=4000&result_ptr=0’ -o /dev/null | <a class="zem_slink freebase/en/grep" title="Grep" rel="wikipedia" href="http://en.wikipedia.org/wiki/Grep">grep width| grep script_id= |<a class="zem_slink freebase/en/sed" title="Sed" rel="wikipedia" href="http://en.wikipedia.org/wiki/Sed">sed ’s/.*script_id=//’|tr -d ”””|tr ”>” ” ”|sed ’s/< .*//’ | tr -d ’\001’-’\011’’\013’’\014’’\016’-’\037’’\200’-’\377’ |sort -n<h6 class="zemanta-related-title">Related articles by Zemanta <ul class="zemanta-article-ul"> <li class="zemanta-article-ul-li"><a href="http://www.slumpedoverkeyboarddead.com/2009/09/17/use-curl-to-monitor-your-vonage-phone-bill/">Use curl to Monitor Your Vonage Phone Bill (slumpedoverkeyboarddead.com) <li class="zemanta-article-ul-li"><a href="http://securewebs.com/yum-updates/">Yum Updates (securewebs.com) <li class="zemanta-article-ul-li"><a href="http://www.gubatron.com/blog/2009/10/10/ubuntudebian-quick-reference-how-to-change-your-servers-utc-timezone-on-the-command-line/">Ubuntu/Debian Quick Reference: How To Change Your Server’s UTC Timezone on the command line (gubatron.com) <li class="zemanta-article-ul-li"><a href="http://blog.amit-agarwal.co.in/2009/11/22/short-information-about-loaded-kernel-modules/">Short Information about loaded kernel modules (amit-agarwal.co.in) <div class="zemanta-pixie"><a class="zemanta-pixie-a" title="Reblog this post [with Zemanta]" href="http://reblog.zemanta.com/zemified/b22b2e85-8107-4f9b-9130-85a8e8ae370b/"><img class="zemanta-pixie-img" src="http://blog.amit-agarwal.co.in/wp-content/uploads/2010/08/reblog_b52.png" alt="Reblog this post [with Zemanta]" /><span class="zem-script more-related more-info pretty-attribution paragraph-reblog">

Downloading all related videos from youtube with youtube-dl

2010-05-16 2 min read Fedora Learning Linux Uncategorized

How many time did you want to see all the videos (related) to the one you were watching on youtube but decided otherwise as the downloaded was not fast enough. Or has it happened to you that you wanted to <a class="zem_slink freebase/en/uploading_and_downloading" title="Uploading and downloading" rel="wikipedia" href="http://en.wikipedia.org/wiki/Uploading_and_downloading">download all the related videos, in either case you can use the youtube-dl utility I blogged about some times back. First you need to get all the <a class="zem_slink freebase/en/uniform_resource_locator" title="Uniform Resource Locator" rel="wikipedia" href="http://en.wikipedia.org/wiki/Uniform_Resource_Locator">URL’s in a <a class="zem_slink freebase/en/text_file" title="Text file" rel="wikipedia" href="http://en.wikipedia.org/wiki/Text_file">text file, and the simplest way to do this is :

Advanced Sed Substitution Examples

2010-04-11 1 min read Bash Linux

This article is part of the on-going Unix Sed Tips and Tricks series. In our previous sed articles we learned — sed printing, sed deletion, sed substitute , sed file write, and sed multiple commands. In this article, let us review some interesting workarounds with the “s” substitute command in sed with several practical examples. I. […]

Read More: <a href="http://www.thegeekstuff.com/2009/10/unix-sed-tutorial-advanced-sed-substitution-examples/">Unix Sed Tutorial: Advanced Sed Substitution Examples

URL: <a href="http://feedproxy.google.com/~r/TheGeekStuff/~3/uN-cE6UOvfA/">http://feedproxy.google.com/~r/TheGeekStuff/~3/uN-cE6UOvfA/<h6 class="zemanta-related-title" style="font-size: 1em;">Related articles by Zemanta <ul class="zemanta-article-ul"> <li class="zemanta-article-ul-li"><a href="http://www.slumpedoverkeyboarddead.com/2009/08/06/vim-video-tutorial/">VIM video tutorial (slumpedoverkeyboarddead.com) <li class="zemanta-article-ul-li"><a href="http://www.slumpedoverkeyboarddead.com/2009/08/22/turn-vim-or-emacs-into-and-ide-with-exuberant-ctags/">Turn Vim or Emacs Into and IDE With Exuberant-Ctags (slumpedoverkeyboarddead.com) <div class="zemanta-pixie" style="margin-top: 10px; height: 15px;"><a class="zemanta-pixie-a" title="Reblog this post [with Zemanta]" href="http://reblog.zemanta.com/zemified/0d68ebaf-6fc5-419a-a31c-7d31bc55ab3f/"><img class="zemanta-pixie-img" style="border: medium none; float: right;" src="http://blog.amit-agarwal.co.in/wp-content/uploads/2010/08/reblog_e8.png" alt="Reblog this post [with Zemanta]" /><span class="zem-script more-related pretty-attribution">