bash Linux

Splitting strings with IFS

Today I want to discuss splitting strings into tokens or “words”. I previously discussed how to do this with the IFS variable and promised a more in depth discussion. Today, I will make the case on WHY to use IFS to split strings as opposed to using a subshell combined with awk or cut.

I wrote this script which reads the /etc/password file line-by-line and prints the username of any user which has a UID greater than 10 and has the shell of /sbin/nologin. Each test function performs this task 10 times to increase the length of the test:

 [root@sandbox ~]# cat #!/bin/bash split_words_cut() { # execute 10 times for i in {0..9} do while read line do # get uid id=$(echo $line | cut -d: -f3) if [[ $id -gt 10 ]] then # get shell shell=$(echo $line | echo $line | cut -d: -f7) if [[ \'/sbin/nologin\' == \"$shell\" ]] then # print username echo $line | cut -d: -f1 fi fi done < /etc/passwd done }

split_words_awk() { # execute 10 times for i in {0..9} do while read line do # get uid id=$(echo $line | awk -F: \'{print $3}\') if [[ $id -gt 10 ]] then # get shell shell=$(echo $line | awk -F: \'{print $NF}\') if [[ \'/sbin/nologin\' == \"$shell\" ]] then # print username echo $line | awk -F: \'{print $1}\' fi fi done < /etc/passwd done } split_words_native() { # execute 10 times for i in {0..9} do while read line do oldIFS=$IFS IFS=: set -- $line IFS=$oldIFS # at this point $1 is the username, $3 # is the uid, and $7 is the shell if [[ $3 -gt 10 ]] && [[ \'/sbin/nologin\' == \"$7\" ]] then echo $1 fi done < /etc/passwd done } echo -e \"---Cut---\" time split_words_cut >/dev/null echo -e \"n---Awk---\" time split_words_awk >/dev/null echo -e \"n---Native---\" time split_words_native >/dev/null

As you can see, using the shell itself is about two orders of magnitude faster than using the subshell awk/cut method:

 [root@sandbox ~]# ./
Learning Linux

Understand Awk Variables with 3 Practical Examples


\"\" This article is part of the on-going Awk Tutorial and Examples series. Like any other programming languages, Awk also has user defined variables and built-in variables. In this article let us review how to define and use awk variables. Awk variables should begin with the letter, followed by it can consist of alpha numeric characters or underscore. Keywords […]

Read More: Awk Tutorial: Understand Awk Variables with 3 Practical Examples


Short Information about loaded kernel modules

There are couple of ways you can find the information on the loadable kernel modules. All these would always involve calling lsmod to get the loaded kernel modules and then calling modinfo to get the info on the loaded modules. I will show you with examples:

$ lsmod
Module                  Size  Used by
cdc_acm                19616  0
vfat                    8744  9
fat                    41836  1 vfat

The above output is for the list of the modules loaded on the kernel.

$ modinfo vfat

filename:       /lib/modules/
author:         Gordon Chaffee
description:    VFAT filesystem support
license:        GPL
srcversion:     48F6DF1D674F0E1325466C9
depends:        fat
vermagic: SMP mod_unload 686

modinfo will list the filename, authour and other related info for the modules. So if you wanted to get the info for all the modules, then the easiest way would be to do :

lsmod|xargs modinfo

But as you can see, this will list all the info on  the module, which is not really required. So there were couple of posts on commandlinefu, that takes care of displaying only the required information, viz. filename, authour and dependency. Here are the commands:

lsmod | sed \’1d\’ | cut -d\’ \’ -f1 | xargs modinfo | egrep \’^file|^desc|^dep\’ | sed -e\’/^dep/s/$/\\n/g\’

Quite raw method where we grep the required fields and susbstitute the end with newline for the last patter \":)\"

lsmod | sed -e \’1d\’ -e \’s/\\(\\([^ ]*\\) \\)\\{1\\}.*/\\2/\’ | xargs modinfo | sed -e \’/^dep/s/$/\\n/g\’ -e \’/^file/b\’ -e \’/^desc/b\’ -e \’/^dep/b\’ -e d

Quite same as above with only exception of using one sed command rather than egrep and sed.

modinfo $(cut -d\’ \’ -f1 /proc/modules) | sed \’/^dep/s/$/\\n/; /^file\\|^desc\\|^dep/!d\’

Directly using the modinfo commands without xargs and passing the arguments with cut command and then using sed to display only the required fields.

awk \'{print $1}\’ \”/proc/modules\” | xargs modinfo | awk \’/^(filename|desc|depends)/\’

using awk to check the /proc/modules and then displaying the fields with awk