Improving DNF Update Speeds with max_parallel_downloads

When managing a Fedora or RHEL-based system, downloading updates can sometimes be time-consuming, especially when dealing with a slow network connection or multiple large packages or sometimes too many small packages 😀. By default, the DNF package manager downloads 1/3 package at a time, which can slow down the process. However, you can significantly improve download times by tweaking the max_parallel_downloads setting.

What is max_parallel_downloads?

max_parallel_downloads is a configuration option in DNF that controls how many packages are downloaded simultaneously during system updates. By increasing the number of parallel downloads, you can take full advantage of your available network bandwidth and reduce the overall time needed for downloading updates.

Bash script performance issue and remediation

I was looking at writing some output to files. The output did not vary much but the number of lines was huge. It was long time back so I dont remember the context of the requirement but I needed to write few million lines of text. The quickest way to do this was bash so I wrote a simple script that could do this. To my astonishment, I could see the script much more time than expected, so here is how I debugged the issue and fixed it.

scripting – performance improvement with file open

Sometimes just one line of code can make all the difference. I will show you with example.

Here is script with 2 functions. Both are writing some lines of text to a file. First function, “a” I have used the redirection to write to file. Function “b”, I have opened a file descriptor with “>” before going into the loop and use that reference for writing to the file. (This concept remains same for any scripting or programming language).

glances – new way to look at contention

glances is like top/htop but little different. It shows you sort based on contention (smartly and automatically) unless you change that and hence if you just want to check what is biggest bottleneck in system, then head over to glances quickly. Here is quick description from dnf info command

Name : glances

Version : 2.11.1

Release : 2.fc28

Arch : noarch

Size : 3.2 M

Source : glances-2.11.1-2.fc28.src.rpm

Repo : @System

From repo : fedora

Summary : CLI curses based monitoring tool

URL : https://github.com/nicolargo/glances

License : GPLv3

Description : Glances is a CLI curses based monitoring tool for both GNU/Linux and BSD.

Glances uses the PsUtil library to get information from your system.

It is developed in Python.

change the output format for time command

If you are doing some scripting and using ‘time’ command, then you know sometimes it becomes difficult to capture the output as the output would be something like this:

: amit ; time ls

real    0m0.002s
user    0m0.000s
sys 0m0.001s

So, it is better to change that format. Here is simple example:

  <td>
    <div class="text codecolorer">
      &nbsp;
    </div>
  </td>
</tr>

TIMEFORMAT=’real %3R user %3U sys %3S pcpu %P’ time ls

amit ;  TIMEFORMAT=’real %3R user %3U sys %3S pcpu %P’ time ls

0.00user 0.00system 0:00.00elapsed 0%CPU (0text+0data 2432max)

0inputs+0outputs (0major+109minor)pagefaults 0swaps

amit ; TIMEFORMAT=’real %3R user %3U sys %3S’

amit ; time ls
real 0.001 user 0.001 sys 0.000

amit ;

amit ; TIMEFORMAT=’TIMEOUTPUT = real %3R user %3U sys %3S’

amit ; time ls
TIMEOUTPUT = real 0.001 user 0.001 sys 0.000

amit ;

ionice – renice you IO activity for the process.

ionice is utility provided by the package util-linux. Description of util-linux:

Description : The util-linux package contains a large variety of low-level system

utilities that are necessary for a Linux system to function. Among

others, Util-linux contains the fdisk configuration tool and the login

program.

 

With the help of this utility, you can set the scheduler priority for your IO scheduling upto RealTime, but be careful that can make other process sluggish. Example use:

perl is faster than bash in some cases.

Some days back, I had to generate some data to be uploaded to a database. As usual I assumed that bash should be faster and hence wrote the script to create the files in bash. But I found that even after 5 hours I was only 10% done with the data generation. Now that would mean that it would take around 50 hours to complete the data generation. Something did not look correct to me and I asked one of my colleague. He suggested I do a strace.