Monday, January 29, 2018

Removing audio noise from a video

The basic steps are to extract the audio, remove noise, and then re-insert the noise-free audio back
into the video wrapper.

Step I: Extract the noisy audio

1. Open Audacity

2. Open the video file in Audacity

3. Select a region of pure background noise (which should be periods of silence). Then use the Audacity menu to select Effects -> Noise reduction

4. Select Get Noise Profile (Important: don’t click ok)

5. Then go back to the waveform and select the entire waveform

6. Again select Effects -> Noise reduction

7. But this time just click ok

8. Audacity will work on the file to reduce the noise. Check the region selected before to see if noise is removed/reduced

9. Repeat the steps #3 to #7 as many times as necessary till all noise is gone

10. Now Export the file to mp3. Make to change the extension as Audacity uses “aiff” even when the file type is selected to be mp3.

Step II: Remove audio from video file

1. Go to the folder containing the video file.

2. Strip audio from the video file and store it in a temporary video file called temp.mp4 using the following command

$ ffmpeg -i video.mp4 -an -vcodec copy temp.mp4

temp.mp4 will have no audio

Step III: Add noise-free audio back to the video file

1. Copy the noise-reduced audio file from Audacity to this folder. Let’s say the audio file is called audio.mp3
2. Add this audio file to the temporary audio-free video file temp.mp4

$ ffmpeg -i temp.mp4 -i audio.mp3 -vcodec copy -acodec copy final.mp4

final.mp4 will contain the audio with no noise.

Wednesday, January 24, 2018

dpkg: error: duplicate file trigger interest for filename `/usr/lib/gio/modules' and package `libglib2.0-0:i386'

The dpkg trigger file has duplicates. We would need to manually remove all of them.

$ cd /var/lib/dpkg/triggers/
$ sudo vim File

/usr/share/octave/packages octave
/usr/lib/libreoffice/share/extensions libreoffice-common
/usr/lib/x86_64-linux-gnu/gio/modules libglib2.0-0
/usr/lib/gio/modules libglib2.0-0
/usr/share/glib-2.0/schemas libglib2.0-0
/usr/lib/i386-linux-gnu/gio/modules libglib2.0-0:i386
/usr/lib/gio/modules libglib2.0-0:i386
/usr/share/glib-2.0/schemas libglib2.0-0:i386
/etc/dbus-1/system.d dbus/noawait
/usr/share/dbus-1/system-services dbus/noawait
/usr/lib/vlc/plugins vlc-nox/noawait

Notice the duplicates in bold. Remove them manually.
If there are many duplicates then a quick way to see the duplicates is to run

$ cat File | cut -d" " -f 2 | uniq -c

      1 octave
      1 libreoffice-common
      3 libglib2.0-0
      3 libglib2.0-0:i386
      2 dbus/noawait
      1 vlc-nox/noawait
      2 libgdk-pixbuf2.0-0
      1 udev/noawait
      1 cups/noawait
      2 systemd/noawait

The number in front of each package gives the duplicate count.

After removing duplicates run

$ sudo apt-get install -f

to install missing dependencies

Monday, October 2, 2017

OK GOOGLE on Samsung Galaxy S7 doesn’t work

To make Ok Google detection work on Galaxy S7 (Galaxy series phones) we need to perform a couple of steps.

1. As long as Samsung S voice is enabled, it interferes with Ok Google detection.

So disable S Voice first.

Goto settings -> Apps -> S Voice -> Disable. If needed Force Stop the app.

2. Make sure S voice is no longer the default app for voice commands. Otherwise the phone will start it again instead of Google assistant.

Goto settings -> Apps -> Default apps -> Device assistance app

Make sure that the device assistance app is Google.

3. Make sure “Ok Google” is turned ON.

Tap the google search bar to go to the Google Now app. Tap the big G at the top left corner and then the three bars at the top left corner and choose settings -> Voice -> OK Google detection. Make sure that Ok Google detection is turned on and Trusted voice is turned on too. Train voice model.

4. Restart the phone. This is a crucial step.

Saturday, May 20, 2017

Common questions on climate change

1.  Are renewable energy systems even viable or are they just an Utopian dream of impractical scientists? And even if they are viable wouldn’t it cost an arm and a leg to make them viable?

Among the multitude of scientific journal articles, which answers both these questions separately, one article answers both these questions succinctly in a single paper.

In 2011 Jacobson et al. proposed a 100% WWS (wind, water, solar) energy system which can supplant the current fossil fuel industry. They found that production cost of WWS is exactly same as current fossil fuel-derived energy systems and that includes costs of production, storing, and transmission (with transmission losses). Their conclusion is that “ … barriers to a 100% conversion to WWS power worldwide are primarily social and political, not technological or even economic.”

2. Which countries invest the most in renewables?

The current standing (May, 2017) is:

3. What is geothermal energy?

Geothermal Energy is a viable a source of renewable energy. A NatGeo documentary Steam Drilling (2012) explores the steamy world of geothermal energy production, its challenges, and its tremendous potential as a renewable energy source.

The only problem with geothermal energy is its location dependence. Only a few places on Earth are fortunate enough to have access to geothermal energy reserves. Krafla Power Station in Iceland is one such place. Till now it remains unique in its production of geothermal energy.

4. Many liberals claim that 97% of climate scientists agree that climate change is real and that it is man-made while conservatives say that it is a made-up number and the actual consensus is much lower. Who is right and where did this number come from?

The 97% comes from a series of surveys and meta-analysis done by climate scientists on the level of consensus in their own community. See this short video to get a quick update on how the number is arrived at or read the article at skepticalscience. However, the most convincing argument always come from a paper published in a reputed a peer-reviewed journal.  And that argument can be found in the recent article by Cook J et al 2016 Consensus on consensus: a synthesis of consensus estimates on human-caused global warming Environ. Res. Lett. 11 048002 IOPscience.

Thursday, December 15, 2016

Command-line password generator for Linux

Linux has its own cryptographic generator which can easily be harnessed by the common user to generate extremely strong passwords. This password generator can be found at /dev/random. /dev/random generates pseudorandom numbers based on the available entropy of the system (see wiki page). A derivate of /dev/random called /dev/urandom does the same thing but it is more accessible while being slightly (theoretically) less secure. So what stops us from grabbing outout from /dev/urandom using cat? An output of the content of /dev/urandom will demonstrate that it is practically gibberish with very few usable random characters. The output of

cat /dev/urandom 

should convince the user.  However, it is possible to trim out the unwanted characters and only keep alphabets, numbers, and special characters with a simple filtering by the tr command.

cat /dev/urandom | head -c 10 | tr -dc "A-Za-z0-9_!"

This command will produce the desired output but it will generate very small passwords, which are truly random but which do not meet the length criteria in most cases. Therefore, this procedure needs to be repeated a large number of times until the required length of random passwords can be generated. It is better to write a bash script which will automate this procedure.

The following bash script generates random passwords and allows the user to control the length and number of random passwords that are generated. It also allows special characters to be included or excluded with a switch (argument). Since I am always in a hurry and don't like passing too many arguments, I wrote the script so that it can accept arguments in a shorthand manner in addition to the longhand manner (Unix style) of passing arguments. Shorthand way: ./ 10 2 1 will generate 2 random passwords of length 10 and they will all include special characters.

~$ ./ 10 2 1


Or I could use the longhand (Unix-style) way of passing the arguments: ./ -l 10 -n 2 -s 1

~$ ./ -l 10 -n 2 -s 1


The result will be the same. I have included the full code below which is well commented. I also provide explanation and cite sources below the code.


# generate random passwords using urandom CSPRNG.  

# v5.2 by Jones

Usage: ./ [options ...]
  -l password length. DEFAULT=16
  -n number of passwords to generate. DEFAULT=4
  -s special characters. DEFAULT=1. Set to 0 to disable selecting _!%
  -h help
Example 1: ./ -l 8 -n 2 -s 0

./ [password length] [number of passwords to generate] [include special characters] 

Example 2: ./ 8 2 0   "

if [[ "$1" =~ ^[0-9]+$ ]]; then  

 # Second and third argument might be empty which means $num and $special
 # will be empty at this point too. Set the values of $num and $special 
 #to the default values in case they are empty
 [ "$2" == "" ] && num=4
 [ "$3" == "" ] && special=1
 # If user uses arguments then use getopts.
 while getopts l:n:s:h option
  case "${option}"
          l) length=${OPTARG};;
          n) num=${OPTARG};;
          s) special=${OPTARG};;
          h) echo "$usage" ;;


# check if the first argument is an integer. If it is not then throw error. 
#=~ matches REGEXP ^[0-9]+$ which is one or more occurences of the numbers 0-9
if [[ !("$length" =~ ^[0-9]+$ )]]; then
 echo "Enter numbers only for the password length"
 echo "$usage"

# check if the second argument is an integer. If it is not then throw error.
if [[ !("$num" =~ ^[0-9]+$) ]]; then
 echo "Enter numbers only for the number of passwords to generate"
 echo "$usage"

# check if the third argument is an integer. If it is not then throw error. 
if [[ !("$special" =~ ^[0-1]+$) ]]; then
 echo "Enter only 0 or 1 for the third argument."
 echo "$usage"

if [[ $special == "1" ]]; then # if special chars needed: DEFAULT
# echo "first loop for 1"
 while [ $i -lt $num ] 
 # Keep looping till next password, which satisfies the required criteria, 
 # is successfully generated. May require many more passess for stricter 
 # requirements, usually smaller length passwords.
 pass=`tr -dc "A-Za-z0-9_!%" < /dev/urandom | head -c ${length} | xargs`

 if [ `echo $pass | grep "%" | grep "[0-9]" ` ]; then
   echo $pass
else  # if special chars are NOT needed
# echo "second loop for 0"
 while [ $i -lt $num ]
 pass=`tr -dc "A-Za-z0-9" < /dev/urandom | head -c ${length} | xargs`

 if [ `echo $pass | grep "[0-9]" ` ]; then
   echo $pass


Parsing the command-line

The block of code which parses the command-line options using getopts is found at lifewire. The post at that link provides a clear explanation of what the block of code while getopts l:n:s:h option does. So I will only mention that the while loop parses through the command-line arguments and if it finds any matches to the options that the script expects, then it assigns the value passed to the option to the corresponding variables, defined in the script.

Checking validity of arguments passed

Since the script can only accept numeric arguments, their values need to be checked for non-numeric entries. If found an error message is displayed. To check if the values are numeric we can treat the value as a string and verify that every character lies between 0-9. This ensures that there are no non-numeric characters in that string which in turn implies that the string is a number. "$num" =~ ^[0-9]+$  does exactly that. =~ operator is used for REGEX matching of lvalue $num to rvalue ^[0-9]+$ (which is a REGEX that matches every character to characters between 0-9). A similar test is applied for the special variable which acts as a switch to include or exclude special characters. However, it only takes values 0 or 1. Hence its REGEX is ^0-1+$. For more information on regular expressions see this quick reference.

Sunday, November 27, 2016

Turning normal HDTV into Smart HDTV

One word ChromeCast. That will do it. There is no need to pay a premium for a Smart TV when Chromecast can outperform any of them for a fraction of the cost. Also, there are reports of some Smart TVs stealing user browsing behavior for commercial purposes. The worst was the claim that some TVs secretly listen to the conversation of the people in the room. I cannot verify the truthfulness of these reports but all these are viable possibilities if a malicious app has been secretly installed in the Smart TV. So I would rather play it safe and use Chromecast (since Google has all my data anyways). So we need to buy the Chromecast device and install the Chromecast app on the phone and that will transform the non-smart TV into an entertainment hub. However, just installing the Chromecast app on the phone does not satisfy a power user. Therefore, I chose to explore the possibilities of "Home entertainment".

1. Streaming movies from phone - Install Chromecast app. This is simple and most straighforward way to use it.

2. Streaming movies from the computer - Install Videostream app in Chrome and play the files straight from the local hard drive. However, one drawback of the free version is that it does not support adding a subtitle file. So the only options are to either pay for it or encode the subtitle files into the video files themselves. I will choose the latter option. We will need to create a mkv container and then embed the video and the subtitle files in that container. Videostream supports mkv files so it will stream the video to the TV along with the embedded subtitles.

The embedding can be done with a single command

mkvmerge -o output.mkv video.mp4

However, if there is a lag in the subtitle file and it needs to be resynced with the video file then a couple of additional steps will be required. I wrote an article about it here.

Saturday, August 13, 2016

Setting up Crafty chess engine on Linux/Ubuntu

Crafty installation is pretty simple. The command

sudo apt-get install crafty

will install crafty at /usr/games/crafty and the book files at /usr/share/crafty/books.bin.

If the book files are not installed they can be manually installed by

sudo apt-get install crafty-books-medtosmall

Sometimes the book are not automatically installed in the home folder and it will result in the following error when crafty is run from the command line.

Machine is NUMA, 1 nodes (6 cpus/node)

unable to open book file [./book.bin].
book is disabled
unable to open book file [./books.bin].
ERROR, unable to open game history file, exiting

This implies that we need to import the book.bin and books.bin which contain a vast array of opening moves into the local ~/.crafty directory. For that, we need to run the However, it needs to be made executable by the normal user first. As root go to the directory /usr/share/doc/crafty and do the following,

cd /usr/share/doc/crafty 
sudo chmod 777 

Then logout from root, go back to the same directory /usr/share/doc/crafty and run as user:


Don't run the last command as root otherwise the .crafty directory will be created in the /root directory! It will spit out a few lines of info. Copy the lines

export CRAFTY_LOG_PATH=/home/<USERNAME>/.crafty
export CRAFTY_RC_PATH=/home/<USERNAME>/.crafty
export CRAFTY_TB_PATH=/home/<USERNAME>/.crafty
export CRAFTY_BOOK_PATH=/home/<USERNAME>/.crafty

to .bashrc file and then

source .bashrc

After that source

source /home/<USERNAME>/.crafty/env_settings

Run Crafty. It should be able to see the book files now.

Removing audio noise from a video

The basic steps are to extract the audio, remove noise, and then re-insert the noise-free audio back into the video wrapper. Step I: Ext...