Working with dates and timestamps in GNU Linux

With the GNU date command line utility.

Print time as seconds from the epoch, or, “seconds since 1970-01-01 00:00:00 UTC” :

$ date +%s
1365439166

Convert timestamp or epoch dates into familiar day and time :

$ date --date='@1365439166'
Mon Apr 8 11:39:26 CDT 2013

This works under Debian/Ubuntu, and may work under other Unix/Linux systems.

sort files by size

du -sh * | sort -h

Output should be sorted like this (K, M, G for Kilobytes, Megabytes, Gigabytes):

8.0K totem.txt
8.0K em.vlan.txt
20K dot.png
68K pid5367.log
104K dmesg.txt
384K hwinfo.txt
1.2M textarchive.tar.gz
2.9M Archive
78M minicom.cap
226M vorbis
688M 09283.ar
1.2G VirtualBox VMs
2.5G Downloads

streaming tar

Because I always forget the syntax:

tar cvf - tftpboot/ | ssh userone@10.11.12.13 "cd /home/userone/store_old/ ; tar xvpf - "

There are many variations, and this one includes the verbose flag. (-v)

system uptime

Juniper/JunOS M-series:

rem10> show system uptime
Current time: 2013-02-13 13:40:10 CST
System booted: 2013-02-13 03:07:33 CST (10:32:37 ago)
Protocols started: 2013-02-13 03:09:23 CST (10:30:47 ago)
Last configured: 2013-02-13 03:21:06 CST (10:19:04 ago) by nim42
1:40PM up 10:33, 1 user, load averages: 0.00, 0.02, 0.00

Changing line-endings from DOS-style (Windows) to Unix-style (Linux) the easy way with dos2unix

I’m always forgetting about the dos2unix utility.
Probably because I only need once every couple of years.

dos2unix brokedybroke_file.sh

For the Ubuntu package, here’s the abbreviated apt-cache show dos2unix :

Description-en: convert text file line endings between CRLF and LF
This package contains utilities dos2unix, unix2dos, mac2unix,
unix2mac to convert the line endings of text files between UNIX (LF),
DOS (CRLF) and Mac (CR) formats.
.
Text files under Windows and DOS typically have two ASCII characters
at the end of each line: CR (carriage return) followed by LF (line
feed). Older Macs used just CR, while UNIX uses just LF. While most
modern editors can read all these formats, there may still be a need
to convert files between them.
.
This is the classic utility developed in 1989.
Homepage: http://freshmeat.net/projects/dos2unix

Where does an Ubuntu package get installed?

For Debian-based systems,

“You can discover where a package puts its ‘stuff’ by running:
dpkg -L 'packagename'

From the dpkg manpage, here’s the -L option:

-L, --listfiles package-name...
List files installed to your system from package-name.

Credit to Rooke at Askubuntu.

Stripping leading whitespace and comments with sed

In trying to consolidate some bash aliases (found in .bashrc and .bash_aliases), I needed to strip out comments and leading whitespace to sort for unique lines.

To strip spaces, I used
egrep alias .bashrc | sed 's/^[ ]*//'

To remove spaces and tabs in the whitespace (see info on tabs after this example):
egrep alias .bashrc | sed 's/^[ ]*//'
NOTE: In the expression, sed 's/^[ ]*//' , the expression in the brackets is actually a space followed by a tab. At the command line, I had to create the tab by typing Control-V and then hitting TAB on the keyboard. You may need to represent tabs in a different fashion.

To get rid of leading whitespace, comments, and indented comments, I settled on
egrep alias .bashrc | sed 's/^[ ]*//' | egrep -v '^#'

To gather and sort aliases from both .bash and .bash_aliases, then sending to a temporary file:
egrep -h alias .bash[r_][ca]* | sed 's/^[ ]*//' | egrep -v '^#' | sort -u >> tmp

The -h flag tells the first grep not to print the file name.

Getting ‘Scroll Lock’ to work under Linux

I’ve got a couple of keyboards with Scroll Lock keys that are not functional by default under Linux.
For my Ubuntu systems, the following procedure worked to fix the problem. After this, the LED for scroll lock lights and the key function as expected.

The steps are as follows:

  1. Find an unused modifier with xmodmap -p
  2. Use xmodmap to assign Scroll_Lock to this modifier
  3. Make the change persistent by adding it to your .Xmodmap file

First, use xmodmap -p to identify an unused mod key.
For example,

$ xmodmap -p
xmodmap: up to 4 keys per modifier, (keycodes in parentheses):

shift Shift_L (0×32), Shift_R (0x3e)
lock Caps_Lock (0×42)
control Control_L (0×25), Control_R (0×69)
mod1 Alt_L (0×40), Alt_R (0x6c), Meta_L (0xcd)
mod2 Num_Lock (0x4d)
mod3
mod4 Super_L (0×85), Super_R (0×86), Super_L (0xce), Hyper_L (0xcf)
mod5 ISO_Level3_Shift (0x5c), Mode_switch (0xcb)

mod3 is the open modifier. So, we’ll use it with xmodmap -e :

xmodmap -e "add mod3 =Scroll_Lock"

Capitalization is important, so “Scroll_Lock” has to be written exactly as printed.
You can have a space, or no space, after the equal sign.

Check that it works, then add the command content to your home directory (file does not have to exist):

echo "add mod3 =Scroll_Lock" >> ~/.Xmodmap

send man page to text file

Printing a manpage to a text file (perhaps more portable for cross-platform or bedtime reading):

generic Unix/Linux, using `col`:
man rcs | col -b > /tmp/man_rcs.txt

These alternatives would require that you reset PAGER or MANPAGER:
alternative BSD :
export MANPAGER=cat
man pf.conf > man_pf.conf.txt

alternative BSD :
export PAGER=cat
man pf.conf > man_pf.conf.txt

Ubuntu, Debian, Mint, etc. allow all of these alternatives:

-P pager, --pager=pager
Specify which output pager to use. By default, man uses pager -s. This option overrides the $MANPAGER environment variable, which in turn overrides the $PAGER environment variable. It is not used in conjunction with -f or -k.

recovering a complete config from rancid

From time to time, I need to access a complete config stored by the rancid application under CVS, so I can get a good look (more than just the line diffs) at the configuration.

Here are the steps I use to recover an older version of configuration stored by the rancid process in a CVS structure.


1) Find the group that contains the config you want, and get the ‘CVSROOT’ directory info contained in the Root file. This should be in a directory that corresponds to the rancid base install directory — BASEDIR/var/GROUPNAME/configs/CVS/Root. My install base is /usr/local/rancid, so to get a config from my atlanta-zone5 group, I need to look in /usr/local/rancid/var/atlanta-zone5/configs/CVS/Root.

e.g.,
$ cat /usr/local/rancid/var/atlanta-zone5/configs/CVS/Root

2) Next, set and export the CVSROOT variable that you found in step 1.

e.g.,
$ export CVSROOT=/usr/local/rancid/var/CVS

3) Find the version of the file (the configuration revision) you want to get back. This can be done with various CVS front ends, or the command line. For the command line CVS command, go to the configs directory of the group containing the router/switch config you want, then use the ‘cvs log’ command. For example:
$ cd /usr/local/rancid/var/CVS/atlanta-zone5/configs/

4) Look through the log for the version that corresponds to the date you need:
$ cvs log bgp2-rtr | more

5)Go to the directory in which you’ll expand the old config.
$ cd /tmp (or some other directory where you won’t collide with existing files)

6) Check out the version you need using the ‘cvs co’ command in this form:

$ cvs co -r 1.21 GROUP/configs/YOUR_ROUTER
e.g.,
cvs co -r 1.21 atlanta-zone5/configs/bgp2-rtr

The group directory will be restored, and you will have to descend into it to find the exact config you need.

$ cd atlanta-zone5/configs
$ more bgp2-rtr

7) Enjoy your recovered config. Repeat as necessary.

Next Page »