Wednesday, April 28, 2010

All You Should Know About: lsof

Lsof follows Unix philosophy closely. It does just one task and it does it perfectly -- it lists information about files opened by processes. An open file may be a regular file, a directory, a NFS file, a block special file, a character special file, a shared library, a regular pipe, a named pipe, a symbolic link, a socket stream, an Internet socket, a UNIX domain socket, and many others. Since almost everything in Unix is a file, you can imagine how incredibly useful lsof is!

How to use lsof?

In this article I will try to present lsof based on as many use cases as I can think of. Let's start with the simplest (that you probably already know) and proceed to more complicated ones.

List all open files.

# lsof
Running lsof without any arguments lists all open files by all processes.

Find who's using a file.

# lsof /path/to/file
With an argument of a path to a file, lsof lists all the processes, which are using the file in some way.

You may also specify several files, which lists all the processes, which are using all the files:

# lsof /path/to/file1 /path/to/file2
Find all open files in a directory recursively.

# lsof +D /usr/lib
With the +D argument lsof finds all files in the specified directory and all the subdirectories.

Note that it's slower than the usual version with grep:

# lsof | grep '/usr/lib'
It's slower because +D first finds all the files and only then does the output.

List all open files by a user.

# lsof -u Dejavu
The -u option (think user) limits output of files opened only by user Dejavu.

You can use comma separated list of values to list files open by several users:

# lsof -u rms,root
This will list all the files that are open by users rms and root.

Another way to do the same is by using the -u option twice:

# lsof -u rms -u root
Find all open files by program's name.

# lsof -c apache
The -c option selects the listing of files for processes whose name begins with apache.

So instead of writing:

# lsof | grep foo
You can now write the shorter version:

# lsof -c foo
In fact, you can specify just the beginning part of the process name you're looking for:

# lsof -c apa
This will list all the open files by a processes whose starts with apa.

You can also specify several -c options to output open files by several processes:

# lsof -c apache -c python
This will list all open files by apache and python.

List all open files by a user OR process.

# lsof -u Dejavu -c apache
Lsof options can be combined. The default is to OR between options. It means it will combine outputs of -u Dejavu and -c apache producing a listing of all open files by Dejavu and all open files by apache.

List all open files by a user AND process.

# lsof -a -u Dejavu -c bash
Notice the -a option. It combines the options with AND. The output listing is files opened by bash, which is run under Dejavu user.

List all open files by all users EXCEPT root.

# lsof -u ^root
Notice the ^ character before root username. It negates the match and causes lsof print all open files by all users who are not root.

List all open files by the process with PID.

# lsof -p 1
The -p option (think PID) filters out open files by program's id.

Remember that you can select multiple PIDs by either comma separating the list or using multiple -p arguments:

# lsof -p 450,980,333
This selects processes with PIDs 450, 980 and 333.

List all open files by all the processes EXCEPT process with PID.

# lsof -p ^1
Here the negation operator ^ is used again. It inverts the list and does not include process with PID 1.

List all network connections.

# lsof -i
Lsof with -i option lists all processes with open Internet sockets (TCP and UDP).

List all TCP network connections.

# lsof -i tcp
The -i argument can take several options, one of them is tcp. The tcp option forces lsof to list only processes with TCP sockets.

List all UDP network connections.

# lsof -i udp
The udp option causes lsof to list processes with UDP sockets.

Find who's using a port.

# lsof -i :25
The :25 option to -i makes lsof find processes using TCP or UDP port 25.

You may also use service port name (found in /etc/services) rather than port number:

# lsof -i :smtp
Find who's using a specific UDP port.

# lsof -i udp:53
Similarly, to find who's using a TCP port, use:

# lsof -i tcp:80
Find all network activity by user.

# lsof -a -u hacker -i
Here the -a option combines -u and -i to produce listing of network file usage by user hacker.

List all NFS (Network File System) files.

# lsof -N
This option is easy to remember because -N is NFS.

List all Unix domain socket files.

# lsof -U
This option is also easy to remember because -U is Unix.

List all files for processes with a specific group id.

# lsof -g 1234
Process groups are used to logically group processes. This example finds all files opened by processes with PGID 1234.

List all files associated with specific file descriptors.

# lsof -d 2
This lists all files that have been opened as file descriptor 2.

You may also specify ranges of file descriptors:

# lsof -d 0-2
This would list all files with file descriptors 0, 1 and 2.

There are also many special values, such as mem, that lists memory-mapped files:

# lsof -d mem
Or txt for programs loaded in memory and executing:

# lsof -d txt
Output PIDs of processes using some resource.

# lsof -t -i
The -t option outputs only PIDs of processes. Used together with -i it outputs PIDs of all processes with network connections. It's easy to kill all processes that use network:

# kill -9 `lsof -t -i`
Repeat listing files.

# lsof -r 1
The -r option makes lsof repeatedly list files until interrupted. Argument 1 means repeat the listing every 1 second. This option is best combined with a narrower query such as monitoring user network file activity:

# lsof -r 1 -u john -i -a
How to install lsof?

Lsof comes preinstalled on many Unix systems. If your system doesn't have it, try to install it from the source.

BSD supplies its own utility that does similar things, it's called fstat.

For the full documentation of lsof see the man lsof page or type lsof -h for a small cheat sheet.

Have fun with lsof!

Sunday, April 25, 2010

Fixing Windows XP Master Boot Record (MBR)

 

fixmbr 

After installing a Linux distro on a Windows XP machine, the GRUB boot loader, or other Linux boot loader, will appear when the PC boots. It gives you the option of which operating system to start. GRUB usually makes Linux the default OS, so it will start automatically when the PC is turned on.

If you ever delete the Linux distro and don’t have Windows set as the OS that will boot when the PC is powered on, Windows might not start and will give an error message.

An easy way to fix the Windows XP master boot record is to use the Windows Install CD.

1) Insert the Windows XP Install CD and reboot the computer.
2) When “Press any key to boot from CD” appears, press any key.
3)When the “Welcome to Setup” screen appears, press R to enter the Recovery  Console.
4)When asked which Windows installation to log onto, enter the number for the correct Windows install, then press Enter.
5)Type the Administrator password or just press Enter if there isn’t one.
6)Type FIXMBR and press Enter.
7) It will ask if you are sure you want to write a new MBR. Type y and press Enter.
It should say “The new master boot record has been successfully written”. Type exit and press Enter to reboot the computer.
8) Remove the Installation CD as soon as the computer powers up so it won’t boot from the CD again.

fix_mbr

The computer should now boot into Windows.

Thursday, April 1, 2010

Birth of The Linux Kernel :

 

In the early days of Unix, universities used Unix as a vehicle for teaching computer science students about operating systems. When AT&T asserted its proprietary claim to Unix, universities needed a replacement. Andrew Tannenbaum created a Unix-like operating system called MINIX, which became popular as a teaching tool. However, unlike Unix, MINIX was designed primarily as a pedagogical tool and performed relatively poorly. In 1990, Finnish computer science student Linus Torvalds began work on a memory manager for Intel-architecture PCs. At some point he realized that his work could be extended to operate as a Unix kernel. In August 1991, he posted his work-in-progress to the Internet newsgroup comp.os.minix, inviting others to request features to be considered for implementation:

             tuxbday

 

From: torvalds@klaava.Helsinki.FI (Linus Benedict Torvalds)

Newsgroups: comp.os.minix

Subject: What would you like to see most in minix?

Summary: small poll for my new operating system

Message-ID: <1991Aug25.205708.9541@klaava.Helsinki.FI>

Date: 25 Aug 91 20:57:08 GMT

Organization: University of Helsinki

Hello everybody out there using minix -I'm doing a (free) operating system (just a hobby, won't be big and professional like gnu) for 386(486) AT clones. This has been brewing since april, and is starting to get ready. I'd like any feedback on things people like/dislike in minix, as my OS resembles it somewhat(same physical layout of the file-system (due to practical reasons)among other things). I've currently ported bash(1.08) and gcc(1.40), and things seem to work. This implies that I'll get something practical within a few months, and I'd like to know what features most people would want. Any suggestions are welcome, but I won't promise I'll implement them :-)

Linus (torvalds@kruuna.helsinki.fi)

PS. Yes - it's free of any minix code, and it has a multi-threaded fs.

It is NOT protable [sic](uses 386 task switching etc), and it

probably never will support anything other than AT-harddisks, as that's all I have :-(.

 

Torvalds called his operating system kernel Linux, for Linus’s Minix. Unix programmers eagerly offered help in developing Linux. Because Stallman’s GNU project had completed almost all of the components needed for its Unix like operating system except the kernel, Linux and GNU were a natural marriage.In 1994, about three years after Torvalds’s posting, Linux 1.0 was released under the terms of the GPL. Already, Linux had about 100,000 users.

 

AT PRESENT THE LINUX DEVELOPMENT PROCESS IS GOING LIKE THIS :

linux_dev_process_final

 
Things You Should Know About Linux !!!