Using DD to backup your partitions, or Clonezilla.

A well-written article on how to backup an entire partition using DD.

Another way to do this is to use CloneZilla.

Places to buy systems with Linux pre-installed

Places to buy systems with Linux pre-installed (which means the hardware is Linux compatible!). There may be others, this is just a short reference.

1) Dell: Dell is currently selling four laptop systems starting from $549 to $1049; the linux desktop system available starts at $448. All these system are pre-installed with Ubuntu 8.04. here:

2) system76: They are Mark Shuttleworth’s personal favorite Linux pre-installed reseller. Like Dell they only sell pre-installed Ubuntu Linux. But they have a wider choice in systems and hardware configurations. Their cheapest laptop starts at $869. More information here.

3) The Linux Laptop Company: As the name suggests they are specialized in selling pre-installed linux laptops; like the previous two vendors their distro of choice is Ubuntu 8.04. They currently sell four models of laptops and the price starts at $699 to all the way up to $1299. More information here.

4) ASUS EeePC: Based on Xandros Linux Asus EeePC is perhaps the most widely used pre-installed linux subnotebook ever sold. There are several resellers for ASUS EeePC, but my favorite is newegg; where the listed price for 1000H 40G is $669. You can get more information about the system here.

5) Linux Emporium: Based on UK, they are Thinkpad resellers with three choices of Linux distro. Their systems can be configured for Ubuntu, Suse or Fedora. Prices start at £376. Get more information here.

6) Linux Certified: They sell a wide variety of laptops with six choices of pre-installed Linux Distros. Fedora 8, Ubuntu 8.04, open SUSE 11, RHEL 5, CenOS 5, Oracle Linux. Their LC2000 series laptops start $699. here.

Source: LinuxHaxor.

Make a simple, web-based streaming server with Ubuntu: GnuMP3d.

Source: UbuntuGeek

The GNU MP3/Media Streamer is a simple application which makes it possible to navigate through your audio and video collection via a browser, and stream playlists across a network.GNUMP3d is a streaming server for MP3s, OGG vorbis files, movies and other media formats.The software supports browsing, searching, and streaming all via your browser with support for MP3, OGG Vorbis, WMA and many other types of audio files.

Click here for more info and configuration information.

GnuMP3d home site.

How to make your Linux harder, better & faster.

Some of these are to be taken with a grain of salt,

your mileage may vary:

_______________

Harder:

Anatomy of Security-Enhanced Linux (SELinux)

Understanding AppArmor

grsecurity

What is Linux Security Module?

10 Basic Linux Security Tips to Implement

The Best Linux Security Tools

Linux Audit and Intrusion Detection

The Unix security audit and intrusion detection tool

Top 100 Network Security Tools

The short life and hard times of a Linux virus

LIDS Secure Linux System

10 ways to secure your Linux desktop


Better:

Beautiful Conky Setup

Building a Better (Linux) GPU Benchmark

Last-Exit: A Better Linux Client for LastFM

The best Linux system repair distribution

A Better Introduction to Linux User Interface

Better Linux Dock


Faster:

How to make Ubuntu extremely fast

Booting Linux in five seconds

Improving boot time on a general Linux distribution

Reboot Linux faster using kexec

Make Linux Faster and Smoother

A better, faster desktop search for Linux

10 ways to make Linux boot faster

Fast file copy in Linux

How fast does Arch Linux install? (video)

x2x is a software alternative to a KVM switch

From Linux.com:

Unless you have fully embraced the virtualization movement, you probably have more than one machine in your home or office, particularly if you run more than one operating system, and you probably have more than one keyboard and mouse on your desk. If you would like to regain some desk space without having to purchase a KVM switch, x2x may be the solution. Simply put, x2x takes advantage of the X Window System’s ability to run a display over a TCP/IP network. In this case, you are not running a display as much as you are taking charge of another display’s mouse and keyboard.

More can be found here.

How to install Ubuntu Linux without burning a CD

Extract the files in the .ISO to a USB stick and boot off that, then on-bootup, install from the USB key, quite simple.

Here’s the full howto.

Encrypting containers (or partitions) with Cryptsetup and LUKS

Source: feraga.com

Howto use Cryptsetup with LUKS support. (Debian Linux).

This is a short howto to describe the basic usage of Device-Mapper, DM-Crypt, and Cryptsetup to mount and use encrypted partitions and container files.

This is partially in response to the recent articles about the numbers of USB flash thumbdrives that are regularly lost. If we learn to use encryption then that statistic is just sad but not worrying. (see The problem of lost USB flash thumbdrives)

Device Mapper and DM-CryptStarting in version 2.6, the Linux kernel started providing the Device-Mapper interface. This interface allowed for the creation of layers of virtual block devices ontop of real block devices. These devices are used for things like RAID formats, snapshot or encryption. The DM-Crypt is the module for Device-Mapper that provides access to the cryptographic functions.

CryptsetupCryptsetup is the primary userland tool for creating and managing encrypted partitions and containers for DM-Crypt.

Linux Unified Key Setup (LUKS)LUKS provides a standard on-disk format for encrypted partitions to facilitate cross distribution compatability, to allow for multiple users/passwords, effective password revocation, and to provide additional security against low entropy attacks. To use LUKS, you must use an enabled version of cryptsetup. To the authors knowledge currently only Debian (Etch, Lenny and Sid), Ubuntu and Gentoo offer LUKS enabled versions of cryptsetup in their repositories.

Creating a New Encrypted Container File or Partition

Create the Container and Loopback Mount itFirst we need to create the container file, and loopback mount it.

root@host:~$ dd if=/dev/urandom of=testfile bs=1M count=10
10+0 records in
10+0 records out
10485760 bytes (10 MB) copied, 1.77221 seconds, 5.9 MB
root@host:~$ losetup /dev/loop/0 testfile
root@host:~$

Note: Skip this step for encrypted partitions.
luksFormatBefore we can open an encrypted partition, we need to initialize it.

root@host:~$ cryptsetup luksFormat /dev/loop/0

WARNING!

========

This will overwrite data on /dev/loop/0 irrevocably.


Are you sure? (Type uppercase yes): YES

Enter LUKS passphrase:

Verify passphrase:

Command successful.

root@host:~$

Note: For encrypted partitions replace the loopback device with the device label of the partition.
luksOpenNow that the partition is formated, we can create a Device-Mapper mapping for it.

root@host:~$ cryptsetup luksOpen /dev/loop/0 testfs
Enter LUKS passphrase:
key slot 0 unlocked.
Command successful.
root@host:~$

Formating the FilesystemThe first time we create the Device-Mapper mapping, we need to format the new virtual device with a new filesystem.

root@host:~$ mkfs.ext2 /dev/mapper/testfs
mke2fs 1.39-WIP (09-Apr-2006)
Filesystem label=
OS type: Linux
Block size=1024 (log=0)
Fragment size=1024 (log=0)
2432 inodes, 9724 blocks
486 blocks (5.00%) reserved for the super user
First data block=1
2 block groups
8192 blocks per group, 8192 fragments per group
1216 inodes per group
Superblock backups stored on blocks:
8193

Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 34 mounts or
180 days, whichever comes first. Use tune2fs -c or -i to overri
root@host:~$

Mounting the Virtual DeviceNow, we can mount the new virtual device just like any other device.

root@host:~$ mount /dev/mapper/testfs /mnt/test/
root@host:~$

Mounting an Existing Encrypted Container File or Partition

root@host:~$ losetup /dev/loop/0 testfile
root@host:~$ cryptsetup luksOpen /dev/loop/0 testfs
Enter LUKS passphrase:
key slot 0 unlocked.
Command successful.
root@host:~$ mount /dev/mapper/testfs /mnt/test/
root@host:~$

Note: Skip the losetup setup for encrypted partitions.

Unmounting and Closing an Encrypted Container File or Partition

root@host:~$ umount /mnt/test
root@host:~$ cryptsetup luksClose /dev/mapper/testfs
root@host:~$ losetup -d /dev/loop/0
root@host:~$

Note: Skip the losetup setup for encrypted partitions.

Handling Multiple Users and Passwords

The LUKS header allows you to assign 8 different passwords that can access the encyrpted partition or container. This is useful for environments where the CEO & CTO can each have passwords for the device and the administrator(s) can have another. This makes it easy to change the password in case of employee turnover while keeping the data accessible.

Adding passwords to new slotsroot@host:~$ cryptsetup luksAddKey /dev/loop/0
Enter any LUKS passphrase:
Verify passphrase:
key slot 0 unlocked.
Enter new passphrase for key slot:
Verify passphrase:
Command successful.
root@host:~$

Deleting key slotsroot@host:~$ cryptsetup luksDelKey /dev/loop/0 1
Command successful.
root@host:~$

Displaying LUKS Header Information

root@host:~$ cryptsetup luksDump /dev/loop/0
LUKS header information for /dev/loop/0

Version: 1
Cipher name: aes
Cipher mode: cbc-essiv:sha256
Hash spec: sha1
Payload offset: 1032
MK bits: 128
MK digest: a9 3c c2 33 0b 33 db ff d2 b9 dc 6c 01 d6 90 48 1d c1 2e bb
MK salt: 98 46 a3 28 64 35 f1 55 f0 2b 8e af f5 71 16 64
3c 30 1f 6c b1 4b 43 fd 23 49 28 a6 b0 e4 e2 14
MK iterations: 10
UUID: 089559af-41af-4dfe-b736-9d9d48d3bf53

Key Slot 0: ENABLED
Iterations: 254659
Salt: 02 da 9c c3 c7 39 a5 62 72 81 37 0f eb aa 30 47
01 1b a8 53 93 23 83 71 20 03 1b 6c 90 84 a5 6e
Key material offset: 8
AF stripes: 4000
Key Slot 1: DISABLED
Key Slot 2: DISABLED
Key Slot 3: DISABLED
Key Slot 4: DISABLED
Key Slot 5: DISABLED
Key Slot 6: DISABLED
Key Slot 7: DISABLED
root@host:~$

Linux Reference Sheets

The 1 page Linux CommandManual

Linux Command Reference

Linux Security Quick Reference

The Gimp Hotkey Reference

Gmail Account Hacking Tool

Source: hungry-hackers.com

Gmail Account Hacking Tool:

A tool that automatically steals IDs of non-encrypted sessions and breaks into Google Mail accounts has been presented at the Defcon hackers’ conference in Las Vegas.

Last week Google introduced a new feature in Gmail that allows users to permanently switch on SSL and use it for every action involving Gmail, and not only, authentication. Users who did not turn it on now have a serious reason to do so as Mike Perry, the reverse engineer from San Francisco who developed the tool is planning to release it in two weeks.

When you log in to Gmail the website sends a cookie (a text file) containing your session ID to the browser. This file makes it possible for the website to know that you are authenticated and keep you logged in for two weeks, unless you manually hit the sign out button. When you hit sign out this cookie is cleared.

Even though when you log in, Gmail forces the authentication over SSL (Secure Socket Layer), you are not secure because it reverts back to a regular unencrypted connection after the authentication is done. According to Google this behavior was chosen because of low-bandwidth users, as SSL connections are slower.

The problem lies with the fact that every time you access anything on Gmail, even an image, your browser also sends your cookie to the website. This makes it possible for an attacker sniffing traffic on the network to insert an image served from http://mail.google.com and force your browser to send the cookie file, thus getting your session ID. Once this happens the attacker can log in to the account without the need of a password. People checking their e-mail from public wireless hotspots are obviously more likely to get attacked than the ones using secure wired networks.

Perry mentioned that he notified Google about this situation over a year ago and even though eventually it made this option available, he is not happy with the lack of information. “Google did not explain why using this new feature was so important” he said. He continued and explained the implications of not informing the users, “This gives people who routinely log in to Gmail beginning with an https:// session a false sense of security, because they think they’re secure but they’re really not.”

If you are logging in to your Gmail account from different locations and you would like to benefit from this option only when you are using unsecured networks, you can force it by manually typing https://mail.google.com before you log in. This will access the SSL version of Gmail and it will be persistent over your entire session and not only during authentication.

(To turn on SSL in GMail, go to SETTINGS, GENERAL, and at the bottom make sure to select ALWAYS USE HTTPS under the BROWSER CONNECTION item.)

Creating global keyboard shortcuts in GNOME

The GNOME desktop environment is the default for Debian (and one of it’s more popular distros, Ubuntu), and is one that I use every day. One thing that I always have a hard time remembering is how to setup global keyboard shortcuts, so this quick guide will document the process.

There is a simple applet (reached via the menu “System | Preferences | Keyboard Shortcuts”) which allows you to specify which keys should be used for common activities.

Using the applet you can, for example, setup the keybinding Ctrl-Alt-t to run a terminal. However the list of actions is static, and it isn’t possible to add a new command there.

If you wish to bind Ctrl-Alt-e to launching emacs you’re out of luck – to do this you must use the gconf-editor.

Launch the editor by opening a terminal and running:

skx@pies:~$ gconf-editor

Using the gconf-editor we will need to do two things:

  • Define a new command.
  • Define the shortcut which should be used to execute that command.

So first of all navigate to “apps | metacity | keybinding_commands“. Once there in the righthand pane select “command_1” and change the value to /usr/bin/emacs.

Now we need to define the shortcut which will execute it, we do that by visiting the key “apps | metacity | global_keybindings“. Choose “run_command_1” and set the value to be “<Control><Alt>e”.

Now you should find that pressing Ctrl+Alt+e will launch emacs.

I hope in the future the keyboard shortcut application will allow you to define arbitrary commands and their associated shortcuts.

How to properly backup your system using TAR

Source: help.ubuntu.com

Preparing for backup

IconsPage/IconHint2.png Just a quick note. You are about to back up your entire system. Don’t forget to empty your Wastebasket, remove any unwanted files in your /home directory, and cleanup your desktop.

  • Depending on why you’re backing up, you might want to:

    • Delete all your emails
    • Clear your browser search history
    • Wipe your saved browser personal details

      IconsPage/IconNote.png If you are not worried about the security concerns, this step is not necessary. Many users explicitly want backups of their email and browser settings.

  • Unmount any external media devices, and remove any CDs/DVDs not needed for the backup process.
    • This will lessen the amount of exclusions you need to type later in the process.

Continue reading

Yahoo & Google letting you opt out of ads!

Source: Techcrunch (Click for the full article.)

From the article . . .

All of a sudden, Yahoo and Google want to make it easy for you to opt out of their ad targeting on both their sites and across the Web. Yahoo announced a new one-click opt-out policy today, and Google made it possible to opt out of both Google and Doubleclick ad targeting with one click yesterday.

At least Yahoo was honest enough to come out and say that the new policy was a direct response to Congressional scrutiny over the intrusiveness of online advertising and behavioral targeting. Google’s announcement was buried in a blog post about Doubleclick cookies.

. . . if you don’t want those cookies, you can decline Yahoo’s here and Google’s here.

VMware ESXi now available for FREE.

Now, you can virtualize your operating systems without the need for a full-fledged host to run. VMware’s famous ESX platform (a small 32MB footprint which in turn hosts the virtual operating systems) is now available for FREE, where one can them devote most of the system resources for multiple VMs on one machine, without the need for a fully-installed host OS to be present.

Click here for more information from VMware.

How to move Linux to a new hard drive

Source: PenguinPetes.com

1. You’ll need to connect the new drive onto the IDE cable. Most IDE cables have two plugs for hard drives. The headache comes with determining which drive is a master and which a slave, so the computer knows which one to boot to. No two hard drive manufacturers have the same standard for setting those tiny little jumper pins on the unit, so you’ll have to read whatever diagram they have and fiddle with tweezers moving them around and such.

Also, most PCs allow you to select boot order from the BIOS, or with what order you go by on the cables. Our troubles begin when we have a drive with the pins in the wrong place and a BIOS that disagrees with it. You’ll know that you have it all straightened out when you have your original drive as “/dev/hda” and your new drive as “/dev/hdb”. Start it up and test for this, then shut it back down. (Lyle’s edit: Often these days, hard disks are now labelled /dev/sdX, where X are letters of the alphabet, starting with ‘a’.)

2. Disconnect the original drive (so you don’t accidentally mess it up!) and boot the machine from your choice of any live CD distro or “Tom’s root-boot floppy“. Tomsrtbt has the benefit of being fast, having all the tools you need for disk maintenance, and virtually guaranteed to run on any old toaster you have. It’s saved my bacon dozens of times, so another big, whooping shout out to Tom. (Lyle’s edit: You can also use GParted’s Live-CD, very useful).

3. If it’s a used drive, use “fdisk /dev/hdb” to delete partitions. fdisk is really simple. The commands are all one-letter.

  • m – The “manual”, it will print all the commands available.
  • p – Prints the partition table.
  • d – Deletes a partition. It will prompt you for a number, if there’s more than one.
  • w – Will write the changes to the drive permanently.
  • q – Quits fdisk.

After this, I always use “dd if=/dev/zero of=/dev/hdb” to wipe the new hard drive completely. If it’s factory-new, you can skip these steps. Using “dd” will take a while for a large drive. Figure roughly one hour per ten Gigabytes. Leave it run while you go play Sudoku.

4. Now you will want to set up the new drive’s partitions with fdisk. Once again, use “fdisk /dev/hdb” and type your letters; Linux distros will usually need partition #1 to be the boot section, #2 to be swap, and #3 to be the “/” root directory. But these differ. Some distos want to split up the root partition into multiple sub-partitions, with “/usr/” or “/home/” going on a separate partition.

  • n – New partition. You will be prompted to answer whether it’s to be a primary or extended partition, enter a partition number, and specify the beginning and ending blocks. For anything less than four partitions per disk, make them all primary.
  • a – make a partition “bootAble”. Usually only the first partition – boot, naturally!
  • t – Change a partition’s system Type. By default, fdisk on Linux will make every partition type #83 – native Linux. The only one you will have to change is the swap one, to #82.

Here is a typical Linux setup from a different machine:

Disk /dev/hda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

   Device Boot      Start         End      Blocks   Id  System
/dev/hda1   *           1           9       72261   83  Linux
/dev/hda2              10         140     1052257+  82  Linux swap
/dev/hda3             141        1550    11325825   83  Linux
/dev/hda4            1551        9729    65697817+  83  Linux

Check the start and end cylinders for each partition. The boot partition (#1) only needs to go from 1 to about 9, since the boot section only needs a few MBs. The swap partition… is the subject of controversy. Nobody ever knows exactly what to make it. Just take a guess close to what most systems seem to install by default and go from there. The rule of thumb is “twice your RAM size”, but in real life I’ve found that the minimum you need should be about 256MB (1/4th of a Gig) and the most about a Gig. For a home system, anyway.

Because I have to have huge amounts of disk space for home (backing up the website, archiving my graphics art and writing work, emulating systems), I have 62 Gigabytes here for /home/ and only 11 Gigs for the “/” partition, which is extremely generous for most Linux installs even with more stuff on it than anybody would need.

5. Anyway. Now that you have the disk partitioned, make a file system on all partitions with type #83. “mke2fs /dev/hda1” and do the same for /dev/hda3… etc.

6. Now you’re ready to shut down, reconnect the main hard drive, and boot back to your main drive. Mount each partition from both disks in tandem – “mount -t ext2 /dev/hda1 /mnt/hda1” and “mount -t ext2 /dev/hdb1 /mnt/hdb1” for instance. Then copy everything from one disk to the other, one set of partitions at a time, repeating for hda3, hda4, etc. If you use “cp -pR /mnt/hda1 /mnt/hdb1”, that will copy everything Recursively (R) (all folders, subfolders, and files for the whole file tree), and preserving permissions (p). Do not copy the swap partition, of course!

7. And… one last step. You need to use either Grub or Lilo to write to the Master Boot Record of the new hard drive. I use Grub, since… well, I dunno, I like it! And it’s flexible. The only problem with grub is that its syntax is so alien. Here’s a good tutorial on Grub. My only habit is to mount the old boot partition and make a copy of the menu.lst file to somewhere else, then after I use Grub to create the bootable record on the new disk, I copy over the menu.lst file to it. Providing, that is, that the old system also used Grub and I’m not changing anything.

8. Finally, shut down yet again, remove the old hard drive, change the new hard drive so it will be the master disk instead of the slave, checking jumper pin arrangement on the drive, cable position on the IDE cable, and BIOS boot order. Congratulations, you did it! But do check to make sure the system is stable and no file data has been lost before you go wiping the old hard drive… this is why we copied files instead of moving them!

This was just a quick overview of the process. There’s a lot of details for each of the steps. Consult documentation for specifics, and if you’re not at least competent with the tools involved, try practicing on a scratch monkey computer until you get the hang of it without risking your valuable data.

And be sure to check the comments for all of the corrections left by visitors pointing out where I goofed this up!

NVidia G84 and G86 chips fail, consistently.

Don’t order any Laptops or PC’s with these NVidia chips … they fail.

Related: Ten Dell models have defective Nvidia GPUs.

(HP and ASUS and all other vendors are equally affected as the NVidia chips themselves are faulty).

Command Line Magic: Part II

Fun with rename:

1) To rename filenames with CAPS characters to lowercase:

rename ‘y/A-Z/a-z/’ *

2) To remove <spaces> from filenames:

rename ‘s/ /-/g’ *

3) To strip files with the .bak extension:

rename ‘s/\.bak$//’ *.bak

Free: The free command displays the amount of free and used memory in the system. For example, free -m gives the information using megabytes, which is probably most useful for current computers.

lsb_release -a: The lsb_release command with the -a option prints version information for the Linux release you’re running

sed: The sed (or Stream EDitor) command allows search and replace of a particular string in a file. For example, if you want to find the string “cat” and replace it with “dog” in a file named pets, type
sed s/cat/dog/g pets.

grep: An excellent tutorial.

Running commands sequentially in one command:
If you need to execute multiple commands in sequence but don’t need to pass output between them, there are two options based on whether or not you want the subsequent commands to run only if the previous commands succeed or not. If you want the commands to run one after the other regardless of whether or not preceding commands succeed, place a ; between the commands.

For example, if you want to get information about your hardware, you could run lspci ; lsusb which would output information on your PCI buses and USB devices in sequence.

However, if you need to conditionally run the commands based on whether the previous command has succeeded, insert && between commands. An example of this is building a program from source, which is traditionally done with ./configure, make, and make install.

The commands make and make install require that the previous commands have completed successfully, so you would use ./configure && make && sudo make install. Usually you’ll need an SUDO, for the last command, so make sure you’ve authenticated SUDO prior to running that command.

Learning the Linux Shell: A great tutorial.

How to record Skype calls (Ubuntu)

Source: Ubuntu Wiki.

Recording Skype calls

NOTE: This howto uses examples geared toward recording with gnome-sound-recorder. If you are using KDE, go to the bottom of this howto and click on the original link. There you will find additional info on using kmix and krecord. The basic howto is still the same.

What you’ll needed

  • ALSA utilities(alsamixer) found in alsa-utils
  • Audio recording software gnome-sound-recorder(Already installed in Hardy)
  • A version of Skype that supports ALSA(I installed from the Skype website in July 08 and that version or newer should be fine.)

How to configure

  • Make sure Skype is setup to use ALSA. Tools>Options>Sound Devices>Audio
  • Set up ALSA mixer
    • Open a terminal(Applications>Accessories>Terminal) and run alsamixer
    • Use your R/L arrow keys to set the mic input level.
    • You can optionally set the mic boost on by using the R/L arrow keys and then the space key.
    • Now hit the TAB key to see the Capture options window.
    • Now use the R/L arrow keys to navigate to Mix. Use your space key to select it. A red L R CAPTUR should appear.
    • Now use the R/L arrow keys to navigate to th Capture option, use your space key to select it. Again, you should see a red L R CAPTUR. Choose the level here using the up/down arrows.
    • Press ESC to close alsamixer
  • Now launch gnome-sound-recorder(Applications>Sound & Video>Sound Recorder)
  • Set Record from input to Capture in gnome-sound-recorder

Testing

It is best to test record a call before using it to record something important. You can do this by hitting the record button in gnome-sound-recorder and then use Skype to make the test call to Skype testing service. After finishing the call, hang up and stop recorder. Now save the file too your desktop as an ogg file. You can now use any audio playback device you have to play the recording. If all works correctly you should hear your recording. Note: You can now use an app. like audacity to edit your file or change the format, or both.

I got this howto here: [WWW] http://porpoisehead.net/hi/?q=node/23 There are some further instructions on adjusting the levels, etc. and using in KDE.

last edited 2008-07-16 22:14:39 by JohnCrawford

10 Essential tips of Linux Admins

Tips from IBM, not all that complicated, but worth looking at.

Other tips include…

Note: Some of these links requires registration.

Downloading torrents from command line

Source: LinuxHaxor.net

Essentially RTorrent, CTorrent and Transmission-CLI.

Command Line Magic: Part I

Source for this great article: Linux.com

In this context, a one-liner is a set of commands normally joined through a pipe (|). When joined by a pipe, the command on the left passes its output to the command on the right. Simple or complex, you can get useful results from a single line at the bash command prompt.
For example, suppose you want to know how many files are in the current directory. You can run: 

ls | wc -l

That’s a very simple example — you can get more elaborate. Suppose you want to know about the five processes that are consuming the most CPU time on your system:

ps -eo user,pcpu,pid,cmd | sort -r -k2 | head -6

The ps command’s o lets you specify the columns that you want to be shown. sort -r does a reverse order sort with the second column (pcpu) as reference (k2). head gets only the first six lines from the ordered list, which includes the header line. You can place pcpu as the first column and then omit the k2 option because sort by default takes the first column to do the sort. That illustrates how you may have to try several approaches on some one-liners; different versions and ways to manipulate the options may produce different results. A common situation for Linux administrators on servers with several users is to get quick ordered user lists. One simple way to get that is with the command:

cat /etc/passwd | sort

If you just need the username, the above command returns too much information. You can fix it with something like this:

cat /etc/passwd | sort | cut -d":" -f1

The sorted list is passed to cut, where the d option indicates the field’s delimiter character. cut breaks into pieces each line, and the first field f1 is the one that you need to display. That’s better; it shows only usernames now. But you may not want to see all the system usernames, like apache, bin, and lp. If you just want human users, try this:

cat /etc/passwd | sort | gawk '$3 >= 500 {print $1 }' FS=":"

gawk evaluates each line from the output piped to it. If the third field — the UID — is equal or greater than 500 (most modern distros start numbering normal users from this number) then the action is done. The action, indicated between braces, is to print the first field, which is the username. The separator for field in the gawk command is a colon, as specified by the FS option. Now suppose you have a directory with lots of files with different extensions, and you want to back up only the .php files, calling them filename.bkp. The next one-liner should do the job:

for f in *.php; do cp $f $f.bkp; done

This command loops through all the files in the current directory looking for those with .php extensions. Each file’s name is held in the $f variable. A simple copy command then does the backup. Notice that in this example we used a semicolon to execute the commands one after another, rather than piping output between them. What about bulk copy? Consider this:

tar cf - . | (cd /usr/backups/; tar xfp -)

It creates a tar package recursevely on the current directory, then pipes this package to the next command. The parenthesis creates a temporary subshell, changes to a different directory, then extracts the content of the package, which is the whole original directory. The p option on the last tar command preserves file properties like time and permissions. After completion, the shell context will be at the original directory. A variant on the previous one-liner lets you do the same kind of backup on a remote server:

tar cf - . | ssh smith@remote.server tar xfp - -C /usr/backup/smith

Here, the command establishes an SSH remote session and untars the package with the C option, which changes the directory, in this case to /usr/backup/smith, where the extraction will be made.

grep and gawk and uniq, oh my!

Text processing is a common use for one-liners. You can accomplish marvelous things with the right set of commands. In the next example, suppose you want a report on incoming email messages that look like this:

cat incoming_emails 2008-07-01 08:23:17 user1@example.com 2008-07-01 08:25:20 user2@someplace.com 2008-07-01 08:32:41 somebody@server.net 2008-07-01 08:35:03 spam not recived, filtered 2008-07-01 08:39:57 user1@example.com ...

You are asked for a report with an ordered list of who received incoming messages. Many recipients would be repeated in the output of the cat command. This one-liner resolves the problem:

grep '@' incoming_email | gawk '{print $3}' | sort | uniq

grep filters the lines that contains a @ character, which indicates an email address. Next, gawk extracts the third field, which contains the email address, and passes it to the sort command. Sorting is needed to group the same recipients together because the last command, uniq, omits repeated lines from the sorted list. The output is shown below. Most text processing one-liners use a combination of grep, sed, awk, order, tr, cut, uniq, and other related commands.

somebody@server.net user1@example.com user2@someplace.com

If you like any of these one-liners but think they’re too long to type often, you can create an alias for the command and put it in your .bashrc file. When you log in your session, anything inside this file will be run, so your personal aliases would be ready at anytime.

alias p5="ps -eo pcpu,user,pid,cmd | sort -r | head -6"

You can certainly create better and simpler variations of all of the commands in this article, but they’re a good place to start. If you are a Linux system administrator, it’s good practice to collect, create, and modify your own one-liners and keep them handy; you never know when are you going to need them. If you have a good one-liner, feel free to share it with other readers in a comment below.

Advanced file permissions in Linux

Source: blog.sontek.net

This is an excellent treatment on advanced file permissions in Linux, though centered on OpenSUSE, it’s still very helpful and applicable to any distro.

Though John credits this source for working with advanced Linux permissions in Ubuntu under Nautilus (Gnome).

Useful less-used Linux commands

1. lsb_release -a

2. The watch command.

3. Command line intro summary, with some rarely used switches (TLDP).

4. If you’re using a system which has a lot of users, and you’d like to see who has started a particular script, daemon, or binary, then the pstree utility is very helpful. It draws a tree of all currently running processes – allowing you to see which processes are related.

ThisIsSand.com

Now here’s something you don’t see every day. Click the little grey box in the upper left for instructions — then have fun, I did.

A gallery of what people have done resides here.

SSHMenu – Like Putty, but for Gnome!

I generally prefer SSH’ing to hosts from the CLI (Command Line Interface) though there are times when a GUI is preferable, especially when there’s a variety of settings specific to each session that might be desired.

SSHMenu is much like Putty, in that you can set up individual address book style destinations to SSH into, and save settings for each. While scripting them from the CLI and applying a chmod +x, I can see the benefit of using a centralized GUI.

From the site:

Each menu option will open an SSH session in a new terminal window. You can organise groups of hosts with separator bars or sub-menus. You can even open all the connections on a submenu (in separate windows or tabs) with one click.

Here’s a killer feature: imagine if every time you connected to a production server the terminal window had a red-tinted background, to remind you to tread carefully. Using terminal profiles, SSHMenu allows you to specify colours, fonts, transparency and a variety of other settings on a per-connection basis. You can even set window size and position.

It also leverages Gnome terminal profiles, so you can set up unique profiles to specify transparency, text color and background to remind you of which SSH Host you’re in. It’s written in Ruby, so once you add the source to your /etc/apt/sources.list you’ll need to sudo apt-get update, and import their GPG keys, and re-update again, then install.

Sources: Linux.com, SSHMenu.

Firefox 3 to be released on June 17 . . . a field guide.

Firefox 3 promises to be a great release. I do hope all the plugins and addons we enjoy using are ready for it.

Debbie Richardson has written a great field guide for Firefox 3 with all it’s improvements itemized.

From the guide, one of the most notable improvements: “Firefox 3 is the fastest, slimmest version of Firefox yet. Speed tests are showing a 2-4x improvement over Firefox 2 and 9x over Internet Explorer 7. Memory usage tests measure that Firefox 3 is 2x more efficient than Firefox 2 and 4.7x more efficient than IE7. There’s been a tremendous focus on performance for this release, and an incredible amount of effort has gone in to achieving these numbers.

Click here for in-depth details on the major improvements in Firefox’s memory usage.

Load more