Tag Archives: kvm

Red Hat Certification – RHCE – KVM via CLI

In a previous post while preparing for RHCSA I installed kvm post-installation, via the GUI.

But how to install, configure and use it only from the CLI?

Virt-Manager

http://virt-manager.org/page/Main_Page has some details

As a test-machine I’m using a server with Scientific Linux 6.2 (with virtualization enabled as seen by ‘cat /proc/cpuinfo|grep vmx’).

None of the Virtualization Groups are installed, as seen by ‘yum grouplist’. While doing that you’ll find four different groups. You can use

yum groupinfo "Virtualization Client"

or correspondingly to get more information about the group.

yum groupinstall Virtualization "Virtualization Tools" "Virtualization Platform" "Virtualization Client"

This installs a lot of things. Libvirt, virt-manager, qemu, gnome and python things.

lsmod|grep kvm
service libvirtd start
lsmod|grep kvm

This also sets up a bridge-interface (virbr0).

Now, how to install a machine or connect to the hypervisor?

How to get console?

ssh -XYC user@kvmserver
virt-manager

did not work.

On the client you could try to do:

yum groupinstall "Virtualization Client"
yum install libvirt
virt-manager

Then start virt-manager and connect to your server. However this didn’t work for me either. Is virtualization needed on the client too?

Noit is not, first: check if Virtualization is enabled on the server. Look in /var/log/messages for

kernel: kvm: disabled by bios

If it says that you’ll need to go into BIOS / Processor Options / and enable Virtualization.

Then you can start virt-manager, check that you can connect to the KVMserver.

Copy a .iso to /var/lib/libvirt/images on the server.

Re-connect to the kvm-server in virt-manager.

Add a new VM called test. Using 6.2 net-install and NAT network interface. This may take a while.

Pointing the VM to kvm-server where a httpd is running (remember firewall rules) and an SL 6.2 is stored. Installing a Basic Server.

OK, we could use virt-manager, it’s quite straight-forward and doesn’t require any edits of config files at all.

Moving on to virsh.

To install a vm you use ‘virt-install’.

You can get lots of info from ‘virsh’

virsh pool-list
virsh vol-list default
virsh list
virsh list-all
virsh dumpxml test > /tmp/test.xml
cp /tmp/test.xml /tmp/new.xml

Edit new.xml

change name to new and remove line with UUID

virt-xml-validate /tmp/new.xml
virsh help create
virsh create --file /tmp/new.xml
virsh list

This creates a new VM that uses the same disk and setup. But, if you shut down this new domain, it will disappear from virsh list –all and the list. To keep it you need to define it first:

virsh define --file /tmp/new.xml
virsh start new

This can become quite a bit more complicated. You would probably want to make clones (virt-clone) or snapshots (virsh help snapshot) instead of using the same disk file.

Making your own .xml from scratch looks fairly complicated. You could use ‘virt-install’ however.

virt-install --help
virt-install -n awesome -r 1024 --vcpus 1 --description=AWESOME --cdrom /var/lib/libvirt/images/CentOS-6.2-x86_64-netinstall.iso --os-type=linux --os-variant=rhel6 --disk path=/var/lib/libvirt/images/awesome,size=8 --hvm

For this the console actually works while running ‘virt-install’ over ssh on the kvm-server.

To make edit to a vm over ssh:

virsh edit NAMEOFVM

Red Hat Certification – RHCSA – Preparation

https://github.com/mconigliaro/RHCE-Cheat-Sheet

Found this “cheat sheet” for RHCE. Sure it doesn’t specifically say RHCSA but honestly there’s a lot of good commands in there. Some things obviously might be too advanced for RHCSA, such configuring an dns/named service. But it might be good as a reference.

The objectives of the RHCSA exam: https://www.redhat.com/certification/rhcsa/objectives/. I copied the ones I’m unsure about below.

I think definitely it would be a good idea to go through these objectives before taking the exam, and if you have time – do each step as well!

There’s a bunch of things there that I’m not sure about or know how to do. I’m attending a five day RHCSA rapid track course, so we should be able to go through the stuff I don’t know there, but doesn’t hurt to do a little preparation!

This post is about: me going through each objective and trying to accomplish it. Writing it down is for you, but mostly for me :) If you have any questions there is the comment field below.

The lists are the objectives, first level is the actual objective while the sub-lists are commands, thoughts and comments.

I’m writing this and updating it as I go along. It’s purpose is to prepare for the exam, without using any ‘cheats’ like trying to find out labs/questions that comes on the exam.

Understand and Use Essential Tools

  • Access remote systems using ssh and VNC
    • In each VM’s setting you can specify port etc to the VM.
    • vnc client:
  • Create hard and soft links
  • adjust process priority with renice,
    • renice
    • nice
    • top #to view, also in ps -fe -o pid,comm,nice
    • /etc/security/limits.conf
  • Access a virtual machine’s console
    • Open virt-manager and open the VM.
  • Start and stop virtual machines
    • Open virt-manager and stop/start there.
    • CLI: virtsh.

Add virtualization post-install.

To test: installing with only Desktop.
Packages, modules, services?

After install ‘lsmod|grep kvm’ doesn’t show anything.
Went into Add/Remove Software and added stuff under ‘Virtualization’.
After install, just trying to start virt-manager doesn’t work. It asks if libvirt service is running. ‘service libvirtd start’. Then virt-manager starts and finds the qemu. No need to reboot as ‘chkconfig –list|grep libv’ shows that they start on boot. Booting a machine after this works.

Configure Local Storage

  • Create and configure LUKS-encrypted partitions and logical volumes to prompt for password and mount a decrypted file system at boot
    • You can set this up while installing the system.
    • /etc/crypttab
    • /etc/fstab still necessary
  • Configure systems to mount file systems at boot by Universally Unique ID (UUID) or label
    • fstab: LABEL= and UUID=
    • Find label/UUID with blkid, set label with e2label.
  • Add new partitions, logical volumes and swap to a system non-destructively
    • # non-destructively? so without making the system unbootable?

You can format, partition a drive and encrypt it after install. In desktop you can go to places and find the drive in there, that will open a dialogue where you put in the password and tada. After that you can hit ‘df -h’ to get the UUID and mountpoint. This you then put in /etc/crypttab. Don’t forget to add it to /etc/fsstab too. But, be careful here. I managed to screw it up so much that it wouldn’t even boot anymore.

This is a great guide for how to set up a LUKS partition and mount it on boot.
Works for partitions created outside install.

When I did ‘custom layout’ in install and set up encryption, it appears to take a lot longer to encrypt/format. If doing this in the exam I’d consider making a small partition. Especially not a 16GB one. It took ~15minutes in comparison to 5s.
It was however fast to create with cryptsetup post-install.
If you do decide to split up the filesystem (perhaps one partition per VM) then you’ll need to set appropriate selinux settings to make it work.

Create LUKS partition to boot from post-install

During install:

vdisk in vmware of 20GB.
One partition of 500MB for /boot
One swap of 512MB
One pg of 10GB, VG of the same, and lv for /
Keeping available space of about 9GB.

After boot:

  1. fdisk -c -u /dev/sda
  2. n, p, 4, enter, enter, t, 4, 83, w
    1. new partition, primary, partition 4, starting, end (space), set type, partition 4, type 83, write
  3. some error, but fdisk -l shows the new partition /dev/sda4
  4. rebooted (tool advised to)
  5. cryptsetup luksFormat /dev/sda4
  6. cryptsetup luksOpen /dev/sda4 luksdrive
  7. ls /dev/mapper/ will show luksdrive in there.
  8. mkfs.ext4 /dev/mapper/luksdrive
  9. edit /etc/crypttab and add: /dev/mapper/luksdrive /dev/sda4
    1. man crypttab
  10. edit /etc/fstab and add: /dev/mapper/luksdrive /mnt/luksdrive ext4 defaults 1 2
    1. man fstab
  11. mkdir /mnt/luksdrive
  12. mount -a
  13. cd /mnt/luksdrive
  14. try a reboot

Mount filesystem based on UUID or label

By UUID: If you for example like above have created another partition and encrypted it and added it to fstab. You could just hit ‘blkid’ to get the UUID of the partition. Then you can change the /dev/mapper/luksdrive on the fstab into UUID=12354-515-51-5. To try it out, hit mount -a.

By label: set it with ‘e2label /dev/mapper/luksdrive lukslabel’. Then in fstab add LABEL=lukslabel instead of /dev/mapper/luksdrive. To view label hit: blkid. If there is none set, it’s not shown.

Create and Configure File Systems

  • Mount, unmount and use LUKS-encrypted file systems
    • cryptsetup luksOpen /dev/sda4 luksdrivelabel
    • mount -t filesystem /dev/mapper/luksdrivelabel /mnt/luksdrive
    • touch /mnt/luksdrive
    • umount /mn/luksdrive
  • Mount and unmount CIFS and NFS network file systems
    • mount -t nfs -o rw host:/remotedir /mnt/nfs
    • mount -t cifs //server/share /mnt/cifs –verbose -o user=username
    • umount /mnt/dir
  • Extend existing unencrypted ext4-formatted logical volumes
  • Create and configure set-GID directories for collaboration
    • A chmod on a directory that changes group owner of all files under that directory, into the same as the directory.
    • mkdir /share
    • touch /share/1
    • chgrp wheel/share
    • chmod g+s /share
    • touch /share/2
    • ls -l /share/
  • Create and manage Access Control Lists (ACLs)
    • first you need to add acl on the file system in /etc/fstab
    • getfacl
    • setfacl -m g:wheel:rw /path/file

Mount NFS file system

First, we need to set up an nfs server, this is not part of RHCSA though.

on server:
mkdir /nfs;chmod a+w /nfs
Make sure nfs-utils and rpcbind are installed.
chkconfig –list  # check nfs, nfslock and rpcbind are on
edit /etc/export # /nfs IP/netmask(rw,sync,no_root_squash)
setsebool -P nfs_export_all_rw
check /etc/hosts.allow and .deny
starts services

on client:
mkdir /mnt/nfs
mount.nfs 192.168.0.17:/nfs /mnt/nfs -v -w
or
mount -t nfs -o rw 192.168.0.17:/nfs /mnt/nfs

ACL on filesystem

  • mount # see options on your filesystem
  • vi /etc/fstab # change ‘defaults’ to the what you saw in ‘mount’ and add acl, comma separated
  • mount -o remount / # use this to remount /. Or you could reboot. Hard to unmount / if you are using it.
  • mount # now it has rw,acl
  • getfacl /root/install.log
  • setfacl -m g:wheel:rw /root/install.log
  • getafcl /root/install.log

Extend existing unencrypted ext4-formatted logical volumes

Deploy, Configure and Maintain Systems

  • Configure systems to boot into a specific runlevel automatically
    • /etc/inittab
  • Install Red Hat Enterprise Linux automatically using Kickstart
  • Configure a physical machine to host virtual guests
  • Install Red Hat Enterprise Linux systems as virtual guests
  • Configure systems to launch virtual machines at boot

Installed SLC6.1 in a VM. This time I chose both Virtual Host and Desktop Environment and X11 for packages. In VMWare Workstation 8 and the settings for the VM, do enable ‘virtualization’ in the processor options or you cannot virtualize inside the VM. It’s a lot easier to setup/install VM if you have a desktop GUI. Especially the part about you getting access to the console.

Post-install there is a GUI tool in the menu that you can use to install a VM and configure VM-stuff.

By default the virtual machine starts on boot.
In chkconfig –list. There is an entry called ‘libvirt-guests’. This is a fairly complex script that looks where the VMs are installed and boots them. You can go into the settings of the VM in the GUI and enable it to boot when the host boots.
By the way, if there are issues during boot, see /var/log/boot.log

Install a VM via an http server.

yum install php

This installs httpd with php-support.

firewall

Add port 80 in the firewall: iptables-save > fwrules. Copy the one with port 22, paste and add port 80. iptables-restore < fwrules.

To keep the rules on reboot:

/etc/init.d/iptables save

Copy DVD into your web root:

This assumes that the DVD is mounted automagically which it does for me.

sudo mkdir /var/www/html/SL6; sudo cp -pR /media/nameofdisk/* /var/www/html/SL6

If you use the -p that means it preserves the read/write permissions on the files, beceause it’s mounted as a CD/DVD that means the files are read-only. If you want to do changes don’t use the -p or you’ll have to change that stuff later.

To set SELINUX context:

chcon -R --reference=/var/www /var/www/html/SL6.

Install from HTTP

Launch the virtualization manager. Create new VM. Name and network transfer, point to your httpd. RAM, disk space. Chose network interface – I only had NAT. (if you follow my guide below you’ll need to set static IP settings). After that the machine boots and you get a console. It starts graphical and then install continues as usual. If you want to see which IP your VM in the VM gets you can look in the access_log in /var/log. By default it got an address in 192.168.122.* range. If you set too little memory you cannot get the kdump.

Bridged networking

follow this guide (incomplete) or one on linuxtopia or on libvirt wiki

  1. ifdown eth0
  2. cd /etc/sysconfig/network-scripts
  3. cp ifcfg-eth0 ifcfg-bridge0
  4. edit ifcfg-eth0 and add ‘BRIDGE=”bridge0″ ‘
  5. edit ifcfg-bridge0 and set ‘DEVICE=”bridge0″ ‘, ‘TYPE=”Bridge” ‘, ‘DELAY=”0″ ‘
  6. TYPE needs to be Bridge, capital B.
  7. ifup eth0
  8. ifup bridge0
  9. ifconfig bridge0 192.168.0.17
  10. add a rule similar to -A INPUT -i bridge0 -j ACCEPT in the iptables (don’t forget to save/restart iptables)
  11. edit /etc/resolv.conf with ‘nameserver ip.ip.ip.ip’.
  12. /etc/sysctl.conf and enable ip_forwarding. Reboot or sysctl -p /etc/sysctl.conf
  13. consider adding static IP addresses in ifcfg-bridge0. My DHCP didn’t work, probably because of some configration in VMWare Workstation. BOOTPROTO=”static”, IPADDR, NETMASK, GATEWAY, NM_CONTROLLED=”no”, ONBOOT=”yes”.

Installing with the help of kickstart

First, copy the /root/anaconda-ks.cfg to /var/www/html/SL6/ks.cfg. Also set permissions to the file as appropriate.
Then open that file in system-config-kickstart. You probably want to change some stuff. For HTTP server install set server to: 192.168.0.17 and path to SL6. That’s if your path is . And of course add the whole URL to the ks.cfg. Remove virtualization packets. Change hdd layout stuff, you probably have less space available this time. Change URL to repository. Mine was still set to CD/ROM so had to manually set that during boot. Got two questions during the install: do you want to overwrite what’s on the disk? And, reboot? at the end of install. Consider removing these to speed up install.
Also, I could not log on after first reboot. Even though I kept the root password as is.

In system-config-kickstart: Set it to clear MBR, initialize labels and also to autoreboot upon completion.
For root password you need to manually enter, you can set it to plaintext. Set setupagent to disabled for a completely automatic install.
Repository you cannot change in system-config-kickstart.
Manually edit the ks.cfg.

repo –name –baseurl=http://192.168.0.17/SL6
user –name user –plaintext –password 112233

Last one creates a user called user with pw 112233.

How-to Boot into CD in VM in qemu

Download the .iso.
Add new storage hardware, make it an IDE CD-ROM, hit add existing storage and select the .iso, set type to ‘raw’. Change boot order.

Manage Users and Groups

  • adjust password aging for local user accounts
    • chage
  • Set enforcing and permissive modes for SELinux
    • sestatus to see current setting
    • /etc/selinux/config # for settings
    • Command to set it ‘on the fly’:
  • List and identify SELinux file and process context
    • files: ls -Z
    • processes: ps -e fZ
  • Restore default file contexts
    • chcon -R –reference=/var/www/html /var/www/html/SL6
    • chon -t usr_t /var/www/html/SL6
    • restorecon -v /var/www/html/SL6
  • Use boolean settings to modify system SELinux settings
    • setsebool
    • to find the available settings: getsebool -a
  • Diagnose and address routine SELinux policy violations
    • Tool ‘sealert’. Logs are in /var/log/audit
    • There is also a GUI tool.

Scientific Linux 6 – Basic Setup

Not allowing root to log in

By default sshd is running on SL6 and you can ssh in with ‘root’.

Probably a good idea to change this in /etc/ssh/sshd_config

permitrootlogin no

But first, create a user that can log in.

useradd mart
passwd mart

Then you can change sshd_config and ‘service sshd reload’.

Then you can ssh in and either hit’ su -‘ to get root access.
Or, if you hit ‘visudo’ and add your user. You can later just type ‘sudo bash’ to get a root bash shell.

Firewall

iptables -L to view the firewall setup, note that there is a ‘virbr0’ interface that has forwarding rules. This is probably for NAT or bridging for potential virtual machines, and was probably created when we chose ‘Virtual Host’ as the role for the system.
iptables-save : another view that may be easier to understand. This you can put in ‘file’ and then hit iptables-restore < file.

Slow before you get the login prompt while ssh-ing?

To see what is happening, ssh in with ‘ssh -v ip’.
In my case I saw

 debug1: An invalid name was supplied
Cannot determine real for numeric host address

A little googling showed me that this is because your machine doesn’t have a name lookup for that IP. So go ahead and add one in /etc/hosts and then it will be fast.

Notice that your ssh stops working after a while? Doesn’t accept input?

If so, add this to your ~/.ssh/config file:

Host *
   ServerAliveInterval 60

Make sure there is at least a space on the second line. I have three :p
You can change the * to a specific domain if you do not want to do this on all your boxes.
If the file doesn’t exist, create it.

Run sshd on a second port.

  1. Edit /etc/ssh/sshd_config
  2. Add a line saying: Port 6666
  3. look in /etc/hosts.allow (any entries? good)
  4. iptables-save > ~/fwrules
  5. vi fwrules
  6. copy the –dport 22 line and paste a new one above the -j REJECT lines (vi commands: yy and P)
  7. change the 22 to 6666 (vi commands: x for delete, R for replace mode. :wq! to save and quit)
  8. iptables-restore < ~/fwrules
  9. /etc/init.d/sshd restart

If you want you can hit: iptables -L or iptables-save.
These will also show the current iptables rules.
See ip6tables for IPv6 rules.

Now the port is running on another non-standard port (you could set it to whatever you want, as long as it’s lower then 65536 and preferably higher than 1024). This might be good for security reasons. You could still have port 22 open for access from your internal network (see adding a -s ip.add.r.ess on the row in the iptables rules) and the other one accessible from the internet or maybe even a specific network / address on the internet for even more security.

Install Scientific Linux 6 in VMWare Workstation

Time for some more Linux testing.

The reason for this is because I think I will go ahead and try to study for the RHCSA – Red Hat Certified System Administrator. Work might send me to a course in December, probably wise to play around with it before this.

So here we go.

Scientific Linux (SL) is a free clone of Red Hat Enterprise Linux (RHEL). Historically it’s been updated faster than CentOS. It’s same as Enterprise Linux (EL) – it’s re-compiled from source.

New VM, \SL-61-x86_64-2011-07-27-Install-DVD.iso, RHEL6 64-bit. 1 Core, 2G RAM, NAT, LSI Logic, New virtual disk, SCSI, 20G. Then boot the VM.

SL.org has this in pictures.

First thing you see is the Grub menu:

  1. Install or Upgrade
  2. Install with basic video driver
  3. Rescue
  4. Boot from local drive
  5. Memtest (I like that memtest is pretty standard now)

Chose 1. Next screen is a graphical interface where you click and write, so you need keyboard/mouse. Next screen asks if you want local disks or external storage (fc, iSCSI, or zFCP – for system Z). Hostname: SL1.localdomain.

Create disks. Custom/full size. xfs/encryption/lvm cannot be used for boot volumes.

Role: Virtual Host (I want to try KVM). Enabling SL 6.1 and SL 6.1 Security Updates repositories.

Pinging to something on the Intertubes work from the start.

More posts coming with more fun stuff :)