Posted by Tres
Fri, 25 Apr 2008 10:03:00 GMT
After Edgy, Ubuntu (6.10) replaced init with upstart. Upstart uses files located in /etc/event.d/ to do what old /etc/inittab entries used to do.
Taking a peek in /etc/event.d/ you’ll see that different run levels are represented by different files. In addition, tty entries are maintained here.
If you’re running Xen, and are having problems with your console access, you’ll need to create /etc/event.d/xvc0 and insert the following:
# xvc0 - getty
#
# This service maintains a getty on xvc0 from the point the system is
# started until it is shut down again.
start on runlevel 2
start on runlevel 3
stop on runlevel 0
stop on runlevel 1
stop on runlevel 4
stop on runlevel 5
stop on runlevel 6
respawn
exec /sbin/getty 38400 xvc0
If you want to retain the ability to hop into the domU without needing to authenticate, make sure that you bypass getty login and spawn a shell:
exec /sbin/getty -n -l /bin/bash 38400 xvc0
Posted in Linux, Sysadmin, Xen, Ubuntu | Tags console, Ubuntu, Xen
Posted by Tres
Fri, 25 Apr 2008 06:45:00 GMT
If you’re trying to start up an Ubuntu environment and seeing
* Starting OpenBSD Secure Shell server sshd PRNG is not seeded
when Ubuntu tries starting OpenSSH, make sure that the /dev/random and /dev/urandom entries have read+write permissions for user,group and other.
chmod 666 /dev/urandom /dev/random
Posted in Linux, Sysadmin, Ubuntu | Tags ssh, Ubuntu
Posted by Tres
Sun, 20 Apr 2008 00:36:00 GMT
Here’s a little ruby hackery to do convert the output we got here into something like this:
MB total: 2432.0
MB used: 479.6953125
MB free: 1952.3046875
#!/usr/bin/env ruby
require 'optparse'
options = {}
outprint = {}
ot = String
opts = OptionParser.new do |opts|
opts.on("-p X", "--path X", String, "path to partition for host") do |path|
options[:path] = path
end
end
opts.parse!(ARGV)
output = `dumpe2fs -h #{options[:path]}`
output.squeeze!(" ")
output.each do | line |
line.grep(/Block count:/) { | total | outprint[:total_label] ,outprint[:total_data] = total.chomp.split(/\s*\:\s*/) }
line.grep( /Free blocks:/) { | free | outprint[:free_label], outprint[:free_data] = free.chomp.split(/\s*\:\s*/) }
line.grep(/Block size:/) { | size | outprint[:block_size_label], outprint[:block_size_data] = size.chomp.split(/\s*\:\s*/) }
line.grep(/Reserved block count:/) { | reserved | outprint[:reserved_label], outprint[:reserved_data] = reserved.chomp.split(/\s*\:\s*/) }
end
mb_available = ( outprint[:block_size_data].to_f / 1048576 * outprint[:total_data].to_f )
mb_free = ( outprint[:block_size_data].to_f / 1048576 * outprint[:free_data].to_f )
mb_used = ( mb_available.to_f - mb_free.to_f )
puts "MB total: #{mb_available}"
puts "MB used: #{mb_used}"
puts "MB free: #{mb_free}"
The ruby script takes a single argument, the path to the device that is going to be looked at. It can be passed with either a -p or –path.
Posted in Linux, Ruby, Sysadmin, Xen, Development | Tags DomU, memory, monitor, ruby, script, Xen
Posted by Tres
Thu, 14 Feb 2008 06:03:00 GMT
Every once in a while, I need to say my old mantra, “it’s always the little things.” The biggest of problems always seem to boil down to the littlest of problems in the world of *nix. Here’s another example.
If you’ve upgraded your version of Xen on CentOS/Red Hat Enterprise Linux from the included Xen 3.0.x to the latest available from XenSource (3.2 at this time), you may run into something like this in /var/log/xen/xend.log when trying to start xend:
INFO (SrvDaemon: ) Xend Daemon started
INFO (SrvDaemon: ) Xend changeset: unavailable.
INFO (SrvDaemon: ) Xend version: Unknown.
ERROR (SrvDaemon: ) Exception starting xend ((13, 'Permission denied'))
...
Error: (13, 'Permission denied')
and something like this in /var/log/xen/xend-debug.log
sysctl operation failed -- need to rebuild the user-space tool set?
Exception starting xend: (13, 'Permission denied')
The error in the debug log is misleading. I spent my time looking for duplicate installs of xen tools that may have been installed. My theory was that the Red Hat supplied Xen RPM installed files that the XenSource SRPM spec file puts in a different spot, or some other bookkeeping issue related to rpm -Uvh on the xen RPM files.
It turns out I was right, but was looking at the wrong source of the issue. The Xen tools were indeed mismatched, but it was to the running kernel. The xen RPM installed the tools and the kernel and did everything it was supposed to, but it didn’t update grub.conf to boot with the updated xen kernel files that were installed.
“It’s always the little things.” It’s often embarrassing, and this is no exception, but I thought I’d put this out there so if someone else gets caught looking at the wrong source of a mismatched user-space tool set error, they can avoid the trouble of looking for rogue tool installations and get on with business. Just update your /boot/grub/grub.conf with something like the following (<DANGER, WILL ROBINSON>:needless to say, copying and pasting grub.conf entries without verifying them can land you in a heap of trouble if you don’t have local access to your server</DANGER, WILL ROBINSON>):
title CentOS (xen-3.2)
root (hd0,0)
kernel /xen.gz ro root=/dev/vol00/root dom0_mem=256M
module /vmlinuz-2.6.18-53.1.13.el5xen ro root=/dev/vol00/root
module /initrd-2.6.18-53.1.13.el5xen.img
<DANGER>:This entry is very much dated, and uses the latest CentOS 5.x kernel as of this writing</DANGER>. The thing that won’t change, and will continue to be valid no matter what version of Xen you build and install later, is the kernel line:
kernel /xen.gz
The XenSource RPM & SRPM will build things so that /boot/xen.gz is a symlink to the latest version installed.
Posted in Linux, Red Hat Enterprise Linux, Sysadmin, Xen | Tags api, upgrade, Xen, xensource, 3.2
Posted by Tres
Sun, 23 Dec 2007 10:55:00 GMT
So you try and log in to a domU and xen says it can’t open a tty?
[tres@calliope ~]$ sudo xm console xen-domu.vm
xenconsole: Could not open tty `/dev/pts/15': No such file or directory
This error normally happens when xenconsoled is no longer running. You can quickly verify by looking for the xenconsoled process:
[tres@calliope ~]$ ps auxwww | grep xenconsoled
[tres@calliope ~]$
If you don’t find it, just run
/etc/init.d/xend start
Posted in Linux, Sysadmin, Xen | Tags DomU, tty, xenconsoled
Posted by Tres
Fri, 21 Dec 2007 12:16:00 GMT
After doing an apt-get upgrade on a Gutsy VM recently, I found that ye olde 4gb seg fixup messages had found their way back into dmesg and were all over the console. I quickly apt-get install libc6-xen only to be told that it didn’t exist…
Hmm…
A little investigating and I found that Gutsy didn’t ship with Xen libc, and as of this writing, they are still listed as Proposed.
So the choice was pretty clear, either mv /lib/tls or downgrade the libc we just upgraded to (libc6-xen 2.6.1-1ubuntu10) so it matched with the available lib6c xen package (libc6-xen 2.6.1-1ubuntu9). Personally, I think having the package installed is the right way to go, so I did:
apt-get remove libc6
apt-get install libc6=2.6.1-1ubuntu9 libc6-xen=2.6.1-1ubuntu9
Posted in Linux, Sysadmin, Xen, Ubuntu | Tags fixup, gutsy, libc6, Ubuntu, upgrade
Posted by Tres
Thu, 20 Dec 2007 15:05:00 GMT
The Hyperic documentation says you should just need to install xorg-x11-libs and libXp to get Hyperic server running on CentOS5. That’s not quite true. In order to get things running I also needed to install xorg-x11-utils.
Posted in Linux, Red Hat Enterprise Linux, Sysadmin | Tags hyperic, libXp, server
Posted by Tres
Tue, 18 Dec 2007 03:36:00 GMT
Later versions of Ubuntu can be fixed with the easy
apt-get install libc6-xen
But when you run this on 6.06 LTS, the package is unavailable. In order to fix this problem, you just need to do this:
echo 'hwcap 0 nosegneg' > /etc/ld.so.conf.d/libc6-xen.conf && ldconfig
Of course, any processes that were started before this fix was made will still write the following to syslog:
4gb seg fixup, process xxxx(pid xxxx), cs:ip 73:xxxxxxxx
Posted in Linux, Xen, Ubuntu | Tags fixup, libc, Ubuntu, Xen
Posted by Tres
Sun, 16 Dec 2007 19:08:00 GMT
If you’re having issues with SSHD not starting normally, but are able to start it using the debug flag /usr/sbin/sshd -d, the problem could very likely be permissions of the /dev/null device file. Check /var/log/secure for the following messages:
Dec 15 22:51:18 server sshd[2128]: fatal: daemon() failed: No such device
If you see messages like these, try the following as root:
rm /dev/null
mknod /dev/null c 1 3
Posted in Linux, Sysadmin | Tags sshd
Posted by Tres
Sat, 20 Oct 2007 13:41:00 GMT
If your LVM partition is running out of room, it’s very easy to add more disk space; use a combination of lvextend, fsck and resize2fs to make that partition bigger.
First, let’s make the logical volume bigger:
[root@calliope mnt]# lvextend -L+1G -r /dev/vol00/partition
In this example, we made the /dev/vol00/partition logical volume 1 gigabyte bigger (-L+1G is where we set how much bigger we want to make it).
The hard part is over, now all we need to do is make sure the extended logical volume doesn’t have errors:
[root@calliope mnt]# e2fsck -f /dev/vol00/partition
And finally, extend the filesystem table to account for the new space:
[root@calliope mnt]# resize2fs /dev/vol00/partition
Posted in Linux, Sysadmin | Tags disk, extend, logical, lvm, manage, partition, size, volume
Posted by Tres
Sat, 20 Oct 2007 12:55:00 GMT
Being able to drop in to your domU via console requires that a terminal be set up in /etc/inittab.
A basic terminal configuration that will allow you to have access to domU is just to set up bash as the terminal ‘device.’
Just add the following to your /etc/inittab to get basic access:
1:2345:respawn:/bin/sh
However, you should beware when using a shell directly; you will lose the ability to send job control keystrokes, such as ctrl + c because there is not controlling terminal for the shell.
Another alternative is to use the xvc0 device instead. This not only gives you all the functionality you expect from a normal terminal based shell, but also has the additional benefit of providing your domU some security. By using xvc0 anyone wanting to use the console to gain access to your system must give valid credentials before they have access to the system (just like logging in locally via console).
Posted in Linux, Sysadmin, Xen | Tags console, DomU, prompt, shell, terminal, Xen
Posted by Tres
Wed, 17 Oct 2007 23:49:00 GMT
If you’re having issues starting Debian domUs that complain about eth devices not being found, try editing etc/udev/rules.d/z25_persistent-net.rules and removing any lines like this:
SUBSYSTEM=="net", DRIVERS=="vif0", ATTRS{address}=="00:16:3a:11:44:32", NAME="eth0
Then restart domU.
Posted in Linux, Sysadmin, Xen | Tags debian, eth0, network, SIOCSIFADDR, Xen
Posted by Tres
Wed, 17 Oct 2007 01:16:00 GMT
This is an issue with TLS libraries on FC6 moving /lib/tls won’t get this fixed. In order to get things working right, you’ll need to do this:
echo "hwcap 0 nosegneg" > /etc/ld.so.conf.d/nosegneg.conf
Then reboot domU. If you don’t reboot, you’ll still see messages like this:
4gb seg fixup, process syslogd (pid 590), cs:ip 73:00135435
For any process that was started before the update was made.
Posted in Linux, Sysadmin, Xen | Tags 6, fedora, fixup, libraries, tls, Xen
Posted by Tres
Mon, 15 Oct 2007 13:40:00 GMT
Setting up a Gentoo Xen DomU is pretty straight-forward. The only gotcha that you may run into is when Gentoo tries to mount /proc and dies with a message that includes:
The "mount" command failed with error: proc already mounted
Fixing this is just a matter of commenting out the section of /sbin/rc that attempts to mount /proc begining at around line 217 like this:
# check_statedir /proc
#
# ebegin "Mounting proc at /proc"
# if [[ ${RC_USE_FSTAB} = "yes" ]] ; then
# mntcmd=$(get_mount_fstab /proc)
# else
# unset mntcmd
# fi
# try mount -n ${mntcmd:--t proc proc /proc -o noexec,nosuid,nodev}
# eend $?
You’ll also need to comment the section attempting to mount /sys as well, or the next time you boot, you’ll get the same error with a different mount point failing.
# if [ "$(get_KV)" -ge "$(KV_to_int '2.6.0')" ] ; then
# if [[ -d /sys ]] ; then
# ebegin "Mounting sysfs at /sys"
# if [[ ${RC_USE_FSTAB} = "yes" ]] ; then
# mntcmd=$(get_mount_fstab /sys)
# else
# unset mntcmd
# fi
# try mount -n ${mntcmd:--t sysfs sysfs /sys -o noexec,nosuid,nodev}
# eend $?
# else
# ewarn "No /sys to mount sysfs needed in 2.6 and later kernels!"
# fi
# fi
Posted in Gentoo, Linux, Xen | Tags DomU, Gentoo, Xen