Posted by Tres
Sat, 03 May 2008 13:49:00 GMT
Installing Postgres on FreeBSD, as always with packages and ports, is a breeze. pkg_add -r postgresql82-server or better yet, portinstall -P databases/postgresql82-server and a quick edit to /etc/rc.conf later, it’s Miller Time… Well almost…
Getting Postgres running in a FreeBSD jail requires that the jail host allows access to FreeBSD’s System V interprocess communication send and receive system calls.
Warning:This breaks down the separation of jailed processes from the host. If you’re paranoid about the security of your host environment, you’ll probably not want to do this – since the same namespace is used for IPC primitives of both the host and in the jail environment as well. This means that someone can potentially send and receive to processes being run in the host environment, or in other jailed environments. There is potential for denial of service, but so long as there are users on a box, there’s always a potential for denial of service, right?
So, to get things running just add the following to /etc/sysctl.conf in the host environment:
security.jail.sysvipc_allowed=1
That will make sure that things start up right whenever the box gets rebooted. To get things running right now, type the command in your terminal window.
sudo security.jail.sysvipc_allowed=1
Posted in FreeBSD, Sysadmin | Tags FreeBSD, Jail, postgresql, virtualization
Posted by Tres
Thu, 01 May 2008 02:12:00 GMT
Here’s a little look at big O notation. It’s by no means complete, but it is an initial overview of what big O is, and why it’s important in Computer Science.
By analyzing the complexity of an algorithm, big O allows the estimation of the resources required to complete an operation with a large data set. On a graph, it defines the asymptotic upper bound for the algorithm.
Big O indicates how the resource consumption shifts with the size of the data. It does not deal with the efficiency of an algorithm, it only deals with efficiency via the fundamental fact that efficiency will be affected by algorithms that require more resources to complete an operation than those that require less resources. It’s a subtle, but meaningful difference when you account for the fact that in big O notation, you’re only using the “dominant term” to define the resources required. So for two constant algorithms, one that takes (i) and one that takes (3i), big O would denote both of these as simply O(i). Of course one is three times more efficient than the other, but big O is not concerned with that, only with the growth rate for larger data sets.
Posted in Development | Tags big O, computer science, notation
Posted by Tres
Wed, 30 Apr 2008 06:38:00 GMT
If you’re experiencing problems with TextMate’s subversion commit window, the diff history or other UI windows just crashing, the problem may be because of Mac Ports.
Check out /var/log/system.log to find out where the crash log was written to. You should see something like this:
Apr 12 18:33:50 blas.phemo.us ReportCrash[1486]: Saved crashreport to /Users/tres/Library/Logs/CrashReporter/mate_2008-04-12-183348_ishmael.crash using uid: 512 gid: 512, euid: 512 egid: 512
Check out the contents of the file and you’ll see something like this:
Process: mate [485]
Path: /Applications/TextMate.app/Contents/SharedSupport/Support/bin/mate
Identifier: mate
Version: ??? (???)
Code Type: X86 (Native)
Parent Process: ruby [471]
Date/Time: 2008-04-29 18:33:48.373 -0700
OS Version: Mac OS X 10.5.2 (9C7010)
Report Version: 6
Exception Type: EXC_BREAKPOINT (SIGTRAP)
Exception Codes: 0x0000000000000002, 0x0000000000000000
Crashed Thread: 0
Dyld Error Message:
Symbol not found: __cg_jpeg_resync_to_restart
Referenced from: /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Versions/A/ImageIO
Expected in: /opt/local/lib/libJPEG.dylib
The important things to look for is the contents of the Dyld Error Message.
If you’re seeing a libjpeg error, it’s due to the installed version of libjpeg that you’ve got through Mac Ports. Luckily this is an easy one to resolve. Just copy the version from /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Resources/libJPEG.dylib to /opt/local/lib/libJPEG.dylib
sudo cp /System/Library/Frameworks/ApplicationServices.framework/Versions/A/Frameworks/ImageIO.framework/Resources/libJPEG.dylib /opt/local/lib/libJPEG.dylib
Posted in OS X, Development | Tags crash, subversion, textmate, window
Posted by Tres
Fri, 25 Apr 2008 10:03:00 GMT
After Edgy, Ubuntu (6.10) replaced init with upstart. Upstart uses files located in /etc/event.d/ to do what old /etc/inittab entries used to do.
Taking a peek in /etc/event.d/ you’ll see that different run levels are represented by different files. In addition, tty entries are maintained here.
If you’re running Xen, and are having problems with your console access, you’ll need to create /etc/event.d/xvc0 and insert the following:
# xvc0 - getty
#
# This service maintains a getty on xvc0 from the point the system is
# started until it is shut down again.
start on runlevel 2
start on runlevel 3
stop on runlevel 0
stop on runlevel 1
stop on runlevel 4
stop on runlevel 5
stop on runlevel 6
respawn
exec /sbin/getty 38400 xvc0
If you want to retain the ability to hop into the domU without needing to authenticate, make sure that you bypass getty login and spawn a shell:
exec /sbin/getty -n -l /bin/bash 38400 xvc0
Posted in Linux, Sysadmin, Xen, Ubuntu | Tags console, Ubuntu, Xen
Posted by Tres
Fri, 25 Apr 2008 06:45:00 GMT
If you’re trying to start up an Ubuntu environment and seeing
* Starting OpenBSD Secure Shell server sshd PRNG is not seeded
when Ubuntu tries starting OpenSSH, make sure that the /dev/random and /dev/urandom entries have read+write permissions for user,group and other.
chmod 666 /dev/urandom /dev/random
Posted in Linux, Sysadmin, Ubuntu | Tags ssh, Ubuntu
Posted by Tres
Sun, 20 Apr 2008 07:26:00 GMT
Sometimes you just need to hand control over to an external command via the exec built-in function in PHP. But
whenever you transfer control of a process to an external command, there’s a chance of your app hanging.
To safely run system commands you should wrap your exec call in the process control functions. The following is an example class that uses process control functions to ensure that a call to an external system command gets returned back.
/**
*
*
*
* @copyright 2008
* @package ProjectX
* @subpackage SystemResources
* @author Tres Wong-Godfrey
*
*/
/**
* Sys handles system execution, ensuring that the external call will be returned to us.
*/
//set up signal callback mechanism
declare(ticks=1);
class Sys {
/**
* Handles the signal alarm timer for the process
*
* @throws TimeoutException when called
* @param integer the signal that will be
*/
public static function alarm( $signal )
{
throw new TimeoutException(__CLASS__.'::'.__FUNCTION__ . ":: a Timeout occured while attempting to run the external system call.");
}
/**
* This sets the timeout for the external command so that we can ensure control gets returned
* to us.
*
* @param int The timeout for the external command in seconds
* @access public
*/
public static function setAlarm($timeout)
{
pcntl_signal(SIGALRM, array($this, 'alarm'));
pcntl_alarm($timeout);
}
/**
* Cancels the previously set alarm
*/
public static function unsetAlarm() { pcntl_alarm(0); }
/**
* This is a modified exec(). It is responsible for setting the
* timeout alarm and handling any invalid returns.
*
* <code>
* $command = "ls /";
* try {
* Sys::execute( $command, 90 );
* } catch( TimeoutException $e ) {
* //handle timeout
* } catch( ExternalApplicationException $e ) {
* //handle problems with external app running
* }
* </code>
*
* @param string $cmd
* @param int Optional. Time in seconds until the process alarm timer expires. Defaults to 10.
* @throws ExternalApplicationException if the external call does not return 0.
* @return string The output from the external command
*/
public static function execute( $cmd, $timeout=10)
{
self::setAlarm($timeout);
$last_line=exec("$cmd 2>&1", $output, $return_value);
self::unsetAlarm();
if ($return_value != 0)
throw new ExternalApplicationException(__CLASS__.'::'.__FUNCTION__ . ":: command $cmd failed. :: output = $output" );
else
return implode("\n", $output);
}
}
Posted in PHP, Development | Tags control, external, pcntl, php, process, run
Posted by Tres
Sun, 20 Apr 2008 00:36:00 GMT
Here’s a little ruby hackery to do convert the output we got here into something like this:
MB total: 2432.0
MB used: 479.6953125
MB free: 1952.3046875
#!/usr/bin/env ruby
require 'optparse'
options = {}
outprint = {}
ot = String
opts = OptionParser.new do |opts|
opts.on("-p X", "--path X", String, "path to partition for host") do |path|
options[:path] = path
end
end
opts.parse!(ARGV)
output = `dumpe2fs -h #{options[:path]}`
output.squeeze!(" ")
output.each do | line |
line.grep(/Block count:/) { | total | outprint[:total_label] ,outprint[:total_data] = total.chomp.split(/\s*\:\s*/) }
line.grep( /Free blocks:/) { | free | outprint[:free_label], outprint[:free_data] = free.chomp.split(/\s*\:\s*/) }
line.grep(/Block size:/) { | size | outprint[:block_size_label], outprint[:block_size_data] = size.chomp.split(/\s*\:\s*/) }
line.grep(/Reserved block count:/) { | reserved | outprint[:reserved_label], outprint[:reserved_data] = reserved.chomp.split(/\s*\:\s*/) }
end
mb_available = ( outprint[:block_size_data].to_f / 1048576 * outprint[:total_data].to_f )
mb_free = ( outprint[:block_size_data].to_f / 1048576 * outprint[:free_data].to_f )
mb_used = ( mb_available.to_f - mb_free.to_f )
puts "MB total: #{mb_available}"
puts "MB used: #{mb_used}"
puts "MB free: #{mb_free}"
The ruby script takes a single argument, the path to the device that is going to be looked at. It can be passed with either a -p or –path.
Posted in Linux, Ruby, Sysadmin, Xen, Development | Tags disk, DomU, monitor, ruby, script, Xen
Posted by Tres
Sat, 19 Apr 2008 14:12:00 GMT
PHP’s implementation of Exceptions works just like you’d expect it to, except for the missing finally statement.
In Java, we could do something like this:
try {
//create a new widget
Widget widget = new Widget();
//here's where we lock the widget before we take action
widget.lock();
widget.doStuf();
} catch( TimeoutException e ) {
widget.fixTimeoutException();
} catch( ConnectionException e ) {
widget.resetConnection();
} catch( PermissionException e ) {
widget.exit();
} catch( FantasticException e ) {
widget.goBallistic();
} finally {
//make sure that we're unlocking the widget since we put a lock on it before
widget.unlock();
}
But when describing the same thing in PHP, we’ve got to do it like this because there’s no finally available:
try{
//create a new widget
$widget = new Widget();
//here's where we lock the widget before we take action
$widget->lock();
$widget->doStuff();
} catch( TimeoutException $e ) {
$widget->fixTimeoutException();
//make sure that we're unlocking the widget since we put a lock on it before
$widget->unlock();
} catch( ConnectionException $e ) {
$widget->resetConnection();
//make sure that we're unlocking the widget since we put a lock on it before
$widget->unlock();
} catch( PermissionException $e ) {
$widget->exit();
//make sure that we're unlocking the widget since we put a lock on it before
$widget->unlock();
} catch( FantasticException $e ) {
$widget->goBallistic();
//make sure that we're unlocking the widget since we put a lock on it before
$widget->unlock();
}
It’s great to have exceptions in PHP, don’t get me wrong, it just took me by surprise that the implementation wasn’t complete.
Posted in PHP, Java, Development | Tags exceptions, finally, missing, php
Posted by Tres
Sat, 19 Apr 2008 08:42:00 GMT
Keeping your classes well organized is an extremely important process once you reach the point where you’re dealing with hundreds of classes and thousands of lines of code. One of the ways that Java implicitly facilitates better organization is by forcing classes into their own files.
Unlike Java, where strict conventions make loading of called classes automatic, with PHP, you can put your classes anywhere, just as long as they’re in a file that the interpreter knows about via an include or require.
This kind of organization is just fine if you’re working on a smaller code base, but quickly breaks down when you’re dealing with a larger project; however,PHP does offer the built-in function __autoload() which automatically attempts to load any class or interface which hasn’t been defined yet. So you can take advantage of single-class files without needing to include (potentially) hundreds of files.
The only thing is, you’ve got to put together the _autoload yourself. Here’s the implementation of _autoload() that I’m currently using. It will load classes, interfaces and configuration files. It uses setincludepath() to define where it should be looking for files.
/**
*
*
*
* @copyright 2008
* @package ProjectX
* @subpackage SystemConf
* @author Tres Wong-Godfrey
*
*/
define( "LIBRARY", "/usr/local/emersius/projectx/library/" );
define( "COMMON_LIBRARY", "/usr/local/emersius/common/" );
define( "CONF_DIR", "/usr/local/emersius/conf/" );
/**
*
* Object autoloader for projectX
*
* @access public
* @param string The name of the class being loaded
*
*/
function __autoload($class)
{
//turn down the volume of error reporting while we're running the autoloader so we don't get warnings from the include_once() calls.
$old_error_reporting_level = error_reporting( 1 );
try {
switch( TRUE ) {
case include_once("class.${class}.php"):
return;
case include_once("intf.${class}.php"):
return;
case include_once( "conf.${class}.php" ):
return;
default:
throw new FileNotFoundException(__CLASS__.'::'.__FUNCTION__ . "Fault occured in ".__FUNCTION__. "can not find incldued library ${class}.\n");
}
} catch( FileNotFoundException $e ) {
die( $e->getMessage() . "\n" . $e->getTraceAsString() );
}
//turn the error reporting back up to what it was
error_reporting( $old_error_reporting_level );
}
/**
* Sets the path where we will look for files when they
* are requested.
*/
set_include_path(
get_include_path()
.PATH_SEPARATOR.LIBRARY
.PATH_SEPARATOR.CONF_DIR
.PATH_SEPARATOR.COMMON_LIBRARY
.PATH_SEPARATOR.COMMON_LIBRARY."/classes/"
.PATH_SEPARATOR.COMMON_LIBRARY."/classes/logging/"
.PATH_SEPARATOR.COMMON_LIBRARY."/classes/exceptions/"
.PATH_SEPARATOR.COMMON_LIBRARY."/classes/models/"
.PATH_SEPARATOR.COMMON_LIBRARY."/interfces/"
);
Posted in PHP, Java, Development | Tags autoload, classes, code, oop, organize, php
Posted by Tres
Sat, 12 Apr 2008 11:48:00 GMT
If you’re trying to monitor disk usage in a Xen domU and are using ext3fs formatted filesystems on LVM partitions, you can use dumpe2fs -h to get an idea of the current disk usage in domU from dom0.
[tres blas.phemo.us ~]$ sudo dumpe2fs -h /dev/vol00/xen_root_img
dumpe2fs 1.39 (29-May-2006)
Filesystem volume name:
Last mounted on:
Filesystem UUID: d18fab79-7123-4289-bd28-222ec8739874
Filesystem magic number: 0xEF53
Filesystem revision #: 1 (dynamic)
Filesystem features: has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file
Default mount options: (none)
Filesystem state: clean
Errors behavior: Continue
Filesystem OS type: Linux
Inode count: 311296
Block count: 622592
Reserved block count: 31129
Free blocks: 336965
Free inodes: 256927
First block: 0
Block size: 4096
Fragment size: 4096
Reserved GDT blocks: 151
Blocks per group: 32768
Fragments per group: 32768
Inodes per group: 16384
Inode blocks per group: 512
Filesystem created: Tue Feb 5 18:57:47 2008
Last mount time: Fri Feb 15 14:42:15 2008
Last write time: Fri Feb 15 14:42:15 2008
Mount count: 6
Maximum mount count: 25
Last checked: Tue Feb 5 19:01:06 2008
Check interval: 15552000 (6 months)
Next check after: Sun Aug 3 20:01:06 2008
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
Inode size: 128
Journal inode: 8
First orphan inode: 32775
Default directory hash: tea
Directory Hash Seed: d3f9829f-e127-427b-be56-4e840a139ccf
Journal backup: inode blocks
Journal size: 64M
So amongst all of the output, there are three lines that provide the magic: Block count:, Block size: and Free blocks:.
It’s easy enough to grab the three lines and then process them however you need to. This is a link to a ruby script that will check domU disk usage from dom0
Posted in Ruby, Sysadmin, Xen | Tags disk, DomU, monitor, usage
Posted by Tres
Fri, 11 Apr 2008 02:56:00 GMT
If you’re managing scripts in subversion, you’ll probably run into a file that was committed without the executable bit set when it was committed.
Changing the permissions in the repository is just a matter of using propset like this:
svn propset svn:executable ON /path/to/file
Posted in Development | Tags management, permissions, repository
Posted by Tres
Wed, 09 Apr 2008 20:50:00 GMT
Here’s a little bash shell foo to figure out how many lines of code are in a specific directory and all its subdirectories.
As written, it looks for java files and excludes blank lines as well as subversion management directories:
totalcount=0
for i in `find . -name "*.java" -print | grep -v ".svn"`
do newcount=`cat $i | grep -v "^$" | wc -l`
totalcount=$(($totalcount+$newcount))
done
echo "totalcount: $totalcount"
Posted in PHP, Ruby, Java, Development | Tags count, line, Source
Posted by Tres
Wed, 27 Feb 2008 16:31:00 GMT
Often times it’s nice to be able to walk through a connection to an IMAP or SMTP server to verify that everything is working correctly. Old hands know this is an easy task with telnet:
(tres blas.phemo.us)$ telnet ilovemymail.com 143
* OK IMAP4 ready
login accountname@myserver.com
Old hands also know that every time you connect this way, your password gets sent unencrypted over the wire. So instead of telnet, try the following:
(tres@blas.phemo.us)$ openssl s_client -connect ilovemymail.com:993
...
* OK IMAP4 ready
Posted in Sysadmin | Tags connect, imap, server, ssl, telnet, test
Posted by Tres
Thu, 14 Feb 2008 06:03:00 GMT
Every once in a while, I need to say my old mantra, “it’s always the little things.” The biggest of problems always seem to boil down to the littlest of problems in the world of *nix. Here’s another example.
If you’ve upgraded your version of Xen on CentOS/Red Hat Enterprise Linux from the included Xen 3.0.x to the latest available from XenSource (3.2 at this time), you may run into something like this in /var/log/xen/xend.log when trying to start xend:
INFO (SrvDaemon: ) Xend Daemon started
INFO (SrvDaemon: ) Xend changeset: unavailable.
INFO (SrvDaemon: ) Xend version: Unknown.
ERROR (SrvDaemon: ) Exception starting xend ((13, 'Permission denied'))
...
Error: (13, 'Permission denied')
and something like this in /var/log/xen/xend-debug.log
sysctl operation failed -- need to rebuild the user-space tool set?
Exception starting xend: (13, 'Permission denied')
The error in the debug log is misleading. I spent my time looking for duplicate installs of xen tools that may have been installed. My theory was that the Red Hat supplied Xen RPM installed files that the XenSource SRPM spec file puts in a different spot, or some other bookkeeping issue related to rpm -Uvh on the xen RPM files.
It turns out I was right, but was looking at the wrong source of the issue. The Xen tools were indeed mismatched, but it was to the running kernel. The xen RPM installed the tools and the kernel and did everything it was supposed to, but it didn’t update grub.conf to boot with the updated xen kernel files that were installed.
“It’s always the little things.” It’s often embarrassing, and this is no exception, but I thought I’d put this out there so if someone else gets caught looking at the wrong source of a mismatched user-space tool set error, they can avoid the trouble of looking for rogue tool installations and get on with business. Just update your /boot/grub/grub.conf with something like the following (<DANGER, WILL ROBINSON>:needless to say, copying and pasting grub.conf entries without verifying them can land you in a heap of trouble if you don’t have local access to your server</DANGER, WILL ROBINSON>):
title CentOS (xen-3.2)
root (hd0,0)
kernel /xen.gz ro root=/dev/vol00/root dom0_mem=256M
module /vmlinuz-2.6.18-53.1.13.el5xen ro root=/dev/vol00/root
module /initrd-2.6.18-53.1.13.el5xen.img
<DANGER>:This entry is very much dated, and uses the latest CentOS 5.x kernel as of this writing</DANGER>. The thing that won’t change, and will continue to be valid no matter what version of Xen you build and install later, is the kernel line:
kernel /xen.gz
The XenSource RPM & SRPM will build things so that /boot/xen.gz is a symlink to the latest version installed.
Posted in Linux, Red Hat Enterprise Linux, Sysadmin, Xen | Tags api, upgrade, Xen, xensource, 3.2
Posted by Tres
Sun, 23 Dec 2007 10:55:00 GMT
So you try and log in to a domU and xen says it can’t open a tty?
[tres@calliope ~]$ sudo xm console xen-domu.vm
xenconsole: Could not open tty `/dev/pts/15': No such file or directory
This error normally happens when xenconsoled is no longer running. You can quickly verify by looking for the xenconsoled process:
[tres@calliope ~]$ ps auxwww | grep xenconsoled
[tres@calliope ~]$
If you don’t find it, just run
/etc/init.d/xend start
Posted in Linux, Sysadmin, Xen | Tags DomU, tty, xenconsoled