Varyonic

Posts Tagged ‘SCM’

Since reading about database refactoring some while ago I’ve been seeking to adopt a tool to support automated tracking and application of database changes.  The best known solution for this is LiquiBase (fka Sundog Refactoring Tool), with over 10,000 downloads of LiquiBase Core recorded on SourceForge as I write this.  In order to adopt LiquiBase on an existing project, however, I was faced with replacing dozens of SQL DDL scripts with LiquiBase specific XML files.  There is a generateChangeLog command, but it has limitations: for example it does not recognize the primary key of Oracle index organized tables, and the output is not arranged anything like the documented best practices.

Instead, for now, I have adopted a newer tool called dbMaintain, which was presented at a Belgian Java User Group last April (see below) and a ServerSide news article last July.  dbMaintain provides similar functionality to LiquiBase, but instead of XML it simply passes through native SQL scripts using JDBC, making it much easier to adopt with confidence on an existing project.  I did run into one known issue with setting the default schema, but I’m happy to say Tim Ducheyne responded to my emails and provided a fixed snapshot within days.  I hope this project gains momentum, and I’m looking forward to the next release.

PS. Other less mature alternatives include SQL based Thoughtworks dbdeploy and Carbon Five’s c5-db-migration.  Another custom XML based solution is MIGRATEdb.  Sony Pictures Imageworks have ported ActiveRecord migrations to Scala and there is a relatively inactive port of ActiveRecord migrations to Groovy called Bering.

Eighteen months have passed since I setup Subversion on Ubuntu and I’ve found multiple repositories are valuable for providing finer grained access control and easier storage management. Here is a revised incantation:

su -
adduser --system --home /srv/svnrepos --gecos "System account to run svnserve" svn
apt-get install subversion xinetd
cat >> /etc/xinetd.d/svn << "EOF"
service svn
{
        port                    = 3690
        socket_type             = stream
        protocol                = tcp
        wait                    = no
        user                    = svn
        server                  = /usr/bin/svnserve
        server_args             = -i -r /srv/svnrepos
}
EOF
/etc/init.d/xinetd restart

# for each repository, eg. myrepo1
$repo=myrepo1
svnadmin create /srv/svnrepos/$repo
chown -R svn:nogroup /srv/svnrepos

# uncomment line to use default password file
vi ~svn/$repo/conf/svnserve.conf

cat >> ~svn/$repo/conf/passwd << "EOF"
[users]
fred = *****
...

Trac is a web-based software project management and bug/issue tracking system. Example publicly accessible sites that use Trac include

• Spring IDE http://springide.org/project
• Trac developers themselves http://trac.edgewall.org/
• One Laptop Per Child http://dev.laptop.org/
• Berkeley Open Infrastructure for Network Computing (BOINC) http://boinc.berkeley.edu/trac used by SETI at Home
• Twisted Matrix http://twistedmatrix.com/trac/report

Getting Trac installed on Solaris 10 is easy, see http://trac.edgewall.org/wiki/TracOnSolaris Question is, what next?

Blastwave packages are installed below /opt/csw (csw = Community SoftWare). /opt/csw/share/doc/trac/INSTALL provides the next steps. I used:

PATH=/opt/csw/bin:$PATH
MANPATH=/opt/csw/share/man:$MANPATH
tracenv=/var/opt/csw/trac
trac-admin $tracenv initenv
tracd --port 8000 $tracenv & # no authentication
firefox http://localhost:8000/trac

For a small number of users the tracd standalone server is good enough. Authentication can be managed with htdigest on Solaris 10 thusly:

/usr/apache2/bin/htdigest -c $tracenv/conf/users.htdigest
mydomain.com fred

Tracd can then be started like this:

PATH=/opt/csw/bin:$PATH
tracenv=/var/opt/csw/trac
nohup tracd --port 8000 --auth *,$tracenv/conf/users.htdigest,mydomain.com $tracenv &

To change the logo upload the new logo to /opt/csw/share/trac/htdocs and modify trac.ini

[header_logo]
…
link = https://trac.mydomain.com/
src = common/mylogo.png
…
[project]
descr = My Trac
footer = Visit the Trac open source project at 
http://trac.edgewall.org/
icon = common/trac.ico
name = My Project
url = https://myproject.mydomain.com/

Trac is much easier to administer with the WebAdmin plugin, which for Trac 0.10.4 requires downloading and installing from source:

easy_install http://svn.edgewall.com/repos/trac/sandbox/webadmin/

To get started a Trac administrator has to be empowered:

trac-admin /var/opt/csw/trac permission add fred TRAC_ADMIN

To simplify account administration we’ll also try an account manger plugin

/opt/csw/bin/easy_install http://trac-hacks.org/svn/accountmanagerplugin/0.10

…and change /var/opt/csw/trac/conf/trac.ini

[components]
webadmin.* = enabled
trac.web.auth.LoginModule = disabled
acct_mgr.api = enabled
acct_mgr.htfile.HtDigestStore = enabled
acct_mgr.web_ui.AccountModule = enabled
acct_mgr.web_ui.LoginModule = enabled
acct_mgr.web_ui.RegistrationModule = disabled
acct_mgr.admin.AccountManagerAdminPage = enabled
[account-manager]
password_format = htdigest
password_store = HtDigestStore
password_file = /var/opt/csw/trac/conf/users.htdigest
htdigest_realm = mydomain.com

Tracd can then be started like this:

PATH=/opt/csw/bin:$PATH
tracenv=/var/opt/csw/trac
nohup tracd --port 8000 $tracenv &

To manage custom fields we’ll add another plugin

easy_install http://trac-hacks.org/svn/customfieldadminplugin/0.10

…and change /var/opt/csw/trac/conf/trac.ini

[components]
customfieldadmin.* = enabled

Getting started with CruiseControl is relatively straightforward, see http://confluence.public.thoughtworks.org/display/CC/Getting+Started+With+CruiseControl

To place an application on CruiseControl it should have an Ant script to build it. The application Ant script does not need to interact with Subversion. The source for the application including the Ant script should be checked into Subversion.

I started to install CruiseControl by downloading and compiling the source, but my version of ant was too old. Instead I downloaded the binary (which ironically includes a newer ant also):

su -
wget http://downloads.sourceforge.net/cruisecontrol/cruisecontrol-bin-2.7.2.zip
unzip -d /opt cruisecontrol-bin-2.7.2.zip

useradd --system --home /srv/cruisecontrol --create-home --gecos "System account to run CruiseControl" cruise
usermod -s /bin/ksh cruise
su – cruise
mkdir checkout logs artifacts
PATH=$PATH:/opt/subversion-1.3.2/bin/:/opt/cruisecontrol-bin-2.7.2
cruisecontrol.sh

Now we need to configure CruiseControl to monitor the Subversion repository. Create config.xml:

Build-myproj.xml contains Ant steps to checkout a clean copy of the application and build it:

To setup the dashboard, running at :8080

su - cruise
cp /opt/cruisecontrol-bin-2.7.2/dashboard-config.xml .
kill `cat cc.pid`
cruisecontrol.sh

There are many ways to do it, but this incantation worked for me on an Ubuntu host. This is an ‘entry level’ setup with one repository and simple authentication. Read the Subversion book and Ubuntu documentation to understand.

su -
apt-get install subversion
adduser --system --home /srv/svn --gecos "System account to run svnserve" svn
svnadmin create /srv/svn
chown -R svn:nogroup /srv/svn

apt-get install xinetd
cat >> /etc/xinetd.d/svn << "EOF"
service svn
{
        port                    = 3690
        socket_type             = stream
        protocol                = tcp
        wait                    = no
        user                    = svn
        server                  = /usr/bin/svnserve
        server_args             = -i -r /srv/svn
}
EOF
/etc/init.d/xinetd restart

# uncomment line to use default password file (~svn/conf/passwd)
vi ~svn/conf/svnserve.conf

cat >> ~svn/conf/passwd << "EOF"
[users]
fred = *****
...
EOF
chmod 600 ~svn/conf/passwd