Trac on Solaris using Apache mod_python and https

If Trac is being used by a distributed team over the internet we want to remove all privileges from unauthenticated users:

for perm in BROWSER_VIEW CHANGESET_VIEW FILE_VIEW LOG_VIEW MILESTONE_VIEW 
REPORT_SQL_VIEW REPORT_VIEW ROADMAP_VIEW SEARCH_VIEW TICKET_CREATE TICKET_MODIFY TICKET_VIEW 
TIMELINE_VIEW WIKI_CREATE WIKI_MODIFY WIKI_VIEW
do
  trac-admin $tracenv permission remove anonymous $perm
  trac-admin $tracenv permission add authenticated $perm
done

We also want to encrypt traffic to the site. To do this I tried stunnel…

/opt/csw/bin/pkg-get -i stunnel

…and placed the following in /opt/csw/etc/stunnel/stunnel.conf

 [https]
accept  = 443
connect = 8000

I also commented out the chroot setup. Once configured all that is required is to run

cd /opt/csw/etc/stunnel 
/opt/csw/bin/stunnel

…and change /var/opt/csw/trac/conf/trac.ini

 [trac]
authz_file =
authz_module_name =
base_url = https://trac.mydomain.com

The bad news is that Trac 0.10.4 does not consistently use base_url, so creating a ticket, for example, redirects the user to an http page.

PATH=/opt/csw/bin:$PATH
tracenv=/var/opt/csw/trac
HTTPS=1; export HTTPS
nohup tracd --port 8000 $tracenv &

To resolve this issue I decided to move from tracd/stunnel to Apache2/mod_python. The default Solaris 10 distribution includes apache2 but not mod_python. Instead I installed mod_python from Blastwave, which in turn automatically installs the Blastwave cswapache2 package below /opt/csw/apache2.

pkg-get install ap2_modpython

We will want to run trac under apache2 using a dedicated account:

groupadd -g 202 trac
useradd -g trac -u 202 -d /var/opt/csw/trac trac
chown -R trac:trac /var/opt/csw/trac

Modified /opt/csw/apache2/etc/httpd.conf

User trac
Group trac
…
RewriteEngine On
RewriteCond %{HTTPS} off
RewriteRule (.*) https://%{HTTP_HOST}%{REQUEST_URI}
…

   SetHandler mod_python
   PythonInterpreter main_interpreter
   PythonHandler trac.web.modpython_frontend
   PythonOption TracEnv /var/opt/csw/trac

Created a self-signed certificate for the site:

cd /opt/csw/apache2/etc
PATH=$PATH:/usr/sfw/bin
/usr/sfw/bin/openssl genrsa -out server.key 2048
/usr/sfw/bin/openssl req -new -x509 -key server.key -out server.crt -days 365 -subj "/C=US/ST=Florida/O=My Company/CN=trac.mydomain.com"

Modified /opt/csw/apache2/etc/extra/httpd-ssl.conf

ServerName trac.mydomain.com
…

   SetHandler mod_python
   PythonInterpreter main_interpreter
   PythonHandler trac.web.modpython_frontend
   PythonOption TracEnv /var/opt/csw/trac

To start Blastwave Apache2 using SMF on Solaris:

svccfg -s cswapache2 setprop httpd/ssl=true
svccfg -s cswapache2 listprop

svcadm enable cswapache2

To check status

svcs cswapache2
svcs –xv

Trac on Solaris

Trac is a web-based software project management and bug/issue tracking system. Example publicly accessible sites that use Trac include

Getting Trac installed on Solaris 10 is easy, see http://trac.edgewall.org/wiki/TracOnSolaris Question is, what next?

Blastwave packages are installed below /opt/csw (csw = Community SoftWare). /opt/csw/share/doc/trac/INSTALL provides the next steps. I used:

PATH=/opt/csw/bin:$PATH
MANPATH=/opt/csw/share/man:$MANPATH
tracenv=/var/opt/csw/trac
trac-admin $tracenv initenv
tracd --port 8000 $tracenv & # no authentication
firefox http://localhost:8000/trac

For a small number of users the tracd standalone server is good enough. Authentication can be managed with htdigest on Solaris 10 thusly:

/usr/apache2/bin/htdigest -c $tracenv/conf/users.htdigest
mydomain.com fred

Tracd can then be started like this:

PATH=/opt/csw/bin:$PATH
tracenv=/var/opt/csw/trac
nohup tracd --port 8000 --auth *,$tracenv/conf/users.htdigest,mydomain.com $tracenv &

To change the logo upload the new logo to /opt/csw/share/trac/htdocs and modify trac.ini

[header_logo]
…
link = https://trac.mydomain.com/
src = common/mylogo.png
…
[project]
descr = My Trac
footer = Visit the Trac open source project at 
http://trac.edgewall.org/
icon = common/trac.ico
name = My Project
url = https://myproject.mydomain.com/

Trac is much easier to administer with the WebAdmin plugin, which for Trac 0.10.4 requires downloading and installing from source:

easy_install http://svn.edgewall.com/repos/trac/sandbox/webadmin/

To get started a Trac administrator has to be empowered:

trac-admin /var/opt/csw/trac permission add fred TRAC_ADMIN

To simplify account administration we’ll also try an account manger plugin

/opt/csw/bin/easy_install http://trac-hacks.org/svn/accountmanagerplugin/0.10

…and change /var/opt/csw/trac/conf/trac.ini

[components]
webadmin.* = enabled
trac.web.auth.LoginModule = disabled
acct_mgr.api = enabled
acct_mgr.htfile.HtDigestStore = enabled
acct_mgr.web_ui.AccountModule = enabled
acct_mgr.web_ui.LoginModule = enabled
acct_mgr.web_ui.RegistrationModule = disabled
acct_mgr.admin.AccountManagerAdminPage = enabled
[account-manager]
password_format = htdigest
password_store = HtDigestStore
password_file = /var/opt/csw/trac/conf/users.htdigest
htdigest_realm = mydomain.com

Tracd can then be started like this:

PATH=/opt/csw/bin:$PATH
tracenv=/var/opt/csw/trac
nohup tracd --port 8000 $tracenv &

To manage custom fields we’ll add another plugin

easy_install http://trac-hacks.org/svn/customfieldadminplugin/0.10

…and change /var/opt/csw/trac/conf/trac.ini

[components]
customfieldadmin.* = enabled

Installing JBoss with Oracle XE on Windows

Both Tomcat and Oracle XML DB use port 8080 by default. Before installing JBoss I first changed the default port for XML DB as suggested here.

call dbms_xdb.cfg_update(updateXML(
            dbms_xdb.cfg_get()
          , '/xdbconfig/sysconfig/protocolconfig/httpconfig/http-port/text()'
          , 8081))
      /

I restarted the Oracle database and verified port 8081 was in use and not 8080.

Hiring advice from Dee Hock

“Hire and promote first on the basis of integrity, second motivation, third capacity, fourth understanding, fifth knowledge, and last and least, experience. Without integrity, motivation is dangerous; without motivation, capacity is impotent; without capacity, understanding is limited; without understanding, knowledge is meaningless; without knowledge, experience is blind. Experience is easy to provide and quickly put to good use by people with all the other qualities.” Dee Hock, founder, VISA

Fast Company Article: The Trillion-Dollar Vision of Dee Hock

Reporting Tools

Having built and populated a reporting database I’ve been looking at reporting tools for the front end. In the past I’ve used Crystal Reports, Oracle Reports and MicroStrategy, but with the BI industry in flux over the past couple of years as a result of acquisitions and new open source businesses, I thought now is a good time to look at alternatives like BIRT, JasperSoft and Pentaho. My requirements are:

  • easy and powerful report designer and engine with relevant examples
  • secure report repository with end user web access including external authentication
  • scheduled report execution and delivery by email
  • value for money given tech savvy in-house support

Starting with report designers I looked at BIRT RCP Report Designer 2.3 vs. JasperSoft iReport 3.0.  BIRT has a field guide included in the online help, report layouts may be fixed or automatic, and the BIRT report engine was described as stronger than JasperReports on a ServerSide forum.  iReport struck me as undistinguished, and the user manuals need to be purchased as they are not available for download, which was irksome to perform a review.  It does, however, provide a ‘pixel perfect’ layout and there were demo samples in jasperreports-3.0.0-project.zip.  The first report I built failed to run, and I could not find any explanation.

Looking at report repositories I compared BIRT Report Server vs. JasperServer 2.1, Pentaho Reporting and OpenReports.  BIRT Report Server was previously Actuate iServer Express, and is still a commercial product costing $1,995/core.  There is a BIRT Deployment kit with repository (but no scheduler) available at $995/core.  There is a 90 day evaluation available and manuals are available online.  JasperServer 2.1 installation was bundled with Apache Tomcat 5.5, MySQL 5.0, Java JDK 1.5 and iReport 2.0.2.  Novell SLES 10.2 is documented as a supported platform.  I found the Pentaho Platform documentation to be jumbled and immature, with different dates more like an online support knowledgebase.  The Pentaho wiki, however, does include articles on integrating BIRT and JasperReports.  Also Pentaho is certified on Novell SLES 10 and has also been installed on Ubuntu.  Finally OpenReports supports multiple report engines, including BIRT, JasperReports and Pentaho.  External integration is supported using Spring Framework, but not well documented.

My conclusions from this research is that these products are still maturing, and I would not want to mix and match components from different products.  I was not persuaded to pay for a commercial BIRT Report Server licence, even if the price is relatively modest compared to competing products a few years ago.  That leaves Pentaho Reporting and JasperServer as the best alternative repositories, and they rate about equal.  In my case I’m focusing on JasperServer as being easier to integrate seamlessly with our own software.

Populating an Oracle tablespace

Suppose that we want to move all the mid-size indexes for a given schema FOO, into their own tablespace. First, let’s list them all:

  SELECT   ext.owner, index_name, SUM (bytes) / (1024 * 1024) MB
    FROM      dba_extents ext
           JOIN
              dba_indexes idx
           ON idx.owner = ext.owner AND idx.index_name = ext.segment_name
   WHERE       ext.owner = 'FOO'
           AND segment_type = 'INDEX'
           AND index_type = 'NORMAL'
GROUP BY   ext.owner, index_name
  HAVING   SUM (bytes) / (1024 * 1024) > 8
ORDER BY   MB DESC

How large does the tablespace need to be?

  SELECT   1.25 * SUM (bytes) / (1024 * 1024) MB
    FROM      dba_extents ext
           JOIN
              dba_indexes idx
           ON idx.owner = ext.owner AND idx.index_name = ext.segment_name
   WHERE       ext.owner = 'FOO'
           AND segment_type = 'INDEX'
           AND index_type = 'NORMAL'
ORDER BY   MB DESC

To populate the tablespace we could use:

  SELECT   'alter index '||ext.owner||'.'||segment_name||' rebuild tablespace '||:new_ts||';'
    FROM      dba_extents ext
           JOIN
              dba_indexes idx
           ON idx.owner = ext.owner AND idx.index_name = ext.segment_name
   WHERE       ext.owner = 'FOO'
           AND segment_type = 'INDEX'
           AND index_type = 'NORMAL'
GROUP BY   ext.owner, segment_name, segment_type
  HAVING   SUM (bytes) / (1024 * 1024) > 8
ORDER BY   SUM (bytes) / (1024 * 1024) DESC

Now suppose we want to move all tables (including index organized) for a given schema FOO into their own tablespace. To list them:

  SELECT   owner, table_name, SUM (bytes) / (1024 * 1024) MB
    FROM   (SELECT   ext.owner, table_name, bytes
              FROM      dba_extents ext
                     JOIN
                        dba_tables tbl
                     ON tbl.owner = ext.owner AND table_name = ext.segment_name
             WHERE   ext.owner = 'FOO'
            UNION ALL
            SELECT   ext.owner, table_name, bytes
              FROM      dba_extents ext
                     JOIN
                        dba_indexes idx
                     ON idx.owner = ext.owner
                        AND idx.index_name = ext.segment_name
             WHERE       ext.owner = 'FOO'
                     AND segment_type = 'INDEX'
                     AND index_type = 'IOT - TOP')
GROUP BY   owner, table_name, bytes
  HAVING   SUM (bytes) / (1024 * 1024) > 8
ORDER BY   MB DESC;

How large does the tablespace need to be?

  SELECT   1.25 * SUM (bytes) / (1024 * 1024) MB
    FROM   (SELECT   ext.owner, table_name, bytes
              FROM      dba_extents ext
                     JOIN
                        dba_tables tbl
                     ON tbl.owner = ext.owner AND table_name = ext.segment_name
             WHERE   ext.owner = 'FOO'
            UNION ALL
            SELECT   ext.owner, table_name, bytes
              FROM      dba_extents ext
                     JOIN
                        dba_indexes idx
                     ON idx.owner = ext.owner
                        AND idx.index_name = ext.segment_name
             WHERE       ext.owner = 'FOO'
                     AND segment_type = 'INDEX'
                     AND index_type = 'IOT - TOP')

To populate the tablespace we could use:

SELECT      'alter table '|| owner|| '.'|| table_name
         || ' move tablespace '|| :new_ts|| ';'
  FROM   (SELECT   owner, table_name
            FROM   dba_tables tbl
           WHERE   owner = 'FOO'
          UNION ALL
          SELECT   owner, table_name
            FROM   dba_indexes idx
           WHERE   owner = 'FOO' AND index_type = 'IOT - TOP')

SELECT   'alter index ' || owner || '.' || index_name || ' rebuild;'
  FROM   dba_indexes
 WHERE   owner = 'FOO' AND status = 'UNUSABLE'

VMware Timekeeping


A hopelessly inaccurate timeclock on my SLES host has been driving me nuts. I eventually found the relevant VMware white paper and used Virtual Center to set the ‘Advanced’ option to ‘Synchronize guest time with host’:

I also modified /boot/grub/menu.lst to add ‘clock=pit’ as an argument to the kernel invocation. Both changes required a reboot of the VM, but at least the issue is resolved now. Why on earth isn’t this option set by default?

CruiseControl and Subversion

Getting started with CruiseControl is relatively straightforward, see http://confluence.public.thoughtworks.org/display/CC/Getting+Started+With+CruiseControl

To place an application on CruiseControl it should have an Ant script to build it. The application Ant script does not need to interact with Subversion. The source for the application including the Ant script should be checked into Subversion.

I started to install CruiseControl by downloading and compiling the source, but my version of ant was too old. Instead I downloaded the binary (which ironically includes a newer ant also):

su -
wget http://downloads.sourceforge.net/cruisecontrol/cruisecontrol-bin-2.7.2.zip
unzip -d /opt cruisecontrol-bin-2.7.2.zip

useradd --system --home /srv/cruisecontrol --create-home --gecos "System account to run CruiseControl" cruise
usermod -s /bin/ksh cruise
su – cruise
mkdir checkout logs artifacts
PATH=$PATH:/opt/subversion-1.3.2/bin/:/opt/cruisecontrol-bin-2.7.2
cruisecontrol.sh

Now we need to configure CruiseControl to monitor the Subversion repository. Create config.xml:


  
  
  
  
 
  
    
      
    
 
    
    
 
    
    
      
    
 
    
    
      
    
 
    
    
 
    
    
    
  

Build-myproj.xml contains Ant steps to checkout a clean copy of the application and build it:

To setup the dashboard, running at :8080

su - cruise cp /opt/cruisecontrol-bin-2.7.2/dashboard-config.xml . kill `cat cc.pid` cruisecontrol.sh

Posts navigation

1 2 3 8 9 10 11 12