pgLoader

One of the weaknesses of PostgreSQL compared to Oracle is the lack of a strong bulk loader utility like sqlldr. Natively Postgres offers only the COPY command, which does not include suppport for a bad file and reject log. EnterpriseDB have acknowledged the need for a better tool but it has not been a priority. There is, however, a Python based utility called pgloader available through pgFoundry that wraps COPY and provides a usable bulk loader. Using Python with Postgres requires psycopg2: for Windows an installable package is downloadable from Stickpeople,  for Ubuntu installation can performed with:

apt-get install tcllib libpgtcl-dev python-psycopg2

pgLoader itself is simply a collection of scripts in some folders so download and untar as follows:

wget http://pgfoundry.org/frs/download.php/2294/pgloader-2.3.2.tar.gz
tar zxvf pgloader-*.tar.gz

Ruby, Python & ODBC

I recently tried extracting data from an Access database using Ruby and ODBC.  The first challenge is that Windows ODBC drivers can be either 32 or 64-bit.  On a 64-bit host only the 64-bit configuration panel is present in the control panel, and the 32-bit panel must be accessed directly by invoking C:WindowsSysWOW64odbcad32.exe.  More seriously, however, the ruby-odbc home page only lists Win32 binaries compiled using Microsoft Visual C, but the popular RubyInstaller for Windows distribution is compiled with MinGW making the two effectively incompatible.

Contrast this with Python which has pyodbc downloads compatible with the canonical distribution from python.org, even if trailing the latest release.  For enterprise applications there is even a commercial ODBC driver supported by egenix.

Both Ruby and Python also have DBI interfaces, but Python DBI has been deprecated, and for Ruby the DBD-ODBC implementation relies on the same underlying ruby-odbc package.  For now it looks like I’ll be sticking with Python for ODBC.

OpenSolaris is Dead

Looks like concerns about Oracle’s Sun acquisition are being fulfilled.  After suing Google using a loophole in the Java license and losing James Gosling Oracle has now cut back significantly on the openness of Solaris development.  Whether or not this was a good business decision this is surely damaging to Oracle’s reputation amongst supporters of FOSS, and great news for Microsoft.  Like MySQL and MariaDB there is already now a community effort sponsored by Nexenta to fork OpenSolaris outside Oracle.

Top IT Business in Southern Florida

Business Leader Media publishes an annual ‘top 100’ small business roster for southern Florida. Tech companies on this list include

  • Datacorp.  A services partner with strong relationships with Microsoft, Cisco and Citrix.
  • United Data Technologies.  A services business with an established customer base in education and government.
  • Bayshore Solutions.  A web design company based in Tampa.
  • AAJ Technologies.  A Microsoft Gold Certified Partner in Fort Lauderdale with an established customer base in healthcare and government.
  • Palindrome Consulting.  A Microsoft Gold Certified development partner based in north Miami.  Also partners with Cisco, Citrix, VMware, HP and Apple.
  • Retail Technology Experts.  A Retail Pro certified partner.
  • Nu Info Systems Inc.  A complete software solutions provider based in Jacksonville, previously in the Inc. 500.
  • QuinnCom A web development company in North Palm Beach.
  • Dedicated IT.  Another Gold Certified Microsoft Partner in West Palm Beach providing a helpdesk and support.
  • International Consulting Group.  Partnerships include IBM, Cisco, Shoretel, Microsoft, Dell and VMware.
  • Kuster Computers.  Provides hosted Microsoft Exchange and desktop installation services.

DD-WRT

DD-WRT is a popular open source router firmware package originally developed for the Belkin WRT54G.  The package caught my attention as it allows a cheap old router to be reused as a client wireless bridge.  The download page has an excellent utility for identifying which routers are compatible.  To flash my router I needed to use TFTP. First I established an Ethernet connection with a static, not dynamic IP address, then I downloaded the Linksys GUI tftp client. There are some comments on the DD-WRT wiki about Windows 7 using CTCP, but I did not have this issue. Instead the challenge with a Belkin router is to initiate the file transfer immediately after the router starts responding after a power cycle.  The other issue I discovered is that not all bin files listed as compatible actually are: some bin files successfully upload but then are ignored.  My problem, however, was resolved by some detailed notes on the wiki which recommended dd-wrt.v24_generic-8184.bin for my hardware.

If the upload is successful the router responds for a few seconds while the upload is flashed, and then the connection is lost as the router restarts and DD-WRT defaults to 192.168.1.1 instead of 192.168.2.1. Reconfiguring the static IP address and browsing to the new default address revealed DD-WRT successfully loaded on my device.

ping-ddrt linksys-tftp

Resizing an LVM swap partition

I tested this on a lightly loaded Ubuntu 8.04 server.

free
swapoff /dev/vg01/lv_swap
lvextend -L+500M /dev/vg01/lv_swap
grep swap /etc/fstab # find the UUID for the existing swap space
# remake the larger swap using the same UUID as before
mkswap -U fc7f369e-c655-4a35-a227-a7c725a2b9bc  /dev/vg01/lv_swap
swapon /dev/vg01/lv_swap
free

linux-iscsi with NetApp

SLES includes an iSCSI initiator, linux-iscsi.  The initiator is configured using a daemon that is managed just like any other service, with a script in /etc/init.d/ and configuration files in /etc. After installing linux-iscsi using YaST I edited /etc/iscsi.conf to set the DiscoveryAddress and ran ‘/etc/init.d/iscsi start’ to start the service (‘chkconfig iscsi on’ ensures it is restarted after a reboot also). Logging onto my NetApp filer I could see the SLES initiator also logged in (‘igroup show’), and was able to create and map a LUN to the host. Back on the host ‘iscsi-ls’ yielded nothing useful, so I downloaded and installed the linux host utilities from NetApp’s support site.  ‘sanlun show all’ now shows the lun-pathname and local SCSI devicename (eg. /dev/sdx) to which it is mapped. Oracle’s Orion benchmark  indicates a random I/O throughput dramatically better for the iSCSI LUN compared to using a VMware virtual disk.

sqlunit & dbUnit

Now that I have my database schema creation and maintenance automated with Incanto and dbMaintain I have been looking at testing tools for stored procedures. Perhaps the best known for Oracle PL/SQL are Feuerstein’s utPLSQL and subsequent Quest Code Tester for Oracle. Other options include plunit, PLUTO and ruby-pl-spec. My main requirement, however, has been to invoke tests from an Ant project continuously integrated using Hudson and for now I have started using sqlunit, which is both mature and portable across different databases, as well as allowing relatively simple testcases to be specified in entirely XML.

My build-common.xml now contains












classname="net.sourceforge.sqlunit.ant.SqlunitTask"
classpathref="sqlunit-lib"/>

In addition to sqlunit I’ve found
dbUnit to be a useful tool for complex data setup prior to running a test.
Initilization of the dbUnit ant task is done as follows:












classname="org.dbunit.ant.DbUnitTask"
classpathref="dbunit-lib"/>

Posts navigation

1 2 3 4 5 6 7 8 9 10 11 12 13
Scroll to top