What Goes round, Comes Round

EuroPython 2010

John Pinner

The Linux Emporium

What am I talking about ?

• How old ideas go out of fashion, and may return.


• In our case:
  • We use an ISAM filing system, not a RDMS
    • Not fashionable.
• For EuroPython, a few NoSQL talks were proposed
  • NoSQL
  • I thought our ISAM-based system would be interesting in the NoSQL context
  • All the other NoSQL talks were withdrawn!

NoSQL

• Non-relational database systems
  • Based on key-value pairs
    • Not NO SQL,
    • but Not Only SQL
• What is an ISAM filing system?
  • Invented by IBM
    • still the basis of DB2
  • key-value files
    • values - flat data file
    • keys - B-tree index to data file
    • very fast look up
  • Like a disk-based Python dictionary
  • Fits the current NoSQL philosophy.

Talk Contents

• How we came to use an ISAM system
  • Proprietory 4GL -> our own GPL'd system
• What our development system comprises


• Demo


• Example systems

How did we get there?

• Once Upon a Time...
• Experimented with RDMSs
  • Mistress
    • became Empress
  • Oracle
• Needed huge resources
• Found a commercial 4GL using ISAM files (Sculptor 4GL)
  • ran comfortably on the hardware of the day
    • NCR Tower
    • Early 386 PCs
  • staggeringly fast by comparison with RDMS systems
  • cross-platform

Sculptor 4GL

• Good in its day
• Fast, both runtime and development
• Portable - Unix, DOS/Windows, Linux
• Its own language and compilers
• Interpreted, running on its own virtual machine
• Proprietory

Sculptor 4GL

• Prohibitive licensing costs
• Authors failed to update it
  • GUI version not available until late 90s,
  • and never ported to Linux
• GUI language slightly 'whacky'
  • Poor tools, eg screen designer
• Now moribund

Where we were

• Masses of Sculptor code in customer systems
  • All running on Unix or Linux
• January 2001 - Sculptor decided not to port to Linux
  • We needed an upgrade path for our customers
• April 2001
  • First Python-based system demonstrated
  • Evolved since

Thyme - What's in a Name

• The Sculptor interpreter is called 'sage'
• We write business software
• The UK leading brand for business software is Sage


• Thyme is not sage.

Thyme Elements

• KFI - the Key File Library
  • Written in C
  • Interfaced to Sculptor files
    • Later extended to our own Thyme-ISAM files


• Thymebase - Python-KFI Interface Module


• Thyme - Python Class Library


• Thyme Development Kit
  • Thyme Custom Widgets
  • makethyme screen genrator
    • master.py
  • Thyme Std Stuff

KFI - the Key File Library

• Provides a C API to ISAM files
  • Currently Sculptor or Thyme ISAM's


• Implements the ISAM filing language in C
  • Programs may be written directly in C
    • Suitable for batch processing and reports
    • eg PayThyme calculation routine


• Main use - makes ISAM files available to Python
  • via thymebase


• Also documents the filing language

Thymebase - Python-KFI Interface

• Lowest level Python interface to ISAM's


• Can use ThymeBase directly
  • Relatively efficient
  • Not necessarily portable to different databases

Thyme - Python Class Library

• Thyme
  • High level Python interface to ISAM's
    • Screens
      • We use PyQt (Qt Designer)
    • Reports
  • Intended to be portable to other database types
  • Experimental implementations:
    • Python pickled dictionaries
    • PostgreSQL files
    • Web-enabled
      • sockets
      • AJAX
• Thyme Custom Widgets
  • Link PyQt screens to Thyme database fields
• Thyme Standard Stuff
  • Provides standard methods for makethyme screens

Thyme Development Kit

• Utilities
  • ISAM file maintenance
    • Thymecheck, etc
    • tedd - maintain file descriptors or 'tables'
• Thyme Custom Widgets
  • Qt Designer Interface
  • thymeuic processor
  • database fields linked to widgets,
    • using magic names and a pre-processor
• MakeThyme Program Generator
  • 'master' program
  • 'standard' mix-in methods
• PyMenu GUI Menu Program
• Tide Development Environment

Pros and Cons

• Cons
  • No SQL
  • Perceived as old-fashioned
    • not 'web-enabled'
• Pros
  • No SQL
    • no corruption by SQL script kiddies
  • Fast development of desktop apps
  • Quick and compatible upgrade path for legacy apps

Plans

• Move to PyQt4
• Port to Python 3
  • thymebase C module done
    • much more Pythonic
• Alternative data platforms

The End

Thanks for listening

• Questions ?