tag:blogger.com,1999:blog-154250532012-01-07T19:24:30.478ZSimpleDBM is an Open Source Embeddable Database Engine written entirely in Java.Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.comBlogger611100tag:blogger.com,1999:blog-15425053.post-52910117752695210472011-01-30T12:44:00.000Z2011-01-30T12:44:51.954Z

After a break, I have resumed work on creating SimpleDBM V2. This version is mostly a refactoring of the existing codebase to ensure: Better project structure - break up the project into smaller modules. Simple IOC container for auto wiring the modules. TypeSystem now integral part of the core, hence some modules can take advantage of the Row structure; in the current version, the TypeSystem is

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-27555735347781645062010-05-02T12:31:00.000Z2010-05-02T12:31:25.846Z

All of the SimpleDBM modules are designed with constructor based dependency injection in mind. But so far, these dependencies have been manually coded. I want to move away from manual setting up of dependencies and was therefore looking for a small IOC container that would serve my needs. Unfortunately, all the available dependency injection frameworks appear to be bloated and huge; PicoContainer

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com2tag:blogger.com,1999:blog-15425053.post-8373171509294414552010-04-25T19:42:00.001Z2010-04-25T19:42:51.245Z

Given below is the boilerplate license notice that will be add to SimpleDBM source files from version 2 onwards. This is based upon the boilerplate used by Mozilla.org. Note that I decided to add LGPL to the mix as well, so that SimpleDBM V2 will be triple licensed. Hopefully that will ensure compatibility with the vast majority of Open Source licenses. /** * DO NOT ALTER OR REMOVE COPYRIGHT

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-55364958776579144222010-04-19T22:10:00.002Z2010-04-19T22:10:39.857Z

The next version of SimpleDBM will be available under the GPLv2 as now, as well as the Apache License. Dual licensing will allow people to use SimpleDBM in more flexible ways.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-61798535354086434502010-04-18T23:52:00.001Z2010-04-18T23:52:32.013Z

I am pleased to finally publish 1.0.18-ALPHA release of SimpleDBM. This release has following changes: A network client server implementation that allows SimpleDBM to run as a standalone database server to which clients can connect remotely. A sample application that demonstrates the use of the network API. The sample implements a simple discussion forum; front end has been created using Google

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-75552050615047347492010-04-18T01:32:00.000Z2010-04-18T01:32:50.012Z

In a previous post I wrote about why I preferred GPL license for SimpleDBM. But I am no longer sure; my intention was always to ensure that SimpleDBM can be used by anyone without worrying about licensing issues, and I have no desire to put restrictions on other people's work. So if someone enhanced SimpleDBM, they should be free to do whatever they like with their enhancement, and although it

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-76059556901336058892010-04-10T22:43:00.001Z2010-04-10T22:46:39.485Z

I have been thinking about how SimpleDBM should evolve. When I started the project my intention was to eventually add support for SQL, but now I can't see this happening in the near term. SimpleDBM is not aimed at competing with other SQL databases; SQL is nice because of the ease with which tables can be queries, joined etc., but implementing an SQL layer is quite a lot of work, which I am not

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-31264199810398701272010-04-06T23:32:00.000Z2010-04-06T23:32:21.536Z

It is taking longer than I anticipated to create a sample application. The main hurdle has been mastering Google Web Toolkit enough to create the user interface. I am hacking from the sample mail application available in the GWT distribution; but the code is increasingly becoming very different. First, here is a screen shot from the web UI. Apologies for the rough edges; I am not a UI developer,

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-79870470774309059022010-03-21T23:57:00.000Z2010-03-21T23:57:32.605Z

I am still testing the new network client server implementation. Apologies for the delay in publishing this enhancement. I am also working on a simple sample web application to illustrate the network API. Learning all about GWT (Google Web Toolkit) which is a lot of fun.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-38484005861732289662009-12-20T11:25:00.000Z2009-12-20T11:25:50.263Z

I am happy to report that an initial implementation of network client server has been checked in. This will allow the SimpleDBM clients to use a thin client which will connect to a remote SimpleDBM server. The client interface is a simplified version of the Database API. The network protocol is based on a simple request/reply model. The messages are passed in the SimpleDBM serialization format.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-44914641015671868982009-10-11T18:09:00.000Z2009-10-11T18:09:02.815Z

I described the implementation of the data dictionary in a previous post. I have updated the implementation to include support for dropping tables. I stated in my last post that the creation of the table definitions would be handled as post commit actions. In the end this wasn't necessary, and the table creation is logged against the page 0 of the virtual container 0. This container is

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-33499296210137786302009-07-04T16:29:00.002Z2009-07-04T16:34:00.579Z

SimpleDBM source code repository has been migrated from Subversion to Mercurial. For a somewhat rude and OTT presentation on the benefits of distributed version control systems over traditional centralized systems see the Linus Torvald's presentation on GIT.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-51614510703415809512009-04-12T15:53:00.003Z2009-04-12T16:06:14.841Z

The SimpleDBM core database engine has been out for a while, and although I am still working on improving the robustness of the code, writing more test cases, etc., it is also time to start thinking about what next.Couple of areas interest me.A simple SQL layer so that it becomes easier to write code against the database. Unlike other projects, I am not interested in a full-blown SQL

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1852219927918505322009-03-31T22:28:00.007Z2009-04-12T15:51:18.238Z

As I mentioned in a previous post, when building SimpleDBM I have used design ideas that I have picked from various technical research papers as well as other OpenSource projects. I would like to list some of the influences here, and acknowledge my indebtedness.Papers from IBM's System-R research project have influenced the role of the database engine in SimpleDBM. I have named the core engine as

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-13758060024850261032009-03-31T22:18:00.002Z2009-03-31T22:27:32.687Z

The next BETA release of SimpleDBM has been delayed due to my desire to re-factor some of the code. The re-factoring has been focused on following:Removing all singletons - even Loggers. Unfortunately, Log4J and other logging packages all use singletons, so this is not completely possible unless I replace the entire logging package.Ensuring that the serialization mechanism uses constructor based

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-86990429324277896272008-08-23T18:55:00.008Z2008-09-18T21:49:49.808Z

The SimpleDBM project started as the DM1 project in August 2001. Initially, I started writing the database code in C. A year later, I ported the code to C++. Here is what I wrote at the time about my switch to C++:I decided to port the product to C++ primarily for following reasons:I wanted to use the automatic construction and destruction facility in C++. I think this is one of the coolest

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-73877572824642556402008-07-09T22:15:00.001Z2008-07-09T22:17:05.437Z

At long last, I have put together the 1.0.8 BETA release. This contains the experimental TypeSystem and Database API module.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-6365866840385299482008-07-03T08:33:00.003Z2008-07-03T09:03:27.644Z

I have chosen to implement option 3 as described in a previous post. For each table, a special container is created, which is used to store not only the table structure but also structures of all associated indexes. Special containers are created for indexes as well but these contain pointers to the table structures. (The reason we need the index definition containers is that during recovery an

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-30529832369099494052008-02-27T21:49:00.003Z2008-02-27T22:15:36.206Z

SimpleDBM project work has been slow for the past couple of months. This is because I have decided to develop SimpleDBM as a plug-in storage engine for Apache Derby, as an alternative to the Derby Store implementation. If anyone is interested in tracking this work, please visit the Apache JIRA system and look for DERBY-3351.There are a few problems to be overcome for this to happen. One of this

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com1tag:blogger.com,1999:blog-15425053.post-60587980565440701672007-12-29T10:35:00.000Z2007-12-29T12:25:45.701Z

It is the classic chicken and egg problem.I would like to implement the system catalog in database tables. However, the system catalog will contain information that is a pre-requisite for working with tables. I do not know how other database implementations handle this cyclic relationship. It is made more difficult for SimpleDBM because the core database engine has no knowledge of the data

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-5609044139348307352007-11-03T01:41:00.000Z2007-11-03T01:50:23.135Z

A new SimpleDBM RSS release (1.0.6 BETA) is out. This release contains an important bug fix to the Buffer Manager. The release can be downloaded from www.simpledbm.org.Along with this release, there is also updated documentation. The User Manual is for developers who wish to use the SimpleDBM RSS component within their own projects. The Developer's Guide is for hackers who want to understand how

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-13831390496747503222007-09-14T23:06:00.000Z2007-11-03T02:33:46.076Z

In my last blog entry, I mentioned the differences in file locking behaviour I found between Sun's Windows JDK and Mac JDK. Well, I recently acquired a Dell server, and am running SLES 10.1 64-bit edition on it. The default Java 5 SDK that comes with SLES 10.1 is IBM's JDK 5 SR3. SimpleDBM fails to pass all its test cases under this JVM. All tests pass successfully when I run using Sun's Java 6

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-76684490879963457512007-09-14T00:24:00.000Z2007-09-14T00:47:31.763Z

Recently bought an iMac - with 24inch screen, and this is now my development platform for simpledbm. I am using JDK 5 and Eclipse 3.3.So far one test case has failed. Upon investigation, found that the behaviour of FileChannel locking is different on the Mac. It seems that Mac JVM allows multiple threads within the same JVM to acquire exclusive locks on a file concurrently - whereas on Windows,

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-84319598182330279732007-06-24T00:16:00.000Z2007-06-24T00:27:27.753Z

At long last, an early Beta release of SimpleDBM is out. Downloads are available from SimpleDBM project website. A User Manual is also available.Due to work pressures, progress has been very slow during the last one year. I am now hoping to build a few sample applications. Testing is still a priority as there are still not enough test cases in some areas, such as the Tuple Manager module, and in

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com3tag:blogger.com,1999:blog-15425053.post-50379379883776678672006-11-29T00:39:00.000Z2006-11-29T01:03:54.656Z

A friend asked me recently why I spent my time implementing a DBMS, when there are already a number of open source databases. It is a good question because clearly I am not breaking any new ground here. In fact, I find myself incapable of inventing great new technology. Most of what I am implementing is well known stuff. I am a software engineer, rather than a scientist, going by the definitions

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com3tag:blogger.com,1999:blog-15425053.post-244227290532692312006-11-28T21:50:00.000Z2006-11-29T00:35:31.542Z

Here is an example of C.A.R.Hoare's statement that premature optimization is the root of all evils.In implementing the Lock Manager in SimpleDBM, I was concerned about the impact thread synchronisation would have on the scalability of the Lock Manager. The Lock Manager uses a hash table to speed up lookups of the locks. Many implementations of Lock Managers synchronize the entire hash table while

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-53038390811813200092006-11-16T13:46:00.000Z2006-11-16T14:58:46.732Z

SimpleDBM's Lock Manager is based upon the algorithms described in Transaction Processing: Concepts and Techniques. Until now, the Lock Manager did not have support for deadlock detection. A simple timeout mechanism was used to abort transactions that were in a deadlock. Yesterday I decided to start implementing a deadlock detector. As I was researching this, I discovered a problem in the way

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1162680619079834262006-11-04T22:39:00.000Z2006-11-04T22:47:27.020Z

SimpleDBM has a new source code repository. I moved the source code from ObjectWeb's repository to Google Code because I found the ObjectWeb repository somewhat slow and unresponsive. The SimpleDBM web pages are still hosted at ObjectWeb.In the past few months, SimpleDBM's source code repository has move from java.net to ObjectWeb to Google Code, which I hope will be its final resting place.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1162431788282213182006-11-02T01:30:00.000Z2006-11-04T22:44:41.413Z

There has been a revision in the package structure within SimpleDBM.In the earlier structure, the API and the implementation of a module were designed to be together. Hence, if there was a module named test, the packages would have been org.simpledbm.rss.test for the API, and org.simpledbm.rss.test.impl for the implementation.The current structure is more favoured towards separating the whole of

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1162147808175361942006-10-29T18:34:00.000Z2006-11-04T22:44:41.346Z

After a break of several months, I am about to start working on SimpleDBM again. I am determined not to add any new functionality until existing functionality is thoroughly tested and fully documented.Currently I am working on improving the unit test cases for the BTree module. This is easily the most complex module in SimpleDBM, and producing comprehensive unit test cases is a significant task

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1150020179896712302006-06-11T09:56:00.000Z2006-11-04T22:44:41.283Z

SimpleDBM is moving to ObjectWeb. The reasons for this move are:ObjectWeb specializes in building middleware solutions. I hope to build a community of users and developers who are like-minded and interested in middleware technologies.Java.net is run by and has too much focus on Sun Microsystems. In some ways it is a propaganda tool for Sun. I wanted a more open environment for SimpleDBM, where

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1144194593934323832006-04-04T23:33:00.000Z2006-11-04T22:44:41.214Z

Due to changes in my work life, I have not been able to devote much time to SimpleDBM in recent weeks. Things are getting back to normal slowly and I am beginning to work on some of the outstanding tasks. The main focus is to provide a usable RSS component, which is feature complete, but needs more test cases, and more testing. Although test coverage is a useful measure of the effectiveness of

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com1tag:blogger.com,1999:blog-15425053.post-1140477047682353152006-02-20T23:03:00.000Z2006-11-04T22:44:41.140Z

Recently completed the implementation of Tuple Scans. Unlike Index Scans which use next key locking to avoid phantom reads, the Tuple Scan implementation does not protect the gap between one tuple and the next. This means that Tuple Scans cannot provide strictly Serializable behaviour. I wonder if this will be an issue later on.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1138412230235327352006-01-28T01:22:00.000Z2006-11-04T22:44:41.080Z

SimpleDBM is licensed under GPL V2 or later. I decided to use GPL because I believe in the values that the GNU movement stands for. It is a pity that so much FUD is generated regarding the GPL, and more pity that there is such a proliferation of OpenSource licences. If GPL was Business Unfriendly, then Linux would never have been successful.When GPL V3 comes out finally, I will adopt it for

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com1tag:blogger.com,1999:blog-15425053.post-1138033653305720962006-01-23T16:19:00.000Z2006-11-04T22:44:41.016Z

This month I am working on finishing the Developer's Guide, and also plan to update the Javadoc documentation. While updating the documentation I realized that I should not have used the term BTree when defining the interface for the Index Module. A BTree is an implementation strategy for Indexes, therefore, it is better to use a more generic term when specifying the interface. I am refactoring

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1137202283371388172006-01-14T01:23:00.000Z2006-11-04T22:44:40.954Z

Some time ago, I promised to write about some of the techniques used in Oracle. A very early draft of a paper on Multi-Version concurrency is now available. It discussed MVCC implementations in PostgreSQL and Oracle.SimpleDBM does not implement MVCC on purpose, as I wanted to understand traditional implementations before attempting to implement MVCC. Perhaps one day, a different version of

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1137114790475776372006-01-13T01:08:00.000Z2006-11-04T22:44:40.892Z

I am ashamed to say that I just discovered LaTeX. Of course, I knew about TeX but never thought I would use it ... well, I have just converted the SimpleDBM Reference Manual to LaTeX, and I love the results. The output is so much better, and the document looks professional. Here's the link to the PDF output.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1136586231783626042006-01-06T22:06:00.000Z2006-11-04T22:44:40.832Z

In a previous post, I blogged about why I favour Checked Exceptions over Unchecked ones. Today, I'll talk about how I am circumventing some of the issues with Checked Exceptions.The big advantage with Checked Exceptions is that the method signature tells you what Exceptions are likely to be thrown. Great as this is, it is also a liability, because any change in the Exception specification can

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1134554583858124742005-12-14T09:50:00.000Z2006-11-04T22:44:40.769Z

Despite my previous resolve, I have started work on the Tuple Manager module this month. I'll talk about some of the issues and challenges in this module.The Tuple Manager module provides a low-level interface for managing persistence of table rows. It is low-level in the sense that this module has no knowledge of what is contained in a table row. I use the term tuple instead of table row, but

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com1tag:blogger.com,1999:blog-15425053.post-1133796659557339682005-12-05T15:20:00.000Z2006-11-04T22:44:40.704Z

The priority for December is to complete the documentation and create sample programs. Both of these are intended to help new developers to get started with SimpleDBM.So far, I have created a BTreeDemo sample - here is a screenshot. The demo program allows a user to interactively manipulate a BTree index. The program creates two threads and allows user to switch from one to the other. This is a

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1132057582573871012005-11-15T11:45:00.000Z2006-11-04T22:44:40.643Z

When designing the interface for the BTree module, there are some conflicting priorities.An important goal is to ensure that the interface is reusable and makes as little assumptions about the content of "keys" and "values" as possible. Some of the points to consider are:a) Should the BTree module be aware of the type system available in the system?b) Should it be aware of how multiple attribute

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1131618863016288802005-11-10T10:16:00.000Z2006-11-04T22:44:40.584Z

In a major refactoring exercise I have split the SimpleDBM modules into two higher level packages. The Latch Manager, the Object Registry and the Util packages all go under the package org.simpledbm.common. The rest of the packages go under org.simpledbm.rss. I decided to use the acronym RSS for the low level data management API in honour of System R. System R called its low level API Research

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1131379093173072092005-11-07T15:51:00.000Z2006-11-04T22:44:40.522Z

BTree scans are now available as well. This means that the BTree implementation is feature complete, although, there are still some areas that need more work.For the rest of November, I am going to concentrate on improving the code, refactoring bits that I don't like, updating documentation, and generally cleaning up the code. All this in preparation for a release of the code towards the end of

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1130936876343206992005-11-02T12:54:00.000Z2006-11-04T22:44:40.451Z

Both insert key and delete key operations are now available. More test cases are being written; in order to test lock conflicts when the same key is concurrently inserted/deleted by different transactions, the new test cases have to use multiple threads. It is harder to debug such test cases, but Eclipse makes it easy. I simply run the JUnit tests in debug mode and set break points at appropriate

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1130460024131713552005-10-28T00:29:00.000Z2006-11-04T22:44:40.389Z

Currently I am busy testing the code for inserting keys into the BTree. My testing methodology is:Write a JUnit test case.Debug the code, stepping through the entire test. Fix any bugs and retest until test runs okay.Out of the 16 test cases written so far, only 4 are on insert - with many more to come. To facilitate testing I need to construct trees with specific properties. The way I have

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1129211325805696052005-10-13T13:40:00.000Z2006-11-04T22:44:40.331Z

I have been working hard on getting the BTree implementation completed. It is about 75% complete, however, delivery will take longer as I need to ensure that adequate test cases are available to verify the correctness of the algorithms. Already, this is probably the largest module. It will also be one of the most complex.As mentioned before, I am implementing the B-link tree algorithm as

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1128213433938602232005-10-02T00:15:00.000Z2006-11-04T22:44:40.267Z

Mainstream Relational Database technology is split into two camps.On the one hand are the systems that are directly or indirectly based upon techniques and algorithms derived from IBM's System R project. Most published research also falls into this group.In the second camp are systems that have been inspired by Oracle. Oracle has a very interesting history. Although Oracle started with the

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1128089959257380912005-09-30T13:21:00.000Z2006-11-04T22:44:40.207Z

One of the design choices I made in SimpleDBM is to build the system as a collection of modules, and keep the modules loosely coupled via interfaces. There is also a hierarchy within the modules; some modules come before others, and provide services used by other modules. It is very important to enforce this hierarchy, because otherwise, the dependencies between the modules would become cyclical

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1128086053289062012005-09-30T13:00:00.000Z2006-11-04T22:44:40.148Z

One constraint I have placed upon myself when coding is that I will not start work on a new module until I have created and executed test cases for the module I am currently working on. This slows me down, but the payoff is tremendous.Writing test cases for a module often takes more time than coding the module itself. I think that this is not unusual, and any project that is serious about testing

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1126825021941764022005-09-15T22:42:00.000Z2006-11-04T22:44:40.086Z

ARIES uses the term page-oriented redo/undo to describe logging that is always applied to a specific page. This means that if some page P1 generated a undo/redo log record L1, then during redo, the redo portion of L1 will be applied to P1, and during undo, the undo portion of L1 will be applied to P1.The term Logical Undo is used to describe logging where the undo action may not always be applied

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1125958604167524062005-09-05T21:39:00.000Z2006-11-04T22:44:40.008Z

I said in a previous post that I favour Checked Exceptions over Unchecked ones. I will try to explain my reasons for doing so.A good discussion of the problems with Checked Exceptions is in this article by Brian Goetz. The main problems can be summarized as:Unnecessary exposure of implementation details.Unstable method signatures.Unreadable code.Exception swallowing.Too much Exception

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1125654138248091452005-09-02T09:35:00.000Z2006-11-04T22:44:39.948Z

Certain BTree operations require Logical Undo to be performed during rollbacks and restart recovery. This is because, in a BTree, keys can move from one page to another, and therefore an undo operation may have to be applied to a different page from the one where the original action took place.To take an example, suppose transaction T1 deletes the key D from page P1. Another transaction T2 comes

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1125617888233446862005-09-01T22:36:00.000Z2006-11-04T22:44:39.880Z

This week I am working on the Space Manager module which will handle page allocation in containers. This module is not so exciting, it is tedious work with lots of little details all of which have to be right for it to work correctly. While working on this, I did have some interesting experiences though.I am a firm believer in Checked Exceptions. I think Unchecked Exceptions should be used very

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1125268973795830232005-08-28T22:08:00.000Z2006-11-04T22:44:39.822Z

After many hours of debugging, I finally have a version of the Transaction Manager that successfully handles the BitMgr Test. The BitMgr is modeled after the OneBit Resource Manager described in Transaction Processing: Concepts and Techniques. It uses a single data page, containing a set of bits that can either be on or off. Despite the simple design, the One BitMgr allows many of the features of

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1124838368685732332005-08-23T22:48:00.000Z2006-11-04T22:44:39.761Z

This week I am working on the Transaction Manager for SimpleDBM. The Transaction Manager is based upon the well known algorithm known as ARIES. There is a freely available paper at ARIES inventor C.Mohan's website that describes ARIES in quite a lot of detail.I wrote a short description of ARIES, which can be found at Apache Derby website.ARIES is based upon the recovery algorithms used in IBM's

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1124757982584160492005-08-23T00:23:00.000Z2006-11-04T22:44:39.703Z

The terms Lock and Log have very specific meaning in a DBMS. A DBMS Lock is a dynamically allocated synchronisation primitive, supporting multiple lock modes, used to manage concurrent access to Table rows. Locks are allocated as part of a transaction and retained until the transaction commits. Unfortunately, the term Lock is also used in a programming language like Java to represent Monitors and

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1124658156993311462005-08-21T20:36:00.000Z2006-11-04T22:44:39.646Z

The Buffer Manager in SimpleDBM uses a number of different locks in order to obtain fine-grained locking. The assumption is that with fine-grained locking, concurrency will be higher and the system will scale better with larger volumes and greater number of CPUs. However, whether this is actually achieved depends largely on the efficiency of the locks. It would be interesting to create a

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1124477988255294472005-08-19T18:57:00.000Z2006-11-04T22:44:39.582Z

Just completed the initial port of the Buffer Manager. Also checked in all the source code in the UNSTABLE_V100 branch.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1124061626065563692005-08-14T23:18:00.000Z2006-11-04T22:44:39.524Z

This week I am working on porting the Buffer Manager module. I expect to upload the code for it by the end of the week.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1124059146413925242005-08-14T22:04:00.000Z2006-11-04T22:44:39.462Z

I had started the DBMS project DM1 back in 2002 using C++. I said then that I had chosen to code DM1 in C++ because of lack of certain features in Java, such as pointers and efficient array handling. It seemed to me that a DBMS needed a language that allowed full control of memory.SimpleDBM is a pure Java project. So what has triggered the change from C++ to Java?Firstly, I no longer think that

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0tag:blogger.com,1999:blog-15425053.post-1124054966600186582005-08-14T21:24:00.000Z2006-11-04T22:44:39.393Z

Welcome to the official weblog for the SimpleDBM project. SimpleDBM is a new project that aims to implement relational database manager in Java. Please visit the SimpleDBM website for further details about the project.

Dibyendu Majumdarhttps://profiles.google.com/104970145946261316025noreply@blogger.com0