FileSystemBasics

This article is based on the following blog post: http://jcoredb.blogspot.de/2012/05/file-system-abstaction.html. It explains some File System basics of jCoreDB.

The File System is the lowest layer in a Database Management System. Most DBMS just access the underlying File System which is provided by the Operating system. Because Database System work with their own page (or Block) sizes, it seems to be necessary to abstract the File System layer by providing a suitable File System API. The best performance regarding this abstraction is indeed achieved if we use the same block size as the one which is used by the underlying OS File System. An other possibility is to use raw access, which means not to use the underlying Operating System's File System. One idea would be to use the VMS FS API by wrapping it with Java classes by using the J(ava) N(ative) I(nterface). More about this idea can be found here: FileSystemVMS. It would be interesting to compare the both approaches regarding the IO performance. But let's at first focus on a File System API which would fit the following requirements:

  • Mount the File System to a specific path
  • Create, Access and Drop containers, whereby a container can just contain some segments
  • Write, Read or Delete a specific block of a segment, a segment is just a set of blocks
  • Manage the free and available blocks.
  • A block has a fixed number of bytes
  • A block has an unique identifier, which means

So a File System is described by using the interface IFileSystem. It has the following methods:

  • String getPath()
  • void setPath(String path)
  • void open()
  • void write(Block block)
  • Block read(BlockId id)
  • BlockId append(Block block, boolean only)
  • void delete(Block blockId)
  • void drop(int containerId)
  • void create(int containerId)
  • IContainer[] getContainers()
  • IContainer getContainer(int id)

Let's explain the 'only' parameter of the append method. If 'only' is set to 'true' then we avoid to find the next free block by asking the underlying free memory management. Instead we only append by using the last free block. If 'only' is false, then the next free block is used.


Behind the scenes it has take care about which blocks are available by doing a basic free memory management. So if a block is free inside a segment which belongs to the container, then the block could be written or overridden.Segments should be provided as necessary. To extend a block means to add a new segment to it. It should be also possible to just append data by avoiding a free memory management overhead during bulk imports. These yet mentioned requirements are part of the IContainer implementation. The IContainer interface has the following method definitions:

  • void create()
  • void open()
  • DataSegment extend()
  • BlockId append(Block block, boolean only)
  • BlockId delete(BlockId blockId, boolean soft)
  • void close()
  • void drop()
  • DataSegment getSegment(int id)
  • String getContainerPath()
  • int getId()
  • DataSegment[] getSegments()
  • int getNumOfSegs();


Some of the method definitions are 'harder' than required. It's a required refactoring step to extract interfaces for Segments as well. However, currently we differ between DataSegments and HeaderSegment. Each data segment has assigned one header segment. The header segment is used to store some information about the the data segment, E.G. the block size, the max. number of blocks and a free memory map. We still have to think about if it makes sense to have a separate segment to store the header information. If you would have the header data inside the data segment then this would cause a lot of seek time inside this segment. The two seperate segment cause that we have two seperate files to read and write from. A raid of disks or a SSD could benefit from that.

Another parameter to explain is the 'soft' one of the method 'delete'. If it is set to 'true' then we just mark the page as free inside the free memory map. If the value is 'false' then we additionally override the related block with an empty one.