Bulk I/O for Java Arrays

This page documents the Bulk I/O extensions implemented in  Titanium for high-performance I/O on Java Arrays.

This library is now available in the tc compiler (starting at versions 0.694 and IF.92), and is currently the fastest way to perform Java Array I/O in Titanium. 

All I/O operations in this library are synchronous (i.e. they block the caller until the operation completes) and work only on Java Arrays. For information on asynchronous I/O or bulk I/O on Titanium Arrays, see the Bulk I/O page.

Please direct any usage questions or difficulties to Dan Bonachea.


Bulk I/O Class Headers
(see descriptions below)

package ti.io;

public class BulkDataInputStream extends java.io.DataInputStream implements BulkDataInput {

// constructors
public BulkDataInputStream(java.io.InputStream in);

// bulk synchronous read
public void readArray(Object primjavaarray, int arrayoffset, int count)     throws java.io.IOException;
public void readArray(Object primjavaarray)                                           throws java.io.IOException;
};

public class BulkDataOutputStream extends java.io.DataOutputStream implements BulkDataOutput {

// constructors
public BulkDataOutputStream(java.io.OutputStream in);

// bulk synchronous write
public void writeArray(Object primjavaarray, int arrayoffset, int count)     throws java.io.IOException;
public void writeArray(Object primjavaarray)                                           throws java.io.IOException;
};

public class BulkRandomAccessFile extends java.io.RandomAccessFile implements BulkDataInput, BulkDataOutput {

// constructors
public BulkRandomAccessFile(String name, String mode)                      throws java.io.IOException;
public BulkRandomAccessFile(java.io.File file, String mode)                  throws java.io.IOException;

// bulk synchronous read
public void readArray(Object primjavaarray, int arrayoffset, int count)     throws java.io.IOException;
public void readArray(Object primjavaarray)                                           throws java.io.IOException;

// bulk synchronous write
public void writeArray(Object primjavaarray, int arrayoffset, int count)     throws java.io.IOException;
public void writeArray(Object primjavaarray)                                           throws java.io.IOException;
};

Bulk I/O Abstract Interfaces
(implemented by the above classes)

package ti.io;

public abstract interface BulkDataInput extends java.io.DataInput {
      // bulk synchronous read
      public void readArray(Object primjavaarray, int arrayoffset, int count)     throws java.io.IOException;   
      public void readArray(Object primjavaarray)                                           throws java.io.IOException;
        }

public abstract interface BulkDataOutput extends java.io.DataOutput {
      // bulk synchronous write
      public void writeArray(Object primjavaarray, int arrayoffset, int count)     throws java.io.IOException;
      public void writeArray(Object primjavaarray)                                           throws java.io.IOException;
        }


Bulk I/O Class Descriptions

Intro

The  BulkDataInputStream, BulkDataOutputStream and BulkRandomAccessFile classes in the ti.io package implement bulk, synchronous I/O. They subclass the 3 classes in java.io that can be used for I/O on binary data (so you can still use all the standard methods you're familiar with), but they add a few new methods that allow I/O to be performed on entire arrays in a single call.  This leads to significantly less overhead, and in practice speedups of over 60x (no mistake!) have been observed for Titanium code that performs a single large I/O using the readArray() and writeArray() methods, rather than many calls to a single-value-at-a-time method like DataInputStream.readDouble();

Because the new file classes are subclasses of the java.io.* libraries of similar name, any method calls you can make on a java.io.RandomAccessFile, you can also make on a ti.io.BulkRandomAccessFile. However, in order to see the performance gains, you need to use the new bulk I/O methods (readArray() and writeArray()) that are added by the bulk subclasses.

These libraries only handles single-dimensional Java arrays of primitive types (i.e. no Titanium arrays, no arrays of reference types). For information on bulk I/O with Titanium Arrays, see the Bulk I/O page.

Constructors

The  BulkDataInputStream, BulkDataOutputStream and BulkRandomAccessFile constructors are identical to the constructors of their superclasses. See the Java 1.0 docs for details.

Bulk I/O Methods

The  BulkDataInputStream, BulkDataOutputStream and BulkRandomAccessFile implement the readArray() and writeArray() methods for performing I/O on arrays of primitive values (see header declarations above). The "Object primjavaarray" parameter is the array you wish to read or write. The methods that take only this parameter will read or write the entire array that you pass to them. This must be asingle-dimensional Java array of primitives, for example,  long[], double[], int[], char[], etc., but not things like String[] (String is a reference type) or double[][] (2-d array) or int [1d] (no Titanium arrays).

You can also pass Java arrays of atomic immutable objects. Atomic immutable objects are immutable objects whose members are all primitives or other atomic immutable objects.

If you wish to only read or write part of an array, you should use the second form of each method and pass the "int arrayoffset, int count" parameters to specify the sub-array to be read or written - the read or write will begin at the array element "arrayoffset" and will continue for "count" elements. 

The readArray() and writeArray() methods return when the I/O has completed or may throw an exception if an I/O error occurs.  The methods will throw an ArrayIndexOutOfBounds exception if arrayoffset > primjavaarray.length or arrayoffset + count > primjavaarray.length. The methods will throw an IOException if primjavaarray is not a single-dimensional Java array of primitives.


Simple Example

import ti.io.*;   // bulk I/O classes reside in this package
import java.io.*;

...

// construct a BulkRandomAccessFile for input + output
BulkRandomAccessFile brafout = new
BulkRandomAccessFile("myoutputfile", "rw");

// construct a buffered BulkDataInputStream
BulkDataInputStream bdisin = new
BulkDataInputStream(
                               new BufferedInputStream(
                                 new FileInputStream("myinputfile")));

int numelems = bdisin.readInt();          // read some header info, using inherited methods
double biasfactor = bdisin.readDouble();

double [] buffer = new double[numelems]; // allocate a buffer to hold input

bdisin.readArray(buffer);
// read a large array of doubles

... // do some work on the array...

brafout.seek(100);       // seek to some destination in the output file

brafout.writeInt(numelems);          // output some header info, using inherited methods
brafout.writeDouble(
biasfactor);

brafout.writeArray(buffer); // write the large buffer of doubles

bdisin.close();
brafout.close();


Extended Examples

bulkperf.ti  - A benchmarking program that provides a performance comparison between the ti.io Bulk I/O classes and the java.io classes

bulkraftest.ti  - A diagnostic tool that exercises all the capabilities of the BulkRandomAccessFile class

bulkddtest.ti  - A diagnostic tool that exercises all the capabilities of the BulkDataInputStream and BulkDataOutputStream classes

bulkglob.ti  - A diagnostic tool that exercises the capabilities of the BulkRandomAccessFile class related to global arrays and global file objects (warning: does NOT work on distributed platforms).

(In order to compile these programs, you'll also need ExnType.ti)


Back to Bulk I/O page