whio  TipsAndTricks

ACHTUNG: AS OF 20110523, THIS PAGE IS NOW MAINTAINED IN THE NEW WIKI: http://whiki.wanderinghorse.net/wikis/whio/?page=TipsAndTricks

See also: whio_dev_HOWTO, whio_dev, whio_stream, whio_epfs

whio Tips and Tricks

The headers

The more important headers are:

#include <wh/whio/whio_dev.h> // whio_dev core API
#include <wh/whio/whio_devs.h> // various whio_dev factories
#include <wh/whio/whio_stream.h> // whio_stream core API
#include <wh/whio/whio_streams.h> // various whio_stream factories

And some of the more obscure ones:

#include <wh/whio/whio_common.h> // data types common to whio_dev/whio_stream, e.g. whio_rc
#include <wh/whio/whio_zlib.h> // de/compress whio_streams
#include <wh/whio/whio_encode.h> // various data en/decoding routines
#include <wh/whio/whio_epfs.h> // the whio_epfs API
#include <wh/whio/whio_udb.h> // the whio_udb API
#include <wh/whio/whio_vlbm.h> // the whio_vlbm API
#include <wh/whio/whio_ht.h> // the whio_ht API

Error checking

The API uses integers almost exclusively for error reporting. All of the integer error values have a symbol in the whio_rc global object. For example:

int rc = whio_foo(...);
if( whio_rc.ArgError == rc ) { ... some argument was wrong ... }
else if( whio_rc.RangeError == rc ) { ... something was out of range ... }
...

The value of whio_rc.OK is defined as zero, but no other integers in whio_rc have published values - use their symbolic names instead.

Remember to clean up...

Though object construction and initialization always require device-specific functions, all whio_dev and whio_stream objects have the same finalization conventions. To destroy a dynamically-allocated device do:

dev->api->finalize(dev);

That will close the device, free all internal resources, and then free the dev object itself.

If dev was not allocated via malloc() then it should not be finalized like that, but must be closed:

dev->api->close(dev);

and then dev itself must be freed using whatever method compliments its allocation. e.g. if it was stack-allocated then it need not (and must not!) be freed.

In-memory i/o devices

We can use the whio_dev API to access arbitrary memory ranges as if they were i/o devices:

// Memory we want to use as storage:
enum { Size = 1024 * 10 };
unsigned char mem[Size];

// Read-write device:
whio_dev * rw = whio_dev_for_memmap_rw( mem, Size );
// Read-only device:
whio_dev * ro = whio_dev_for_memmap_ro( mem, Size );

Or we can create in-memory devices which use an anonymous/internal buffer:

whio_dev * d = whio_dev_for_membuf( Size, 1.5 );
// ^^^^ Size=starting size, 1.5=expand by this much when growing

Use subdevices to partition a device

A subdevice is a whio_dev implementation which fences access to a certain range of another, larger device:

whio_dev * parent = whio_dev_for_filename( "myfile", "r+" );
whio_dev * subdev = whio_subdev_create( parent, 10, 51 );
// Now subdev only has access to the inclusive range (10,50) (aka [10,51))
// within the parent device, but it uses its own coordinate space:
subdev->api->seek( subdev, SEEK_SET, 0 );
subdev->api->write( sub, "hi!", 3 );
// ^^^ will write at the parent device location (10..12).

// The parent device MUST outlive the subdevice:
sub->api->finalize(sub);
parent->api->finalize(parent);

We can use an arbitrary number of subdevices to fence off various ranges of a larger device. Subdevices have no knowledge of each other (and their parent also doesn't know about them), and they may have overlapping ranges.

Subdevices are almost full-featured devices. The glaring omission is that truncate() does not work on a subdevice. The class may eventually be hacked to support truncate in a manner similar to how some RAM devices do (allowing truncation/re-growth only within their defined size).

whio_stream can proxy any whio_dev

We can use the whio_stream interface to provide sequential-only access to a whio_dev device (which is by nature random-access):

whio_stream * str = whio_stream_for_dev( someIODevice, aBool );
// The second param specifies whether the stream takes over
// ownership of the device.

I/O to/from external applications via popen()

We can use the whio_stream API in conjunction with the POSIX popen(2) function to read from or write to an external process (but not both read and write via the same stream).

Here's an example:

#include <wh/whio/whio_streams.h> // whio_stream_for_FILE()

/** popen device destructor. x must be-a (FILE*) opened by popen(). */
static void popen_dtor( void * x )
{
    int rc = pclose( (FILE*)x );
    printf("pclose(@0x%p) rc = %d\n", x, rc );
}

int test_popen()
{
    char const * cmd = "/bin/ls -ltd /*; echo rc=$?";
    FILE * p = popen(cmd, "r" );
    assert( p && "popen() failed!");
    whio_stream * str = whio_stream_for_FILE( p, false );
    // false means we keep ownership of p --->^^^^^^^^
    // We want this because we will install a custom dtor
    // below. The API docs for whio_stream_for_FILE()
    // explain in detail.
    assert( str && "whio_stream_for_FILE() failed!" );
    str->client.data = p;
    str->client.dtor = popen_dtor;
    int rc = whio_stream_copy( str, ThisApp.cout );
    // ^^^^^^ ThisApp.cout is an app-specific whio_stream
    // object which is bound to stdout.
    str->api->finalize(str); // will call popen_dtor(p)
    return rc;
}