
ULOOP -- Loopback block device in userspace
(and a sample for HTTP)
Junjiro Okajima

# $Id: 00readme.txt,v 1.1 2007/08/06 00:38:07 sfjro Exp $


0. Introduction
As you know, there is a Loopback block device in Linux, /dev/loop,
which enables you to mount a fs-image local file.
This is a quick hack adopting a userspace program into /dev/loop.
You can give an empty or non-existing file to /dev/loop backend.
When a process reads from /dev/loop, this dirver wakes a user process
up and passes the I/O transaction to it. A user process makes the
required block ready and tells the driver. Then the driver completes
the I/O transaction.
Also there is sample scripts or usage for diskless nodes working with
aufs. This driver may work with it well.
The name is unrelated to YouTube. :-)


1. libuloop API


2. sample for HTTP
Simple 'make' will build ./drivers/block/uloop.ko and ./ulohttp.
Ulohttp application behaves like losetup(8). Additionally, ulohttp is
an actual daemon which handles I/O request.
Here is a syntax.

ulohttp [-b bitmap] [-c cache] device URL

The device is /dev/loopN and the URL is a URL for fs-image file via
HTTP. The http server must support byte range (Range: header).
The bitmap is a new filename or previously specified as the bitmap for
the same URL. Its filesize will be 'the size of the specified fs-image
/ pagesize (usually 4k) / bits in a byte (8)', and round-up to
pagesize.
The cache is a new filename or previously specified as the cache for
the same URL. Its filesize will be 'the size of the specified
fs-image', and round-up to pagesize.
Note that both the bitmap and the cache are re-usable as long as you
don't change the filedata and URL.

When someone reads from the specified /dev/loopN, or accesses a file
on a filesystem after mounting /dev/loopN, ULOOP driver first checks
the corresponding bit in the bitmap file. When the bit is not set,
which means the block is not retreived yet, it passes the offset and
size of the I/O request to ulohttp daemon.
Ulohttp converts the offset and the size into HTTP GET request with
Range header and send it to the http server.
Retriving the data from the http server, ulohttp stores it to the
cache file, and tells ULOOP driver that the HTTP transfer completes.
Then the ULOOP driver sets the corresponding bit in the bitmap, and
finishes the I/O/request.

In other words, it is equivalent to this operation.
$ wget URL_for_fsimage
$ sudo mount -o loop retrieved_fsimage /mnt
But ULOOP driver and ulohttp retrives only the data (block) on-demand,
and stores into the cache file. The first access to a block is slow
since it involves HTTP GET, but the next access to the same block is
fast since it is in the local cache file. In this case, the behaviour
is equivalent to the simple /dev/loop device.

o Note
- ulohttp requires libcurl.
- ulohttp doesn't support HTTP PUT or POST, so the device rejects
  WRITE operation.
- ulohttp doesn't have a smart exit routine.
- This sample is "proof-of-concepts", do not expect the maturity level
  too much.
- This driver and the sample is developed and tested on linux-2.6.21.3.
- If you impelment other protocols such like nbd/enbd, iscsi, aoe or
  somehting, instead of http, I guess it will be fantastic. :-)

o Usage
$ make
$ sudo modprobe loop
$ sudo insmod ./drivers/block/uloop.ko
$ dev=/dev/loop7
$ ./ulohttp -b /tmp/b -c /tmp/c $dev http://whatever/you/like
$ sudo mount -o ro $dev /mnt
$ ls /mnt
	:::
$ sudo umount /mnt
$ killall ulohttp
$ sudo losetup -d $dev


Enjoy!
