The Concept of Xen micro instances for running Lockers:

We spoke with Jeremy Fitzhardinge from XenSource/Citrix, regarding running Lockers in micro instances of Xen.  Through a lot of discussion, it sounds like it’s at least feasible in theory that very small Xen instances could indeed be used.  We also discussed the possibility of running Xen inside of Xen (to run Xen instances on a platform such as AWS), but it appears that this doesn’t peform well, understandably.

Plug Computers:

I attended a session that talked about the current state of small “plug” computers–defined as those types of computers that are small enough to simply plug into a wall.  These usually range from very small thumb-drive sizes up to a large wall wart or slightly larger.  The DreamPlug was one being demoed, and could be very interesting as a platform to use as a Locker appliance.  The good news is that besides the various networking capabilities of the DreamPlug, it also has an eSATA port on it, meaning it’s simple to plug in a large, high-performance hard drive for lots of storage.  Storage tends to be one of the weakest aspects of plug computers, and being able to add storage as necessary makes this very interesting.  Its price, when released, is said to be around $99.00 USD.

Here are the DreamPlug specs:

• CPU – Marvell Kirkwood 88F6281 @ 1.2GHz speed
• Linux 2.6.3x Kernel
• 512MB 16bit DDR2-800 MHz
• 2MB SPI NOR Flash for uboot
• 2 GB on board micro-SD for kernel and root file system
• 2 x Gigabit Ethernet 10/100/1000 Mbps
• 2 x USB 2.0 ports (Host)
• 1 x eSATA 2.0 port -3Gbps SATAII
• 1 x SD socket for user expansion/application
• WiFi 802.11 b/g
• Bluetooth 2.1 + EDR
• Audio Interfaces
• 5V3A DC power supply

Another plug computer that came across the radar is the Raspberry Pi.  This is a tiny computer, yet is ARM-based and can run Linux.  It’s does not have a lot of storage support other than SD cards, but could be extremely flexible in being able to run one or more connectors and stream data back to a central Locker instance.  I can see this being used with the Arduino stack to get cheap, realtime sensor data from the environment and into Lockers.  Its price is $25.  (Cheaper than the Arduino Uno dev board!)

Here are the Raspberry Pi specs:

• 700MHz ARM11
• 128MB of SDRAM
• OpenGL ES 2.0
• 1080p30 H.264 high-profile decode
• Composite and HDMI video output
• USB 2.0
• SD/MMC/SDIO memory card slot
• General-purpose I/O
• Open software (Ubuntu, Iceweasel, KOffice, Python)

Handling Locker Storage Safely and Quickly:

How to safely scale and store Locker data has been an ongoing discussion within the Locker Project.  Several options are on the table, and a lot depends on the requirements at hand.  For instance, for Singly-hosted Lockers, we will need something that we can host on insecure platform-as-a-service providers such as Amazon AWS.  For this, something like Tahoe LAFS looks like a great contender.

Other requirements that span both Singly-hosted Lockers and self-hosted Lockers are the capabilities of authorization to access, proof of change of data, and versioning of previously-changed data.

So it was interesting to have met up with Brad Fitzpatrick from the Memcached/Danga/LiveJournal world, to chat with us about his new project Camlistore.  He gave us a very quick rundown on various features.  One of note is the key signature of any changes made to the filestore, which allows easy confirmation of who changed what and when.  As it was described to me, it sounds like this same functionality also allows only approved users to view sets and/or subsets of the data.  Lastly, the versioning support that it has could prove to help us with our versioning issues as well.

One use case I found quite compelling was Brad discussing the possibility for a Camlistore (and by extension, a Locker) user to provide a subset of data that other people can write to.  For instance, you could provide a set of photos that you took while attending a party, provide write access to those particular photos to a group of friends who were also at that party, and those other users could add data–such as comments or tags–to your dataset, all using the Camlistore functionality.  This peer-enrichment idea is super exciting to me.

Locker-wide Search:

I met up with Tyler Gillies, who wrote the node-lucene module for Node.js.  This module wraps up the CLucene library and exposes it to a Node instance.  CLucene, for those who aren’t familiar with it, is a C++ port of the Java Lucene library–itself a very mature and powerful information retrieval library.  Internally, Singly has a Locker-wide search application already running, but it’s a prototype and requires the Elasticsearch Lucene implementation to run.  Elasticsearch is amazingly powerful, but it is also very resource-intensive.  We need something smaller, leaner, and faster, and CLucene fits the bill well.

The last night we were there, Singly put in a Locker BIrds of a Feather session where a bunch of us hacked on Locker things together.  I was able to work with Tyler to get up to speed on the state of node-lucene, and to begin contributing back to it with some more features we’ll need for full Locker search.  I’m super excited to get advanced search capabilities in the Locker soon, such as the following:

- Find all of my contacts from the contacts collection that have the e-mail address containing the term “singly.com”

- Find all of my geographic data from any connector or collection for the user “Eric Jennings”

- Find that link I visited recently that talked about the Google V8 garbage collection method, can’t remember if I read it on my phone or on one of my machines

If we do search right, any of thes types of search queries will be available to any application, collection, or connector.

That’s it for the OSCON trip.  Several of us were able to catch up with people we hadn’t seen in years, or have never met other than in IRC.  For those we met, it was great meeting you!  And for those who weren’t able to go, we hope to meet up soon!