Locker Project's Blog » singly

Next steps for the Locker Project

quartzjer — Tue, 01 May 2012 19:46:06 +0000

Since we started this journey, the Locker Project and Singly have progressed side-by-side, with Singly as the hosted experience that sponsors the Locker Project.

After opening Singly’s Locker hosting to developers and getting lots of feedback on all of the possibilities that a hosted Locker could enable, the resounding theme was that developers want to use the API first and foremost. Based on that, the Singly team has been working on an effort to bolster the API aspects of the codebase so that it can support apps at a large scale. There’s been excellent progress in creating a cross-platform, cross-service API that provides merged, normalized and de-duplicated data on which apps can be built.

So now there is the question of how best to support and improve the open-source Locker Project, since it encompasses more than just an API. It’s an effort everyone in this community cares deeply about and has helped create a shared vision for, so we’re calling upon you to help decide what the most powerful direction for the Locker Project should be.

Please help take the time to fill out this survey, and most importantly, thank you all for your ongoing support and hard work!

Synclets

ctide — Fri, 19 Aug 2011 00:48:43 +0000

Temas touched on this a bit in his last blog post, but I wanted to write a more detailed explanation of where things are at for synclets. The short synopsis of synclets is that they are a simple way to pull data from a provider and feed it into the system. Synclets are basic routines that are fed authentication keys and some configuration and uses that information to pull down data from the provider which is funneled back into locker core via JSON. They are a drastic reduction in scope for what’s currently required for a connector. Adding new connectors currently relies on the developer managing a lot of pieces (authentication, running a web server, processing data from the source, feeding that data into Mongo, generating correct events) and since the majority of those pieces are fairly common across all of the connectors, we want to drive towards providing a system that manages these common components.

The first step towards this end was to convert the stable connectors into packages composed of synclets being managed by the common code. This has been completed (we’re still working on cleaning up some of the patterns to get them right, but we have a start checked in that we’re playing with) and they are now ready for people to start poking at. The current plan is to eventually migrate all of the connectors (and all the data your connectors have been collecting!) to synclet powered versions. For the time being, existing connectors won’t be affected by any of the work being done around synclets.

The missing pieces that will tie this work together and make it easy for developers to implement their own synclets mostly surround authentication and creating a UI for managing installed synclets. We will be implementing something like everyauth to manage authentication for synclets at some point in the near future. This will enable us to simplify both the UI, and the implementation, for authentication and allow us to provide authentication keys to any number of synclets that would like to pull data from a provider.

What does this mean to developers hacking on connectors today?

Not a whole lot just yet. We want to spend some more time ensuring that we have this pattern right, and connectors will continue to exist exactly as they are today during that period. If you’re feeling super adventurous, feel free to poke through some of the code in lsyncmanager, and look at the new synclets that are now provided with the connector code for Facebook, Twitter, Github, Foursquare, and Google Contacts. Once the authentication and UI pieces have been baked into the project, we’ll write a more detailed post describing exactly how you would use those pieces to build synclets.

The Locker Project’s First Hackathon, EVAR!

tikva — Tue, 26 Jul 2011 16:59:00 +0000

The Locker Project recently had its very first hackathon at Singly’s headquarters in San Francisco! Nearly 30 developers in the Quantified Self community came out, passionately hacking their way through Locker Project goodness. I speak on behalf on the entire Singly team when I say how appreciative we all are for the loving attention the Locker Project is getting.

The hackathon was super exciting for me and the entire team, working with so many excited developers. We had a ton of fun over beers, tacos and cupcakes, thanks to Cups & Cakes Bakery!

http://www.flickr.com/apps/slideshow/show.swf?v=71649

We learned a bunch about what’s working and what can be better, which we’ve taken to heart. As people went through the process of installing lockers and building connectors and applications, we discovered install issues we weren’t aware of and needs for documentation to make building easier and quicker. The team took the feedback to heart and has been working on documentation on the wiki and tarballs that come prepackaged with all dependencies intact!

Here are some examples of what folks worked on during the evening and since:

The incredible Chris Faulkner made uninstalling application in lockers possible!

Leonard Lin worked on a “geo viewer” application to get his contacts onto a map so when he visits a city, he can see on a map who lives there along with how to get in touch with them. He used the Locker Project’s Contact Collection to pull in his contacts along with the FourSquare Connector to pull in latitude and longitude for each contact and draw markers on his map view. He said he had to “blindly” go through the Contacts Collection to figure out what the geodata was, which was difficult. He said he wants geodata to be a first class Collection and so do we, so we are working on this!

Steve Lloyd worked on building a Connector for Dailymile.com, which is where he gets his running workouts. He wasn’t able to get the Connector to sync data, but he does have it authenticating and pulling data over at: https://github.com/repeatingbeats/Locker/commits/dailymile. He’ll be sending a pull request once he’s complete with it! Steve does pretty heavy work in Node.js/MongoDB in his day job, so didn’t have problems getting started with the code. However, he did find that getting started on a connector was somewhat difficult. He said our documentation is very good in terms of what files we provide, but still thinks our documentation is light on concrete examples for common patterns such as syncing data, storing data, and passing events. He says he knows he’s supposed to do those things, but he doesn’t really know why (beyond the basic premise of the Locker Project) at this point. As a result of his feedback, we’ve created a step-by-step “How to create a Connector” using FourSquare as an example which can be found here: https://github.com/LockerProject/Locker/wiki/Create-a-new-connector.

Paul Oppenheim spent time porting Slackulator, an application he had already made, over to work on the Locker Project. Slackulator tells you how many minutes per day you would waste (er take) to keep up with individual people you are following on Twitter. He was able to pull tweets in and was hoping to get to process them service side and then draw data once stored by got stopped.

Sjors Provoost built a WakeMate Connector. He said the skeleton code was hard to work with, because it was too closely tied to specific ways of doing things (like connecting to APIs and dealing with OAuth). He got far enough to submit a pull request containing his WakeMate work.

Jaisen Mathai: Tried to make a repository viewer for Github’s commits and diffs. They got confused because there isn’t a repository Collection so had to go to the Connector. Said it feels like you should never query the Connector directly, and that you should always query a Collection. It’s fine to get data from connectors in the absence of a collection, it’s just that collections are the preferred.

Here’s a Twitter list of some of the people who attended. The Evenbrite page has a full list of folks too.

Finally, the hackathon went so well, that we’ve decided to have them monthly, so please keep your eyes peeled for news about this!