Things to watch out for in HTML5 IndexedDB as at 21 June 2011 June 21, 2011

I’m between contracts at the moment so taking the opportunity to play with some bleeding edge technology.  With it seeming like everyone’s jumping on the HTML5 bandwagon, even microsoft with windows 8, seemed like a good opportunity to restart my side project playing with the latest web tech.

So there’s a few things to note if you pick up indexedDB..  It’s bleeding edge and to be expected but here’s my experiences over the last week.

IndexedDB is in webkit (chrome) and firefox but not yet in safari.  The database visualisation in the webkit developer tools isn’t linked in yet so you can’t mange the database that way yet.  You can’t delete a database programmatically yet in either chrome or firefox.  If you’re writing unit tests this is going to be a bit of a pain ;).  Also you can’t yet access the indexedDB from webworkers.  At this stage it’s attached to the window.  One of the things I’m playing with is a stemming and text sorting index which was all running via webworkers.  It’s an easy workaround, you just take the results from the webworkers and at a convenient time, merge and store instead of doing it directly.  Still, will be cool when this works.

The other thing I’ve noticed is that it feels very different than other data stores, even other kvp such as cassandra etc.   It really is a javascript data store.   The feeling I get is the asynchronous model is the preferred interaction method which again feels different that other API’s.  I’m still getting the feel, but it feels right for client side javascript.  In my opinion if I had to choose between the sqllite model and this, I’d choose this as a better technology direction for browser based client structured storage.  Sqllite would have just recreated the sql feeling of datastores and I don’t think it would have felt quite right for javascript in the long term.

I’m sure these will be addressed pretty shortly, I’m running chrome alpha and dev channels, ffox 5 and will post back when I notice a change.

Services and Contracts May 19, 2011

I’ve been playing a bit this week with WADL.  The service I’m playing with is a JSON REST service that’s representing the service contract in WADL.  It’s got me thinking about the role for service descriptions in a post WSDL world.  Fundamentally, if I release a service for the world must there be a method of specifying A) the endpoints exposed by a service and B) the data format that it accepts in a published standard.  So really the question is; If I release a service what’s the best way of helping people to write a client that will interact with it?  Or maybe; If the integrators are knife wielding coding maniac’s, what should I do?

First my thoughts on WADL.  WADL works well with XML based services.  The data can be typed with XSD and in combination with param path(an XPATH to the node of interest), has sufficient information to generate an implementation that interacts with that service.  It breaks down when trying to represent JSON (or alternate formats), path being undefined and as JSON doesn’t have an official schema definition, there’s no way of specifying the contract of complex JSON types or the path in the payload.  There’s almost schema’s such as JSONSchema of course and a number of notable others.  So it’s possible with a hodgepodge of almost standards to fully specify a JSON REST service that could be theoretically used for code generation.  The major impedance being tool support.   At the current moment in time with a JSON service you end up with multiple points of truth, the WADL and then the documentation around the JSON payload; what the parameters mean and their business logic +/- sample integrations.

So if WADL isn’t by itself sufficient for JSON, the question is; how do you hand over a JSON service to an integration team and get them to use it effectively?  At the moment this answer seems to be; here’s REST learn it, here’s JSON it’s simple, here’s our documentation about the data we expect.  It’s easy as a experienced developer to expect that these technologies are mastered and it’s a simple 1/2 day task to get it up and running.  However having shipped and supported a product that external developers have had to write an integration to, the lesson I’ve learned, is that it’s never simple enough!  Other developers will not bother to understand the technologies, will not read your documentation and will consider it your problem.   JSON and REST are fundamentally simple building blocks …  once you’ve mastered a number of other technologies and building blocks.  My experience is people writing integrations (mainly in the enterprise space) against your API are time pressed and often the most inexperienced developers.  So how do you cater for them?

The main benefit of WSDL in my experience is code generation.  Integration developers don’t need to understand SOAP, it’s point at the WSDL (WS-I compliant of course) and boom, get a client with a mostly understandable business object model.  Put your app on top, populate with the data and let the generated code take care of the rest.   Does this need to exist for JSON REST services?  The immediate answer is YES, of course! but it runs bit deeper.  The downsides of the WSDL  approach is that it’s a lot of magic.  When things work, it works well.   As soon as a problem crops up it can be an almost impossible task to understand what and where it’s not working.  By not having to fundamentally understand the technology stack and relying on the generated magic the masked complexity becomes an insurmountable problem, complexity always escapes.   The WSDL stack is actually quite deep and complex, the solution to the complexity being code generation, wizards and, well, magic.  JSON REST in my view is a fundamental shift in the solution of the problem.  Not let’s specify this more completely, to ensure better interoperability with another standard and ensure that we can generate software where the complexity is masked; but a re-orientation where with a bit of basic knowledge the problem is sufficiently simple not to require that additional complexity.   If JSON REST services get sufficiently complex in order to require the overhead of complex integration specifications and code generation then to some extent they’ve failed, the technology stack has failed to be sufficiently simple!

So here’s my conclusions.  It’s like most complex problems in software development, a human problem.  I think the choice to use JSON is a choice about the users that you want and expect to use your service.  WSDL may be a more appropriate solution, particularly in the enterprise space.  If shipping a JSON REST service; ship documentation, an example stub program in all of the languages you want to officially support and JSON samples (drop the WADL).  In the best of both worlds of a product you ship BOTH and allow users to self select.  Most of the core concepts are identical and with a little clever architecture in your products service it’s pretty easy to do.