Sunday, July 22, 2012

From Hibernate to MongoDB - The promise of changing the persistence layer.

This is not basically eclipse related, but it comes from my eclipse project, that I recently ported to RAP and now I plan to go multi-tenant with it.

This is not my first attempt in this direction, but this weekend I have a working proof of concept for changing my persistence layer from hibernate/hibernate search to mongodb.

I have about 80 domain classes and a bunch of handmade custom DAOs that are mostly based on the JPA Api. After playing with orientDB (I stopped that because of no "delete-orphan" there) this weekend I made an attempt with the datanucleus mongodb storage engine. To replace hibernate search, I decided to go for elasticsearch, which seems to be straightforward to build a multi-tenant index.

At first I struggled with some classloading issues, the datanucleus enhanced classes need a dependency on the jdo api at runtime. Then one major all-solution refactoring to use the mongodb string-based id on all my entities. I guess that took most of the time. I also changed  many dependend relations to embedded types, so that I can get the performance benefits mongo db promises.

Before I had load times for a single object graph (hibernate/sql server) from 3 to 4 seconds (yes one entity, you now know why I have to change that) and with mongo db and all the embedded collections you can hardly notice load time.

It's just that the relational model is not suited for complex object graphs. Even with massive parallel processing on an sql server cluster I had processing times around 10 minutes for 50k objects. I do not have values on that for mongo (need an importer first), but I guess it will work out.

I still have some exceptions and need to refine the elasticsearch integration, but over all I worked some 15 hours on it and at least I have a proof that it's doable.

3 comments:

Luca Garulli said...

OrientDB supports cascading deletion in latest snapshot. Delete of orphans will be implemented soon.

erdal said...

Looking forward to know your overall performance improvement loading yout 50k objects from mongodb...
And how did you persist your models using hibernate? CDO with hibernate store?

Thomas Kratz said...

Hi Erdal, no it's not CDO, its a plain jpa/pojo layer.

Using Mapstruct with Protobuf3