The road so far….

February 1, 2013

Mongo, the silent killer!!

Filed under: java — Tags: , , — Rahul Sharma @ 6:22 pm

I was pretty much convinced that Mongo db is not the best choice for an application.  But a recent incident makes me believe that it is a silent killerof the application. As you work with it and  increase the amount of data/ concurrency  you realize  its one or more pain points.

Our application was working fine,  and one fine day one of the bugs got reported signup is not working as expected. It was creating user but could not login them. The app was not updated from quite some time, no changes to the schema how could this happen. So then we searched for logs, and it was sparkling clean !!! But still no user creation and other db operations, for some reason. After some debugging we realized that saves are getting called but data is not available in mongo after the operations and surprisingly it does not throw errors. WTF !!!

Googling around how to debug mongo,  leads you to some thing called WriteConcern. It is related to how mongo operations are handled. According to the documentation of MongoTemplate the default WriteConcern does not report back errors. We changed that so safe, which means it will complete that operation and report back errors if any.

<bean id="mongoTemplate" class="org.springframework.data.mongodb.core.MongoTemplate"></pre>
<em id="__mceDel"><constructor-arg name="mongoDbFactory" ref="mongoDbFactory" /></em>
<em id="__mceDel"> <property name="writeConcern">
 <util:constant static-field="com.mongodb.WriteConcern.SAFE"></util:constant>
 </property>
 </bean></em>
<pre>

After we enabled this we got a pretty interesting error while performing save:

“com.mongodb.MongoException: can’t map file memory – mongo requires 64 bit build for larger datasets”

If you google around this, then you realize that on 32-bit machines mongo can only have 2GB of data. Now this is crap !!!

Just 2GB really ???? If you are supposed to be  working with just 2GB why use a database ?? DBs are meant as data warehouses.

The only solution is left is to upgrade it to 64 bit ver.  Going by my previous experiences with mongo I knew that will not be as easy as it sounds. So I cleared some old data and the application was back online.

Mongo is a great product and has good deal of features. But I am sorry to say that it is not a 100 % finished product. Increasingly you see  its pain areas when you put in load and production.

4 Comments »

  1. bhai its very well documented and we all now about it from HI. I think it is your fault, not MongoDB. When you start MongoDB it shows in its console.

    Comment by whyjava — February 1, 2013 @ 10:13 pm

    • Every db has a limit in one way or not. In RDBMS also there is tablespace concept. But just 2gb of space in a db is a joke. I can do better with HSQL I think. I know from HI that there is a limit to documnet 4mb, which is kind of large text.

      Also I am not in favor of going down without throwing errors. This is very bad concept. In the end we resort to writing safe, why not make it default ? I believe thats how applications should behave. If your application does not want, you can turn down the exceptions raised. Here mongo has been used at two places and I am throwing it out due to the good experieces we had from HI.

      aur bhai its not about documentation, but more about choices they made while creating mongodb and mongotemplate.

      Comment by Rahul Sharma — February 2, 2013 @ 5:55 pm

      • Again I will say its the user mistake that you don’t know that MongoDB is async in nature and is not designed for 32 bit machines. Every new technology involves new thinking and new compromises, and as a user you should be aware of that. Sorry bhai, but I don’t agree with you on this. Why you guys choose MongoDB and which db are you guys gonaa use know.

        Comment by whyjava — February 2, 2013 @ 11:15 pm

      • Sure, I am not denying that its a mistake that we choose it, while we could do things in better manner with other standard technology. I believe u and me know it better because of our share of experiences in HI. There also we dropped it in next project, because the postgres did far better job. Thats what I am doing now ! paying the price of the mistake.
        Mongo is like gigaspace, if u can see the pain, both are not complete products. You can make new technology and make compromises, but make sure that these compromises sound like ones. DBs are meant for storage and then one day some db can say that I will store on 512MB on commodity architecture. Does that sound a compromise or a foolish mistake to make ?

        Comment by Rahul Sharma — February 3, 2013 @ 10:43 am


RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

The Shocking Blue Green Theme. Create a free website or blog at WordPress.com.

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: