Junit Recepies : Having common Setup and Teardown

The other day I was refactoring one of our apis. We were removing some of the columns as they were not required. I believe in this exercise dropping the column was the most easy part. After that I removed the corresponding fields from the associated ORM and it was fine for the business related code.  But !! the big one, was my code was not compilable, as the object was being used in a lot of tests and thus my project was all painted RED.

While I was fixing all the required tests I realized most of the tests were on similar lines. We were pushing some data in tests, creating some required resources and then destroying them after the tests were done. Now we have blatantly copied the stuff in every testcase  with total disregard to the DRY principle.

Spring Recepies : URI redirection

We have a application which has gone through a short developments sprints. Things got added and then some got modified or removed etc. During this phase a number of service URLs got added/removed/modified. Now if such an application is getting crawled by Google, the you have a mess to clear about. Clearing such a mess is not easy, some URLs should give 301 to a relevant page which some should show 404 and 500 status errors. More over there can be other requirements from the application like domain/sub-domain redirections eg making http://www.myapp.com to myapp.com, ver1.myapp.com to ver2.myapp.com etc.

Tip ! : How to call static method from JSP

Every application has some user information which it displayed to the logged-in user. My application also has the same information, but at few places it needs to be converted to JSON format as well, besides being displayed. Now either I can pass this JSON information from my Controller or I can somehow reuse the existing information that is being  passed. I choose the second route, but then the next question comes to your mind is how to convert this  on server-side ? on client side etc. ? Since we already had a static method to do  such conversions so I needed to figure out a way how I can call the same static method which could do the conversion on JSP  ?

Answer : Spring EL, is the way to do this. Using this you can access any static method of a class or an Enum on the JSP.

<%@taglib prefix="s" uri="http://www.springframework.org/tags" %>

<s:eval expression="T(com.rahul.devlearnings.MyClass).callStaticMethod()" var="data1" />

<s:eval expression="T(com.rahul.devlearnings.MyEnum).enum.call(data)" var="data2" />

Mongo, the silent killer!!

I was pretty much convinced that Mongo db is not the best choice for an application.  But a recent incident makes me believe that it is a silent killerof the application. As you work with it and  increase the amount of data/ concurrency  you realize  its one or more pain points.

Our application was working fine,  and one fine day one of the bugs got reported signup is not working as expected. It was creating user but could not login them. The app was not updated from quite some time, no changes to the schema how could this happen. So then we searched for logs, and it was sparkling clean !!! But still no user creation and other db operations, for some reason. After some debugging we realized that saves are getting called but data is not available in mongo after the operations and surprisingly it does not throw errors. WTF !!!

In-experienced development = Bowl full of Soup

There are times when things become an uphill task all of a sudden. It happened to me in my last assignment.  I was asked to take charge of an existing product. The product has been under development for about 2 years, and now it needed to fix a few things so that the system can scale well.

It started quite well, as the complete things was based on a licensed product from a vendor. The technology had loads of  docs explaining things in quite detail. So I played around the technology and it performed well.

Adapting in-compatible APIs

In Apache Crunch we are supporting two versions of  Hadoop viz. 1.0.3 and 2.0-alpha. In such situations couple of times you could hit the issue of incompatible APIs. Hadoop version 1.0.3 contains couple of classes in the core-lib which are interfaces in version 2.0-alpha. Now if you need to implement an interface for one of the version how would you go about ? The APIs in the two versions are completely incompatible, so would you drop the idea and start supporting one version ? But that’s a big price to pay, thankfully there are couple of things to your rescue in such cases.

Lets Crunch big data

As developers our focus is on simpler, effective solutions and thus one of the most valued principle is “Keep it simple and stupid”. But with Hadoop  map-reduce it was a bit hard to stick to this. If we are evaluating data in multiple Map Reduce jobs we would end up with code that is not related to business but more related to infra. Most of the non-trivial business data processing involves quite a few of  map-reduce tasks. This means longer tread times and  harder to test solutions.

Google presented solution to these issues in their FlumeJava paper. The same paper has been adapted in implementing Apache-Crunch. In a nutshell Crunch is a java  library which simplifies development on MapReduce pipelines. It provides a bunch of  lazily evaluated collections which can be used to perform various operations in form of map reduce jobs.