The road so far….

April 26, 2010

Lucene : Reuse IndexSearcher rather creating new everytime

Filed under: java, lucene — Rahul Sharma @ 10:26 pm

In Lucene we can create an index using IndexWriter and then to search over this created index we need an IndexSearcher. If the requirement is such that some part of the application is continually updating the index which some other part is using  then the IndexSearcher that is used to search the index needs to be created again, whenever the index is updated. It turns out that every time we create an IndexSearcher, it keeps a snapshot of the index over which it has been opened and then searches over the snapshot only, any subsequent updates to the index are not reflected. In such a case extra collaboration needs to build in place between the elements that are updating and that are searching the index.

We can avoid this by creating the IndexSearcher passively every time, i.e . whenever required, this makes sure that when we are going to search then the latest index will be used. This also creates an over head as we will be additionally opening and closing the searcher every time we do the search. It would be optimal if the searcher can re-load the latest index whenever required. (more…)

April 21, 2010

Spring: Load resources from any location(Classpath Jar, FilePath etc)

Filed under: java — Tags: , — Rahul Sharma @ 11:47 pm

Loading proper resources esp configuration files, properties files etc is an important aspect of any application. In one of my projects I was supposed to load a resource(configuration file) from a dependency jar but the same resource was also required to be loaded from a specified path location in another part of the same application .  We can do this using the ClassLoader‘s and URL for loading the Classpath resource and using File for the file system resource. OR, we can use spring Resource api to get the difference sort out for us. (more…)

April 17, 2010

Confessions of a new Agile Developer

Filed under: Agile — Rahul Sharma @ 8:15 am

I had worked in waterfall model for most of my time and now I started working in Agile model. It has been quite some time now, but the transition has not been one of my best ones.

I joined a team of 8 pros and contrary to my expectation 6 of us had none experience of Agile methodology. So we started under the guidance of our 2 experienced pros. Initially there were few ups and most downs. We found out Agile is not just about sprints and no documentation, there are more subtle aspects which you have to learn the hard way. Here is an account of the what went: (more…)

April 10, 2010

Lucene: prefer IndexWriter.commit over IndexWriter.close

Filed under: java, lucene — Rahul Sharma @ 12:45 pm

In one of the projects we were working with Lucene, for indexing data. We needed to create separate  indexes  depending on some data property. As we were writing to separate indexes we needed to close our latest index and then open the IndexWriter on the new index, if required.

At first this seemed to be the right operation because closing an index would flush the data to the indexed and optimise it. But as we started testing the application we found that it was working considerably slow. When we looked into the documentation of IndexWriter we found out the Close operation is considerably slow. Lucene recommends using the Commit operation over the Close operation if the IndexWriter is required to be used again and again. Our metrics found that the close was about 5 times slower as compared to commit. (more…)

April 3, 2010

Spring:Classpath and the world of Classloaders

Filed under: java — Tags: , — Rahul Sharma @ 1:22 pm

A while ago I re-factored an existing application in a multi module maven project. These modules were being loaded into a Gigaspace container using spring. Gigaspace used multiple classloaders to load the applications at different levels. The re-factored the application was being assembled as dependent jars from one module to other, so I had different xml context files that were being imported using : (more…)

Older Posts »