CUBRID nLucene Configuration Tutorial
The goal of this tutorial is to help you configure nlucene from Cubrid Manager 8.4.0 so that you can use it to search through your database. The example that we will focus on is searching the documents of an Xpress Engine based site. Normally, since there isn't an index, a search would go through all the documents taking a very long time. nLucene is really helpful in this scenario because it creates the index of any table we want to use (in this case xe_documents) and therefore searches become really fast.
Overview
Before we proceed to the steps regarding the installation, we should first list the dependencies. For the purpose of this tutorial, we assume that you have installed XE from http://www.xpressengine.net/. You will also need to download the XE nLucene module version 1.2.
Regardless of you want to use XE or just want to search your own cubrid tables that do not have an index, you will also need to install Cubrid Manager 8.4.0 and nLucene-1.1 from dev.naver.com. Now we are ready to get started.
Step 1 - nLucene Manual Configuration
There are a couple of things that need to be done before nLucene can be configured from Cubrid Manager. First of all, we need to extract all the files from nLucene-1.1 to a specific folder. In this tutorial C:\nLucene-1.1 was used. Under Linux the path should be something like /opt/nLucene-1.1.
After this step, we must rename the folder nLucene-1.1\nLucene_bloc-1.1\modules\lucene_search_bloc-1.1 to nLucene-1.1\nLucene_bloc-1.1\modules\lucene_search_bloc. As you can see, we only have to remove the "-1.1" at the end.
It is preferable to also replace the old version of CUBRID jdbc with the new one in the following locations:
- nLucene-1.1\nLucene_bloc-1.1\modules\lucene_index_bloc-1.1\lib\cubrid-jdbc-8.3.0.0337.jar
- nLucene-1.1\nLucene_bloc-1.1\modules\lucene_search_bloc\lib\cubrid-jdbc-8.2.0.2005.jar
- nLucene-1.1\nLucene_bloc-1.1\modules\lucene_index_bloc-1.1\lib\cubrid-jdbc-8.3.0.0337.jar
- nLucene-1.1\nLucene_bloc-1.1\modules\hosting-mgmt-bloc-1.1\lib\cubrid-jdbc-8.3.0.0337.jar
- nLucene-1.1\nLucene_admin-1.1\webapps\ROOT\WEB-INF\lib\cubrid-jdbc-7.3.12.1004.jar
Step 2 - CUBRID Manager nLucene
Now let's explain step by step how to configure nLucene from Cubrid Manager.
Before getting started download CUBRID Manager nLucene plugin from: http://sourceforge.net/projects/cubrid/files/CUBRID-AppsTools/CUBRID_Manager/8.4.1/CUBRIDManager_nLucene-plugin-8.4.1.0004.zip/download and copy the files within C:\CUBRID\cubridmanager (or where your cubridmanager is installed).
Now, let's connect the host (right click on localhost and click connect host) and then select Configure nLucene like in Figure 1.
Fig. 1: Configure nLucene Option
A screen will appear like the one in Figure 2.
Fig. 2: nLucene Configuration Screen
Now, let's follow each part of the configuration screen starting with the General Information.
In the "Install path" field put the full path where you have extracted the nLucene folder. Afterwards, click on the check nlucene install path link.
In Figure 2 nlucene admin database was already configured. In order to configure it, we used "admin" for each field. If you want, you can change the values to whatever you want, just be sure that you remember the username/password pair as it will be required later on.
In the Hosting Management Configuration we have 2 files to modify:
- sql-map-props.properties
These should be the first 5 lines of the file. "admin/admin" must be changed according to the username/password you selected when creating the nlucene admin database. The other lines should be left intact.
Driver=cubrid.jdbc.driver.CUBRIDDriver #e.g. ConnectionURL=jdbc:CUBRID:x.x.x.x:33000:adminDB::: ConnectionURL=jdbc:CUBRID:127.0.0.1:33000:nlucene_admin_db::: Username=admin Password=admin
- sql-map-config.xml
<!--
<property name="validationQuery" value="${ValidationQuery}"/>
<property name="testOnBorrow" value="${TestOnBorrow}"/>
<property name="testWhileIdle" value="${TestWhileIdle}"/>
<property name="timeBetweenEvictionRunsMillis" value="${timeBetweenEvictionRunsMillis}"/>
--> Practically we remove the commented section and make it active. The file should now look like:
Fig. 3: sql-map-config.xml file (Notice the removal of the comments before and after the lines selected above)
Let's go back to Fig. 2 and look at the "Search Module Configuration". You must place the correct path to where the index files will be stored such as: "C:\nLucene-1.1\index_file_repo_sample".
Then edit search.xml file to look like:
<?xml version="1.0" encoding="utf-8" standalone="no"?> <root> <repository path="C:\nLucene-1.1\index_file_repo_sample"/> </root>
Finally, edit sql-map-props.xml to use the correct url/username/password:
<url>jdbc:CUBRID:127.0.0.1:33000:xe:::</url> <username>dba</username> <password>arnia</password>
xe/dba/pass are the database name, while the user and password are the ones used to connect to that database.
Now go to the Index Modules tab (you can see it in the lower part of the CUBRID manager window next to the Overview tab). Click add module and give it a name. We will use xe_module.
Fig. 4: Module Details
Fill in the fields with the database name, user name, password, broker ip and click connect. Select xe_documents table from the drop box and keep all the fields which are of interest to you. We kept only document_srl, title, content and tags. document_srl we renamed as id.
Modify the SQL Define to:
select A.document_srl as id, A.title, A.content, A.tags from xe_documents as A
There are 2 more modifications that need to be made from service.xml file. You can access it from xe_module.service.xml tab in the lower part of the screen:
Fig. 5: The Index Modules tab
Change XE_DOC to XE_document and set the repository path to where your index_file will be stored. It has to be the same path as the one put in Search Module Configuration, for example "C:\nLucene-1.1\index_file_repo_sample". The file should look like:
Fig. 5: service.xml file example
Go back to the Overview tab and then save everything (File->Save or CTRL+S).
You are ready now to start the bloc server. Just click on "Start Bloc Server" and wait for process to finish. Make sure the xe database is not started before starting the bloc server. If it is stop it and then restart it after the bloc server has been started. The bloc server can also be run using the startup.bat file in C:\nLucene-1.1\nLucene_bloc-1.1\bin.
Note: On Linux, you will have to set the user, password and connection uri manually. Just change them to look similar to the example in Figure 5 using your own username and password of course.
XE Configuration
Go to the XE admin. Select Manage from the left tab and then go to Naver Lucene tab.
Fig. 6: Address configuration
Put the IP address in the first field alongside the port on which the bloc server is running (by default it is 5001). In the second field put only the ip address. Then check the checkbox which tells XE to use nlucene and click submit.
Go to the third tab and put the name of the service before _document (in our case it is XE since the service name is XE_document). Put the path to the Search Module in the second field and in the last one put the index timeout (the time after which the index
is updated) and click submit.
Fig. 7: Index file Configuration
There are 2 modifications that need to be done in the lucene module. First, in lucene.class.php uncomment all the lines
from moduleInstall() function and then change in checkUpdate() function from return false to return true. The top of the file
should look like Figure 8:
Fig. 8: lucene.class.php file
In order for the modifications to be processed, go to the XE manage modules screen and click the Update button next to the lucene module. This makes the XE use the lucene search instead of the integrated search.
One last modification is required in lucene.view.php from lucene XE module. Change the line:
$searchAPI = "lucene_search_bloc-1.0/SearchBO/";
to
$searchAPI = "lucene_search_bloc/SearchBO/";
If you named your search bloc differently than lucene_search_bloc during the nLucene configuration, then just place it there. You can use for example lucene_search_bloc-1.1 which is the default name from the archive.
Go to the main page again and search for "welcome" or any text you have. You will see the results:
Fig. 9: Sample Search
The main difference is that now the search uses nLucene and thus does not go through all the documents when performing a search. The value "1" in the screenshot earlier should be replaced with the interval between index updates.
To perform your own JSON searches using the nLucene plugin, go to the link:
http://localhost:15880/callprocedure.nhn?module=lucene_search_bloc&procedure=searchByMap&resource=SearchBO&host=&port=
and type in the following query:
{"query":"*", "fieldName":"title", indexType:"db", "serviceName":"XE_document", "curPage":1, "numPerPage":10}
You should see at least 1 result (the default page from XE installation).
Conclusion
This was the guide to using nLucene to perform searches within the documents of an XE site. nLucene can be used with any Cubrid table exactly as we used it with xe_documents. You only need to perform your own JSON queries on the link described. We hope you found all the information useful. If you have any difficulties, you can post your questions to java forum for CUBRID.