ShowTable of Contents
The advanced XPages2Eclipse feature "JavaScript servlets" described in this article works similar to a technique called "XAgents", covered by blog posts of Chris Toohey and Stephan Wissel:
In short, JavaScript servlets as well as XAgents are an XPages version of classic Domino web agents or HTTP servlets:
A piece of code is called via a URL from the web browser and produces content by writing text or binary data into an output stream. For XPages development, this code is written in server side JavaScript (SSJS).
The technique is useful when there is a need to programmatically create web content, e.g. web pages in a Domino based Content Management System, Office documents and spreadsheets, captcha graphics for authentication or to implement your own
REST service
.
So if there already is such a mechanism for XPages, why did we develop our own?
Well, when working with those XPages based "agents", we faced several issues that our implementation addresses:
Call code in other databases
Let's say you have an application that consists of at least two databases, one application database and one that stores configuration data. For a clean architecture, you would like to put all configuration related code into the configuration database, including a REST API that you want to call from client side JavaScript code in the application, e.g. to read configuration data or store user preferences.
This works fine on the web, because you can request content of other databases without problems as long as the current user has sufficient access rights.
But unfortunately it's not possible with XPages in the local Notes Client (XPiNC). There you can only call XPages in the currently open database. You would need to move all your configuration code into the application database to make this work, which messes up your architecture, because that code belongs to the configuration database. It gets even worse when you have more than two databases, like in a CRM system with a filing database, an address book, time recording and other things. You just should not need to duplicate the whole code base to make any API and data accessible throughout the system.
The implementation we picked for XPages2Eclipse has a different approach: It uses its own servlet implementation to handle a HTTP request and pass it to your JavaScript or Java code. It supports calling code in other databases (stored locally or on a server), making sure that it can only be called by code in an XPages application and not from a local browser.
Parallel execution of requests
Routing many requests through XPages can lead to bad application performance. The reason is that, up to Lotus Notes/Domino 8.5.2, code in an XPage runs sequentially for a single user. So if you let one XPage do a long running operation (e.g. output a large file or process many documents as part of a REST call), other XPages for the same user will wait until the operation is done (for experts: the XPage code is synchronizing on the HttpSession object while your code is invoked).
Our implementation in XPages2Eclipse does not need such a synchronization. Code execution occurs in parallel, only limited by the number of HTTP threads in the web server. You get maximum throughput.
Cache control for developers
As an XPages developer, you probably have experienced the issue that changes you make in Domino Designer sometimes are not visible after a page reload in the browser or Notes Client. The reason is that the XPages runtime caches the code for performance reasons and you often have to do a "restart task http" on your development server (which also restarts the JVM running XPages). This might be ok for web development on a local Domino server, but it's a real pain for XPiNC development. In that case, you have to restart the Notes client including Domino Designer!
XPages2Eclipse provides (OSGi) console commands that you can use to clear the caches we use: one for SSJS library content (which we execute for our JavaScript servlet technology) and another one for any custom Java class that your code is using. This makes it easy to develop and test an application in a short amount of time.
Performance for server applications
When you are dealing with many small Java or resource files
in your application like a newer Dojo build than your current Notes version provides (managing the files in the Java perspective), you may
experience pretty bad performance when working with XPiNC applications (XPages in the Notes Client) stored on a Domino server.
The reason is that every single file is stored as a
document in the database design and needs to be pulled over the wire the
first time it is requested during a user session.
The XPages2Eclipse classloader supports the generation of a class cache file which speeds up the load time of an application. You can find details about this technique in the
article about the NSF classloader.
Access to Notes Client APIs
Originally, the first idea behind our "JavaScript servlet" implementation was to make the XPages2Eclipse APIs accessible for developers who are not using the normal Dojo/Ajax request mechanism of XPages to call server side code from UI code (e.g. IBM's "XSP" client side library), but prefer other frameworks like Ext.JS or JQuery. As you may know, Dojo and XPages client side runtime code can be disabled with a database property "xsp.client.script.libraries=none", as mentioned in this blog posting:
In such a scenario, you could feed your UI elements through your own REST API and also implement the whole application logic as REST calls - a very clean approach, because the UI is completely separated from the backend code and can be replaced at any time.
We support this approach by making an "eclipseconnection" object available as a global variable to your JavaScript code when it is run in the local Notes Client. Use it in your JavaScript code as if you had created the connection by calling X2E.createConnection() in regular SSJS code.
Since we found out that the JavaScript servlet technology would also be very useful for Domino applications on the web and not just locally (even without the "eclipseconnection" object), we decided to add a Notes.ini switch to make the feature work on Domino servers as well.
For security reasons, it's disabled by default and can be enabled globally and on a per database basis.
Activation on Notes Client/Domino server
Javascript servlets are enabled by default in the local Notes client, protected by security mechanisms so that only code coming from an XPages application can invoke them.
As mentioned before, the feature is disabled on a Domino server by default.
You need to add a Notes.ini variable in order to make it work. To enable the feature for a single database with filepath "path/to/database.nsf", simply define the following variable:
mndX2ERPCEnabled_path_to_database_nsf=true
If we cannot find this variable in the server's Notes.ini, we check if the feature is enabled globally with the following line:
For performance reasons, the result of the variable lookup is cached and not read again until a server restart.
For technical reasons, the URL syntax to access the JavaScript servlet on the Notes client is different than the syntax on a Domino server.
For the Domino server, just append the string "/x2erpc/js" to a database url, followed by the name of a SSJS library and the name of a JavaScript function to call:
http://-hostname_of_server-/-path_to_database-/x2rpc/js/-name_of_ssjs_lib-/-name_of_js_function_in_lib-
for example:
http://www.mindoo.com/office/address.nsf/x2erpc/js/restapi/service
Unfortunately, in the local Notes client, we had to tweak a bit with the syntax, because the client's servlet engine does not allow the same syntax as on the web:
http://127.0.0.1:-dynamic_XPages_Port-/x2erpc/-UTF8_encoded_server-!!-UTF8_encoded_dbpath-/-name_of_ssjs_lib-/-name_of_js_function_in_lib-
for example:
http://127.0.0.1:1234/x2erpc/Server1%5cMindoo!!office%5caddress.nsf/restapi/service
In addition to the UTF-8 encoding, the "/" in the servername "Server1/Mindoo" was replaced with a "\" (%5c).
Sounds complicated?
We provide a small Java class in order to make it easier to generate the right servlet URL syntax in your code:
var jsBaseRpcUrl=com.mindoo.xpages2eclipse.tools.URLHelper.getRpcUrl(database, "js"); // use type "js" to get a URL for JavaScript RPC
var servletUrl=jsBaseRpcUrl+"restapi/service";
Calling the example URLs above could for example lead to the execution of the following method in the SSJS library "restapi":
function x2erpc_service(req,response) {
response.setContentType("text/html");
var writer=response.getWriter();
if (eclipseconnection) {
var pUI=com.x2e.PlatformUIAPI.getUI(eclipseconnection);
pUI.logToStatusBar("JavaScript servlet invoked with path "+req.getPathInfo());
}
writer.println("<html><body>");
writer.println("username: "+session.getUserName()+"<br>);
writer.println("database: "+database.getServer()+"!!"+database.getFilePath()+"<br>");
writer.println("pathinfo: "+req.getPathInfo()+"<br>");
writer.println("method: "+req.getMethod()+"<br>");
writer.println("</body></html>");
}
The method name is the same as specified in the URL, but for security reasons, the text "x2erpc_" is used as a prefix so that only dedicated methods can be called, in this case x2erpc_service.
The argument "req" passed to this method contains the standard
HttpServletRequest with the HTTP request data. "response" is a
HttpServletResponse which is used to return the result type and data.
Here is another example:
The following XPage contains a link to call such a JavaScript servlet based on a dynamically generated URL:
<?xml version="1.0" encoding="UTF-8"?>
<xp:view xmlns:xp="http://www.ibm.com/xsp/core" xmlns:xpages2eclipse="http://www.mindoo.com/x2e">
<xpages2eclipse:apiinit id="apiinit1"></xpages2eclipse:apiinit>
<xp:link escape="true" text="Link to Javascript servlet" id="link1">
<xp:this.value><![CDATA[
#{javascript:var url = context.getUrl();
var jsBaseUrl=com.mindoo.xpages2eclipse.tools.URLHelper.getRpcUrl(database, "js");
url.setPath(jsBaseUrl+"cms/pages/index.html");
return url.toString();}
]]>
</xp:this.value></xp:link>
</xp:view>
This would call the method "x2erpc_pages" in the SSJS library "cms". As usual for any servlet implementation, the additional text "/index.html" in the URL and any query string parameters can be obtained by calling the available methods of the passed "
req" object (e.g. getPathInfo(), getQueryString()).
Please note that for the last example, we had to tweak a bit by creating an absolute URL (including hostname and port) from the current context URL. This works around the (annoying) behaviour of the xp:link tag to automatically add the current database path to relative URLs when the page is rendered.
Security
Now that you know how to enable and use the JavaScript servlet technology, we end this article by discussing the options you have to secure its usage and restrict it to authorized developers.
Execution rights
By default, the JavaScript servlet code runs
with plugin privileges (we call it "elevated mode") and code from all
signers can be executed. This makes it easy in a test environment to try
out all functionality without the need to set up policy control.
For a production environment however, we
strongly recommend
to use the techniques mentioned in the article
"
Use policies to restrict developer use
of XPages2Eclipse API
" to restrict code execution to specific trusted
code signers.
For JavaScript servlets on a Domino server we
provide additional policy preferences that you can use to lower the
execution privileges to those of regular servlets in a servlet
container. This, for example, prevents a developer from accessing the
ClassLoader of classes by calling MyClass.class.getClassLoader() or to shut down the HTTP server's JVM.
To do this, add two variables to the managed settings section of the desktop policy document:
Plugin: com.mindoo.xpages2eclipse
restrictelevationmode=1
runelevated_<Replica ID of database>=false
e.g.
runelevated_C1257889004BEB96=false
Sample application
Our sample application for JavaScript servlets contains two SSJS libraries: the "cms" library acts like a dispatcher of a small content management system: It reads the
pathinfo argument from the servlet request which contains the URL part right behind "/x2erpc/js/-libraryname-/-method-" and tries to find a matching resource on the classpath.
The sample contains both static resource files (like the file index.html that you see when you open the database) and dynamically created content (in this case, we load a Java class from JavaScript and call a method to handle the request.
//list of known mime types
var mimeTypes={
"html" : "text/html",
"htm" : "text/html",
"txt" : "text/plain",
"jpg" : "image/jpeg",
"gif" : "image/gif",
"png" : "image/png",
"css" : "text/css",
"js" : "application/javascript",
"json" : "application/json"
}
//map of mimetypes that contain text content; if the JavaScript servlet
//processes this kind of content, placeholders like %%rpcurl%% are
//replaced by the computed values
var textMimeTypes={
"text/html" : true,
"text/css" : true,
"text/plain" : true,
"application/javascript" : true
}
//mapping between the pathInfo prefix and the corresponding Java resource name path
var pathMappings={
"/ext4" : "/ext4",
"/examples" : "/com/mindoo/cms/resources"
}
//the method converts the incoming path into the resource name that is used for
//the NSF classloader; if the path's prefix is unknown, the method returns an
//empty string
function getMappedResource(path) {
for (var pathPattern in pathMappings) {
if ((path.indexOf(pathPattern+"/")==0) || path==pathPattern) {
var mappedResource=pathMappings[pathPattern];
var newPath=mappedResource+path.substring(pathPattern.length);
return newPath;
}
}
return "";
}
function sendError(req,response,errCode,msg) {
response.sendError(errCode, msg);
}
function debug(msg) {
java.lang.System.out.println(msg);
}
//the method tries to extract the file type from the pathinfo
//and returns a matching mimetype or application/octet-stream if
//the type is unknown
function getMimeType(pathInfo) {
var iPos=pathInfo.lastIndexOf(".");
var extension=iPos==-1 ? "" : pathInfo.substring(iPos+1);
var mimeType=mimeTypes[extension];
if (!mimeType) {
debug("Unknown file type:"+extension);
mimeType="application/octet-stream";
}
return mimeType;
}
//The method is used to check whether content with the
//specified mime type contains text content (for placeholder replacement)
function isTextContent(mimeType) {
if (textMimeTypes[mimeType]==true)
return true;
else
return false;
}
//this dispatch method is called from the X2E servlet to
//dispatch an URL; it first tries to load the content
//via the NSF classloader as static resource (e.g. html file in
//the database design); if the resource cannot be found,
//we then search for a Java class stored under
//the package com.mindoo.x2esamples.cms.content that contains
//code to dynamically generate the content
function x2erpc_dispatch(req, response) {
if (!nsfclassloader) {
sendError(req, response, 500, "NSF class loader variable not found");
}
debug("response:"+response);
var pathInfo=req.getPathInfo();
debug("pathInfo:"+pathInfo);
if (!pathInfo) {
sendError(req, response, 500, "No pathinfo specified");
return;
}
var mappedPathInfo=getMappedResource(pathInfo);
if (!mappedPathInfo) {
sendError(req, response, 500, "No resource mapping found for path "+pathInfo);
return;
}
debug("mappedPathInfo:"+mappedPathInfo);
//try to load path as resource
var resourceStream=nsfclassloader.getResourceAsStream(mappedPathInfo);
if (resourceStream) {
try {
//found a resource; send the right mime type to the browser
var mimeType=getMimeType(pathInfo);
response.setContentType(mimeType);
if (isTextContent(mimeType)) {
//replace placeholder in text based content
var txtContent=""+com.mindoo.cms.tools.StreamUtil.getTextContent(resourceStream, "UTF-8");
var jsrpcBaseUrl=com.mindoo.xpages2eclipse.tools.URLHelper.getRpcUrl(database, "js");
var javarpcBaseUrl=com.mindoo.xpages2eclipse.tools.URLHelper.getRpcUrl(database, "j");
txtContent=txtContent.replace(/\%\%jsrpcurl\%\%/g, jsrpcBaseUrl);
txtContent=txtContent.replace(/\%\%javarpcurl\%\%/g, javarpcBaseUrl);
var writer=response.getWriter();
writer.print(txtContent);
}
else {
//use helper function to transfer the stream content
var outStream=response.getOutputStream();
com.mindoo.cms.tools.StreamUtil.write(resourceStream, outStream);
}
}
catch (e) {
debug(e);
sendError(req, response, 500, "Could not process request "+pathInfo+":"+e);
return;
}
finally {
resourceStream.close();
}
return;
}
else {
debug("getResourceAsStream("+mappedPathInfo+") returned null");
}
var className=mappedPathInfo.substring(1).replace(/\./g,'_').replace(/\//g, '.');
debug("className:"+className);
//now try to find a Java class for a request handler with that name
var clazz;
try {
clazz=nsfclassloader.loadClass(className);
}
catch (e) {
//catch class not found exception
sendError(req, response, 404, "path "+pathInfo+" not found");
return;
}
//create a new handler instance of this class and let is process the request
var handler=clazz.newInstance();
handler.service(req, response);
}
The second library "rest" provides a REST API that is used to read data from a view with 40.000 person documents (thanks to Jake Howlett for his
Fake Names Database).
//JavaScript class that handles the persons rest API request
var PersonRestHandler=function() {
this.service=function(req, response) {
//read start and limit parameter sent by the Ext JS 4 grid
var start=req.getParameter("start");
if (!start)
start=0;
var limit=req.getParameter("limit");
//open People view in database
var peopleView=database.getView("People");
//autoUpdate=false needed for the accelerated ViewNavigator
//used in the RestViewNavigatorFactory class (uses
//new buffered mode in 8.5.2)
peopleView.setAutoUpdate(false);
//to read the view content, we are using a few classes from
//the Domino Extension Library (Apache License) that unify
//several different sorts of view data access
var params=new com.mindoo.cms.tools.extlib.DefaultViewParameters();
var nav=com.mindoo.cms.tools.extlib.RestViewNavigatorFactory.createNavigator(peopleView, params);
//send json mimetype to browser
response.setContentType("application/json; charset=UTF-8");
response.setHeader("Cache-Control", "no-cache");
response.setCharacterEncoding("UTF-8");
//instantiate Jackson library that is used to stream out the JSON code
var jsonFactory=new org.codehaus.jackson.JsonFactory();
var jg = jsonFactory.createJsonGenerator(response.getWriter());
jg.writeStartObject();
//send the total number of entries as "totalCount"
jg.writeBooleanField("success", true);
jg.writeNumberField("totalCount", nav.getTopLevelEntryCount());
//now iterate through the view data and write an array entry for each row
jg.writeArrayFieldStart("data");
var idx=0;
for( var b=nav.first(start,limit); b && idx<limit; b=nav.next(), idx++) {
var unid=nav.getUniversalId();
var lastName=nav.getColumnValue("LastName");
var firstName=nav.getColumnValue("FirstName");
jg.writeStartObject();
jg.writeStringField("id", unid);
jg.writeStringField("firstname", firstName);
jg.writeStringField("lastname", lastName);
jg.writeEndObject();
}
jg.writeEndArray();
jg.writeEndObject();
jg.flush();
nav.recycle();
};
};
function sendError(req,response,errCode,msg) {
response.sendError(errCode, msg);
}
function debug(msg) {
java.lang.System.out.println(msg);
}
//this dispatch method is called from the X2E servlet to
//dispatch the rest service /rest/persons
function x2erpc_persons(req, response) {
//var pathInfo=req.getPathInfo();
//debug("pathInfo:"+pathInfo+", queryString:"+req.getQueryString());
var handler=new PersonRestHandler();
handler.service(req, response);
return;
}
The database contains a page, served by the cms library, that loads the resources of the Ext.JS framework from a JAR file on the classpath and points an Ext Grid to the persons REST service.
The result looks like this:
The grid component has a pretty smart implementation. It only loads chunks of 200 entries from the REST service at a time, and not the whole list of 40.000 view entries.
Summary
JavaScript servlets provide a very powerful and advanced feature of the XPages2Eclipse product. While for most developers it will probably be sufficient to just use the functionality provided by the XPages2Eclipse control in the control palette (e.g. call X2E.createConnection() from normal SSJS code), JavaScript servlets provide a solution for scenarios where performance and flexibility of the standard approach are not good enough.