Build servlets in JavaScript

The advanced XPages2Eclipse feature "JavaScript servlets" described in this article works similar to a technique called "XAgents", covered by blog posts of Chris Toohey and Stephan Wissel:

 

IBM Lotus Notes Domino REST Web Services via XPage XAgents

Web Agents XPages style

 

In short, JavaScript servlets as well as XAgents are an XPages version of classic Domino web agents or HTTP servlets:
A piece of code is called via a URL from the web browser and produces content by writing text or binary data into an output stream. For XPages development, this code is written in server side JavaScript (SSJS).

 

The technique is useful when there is a need to programmatically create web content, e.g. web pages in a Domino based Content Management System, Office documents and spreadsheets, captcha graphics for authentication or to implement your own REST service.

 

Motivation

So if there already is such a mechanism for XPages, why did we develop our own?

Well, when working with those XPages based "agents", we faced several issues that our implementation addresses:

 

Call code in other databases

Let's say you have an application that consists of at least two databases, one application database and one that stores configuration data. For a clean architecture, you would like to put all configuration related code into the configuration database, including a REST API that you want to call from client side JavaScript code in the application, e.g. to read configuration data or store user preferences. 

This works fine on the web, because you can request content of other databases without problems as long as the current user has sufficient access rights.

 

But unfortunately it's not possible with XPages in the local Notes Client (XPiNC). There you can only call XPages in the currently open database. You would need to move all your configuration code into the application database to make this work, which messes up your architecture, because that code belongs to the configuration database. It gets even worse when you have more than two databases, like in a CRM system with a filing database, an address book, time recording and other things. You just should not need to duplicate the whole code base to make any API and data accessible throughout the system.

 

The implementation we picked for XPages2Eclipse has a different approach: It uses its own servlet implementation to handle a HTTP request and pass it to your JavaScript or Java code. It supports calling code in other databases (stored locally or on a server), making sure that it can only be called by code in an XPages application and not from a local browser.

 

Include other JavaScript libraries

To improve code reuse and a modular application architecture, we often missed an include statement that includes shared code from another SSJS library. The standard SSJS environment of Lotus Notes/Domino does not provide such a mechanism.

 

In contrast, XPages2Eclipse does support inclusion of other library content. See the advanced section of this article for details.

Parallel execution of requests

Routing many requests through XPages can lead to bad application performance. The reason is that, up to Lotus Notes/Domino 8.5.2, code in an XPage runs sequentially for a single user. So if you let one XPage do a long running operation (e.g. output a large file or process many documents as part of a REST call), other XPages for the same user will wait until the operation is done (for experts: the XPage code is synchronizing on the HttpSession object while your code is invoked).

Our implementation in XPages2Eclipse does not need such a synchronization. Code execution occurs in parallel, only limited by the number of HTTP threads in the web server. You get maximum throughput.

Cache control for developers

As an XPages developer, you probably have experienced the issue that changes you make in Domino Designer sometimes are not visible after a page reload in the browser or Notes Client. The reason is that the XPages runtime caches the code for performance reasons and you often have to do a "restart task http" on your development server (which also restarts the JVM running XPages). This might be ok for web development on a local Domino server, but it's a real pain for XPiNC development. In that case, you have to restart the Notes client including Domino Designer!

 

XPages2Eclipse provides (OSGi) console commands that you can use to clear the caches we use: one for SSJS library content (which we execute for our JavaScript servlet technology) and another one for any custom Java class that your code is using. This makes it easy to develop and test an application in a short amount of time.

 

 

In such a scenario, you could feed your UI elements through your own REST API and also implement the whole application logic as REST calls - a very clean approach, because the UI is completely separated from the backend code and can be replaced at any time.

We support this approach by making an "eclipseconnection" object available as a global variable to your JavaScript code when it is run in the local Notes Client. Use it in your JavaScript code as if you had created the connection by calling X2E.createConnection() in regular SSJS code.

Since we found out that the JavaScript servlet technology would also be very useful for Domino applications on the web and not just locally (even without the "eclipseconnection" object), we decided to add a Notes.ini switch to make the feature work on Domino servers as well.
For security reasons, it's disabled by default and can be enabled globally and on a per database basis.

Javascript servlets are enabled by default in the local Notes client, protected by security mechanisms so that only code coming from an XPages application can invoke them.
As mentioned before, the feature is disabled on a Domino server by default.
You need to add a Notes.ini variable in order to make it work. To enable the feature for a single database with filepath "path/to/database.nsf", simply define the following variable:

 

 

If we cannot find this variable in the server's Notes.ini, we check if the feature is enabled globally with the following line:

 

mndX2ERPCEnabled=true

 

For performance reasons, the result of the variable lookup is cached and not read again until a server restart.

For technical reasons, the URL syntax to access the JavaScript servlet on the Notes client is different than the syntax on a Domino server.

For the Domino server, just append the string "/x2erpc/js" to a database url, followed by the name of a SSJS library and the name of a JavaScript function to call:

 

http://-hostname_of_server-/-path_to_database-/x2rpc/js/-name_of_ssjs_lib-/-name_of_js_function_in_lib-

 

for example:

http://www.mindoo.com/office/address.nsf/x2erpc/js/restapi/service

Unfortunately, in the local Notes client, we had to tweak a bit with the syntax, because the client's servlet engine does not allow the same syntax as on the web:

 

http://127.0.0.1:-dynamic_XPages_Port-/x2erpc/-UTF8_encoded_server-!!-UTF8_encoded_dbpath-/-name_of_ssjs_lib-/-name_of_js_function_in_lib-

 

for example:

http://127.0.0.1:1234/x2erpc/Server1%5cMindoo!!office%5caddress.nsf/restapi/service

 

In addition to the UTF-8 encoding, the "/" in the servername "Server1/Mindoo" was replaced with a "\" (%5c).

 

Sounds complicated?

We provide a small Java class in order to make it easier to generate the right servlet URL syntax in your code:

 

Calling the example URLs above could for example lead to the execution of the following method in the SSJS library "restapi":

 

The method name is the same as specified in the URL, but for security reasons, the text "x2erpc_" is used as a prefix so that only dedicated methods can be called, in this case x2erpc_service.

The argument "req" passed to this method contains the standard HttpServletRequest with the HTTP request data. "response" is a HttpServletResponse which is used to return the result type and data.   

 

Here is another example:
The following XPage contains a link to call such a JavaScript servlet based on a dynamically generated URL:

 

 

This would call the method "x2erpc_pages" in the SSJS library "cms". As usual for any servlet implementation, the additional text "/index.html" in the URL and any query string parameters can be obtained by calling the available methods of the passed "req" object (e.g. getPathInfo(), getQueryString()).

 

Please note that for the last example, we had to tweak a bit by creating an absolute URL (including hostname and port) from the current context URL. This works around the (annoying) behaviour of the xp:link tag to automatically add the current database path to relative URLs when the page is rendered.

 

As mentioned before, the JavaScript servlet implementation is highly optimized for performance and makes heavy use of caching techniques.
Both the SSJS library code and any custom Java classes that are loaded during code execution are cached without being loaded/checked again until the JVM is restarted.

To make development easier, you can use the following OSGi console commands to flush the caches for single databases:

 

Java Classloader content:

 

To enter the OSGi console commands in the Notes Client, you need to start the client in console mode.
Details can be found here.

On the Domino server, the syntax for the OSGi commands is a bit different:

 

tell http osgi x2ejs show      =>  displays a list of cached libraries
tell http osgi x2ejs drop <n>  =>  remove entry number n displayed via the show command
tell http osgi x2ejs drop all  =>  remove all entries from the cache

tell http osgi x2ecl show      =>  displays a list of databases with cached files

tell http osgi x2ecl drop <n>  =>  remove entry number n displayed via the show command

tell http osgi x2ecl drop all  =>  remove all entries from the cache

This section contains additional information about the technical implementation.

Including code from other libraries

By using the following line in your JavaScript code, you can include code from another SSJS library:


Includes are evaluated when the SSJS library content is loaded, not when it is executed. That's why it's not possible to only include a library if a certain condition is true. Actually, the include statement is replaced with a simple string comparison: we trim each SSJS line to remove whitespace characters, check if the line starts with the string include(" and extract the library name of the library to be included.

Available JavaScript language constructs

XPages2Eclipse is using an alternative JavaScript engine (Mozilla's Rhino engine) to execute the JavaScript code and not the one used by the XPages runtime. This is done, because IBM does not provide programmatic access to their engine.
While most of the code will simply work under the Rhino engine, there are a few things that will not work as expected. The following sections will highlight the most relevant things.

Typed variable declaration

Rhino does not support the declaration syntax for strongly typed variables, e.g.



You need to remove the variable type to make the code snippet work in both engines:

Scope access

JavaScript servlets do not run in the context of an XPages application, so there is no access to JSF/XPages scope maps like sessionScope, viewScope, requestScope or applicationScope.
You can however store user related data in the HttpSession retrieved by calling HttpServletRequest#getSession().

Access to Domino data

As in IBM's JavaScript engine, you can access the Domino environment by using the global variables "session" and "database" which we instantiate and pass to your code.

Formula language

IBM's formula language JavaScript contructs are proprietary and therefore not supported when code is executed with the Rhino JavaScript engine. Use the command "session.evaluate()" to get the result of a formula execution.

Using Java classes and resources stored in the NSF

The classloader used by the Rhino engine supports Java classes and resources stored in the NSF database. Just add them to the NSF project's build path as if you would for standard SSJS content.

Security

Now that you know how to enable and use the JavaScript servlet technology, we end this article by discussing the options you have to secure its usage and restrict it to authorized developers.

Execution rights

By default, the JavaScript servlet code runs with plugin privileges (we call it "elevated mode") and code from all signers can be executed. This makes it easy in a test environment to try out all functionality without the need to set up policy control.
For a production environment however, we strongly recommend to use the techniques mentioned in the article "[CodeSignerRestrictionViaPolicy|Use policies to restrict developer use of XPages2Eclipse API]" to restrict code execution to specific trusted code signers.

For JavaScript servlets on a Domino server we provide additional policy preferences that you can use to lower the execution privileges to those of regular servlets in a servlet container. This, for example, prevents a developer from accessing the ClassLoader of classes by calling MyClass.class.getClassLoader().

To do this, add two variables to the managed settings section of the desktop policy document:

Code signature check

If XPages2Eclipse is configured for restricted code execution, the JavaScript servlet's SSJS library, all included libraries, Java classes and resources are checked to verify that the design elements' signature matches the list of authorized code signers. Code execution and resource loading will fail with an exception if any used code/resource has an unknown signature.

Summary

JavaScript servlets provide a very powerful and advanced feature of the XPages2Eclipse product. While for most developers it will probably be sufficient to just use the functionality provided by the XPages2Eclipse control in the control palette (e.g. call X2E.createConnection() from normal SSJS code), JavaScript servlets provide a solution for scenarios where performance and flexibility of the standard approach are not good enough.