Mindoo Blog - Cutting edge technologies - About Java, Lotus Notes and iPhone

  • New C API method NIFFindByKeyExtended2 in Domino 9.0 improves view lookups

    Karsten Lehmann  6 March 2015 14:48:54
    Once again I am digging into the Notes/Domino C API to implement fast and powerful view lookups. What caused it this time was a customer requirement for "Notes Client style keyboard navigation" in web views, meaning that you type a character, press enter and the cursor position should be moved to the relevant area of the view (e.g. where the content in the first sorted view column is greater or equal letter "L").
    Using standard Java APIs for this purpose (ViewNavigator.getNextSibling(), until we find a greater column value) was much to slow, since the view had about 40,000 entries in the top level.

    We did a lot of experiments in this area in the past (1, 2) writing custom DLLs for Windows/.so libraries for Linux, but this time my plan was to use Java Native Access (JNA) from an Eclipse plugin to build the code more quickly.

    I will try to publish some tool libraries in this area in the near future (as always I am very busy...). But in this post, I just want to document a method that is available in the C API since 9.0 and has not been documented in the C API tookit since then.

    The method is called NIFFindByKeyExtended2 (the C API release notes say NIFFindByKeyExtented2) and is only mentioned a few times on the web, e.g. in Scott Souder's C API toolkit announcement post.

    Here is the method signature:

    STATUS far PASCAL NIFFindByKeyExtended2 (HCOLLECTION hCollection,
                    void *KeyBuffer, DWORD FindFlags,
                    DWORD ReturnFlags,
                    COLLECTIONPOSITION *retIndexPos,
                    DWORD *retNumMatches,
                    WORD *retSignalFlags,
                    DHANDLE *rethBuffer,
                    DWORD *retSequence)


    I asked IBM dev about the method and got the information that it combines NIFFindByKey and NIFReadEntries in one call. In earlier Domino releases, you had to use NIFFindByKey to get the view entry position for a search key and then use NIFReadEntries to read the view data. Unfortunately the view could have changed in the meantime so that the postion could point to a different location.

    This is not the case for NIFFindByKeyExtended2, which is executed atomically with a read lock on the view. For remote calls, the returned data can have up to 64 KB of content. If more data is required, subsequent calls to NIFReadEntries have to be made. For local calls, the main use case for XPages apps, the buffer has no limit.

    Here is the complete method documentation:


    /*        NIFFindByKeyExtended2 - Lookup index entry by "key"
    *
    *        Given a "key" buffer in the standard format of a summary buffer,
    *        locate the entry which matches the given key(s).  Supply as many
    *        "key" summary items as required to correspond to the way the index
    *        collates, in order to uniquely find an entry.
    *
    *        If multiple index entries match the specified key (especially if
    *        not enough key items were specified), then the index position of
    *        the FIRST matching entry is returned ("first" is defined by the
    *        entry which collates before all others in the collated index).
    *
    *        Note that the more explicitly an entry can be specified (by
    *        specifying as many keys as possible), then the faster the lookup
    *        can be performed, since the "key" lookup is very fast, but a
    *        sequential search is performed to locate the "first" entry when
    *        multiple entries match.
    *
    *        This routine can only be used when dealing with notes that do not
    *        have multiple permutations, and cannot be used to locate response
    *        notes.
    *
    * Inputs:
    *
    *        hCollection = Handle of per-user collection context
    *        KeyBuffer = Address of "key" summary buffer
    *        FindFlags = Find flags word (FIND_xxx)
    *        ReturnMask = Mask specifying what information is to be returned on each entry
    *                                        (READ_MASK_xxx).  Information is always returned in the
    *                                        order of the bits which are 1, from bit 0 to bit 15.
    *        retIndexPos = Place to receive index position of (first) matching entry
    *        retNumMatches = Place to receive # entries which match (optional)
    *        retSignalFlags = Place to receive extra sharing warning flags (optional)
    *
    * Outputs:
    *
    *        (Routine) = Error status (ERR_NOT_FOUND if none found at all)
    *        *retIndexPos = Index position of (first) matching index entry.
    *        *retNumMatches = # matching entries
    *        *rethBuffer = Handle of return buffer (may be NULLHANDLE)
    *        *retBufferLength = Length of return buffer
    *        *retSequence = Index Modified Sequence number
    */


    Based on searches on the internet, there also seems to be NIFFindByKeyExtended3, but I don't have information about parameters and purpose for this method yet.


    Comments

    1Nathan T. Freeman  07.03.2015 1:21:18  New C API method NIFFindByKeyExtended2 in Domino 9.0 improves view lookups

    " What caused it this time was a customer requirement for "Notes Client style keyboard navigation" in web views, meaning that you type a character, press enter and the cursor position should be moved to the relevant area of the view (e.g. where the content in the first sorted view column is greater or equal letter "L").

    Using standard Java APIs for this purpose (ViewNavigator.getNextSibling(), until we find a greater column value) was much to slow, since the view had about 40,000 entries in the top level. "

    I'm confused. Why wouldn't you use View.getEntryByKey("L", false) and then move the ViewNavigator to that entry?

    This not to slight this very cool work. It just seems like a simpler path to me and in my experience is quite performant.

    2Karsten Lehmann  07.03.2015 8:17:08  New C API method NIFFindByKeyExtended2 in Domino 9.0 improves view lookups

    It's possible that the search string does not exist in the view, e.g. only the letters "m" and "o". Or I search for "Rad Pill" instead of "Red Pill".

    3Karsten Lehmann  07.03.2015 11:23:44  New C API method NIFFindByKeyExtended2 in Domino 9.0 improves view lookups

    As an additional information, IBM is using the new C method internally in the Notes Java API for View.getDocumentByKey(). So that method should be a bit faster in R9.

    Unfortunately, the greater/less than key X search is not available yet publicly in the Notes Java API (I asked for that 2 years ago). There's a hidden undocumented call in the XPages runtime though that they use for the XPages address picker dialog.

    4Karsten Lehmann  17.03.2015 12:55:55  New C API method NIFFindByKeyExtended2 in Domino 9.0 improves view lookups

    I think this article covers the fact that IBM is using the new C API method for local view lookups:

    { Link }

    5KL  26.05.2015 18:26:45  New C API method NIFFindByKeyExtended2 in Domino 9.0 improves view lookups