JChem Neo4j Cartridge

    {warning} JChem Neo4j Cartridge has been discontinued. Last released version is 20.7.0.

    Installation

    Software requirements

    • Windows (64-bit), Linux

    • Java 11

    • Neo4j 4.4+

    Download

    Download package available upon request. Please use https://chemaxon.freshdesk.com/ or support@chemaxon.com channels for requesting download package.

    Install on Linux

    Choose one of the installers

    • rpm and deb packages unpack JNC to/opt/jnc

    • sh installer provides an interactive installer with graphical user interface or with a command line interface (this can be forced with the -c switch) this installer unpacks by default to /usr/local/jnc

    • Or simply unpack the tar.gz to a desired directory

    We will refer these directories as JNC_DIR in the document.

    Neo4j plugin

    Copy the neo4j plugin to the neo4j plugin directory (copy <JNC_DIR>/plugin/neo4j-jchem-cartridge.jar to <NEO4J_INSTALL_DIR>/plugins/ ) - the sh installer does it automatically. Restart neo4j.

    Licenses

    Copy your Chemaxon JChem Neo4j license (and if you wish to use custom standardization then the Standardizer license as well) to <CHEMAXON_HOME> according to this document.

    If you have installed the jnc application as root, then you need to copy the license file content to /root/.chemaxon/license.cxl

    Start neo4j backend application

    If you have installed the backend application as root then you need to start the service as root as well. In this case use sudo to execute the following programs.

    • You can use <JNC_DIR>/bin/run-jnc script to run the cartridge backend as simple program

    • You can use script <JNC_DIR>/bin/jnc-service script to run cartridge backend as service (the commands it takes: start|stop|restart|status )

    If any error occur the log files are available at <JNC_DIR>/logs directory.

    Install on Windows

    Choose one of the installers

    • exe installer gives you a nice gui to use that installs to <Program Files>/jnc

    • zip is a simple file to unzip wherever you want

    These directories will be called JNC_DIR in the document.

    Neo4j plugin

    Copy the neo4j plugin to the neo4j plugin directory (copy <JNC_DIR>/plugin/neo4j-jchem-cartridge.jar to <NEO4J_INSTALL_DIR>/plugins/ ) - the exe installer does it automatically. Restart neo4j

    Licenses

    Copy your Chemaxon license to <CHEMAXON_HOME> according to this document.

    Start neo4j backend application

    • You can use <JNC_DIR>\bin\run-jnc.exe to run the cartridge backend as a simple program

    • You can use script <JNC_DIR>\bin\jnc-service.exe script to run cartridge backend as service. For this you first needs to install it with --install switch (you can uninstall with --uninstall ), than you can start it with --start (stop with --stop ) To check its state, use the --status switch and run it in the foreground with --run.

    Configuration

    1. Configuring runnables

    You can find two runnable files (jnc-service and run-jnc). Both has their .vmoptions file where you can set parameters for virtual machines. Example content:

    -Xmx4g

    -Xms1g

    -server

    -XX:-UseConcMarkSweepGC

    Adding the above lines would make the Java Virtual Machine to run in server mode, use Concurrent Mark Sweep algorithm for garbage collection and use 1 GB as minimal Heap and 4 GB as maximal heap.

    Find more information here.

    2. Settings of the server

    You can set options for the server in config/application.properties like the port to use, and where to look for settings, etc. You can find general settings here.

    And you can set the following Chemaxon specific settings:

    initOnStart : Should the database initialised on starting the application? Available values: INIT (only new database can be created), OPEN (Only existing databases can be opened), AUTO (exisiting DBs are opened, non existing ones are created.

    updateMode : updateMode gets in action only if the version number has changed. Available values: EXIT (exit if version mode has changed), DROP (drop old data if version mode has changed), REINDEX (keep old data and reindex them in order to work with new version), FORCE_REINDEX (keep old data and forces reindexing at the start of the service)

    jws.db.settings : path to the settings file where additional data are stored

    db.configPath : path to the storage settings

    3. Settings in the database

    A jnc.properties json file is created in the graph.db database folder. Some options can be changed only manually in the file, some options can be manipualted by API functions. Please, be aware that if you copy the database file, the jnc.properties file is travelling with them, because it's in the same directory, this can be an advantage and a disadvantage depending on your workflow.

    Logging

    Some logs are available in neo4j "logs" directory.

    Dbms logging level can be configured in conf/neo4j.conf file. Dbms logging provides information for monitoring purposes. Setting it to level DEBUG, you get detailed information about the processes running at the database.

    dbms.logs.debug.level=DEBUG

    One particularly useful information is the not indexed node ids which are not used during chemical search.

    In the following log message we can see that the node with id 4 is not part of the index while nodes with ids 1,2,3,5 are indexed successfully. (Please note that the fields failedIds and successfulIds are providing information about neo4j database node IDs.)

    2022-05-04 ... DEBUG [...] createdb called with db name test, type sample
    2022-05-04 ... DEBUG [...] {"failedIds":[4],"failedIndexes":[3],"duplicatedIndexes":[],"successfulIds":[1,2,3,5],"successfulIndexes":[0,1,2,4],"duplicatedIds":[],"insertSucce
    ss":[true,true,true,false,true]}

    One can retrieve the problematic node from the database base on the provided id using the following cypher statement:

    MATCH (n) WHERE id(n) = 4 RETURN n;

    The storage server logs to <JNC_DIR>/logs/ folder.

    To enable index creation and search related debug logging, uncomment the following line in <JNC_DIR>config/application.properties

    logging.level.com.chemaxon.jchem=DEBUG

    API Usage

    To be able to use chemical searching functionality you need to create a database using jchem plugin. The database name is called test in all of the following examples, but you can of course use custom name.

    Create a db

    
    call jchem.createdb('test','sample')

    where test is the db name and sample is the db type. Db type defines the business rules and it is (should be) defined in the <JNC_DIR>/config/application.properties file.

    An example sample.type is provided. This type can be copied to a different name and modified if necessary (be sure to provide different typeId for each type).

    Drop a db

    
    call jchem.dropdb('test')

    Get a list of dbs

    
    call jchem.dbs()

    Returns a map where the key is the name of the dbs, the value is the given type.

    Create trigger

    
    call jchem.createTrigger('test', 'molecule', 'molString')

    where test is the db name, molecule is the node label and molString is the property.

    This creates a trigger for node with molecule label and indexing molString property to the test database.

    Drop trigger

    
    call jchem.dropTrigger('test', 'molecule', 'molString')

    where test is the db name, molecule is the node label and molString is the property.

    This drops a trigger provided for node with molecule label which indexing molString property to the test database.

    Get all triggers

    
    call jchem.triggers()

    This call lists all the registered triggers.

    Clean up your db

    Delete all molecule nodes.

    
    MATCH (n:molecule) DETACH DELETE n

    Create chemical nodes

    
    create (n:molecule { molString : 'CCC'})

    Delete node

    
    match (n:molecule { molString : 'CCC'}) delete n

    if you get memory exception

    
    match (n:molecule { molString : 'CCC'}) WITH n LIMIT 100000 delete n;

    Index your nodes

    The molString property contains the structures.

    If you have created triggers (see create trigger API) for the specific database name and molString for the structure property, then adding nodes to the db is automatically triggered, so you won't need this step.

    Otherwise you need to do this call to add molecules to the db:

    
    match (n) with collect(n) as nodes call jchem.addBatch('db_name',nodes,'molString') yield responseCode return responseCode

    You may face a situation when not all compound nodes have the specific property to index. In this case you need to specify only that nodes which has the property to be indexed. So to index all Compound nodes which has molString property, call:

    
    match (n:Compound) WHERE EXISTS(n.molString) with collect(n) as nodes call jchem.addBatch('db_name',nodes,'molString') yield responseCode return responseCode

    Remove nodes from database

    Delete molecules from the database:

    
    match (n) with collect(n) as nodes call jchem.deleteBatch('db_name',nodes) yield responseCode return responseCode

    Substructure search with hit count limit

    
    call jchem.search('db_name','query_string',hit_limit)

    E.g. search for benzene and get the first 10 most relevant hit:

    
    call jchem.search('test','c1ccccc1',10)

    Similarity search with hit count limit

    
    call jchem.search('db_name','query_string',hit_limit, 'sim', sim_threshold)

    E.g. search for 10 most similar compounds to cyclohexane which are over 0.5 similarity threshold. The hits are ordered by similarity:

    
    call jchem.search('test','C1CCCCC1',10, 'sim', 0.5)

    Duplicate search with hit count limit

    
    call jchem.search('db_name','query_string', hit_limit, 'dup') yield node return node

    Utility methods

    Get the current settings:

    
    call jchem.settings()

    Check if a given molecule source is valid (importable):

    
    match (n:molecule) call jchem.canImport(n, 'molString') yield node, booleanValue return node.molString, booleanValue

    Filter nodes with invalid molecule source:

    
    match (n:molecule) with collect(n) as nodes call jchem.filterIncorrectStructures(nodes, 'molString') yield node return node

    Examples

    Simple test

    Create 3 nodes:

    
    create (n:molecule { molString : 'CCC'})
    create (n:molecule { molString : 'C1CCCCC1'})
    create (n:molecule { molString : 'c1ccccc1'})

    Create db:

    
    call jchem.createdb('test','sample')

    Add nodes to db:

    
    match (n) with collect(n) as nodes call jchem.addBatch('test',nodes,'molString') yield responseCode return responseCode

    Search:

    
    call jchem.search('test','c1ccccc1',10)

    Delete nodes from index:

    
    match (n) with collect(n) as nodes call jchem.deleteBatch('test',nodes) yield responseCode return responseCode

    Import your nodes by copying a csv file

    Assume you have example1.csv in the neo4j import directory (/var/lib/neo4j/import/) containing:

    
    CC1=CC(=O)C=CC1=O,1
    S(SC1=NC2=CC=CC=C2S1)C3=NC4=C(S3)C=CC=C4,2
    OC1=C(Cl)C=C(C=C1[N+](o)=O)[N+](o)=O,3
    [O-][N+](o)C1=CNC(=N)S1,4
    NC1=CC2=C(C=C1)C(=O)C3=C(C=CC=C3)C2=O,5

    You can load the csv file as:

    
    LOAD CSV FROM 'file:///example1.csv' AS line
    CREATE (n:molecule { molString: line[0] })

    Or if you have an example2_headers.csv file (in the neo4j import directory as well) with molString and cd_id headers like:

    
    cd_id,molString
    1,CCC
    2,c1ccccc1
    3,C1CCCCC1

    You can load the csv file with header as:

    
    LOAD CSV WITH HEADERS FROM 'file:///example2_headers.csv' AS row
    CREATE (n:molecule)
    SET n = row, n.cd_id = toInteger(row.cd_id), n.molString = row.molString

    Create db, add and delete nodes:

    
    call jchem.createdb('test','sample')
    match (n) with collect(n) as nodes call jchem.addBatch('test',nodes,'molString') yield responseCode return responseCode
    match (n) with collect(n) as nodes call jchem.deleteBatch('test',nodes) yield responseCode return responseCode

    Chemical Search

    Substructure searching for three consecutive carbon atom with maximum 10 hits:

    
    call jchem.search('test','CCC',10)

    Returns

    "node"
    {"cd_id":1,"molString":"CCC"}
    {"cd_id":3,"molString":"C1CCCCC1"}
    {"molString":"CC1=CC(=O)C=CC1=O"}
    {"molString":"NC1=CC2=C(C=C1)C(=O)C3=C(C=CC=C3)C2=O"}

    Similarity search with cyclohexane and limiting the hit count:

    
    call jchem.search('test','C1CCCCC1',10, 'sim', 0.1)

    Returns

    "node"
    {"cd_id":3,"molString":"C1CCCCC1"}
    {"cd_id":1,"molString":"CCC"}
    {"cd_id":2,"molString":"c1ccccc1"}
    {"molString":"CC1=CC(=O)C=CC1=O"}

    The previously provided very simple chemical structure set is not enough for the following examples. Please use a larger dataset which contains molecule pairs with similarity values over 0.9, 0.8, 0.5. If you have more than 100K molecule nodes, then run these statements in batches (50-100K node at once) to avoid memory problems.

    Delete all relationships

    
    start r=relationship(*) delete r;

    Add relationship between nodes with different similarities with only one relationship between nodes

    
    match (n) call jchem.search('test',n.molString,0,'sim',0.9) yield node where n<>node AND not exists ((n)-[]->(node)) create (n)-[r:SIM_90]->(node)
    match (n) call jchem.search('test',n.molString,0,'sim',0.8) yield node where n<>node AND not exists ((n)-[]->(node)) create (n)-[r:SIM_80]->(node)
    match (n) call jchem.search('test',n.molString,0,'sim',0.5) yield node where n<>node AND not exists ((n)-[]->(node)) create (n)-[r:SIM_50]->(node)

    Add relationship between nodes with different similarities with multiple relationships between nodes

    
    match (n) call jchem.search('test',n.molString,0,'sim',0.9) yield node where n<>node create (n)-[r:SIM_90]->(node)
    match (n) call jchem.search('test',n.molString,0,'sim',0.8) yield node where n<>node create (n)-[r:SIM_80]->(node)
    match (n) call jchem.search('test',n.molString,0,'sim',0.5) yield node where n<>node create (n)-[r:SIM_50]->(node)

    Find the most relevant substructure hits over 0.9 similarity

    
    call jchem.search('test','C1CCCCC1',5) yield node with node MATCH (node)-[r:SIM_90]-(n) RETURN n,r,node;