DB Web Services

DB Web Services provide methods for storing and searching chemical structures in a persistent database, currently in MapDB. MapDB works with collections, but in this documentation the terminology table is used instead of collection.

There are methods provided for

  • creating / deleting tables

  • inserting / deleting / modifying structures and data in the tables

  • executing duplicate, substructure, full fragment and similarity searches

This documentation describes installation, administration and usage of DB Web Services.

DB Web Services make possible to reach the chemical searching functionality provided by ChemAxon's products JChem Base, JChem Choral, JChem PostgreSQL Cartridge, JChem Oracle Cartridge.

DB Web Services application is available in two modes:

  1. As part of a microservices system

  2. As standalone web application

Microservices system mode

In microservices system mode the DB Web Service runs together with Config service, Discovery service and Gateway service. These three services are mandatory and optionally other services can also be part of the system. All configuration must be done in the Config module.

The default configuration applies to the microservices system mode.

The web application runs on host <server-host> and listens on port <gateway-server-port>.

Standalone web application mode

In standalone web application mode the DB Web Service runs alone, without the Config service, Discovery service and Gateway service (however the installer installs them, as well).

The default configuration must be changed according to the standalone web application mode; set

eureka.client.enabled=false in application.properties file, and

spring.cloud.config.failFast=false ,

spring.cloud.config.enabled=false and

spring.cloud.config.uri= (so to empty) in bootstrap.properties file.

All configuration must be done in the DB module.

The web application runs on host <server-host> and listens on port <server-port>.

Download

See here.

Software requirements

See here

Installation

See here.

Module is installed into folder: jws/jws-db

Licenses

See here.

Additionally, IUPAC naming plugin license is required for the usage of Name file format.

Configuration

Default configuration:

application.properties

server.port=8062

logging.file=../logs/jws-db.log

eureka.client.enabled=true

set eureka.client.enabled=false to switch to standalone DB Web Services application mode

initOnStart=AUTO

initOnStart can be INIT|AUTO|OPEN
INIT: the existing database is deleted, and a new empty one is created
AUTO: existing database is started, in case of non-existing database a new empty one is created
OPEN: existing database is started, in case of non-existing database error is thrown.

updateMode=EXIT

updateMode gets in action only if the version number has changed
updateMode can be EXIT|DROP|REINDEX
EXIT: exit if version mode has changed
DROP: drop old data if version mode has changed
REINDEX: keep old data and reindex them in order to work with new version
FORCE_REINDEX: keep old data and forces reindexing at the start of the service

search.wallTimeLimitSeconds=3600
com.chemaxon.zetor.settings.indexDir=data/chemical-data/store
com.chemaxon.zetor.settings.scheme=mapdb
com.chemaxon.zetor.settings.forcePurge=true

com.chemaxon.zetor.types[0].version = 1
com.chemaxon.zetor.types[0].typeName = sample
com.chemaxon.zetor.types[0].typeId = 1
com.chemaxon.zetor.types[0].tautomerHandlingMode = OFF
com.chemaxon.zetor.types[0].standardizerConfig = aromatize

com.chemaxon.zetor.types[1].version = 1
com.chemaxon.zetor.types[1].typeName = taumol
com.chemaxon.zetor.types[1].typeId = 2
com.chemaxon.zetor.types[1].tautomerHandlingMode = GENERIC
com.chemaxon.zetor.types[1].standardizerConfig = aromatize

Here are the molecule types defined. You can delete, modify or add new molecules types.

version must be 1
typeName must be unique
typeID must be unique integer
tautomerHandlingMode can be OFF or GENERIC or CANONIC_GENERIC_HYBRID (available from version 20.10)
standardizerConfig can be made of action strings or the content of a standardizer configuration xml file
standardizerConfig=aromatize:b..removeExplicitH

Important: Indexing of the types array must always be sequential 0, 1, 2, ....

All changes take effect only if initOnStart is set to INIT, and the application is re-started. Take care, the existing database will be deleted.

com.chemaxon.zetor.additional.scheme=mapdb

com.chemaxon.zetor.additional.indexDir=data/extra-data/

Performance tuning can be executed by doing some cache related settings. Please use our JChem Engines cache and memory calculator page (take into account that superstructure search is not available) to calculate the appropriate settings.

All properties calculated by the calculator must be copied into the application.properties file (not into the file specified on the calculator page) like:

com.chemaxon.zetor.settings.label.cachePolicy=DISABLED
com.chemaxon.zetor.settings.label.cachedObjectCount=0
com.chemaxon.zetor.settings.molecule.cachePolicy=DISABLED
com.chemaxon.zetor.settings.molecule.cachedObjectCount=0
com.chemaxon.zetor.settings.fingerprint.cachedObjectCount=1320000

The infix '.runtime' in keys was used in version 19.10 and earlier:

com.chemaxon.zetor.settings.runtime.label.cachePolicy=DISABLED
com.chemaxon.zetor.settings.runtime.label.cachedObjectCount=0
com.chemaxon.zetor.settings.runtime.molecule.cachePolicy=DISABLED
com.chemaxon.zetor.settings.runtime.molecule.cachedObjectCount=0
com.chemaxon.zetor.settings.runtime.fingerprint.cachedObjectCount=1320000

bootstrap.properties

spring.cloud.config.failFast=true
spring.cloud.config.uri=${CONFIG_SERVER_URI:http\://localhost\:8888/}
spring.cloud.config.retry.initialInterval=3000
spring.cloud.config.retry.multiplier=1.2
spring.cloud.config.retry.maxInterval=60000
spring.cloud.config.retry.maxAttempts=100

For more settings possibilities see spring documentation page.

Search logging

Debug level of search logging can be set in the files configuring the JVM options:

jws-db-service.vmoptions
run-jws-db.vmoptions

by adding line

-Djchem.debug=true

High Availability (HA)

HA and load balancing is provided for DB Web Services, for the only stateful web service in JChem Microservices.

Running more instances of the db service ensures HA and load balancing.

In HA mode, Hazelcast is used for distributed caching of the data.
It can be optionally set that all nodes present in the system have their own cache ('Near' cache). By default, near cache is switched on.

Features when near cache is switched on:

  • quicker structure search

  • increased memory usage

  • in the case of data update, the system eventually becomes inconsistent; it can take a few seconds to get back again consistent.

Requirements

HA mode needs PostgreSQL database installed, user and database created

Configure HA mode

HA mode must be configure in /jws-config/common-config/application.properties file

Example configuration:

/jws-config/common-config/application.properties

com.chemaxon.zetor.settings.indexDir=data/chemical-data/store
com.chemaxon.zetor.settings.scheme=crdb
com.chemaxon.zetor.settings.forcePurge=true
com.chemaxon.zetor.settings.crdb.sqlBuilderProvider=POSTGRESQL
com.chemaxon.zetor.settings.crdb.jdbcUrl=jdbc:postgresql://localhost:5432/zetor
com.chemaxon.zetor.settings.crdb.user=chemaxon
com.chemaxon.zetor.settings.crdb.password=chemaxon

com.chemaxon.zetor.additional.scheme=crdb
com.chemaxon.zetor.additional.indexDir=data/extra-data/
com.chemaxon.zetor.additional.crdb.sqlBuilderProvider=POSTGRESQL
com.chemaxon.zetor.additional.crdb.jdbcUrl=jdbc:postgresql://localhost:5432/zetor
com.chemaxon.zetor.additional.crdb.user=chemaxon
com.chemaxon.zetor.additional.crdb.password=chemaxon

Load balanced example

Here you find a load balanced example application on GitHub.

Running the server

Prerequisites in case of microservices system mode:

  1. Config service is running

  2. Discovery service is running

  3. Gateway service is running


Run the service in command line in folder jws/jws-db/:

jws-db-service --start (on Windows)

jws- db -service start (on Linux)

or

run-jws- db .exe (on Windows)

run-jws- db (on Linux)

API Documentation

Find and try out the API on the Swagger UI.

Mode

URL of Swagger UI

default URL of Swagger UI

microservices system

<serverhost>:<gateway-port>/jwsdb/API/

localhost:8080/jwsdb/API/

standalone web application mode

<serverhost>:<server-port>/API/

localhost:8062/API/

Demo site

For detailed description check out the JWS DB demo site:

https://jchem-microservices.chemaxon.com/jwsdb/api/index.html

Usage

The guidelines, examples on the Demo site or on the Swagger UI API documentation of your installed module display the methods and syntax implemented for reaching the essential chemical searching functionalities of JChem Base.

Molecule type information

DB Web Services provides method for getting the available molecule types.

Every table has a Molecule type: this is a descriptor that is used by the search engine. It contains information about how structures are handled during search. The application has two very simple built in types called: sample (search with aromatization) and taumol (tautomer search). See the type definitions in the application.properties file.

Store and search molecules and non-chemical data

Table operations

Structure Insert/Delete methods

Duplicate search methods

Substructure search methods

Similarity search methods

Timeout

From version 19.25 the default timeout ( hystrix.command.default.execution.isolation.thread.timeoutInMilliseconds ) is taken into account during indexing. If an indexing job is started already and timeout occurs, then in the background the indexing continues. and the user is informed in the message of the error response about it: "this process has started already will finish in the background" . If the process was only waiting for other indexing processes to finish, then "this process was cancelled" message appears in the error message.

The default timeout is 25000 milliseconds.

We stop the process slightly before the end of the timeout (default: 300 milliseconds, can be set with zetor.indexing.timeoutDifferenceInMilliseconds property) so we have a chance to provide the answer correctly.

Data recovery

The data apparently stored in tables (collections) are stored in the file system under jws-db/data/ folder. The content of that folder must be stored as backup. Furthermore, the application.properties file(s) also should be saved.