Page tree


DB Web Services application provides methods for storing and searching chemical structures in a persistent database, currently in MapDB. MapDB works with collections, but in this documentation the terminology table is used instead of collection.


There are methods provided for

  • creating / deleting tables
  • inserting / deleting / modifying structures and data in the tables
  • executing duplicate, substructure and similarity searches

This documentation describes installation, administration and usage of DB Web Services.

DB Web Service is available in two modes:

  1. As part of a microservices system
  2. As standalone web application

Microservices system mode

In microservices system mode the DB Web Service runs together with Config service, Discovery service and Gateway service. These three services are mandatory and optionally other services can also be part of the system. All configuration must be done in the Config module.

The default configuration applies to the microservices system mode.

The web application runs on host <server-host> and listens on port <gateway-server-port>.

Standalone web application mode

In standalone web application mode the DB Web Service runs alone, without the Config service, Discovery service and Gateway service (however the installer installs them, as well).

The default configuration must be changed according to the standalone web application mode; set eureka.client.enabled=false in application.properties file and spring.cloud.config.failFast=false and spring.cloud.config.uri= (so to empty) in bootstrap.properties file.

All configuration must be done in the DB module.

The web application runs on host <server-host> and listens on port <server-port>.

Download

See here.

Software requirements

See here

Installation

See here.

Module is installed into folder: jws/jws-db

Licenses

See here.

Additionally, IUPAC naming plugin license is required for the usage of Name file format.

Configuration

Default configuration:

application.properties
server.port=8062
logging.file=../logs/jws-db.log

eureka.client.enabled=true

set eureka.client.enabled=false to switch to standalone DB Web Services application mode 
initOnStart=AUTO

initOnStart can be INIT|AUTO|OPEN

INIT: the existing database is deleted, and a new empty one is created

AUTO: existing database is started, in case of non-existing database a new empty one is created

OPEN: existing database is started, in case of non-existing database error is thrown.

updateMode=EXIT

updateMode gets in action only if the version number has changed

updateMode can be EXIT|DROP|REINDEX

EXIT: exit if version mode has changed

DROP: drop old data if version mode has changed

REINDEX: keep old data and reindex them in order to work with new version

search.wallTimeLimitSeconds=3600


com.chemaxon.zetor.settings.indexDir=data/chemical-data/store

com.chemaxon.zetor.settings.scheme=mapdb

com.chemaxon.zetor.settings.forcePurge=true







com.chemaxon.zetor.types[0].version = 1

com.chemaxon.zetor.types[0].typeName = sample

com.chemaxon.zetor.types[0].typeId = 1

com.chemaxon.zetor.types[0].tautomerHandlingMode = OFF

com.chemaxon.zetor.types[0].standardizerConfig = aromatize


com.chemaxon.zetor.types[1].version = 1

com.chemaxon.zetor.types[1].typeName = taumol

com.chemaxon.zetor.types[1].typeId = 2

com.chemaxon.zetor.types[1].tautomerHandlingMode = GENERIC

com.chemaxon.zetor.types[1].standardizerConfig = aromatize

Here are the molecule types defined. You can delete, modify or add new molecules types.

version must be 1

typeName must be unique

typeID must be unique integer

tautomerHandlingMode can be OFF or GENERIC

standardizerConfig can be made of action strings or the content of a standardizer configuration xml file

standardizerConfig=aromatize:b..removeExplicitH


Important: Indexing of the types array must always be sequential 0, 1, 2, ....


All changes take effect only if initOnStart is set to INIT, and the application is re-started. Take care, the existing database will be deleted.

com.chemaxon.zetor.additional.scheme=mapdb

com.chemaxon.zetor.additional.indexDir=data/extra-data/


Performance tuning can be executed by doing some cache related settings. Please use our JChem PostgreSQL Cartridge Xmx Calculator page (take into account that superstructure search is not available) to calculate the appropriate settings.

All properties calculated by the Xmx calculator must be copied into the application.properties file (not into the file specified on the calculator page) and the prefix 'com.chemaxon.jchem.psql'  in keys must be substituted with prefix 'com.chemaxon.zetor.settings' like

com.chemaxon.zetor.settings.runtime.label.cachePolicy=DISABLED

com.chemaxon.zetor.settings.runtime.label.cachedObjectCount=0

com.chemaxon.zetor.settings.runtime.molecule.cachePolicy=DISABLED

com.chemaxon.zetor.settings.runtime.molecule.cachedObjectCount=0

com.chemaxon.zetor.settings.runtime.fingerprint.cachedObjectCount=1320000

bootstrap.properties

spring.cloud.config.failFast=true

spring.cloud.config.uri=${CONFIG_SERVER_URI:http\://localhost\:8888/}

spring.cloud.config.retry.initialInterval=3000

spring.cloud.config.retry.multiplier=1.2

spring.cloud.config.retry.maxInterval=60000

spring.cloud.config.retry.maxAttempts=100

For more settings possibilities see spring documentation page.

Running the server

Prerequisites in case of microservices system mode:


  1. Config service is running
  2. Discovery service is running
  3. Gateway service is running


Run the service in command line in folder jws/jws-db:

jws-db-service --start (on Windows)

jws-db-service start (on Linux)

or

run-jws-db.exe (on Windows)

run-jws-db (on Linux)

API Documentation

Find and try out the API on the Swagger UI.

ModeURL of Swagger UIdefault URL of Swagger UI
microservices system<serverhost>:<gateway-port>/jwsdb/API/ localhost:8080/jwsdb/API/
standalone web application mode<serverhost>:<server-port>/API/localhost:8062/API/

Demo site

For detailed description check out the JWS DB demo site:

https://jchem-microservices.chemaxon.com/jwsdb/api/index.html

Usage

Molecule type information

DB Web Services provides the following methods for getting the available molecule types.

Every table has a Molecule type: this is a descriptor that is used by the search engine. It contains information about how structures are handled during search. The application has two very simple built in types called: sample (search with aromatization) and taumol (tautomer search).

MethodDescriptionExample Response
GET/rest-v1/moleculeType
This service lists the available molecule types.
See how to create new molecule types.
[
 { 
  "name": "sample", 
  "typeId": 1, 
  "tautomerHandlingMode": "OFF", 
  "standardizerConfig": "aromatize", 
  "fpLengthInBits": 512, 
  "fpEdges": 7, 
  "fpOnes": 2, 
  "fpPageSize": 1250, 
  "querySegmentStored": false, 
  "targetSegmentStored": true 
 } 
]

Store and search molecules and non-chemical data

Table operations

MethodDescriptionExample use caseExample requestExample response
GET/rest-v1/db/additional/tablesLists the names of the existing tables and their molecules types
/rest-v1/db/additional/tables
[
{
"tableName": "demoTable",
"moleculeType": "sample"
}
]
POST/rest-v1/db/additional/createTable/{tableName}Creates a new table (of molecule type provided by parameter typeName)Create a new table name MYTABLE which uses SAMPLE molecule type/rest-v1/db/additional/createTable/mytable?typeName=sample
DELETE/rest-v1/db/additional/dropTable/{tableName}Drops the specified tableDrop table named MYTABLE/rest-v1/db/additional/createTable/mytable
POST/rest-v1/db/additional/{tableName}/importFromFile/{fileName}

Imports all data from the file, to the table. The file can be a reference to the .json, or a .zip archive, where first item is the .json

The folder where the file to be imported is found and the batch size of the import can be defined in application.properties file.

Default folder:

com.chemaxon.webservices.db.import_export.dir=data/export

Default batch size:

com.chemaxon.webservices.db.import_export.importBatchSize=5000

Import data from myfile.json to MYTABLE.




/rest-v1/db/additional/mytable/importFromFile/myfile.json
POST/rest-v1/db/additional/{tableName}/exportToFile

Exports all data in the table to a Json or zip  file on the file system.

The folder where the exported file is saved can be defined in the application.properties file.

Default folder:

com.chemaxon.webservices.db.import_export.dir=data/export

Exports all data in table MYTABLE to a json file in the file system.

/rest-v1/db/additional/mytable/exportToFile
C:\Program Files\ChemAxon\JChem Microservices\jws-db\data\export\jwsexport-mytable-20181205-1429.json


Structure Insert/Delete methods

MethodDescriptionExample use caseExample requestExample response
POST/rest-v1/db/additional/{tableName}/{id}Inserts one molecule with specified unique ID and with its additional data into the specified table. Uses JSON format.
/rest-v1/db/additional/mytable/1
{ 
"id": 1,
"molecule": "CCO",
"additionalData":
{
"name": "etanol",
"mass": "46.042"
"available": true
}
}

DELETE/rest-v1/db/additional/{tableName}/{id}Deletes the molecule specified by ID from the specified table 
/rest-v1/db/additional/mytable/1

POST/rest-v1/db/additional/{tableName}/batchInsertInserts batch of molecules with their specified unique IDs and additional data into the specified table. Uses JSON format.
/rest-v1/db/additional/mytable/batchInsert
[
{
"id": 1,
"molecule": "CCO",
"additionalData":
{
"name": "etanol"
}
},
{
"id": 2,
"molecule": "C"
}
]

POST/rest-v1/db/additional/{tableName}/batchDeleteDeletes the molecules specified by IDs from the specified table 
/rest-v1/db/additional/mytable/batchDelete
[ 1, 2 ]

GET/rest-v1/db/additional/{tableName}/{id}

Retrieves the data having the specified molecule ID from the specified table.


/rest-v1/db/additional/mytable/1
{ 
"id": 1,
"molecule": "CCO",
"additionalData":
{
"name": "etanol"
}
}
GET/rest-v1/db/additional/{tableName}/{id}/moleculeRetrieves molecule string having the specified molecule ID.
/rest-v1/db/additional/myTable/1/molecule?format=smiles
CN1C=C(C=C1C)C1=CSC(N=C(N)N)=N1
GET/rest-v1/db/additional/structureCount/{tableName}Retrieves the count of chemical structures stored in the specified table.
/rest-v1/db/additional/structureCount/myTable
12345


Duplicate search methods

MethodDescriptionExample use caseExmaple requestExample response
GET/rest-v1/db/additional/{tableName}/duplicateSearches for limited number of duplicate hits of the specified query  and additional data in the specified table
/rest-v1/db/additional/myTable/duplicate?structure=C
[ { 
"id": 2,
"molecule": "C"
} ]
POST/rest-v1/db/additional/{tableName}/duplicateSearches for limited number of duplicate hit structures of the specified query  and additional data in the specified table. Uses JSON format.
/rest-v1/db/additional/myTable/duplicate
{
"hitCount": 10,
"molecule": "c"
}
[ 
{
"id": 2,
"molecule": "C"
}
]
POST

/rest-v1/db/additional/{tableName}/ids/duplicate

Searches for limited number of duplicate hit IDs of the specified query in the specified table. Uses JSON format.

Recommended use case

if it is enough to receive only the IDs of the hits, the structures and additional data are not needed

/rest-v1/db/additional/myTable/ids/duplicate
{
"hitCount": 10,
"molecule": "c"
}
[ 2 ]


Substructure search methods

MethodDescriptionExample use caseExample requestExample response
GET/rest-v1/db/additional/{tableName}/substructureSearches for limited number of substructure hits of the specified query  and additional data in the specified table
/rest-v1/db/additional/myTable/substructure?structure=C
[
{
"id": 1,
"molecule": "CCO",
"additionalData":
{
"name": "etanol"
}
},
{
"id": 2,
"molecule": "C"
}
]
POST/rest-v1/db/additional/{tableName}/substructureSearches for limited number of substructure hit structures of the specified query  and additional data in the specified table. Similarity value can also be retrieved. Uses JSON format.
/rest-v1/db/additional/myTable/substructure
{
"hitCount": 10,
"molecule": "CC"
}
[
{
"id": 1,
"molecule": "CCO",
"additionalData":
{
"name": "etanol"
}
},
{
"id": 2,
"molecule": "C"
}
]
POST
/rest-v1/db/additional/{tableName}/ids/substructure
Searches for limited number of substructure hit IDs of the specified query in the specified table. Similarity value can also be retrieved. Uses JSON format.

Recommended use case:

if it is enough to receive only the IDs of the hits, the structures and additional data are not needed
/rest-v1/db/additional/myTable/ids/substructure
{
"hitCount": 10,
"molecule": "CC"
}
[ 1, 2 ]


Similarity search methods

MethodDescriptionExample use case

GET/rest-v1/db/additional/{tableName}/similaritySearches for limited number of similarity hits of the specified query  and additional data in the specified table

/rest-v1/db/additional/demoTable/similarity?hitCount=10&similarityThreshold=0.8&structure=C

[ 
{
"id": 2,
"molecule": "C",
"additional": {
"similarity": 1.0
}
}
]
POST/rest-v1/db/additional/{tableName}/similaritySearches for limited number of similarity hit structures of the specified query  and additional data in the specified table. Uses JSON format.
/rest-v1/db/additional/demoTable/similarity
{ 
"hitCount": 10,
"molecule": "C",
"similarityThreshold": 0.8
}
[ 
{
"id": 2,
"molecule": "C",
"additional": {
"similarity": 1.0
}
}
]
POST/rest-v1/db/additional/{tableName}/ids/similaritySearches for limited number of similarity hit IDs of the specified query in the specified table. Uses JSON format.

Recommended use case:

if it is enough to receive only the IDs of the hits, the structures and additional data are not needed
/rest-v1/db/additional/demoTable/ids/similarity
{ 
"hitCount": 10,
"molecule": "C",
"similarityThreshold": 0.8
}
[ 2 ]


Data recovery

The data apparently stored in tables (collections) are stored in the file system under jws-db/data/ folder. The content of that folder must be stored as backup. Furthermore, the application.properties file(s) also should be saved.