Text Reader running in the Browser

The demo is located directly on the homepage. Here is an outline how the model deployment was done:

• Trained a scaled-down version of the CRNN model used in the HTRPipeline repository with PyTorch (model size: 2MB)
• Exported to ONNX (2MB)
• Quantized (PTQ) from float32 to int8 (1MB) and calibrated (using percentile method) on a representative set of images
• LSTM stays in float32, Conv is in int8, see plot of neural network below
• Now coming to the web implementation: the RGBA image data taken from the selected html img tag is converted to an array (1D, float32) containing the gray values, which is then wrapped in a tensor (4D, BCHW, float32) which additionally holds the dimension information
• As the images are small, the described pre-processing is fast, even in pure JS (1ms)
• onnxruntime-web is used as the inference engine
• Inference runs directly on the CPU (web assembly) and takes between 10ms and 50ms, depending on the device
• Finally, decoding (with CTC best path algorithm) the output tensor is done in pure JS and takes less than 1ms

Here is a plot of the quantized model as used during inference (dumped from an inference session, plot created with netron.app):

Harald Scheidl