Projects

Text Understanding in Visual Data

Text recognition (traditionally called OCR) is crucial for visual understanding and reasoning in many application scenarios.

We focus on Arbitrary-Shaped Text detection and Recognition and related Natual Language Processing tasks for understanding these text.

Z. Kuang, H. Sun, Z. Li, X. Yue, T.H. Lin, J. Chen, H. Wei, Y. Zhu, T. Gao, W. Zhang, K. Chen, W. Zhang and D. Lin.
MMOCR: A comprehensive toolbox for text detection, recognition and understanding.
Proc. ACM Multimedia (MM) Open Source Software Competition, 2021. [code]

An open-source toolbox for text detection, recognition and understanding.

Text Detection

X. Yue, Z. Kuang, Z. Zhang, Z. Chen, P. He, Y. Qiao and W. Zhang.
Boosting up scene text detectors with guided CNN.
British Machine Vision Conference (BMVC), 2018. (Oral presentation, acceptance rate: 6.5%) [paper]

A general framework for speeding up scene text detection. Demonstrated performance on two state-of-the-art methods, CTPN and EAST.

Geometry normalization networks for accurate scene text detection, rotated detection, oriented detection, sensetime, ocr

Y. Xu, J. Duan, Z. Kuang, X. Yue, H. Sun, Y. Guan and W. Zhang.
Geometry normalization networks for accurate scene text detection.
Proc. International Conf. on Computer Vision (ICCV), 2019. [github] [blog (in Chinese)]

A general framework for handling large geometry (scale and orientation) variances in scene text detection. Demonstrated performance on two state-of-the-art methods, EAST and PSENet.

Fourier contour embedding for arbitrary-shaped text detection, sensetime, ocr

Y. Zhu, J. Chen, L. Liang, Z. Kuang, L. Jin and W. Zhang.
Fourier contour embedding for arbitrary-shaped text detection.
Proc. IEEE Conf. on Computer Vision and Pattern Recognition (CVPR), 2021. [code in MMOCR]

Text Recognition

Our team won the first runner-up in Scene text recognition (Chinese and Latin) Task of ICDAR2019 Robust Reading Challenge on Arbitrary-Shaped Text (ICDAR 2019-ArT). [results]

RobustScanner: Dynamically enhancing positional clues for robust text recognition, sensetime, ocr

X. Yue, Z. Kuang, C. Lin, H. Sun and W. Zhang.
RobustScanner: Dynamically enhancing positional clues for robust text recognition.
Proc. European Conf. on Computer Vision (ECCV), 2020. (Acceptance rate: 27%) [paper] [blog (in Chinese)] [code in MMOCR]

RobustScanner has been used in solutions of 1st place winner of several challenges, such as CVPR 2021 RetailVision Product Pricing in the Wild Challenge (see report), ECCV 2022 Out of Vocabulary Scene Text Understanding (see report)

Stats