Day 62 – TextStyleBrush by Facebook AI Research Team

We have seen a previous blog posts on MNIST Largest Handwritten Digits Database, EMNIST Dataset Handwritten Character Digits, and Code Example: Import EMNIST Dataset and Print Handwritten Letters.

Today, we will see a very different AI model introduced by Facebook AI Research Team called Text StyleBrush. TextStyleBrush can copy the style of text in a photo, translate, edit, and reply to a text in images.  It is a self-supervised AI model, and it is like a photo-realistic translation of languages in augmented reality (AR) (mentioned in Facebook).  This AI model helps to detect any potential misuse of text translation and deepfake attacks.

Usage of TextStyleBrush:

1. Show the image of text written in a language that can translate into English.
2. Write words or sentences in any font and show some writing style, then those written words or sentences are changed into the style.
3. Create personalized messaging and captions
4. Real-world translation of street signs using AR.

What does Facebook AI Research Team say about TextStyleBrush?

It works similar to the way style brush tools work in word processors, but for text aesthetics in images. It surpasses state-of-the-art accuracy in both automated tests and user studies for any type of text. Unlike previous approaches, which define specific parameters such as typeface or target style supervision, we take a more holistic training approach and disentangle the content of a text image from all aspects of its appearance of the entire word box. The representation of the overall appearance can then be applied as one-shot-transfer without retraining on the novel source style samples.

Source: Facebook AI

Architecture of TextStyleBrush:

However, the general architecture is based on the StyleGan2 model but added control output of desired text content & style and unique nature of styled text images.

Image Source: Facebook AI

Final conclusion on TextStyleBrush:

This will change the way we see image text in the future, reduce photorealistic translation, creative self-expression, helps deepfake text attacks.