MyJournals Home  

RSS FeedsEntropy, Vol. 23, Pages 1614: TIPS: A Framework for Text Summarising with Illustrative Pictures (Entropy)


30 november 2021 17:11:04

Entropy, Vol. 23, Pages 1614: TIPS: A Framework for Text Summarising with Illustrative Pictures (Entropy)

We propose an algorithm to generate graphical summarising of longer text passages using a set of illustrative pictures (TIPS). TIPS is an algorithm using a voting process that uses results of individual “weak” algorithms. The proposed method includes a summarising algorithm that generates a digest of the input document. Each sentence of the text summary is used as the input for further processing by the sentence transformer separately. A sentence transformer performs text embedding and a group of CLIP similarity-based algorithms trained on different image embedding finds semantic distances between images in the illustration image database and the input text. A voting process extracts the most matching images to the text. The TIPS algorithm allows the integration of the best (highest scored) results of the different recommendation algorithms by diminishing the influence of images that are a disjointed part of the recommendations of the component algorithms. TIPS returns a set of illustrative images that describe each sentence of the text summary. Three human judges found that the use of TIPS resulted in an increase in matching highly relevant images to text, ranging from 5% to 8% and images relevant to text ranging from 3% to 7% compared to the approach based on single-embedding schema.

67 viewsCategory: Informatics, Physics
Entropy, Vol. 23, Pages 1613: Towards a Measure for Characterizing the Informational Content of Audio Signals and the Relation between Complexity and Auditory Encoding (Entropy)
Entropy, Vol. 23, Pages 1631: Impulsive Reaction-Diffusion Delayed Models in Biology: Integral Manifolds Approach (Entropy)
blog comments powered by Disqus
The latest issues of all your favorite science journals on one page


Register | Retrieve



Copyright © 2008 - 2022 Indigonet Services B.V.. Contact: Tim Hulsen. Read here our privacy notice.
Other websites of Indigonet Services B.V.: Nieuws Vacatures News Tweets Nachrichten