Blog

Natural language search using generative AI

  • 22 November 2023
  • 3 replies
  • 302 views
Natural language search using generative AI
Remco Kort
Rank
Userlevel 3
Badge +1

Search is one of the most used functions of any software application. But there are many ways to implement it. In this video I want to highlight a new way of implementing search using the large language model embedding integration that the Thinkwise Software Factory provides. This allows users to search by using natural language.

 

 

3 replies

Freddy
Userlevel 5
Badge +16

Hi @Remco Kort, 

Do you have more information on how to create the base vectors for the searchable content? We have for example a database full of knowledge items and services..  each consists of a title, intro and full description. 

I have questions doubts on how to structure the embedding request..  

  • TW only support one string text input right?  So in this case I should concatenate title, intro and full description? 
  • How does TW/OpenAI treat HTML tags? Do they need to be stripped upfront? 
  • There is max input (tokens), are these characters?  If so what need to happen when you have an item that supersedes this max input? 
  • Is there a smart way to validate if a vector is still up to date? Or you just need to track updates on the content that was embedded? 

Do you have more input or information to share? 

LEF - The future favors the brave. [lef.tec.br]
Remco Kort
Rank
Userlevel 3
Badge +1

Hi @Remco Kort, 

Do you have more information on how to create the base vectors for the searchable content? We have for example a database full of knowledge items and services..  each consists of a title, intro and full description. 

I have questions doubts on how to structure the embedding request..  

  • TW only support one string text input right?  So in this case I should concatenate title, intro and full description? 
  • How does TW/OpenAI treat HTML tags? Do they need to be stripped upfront? 
  • There is max input (tokens), are these characters?  If so what need to happen when you have an item that supersedes this max input? 
  • Is there a smart way to validate if a vector is still up to date? Or you just need to track updates on the content that was embedded? 

Do you have more input or information to share? 

  1. Correct, i think that would probably work for your use case. But I believe that in the ESG app made by Thinkwise they use larger amounts of data, so maybe ask them (Sander Kiesbrink, Anne Buit) as well.
  2. As far as i am aware thinkwise will send the process action input to openAI. So I imagine that html will be send to openAI. I have not done this personally so I don't know how openAI would treat this data.
  3. If you have an item that exceeds the max amount of tokens you will most likely need to find a way to chop the data up into smaller chunks. How to do this will depend on your specific data, maybe cut it into chapters, paragraphs, etc. Tokens are not the same as characters, for more information see: https://help.openai.com/en/articles/4936856-what-are-tokens-and-how-to-count-them
  4. I currently just regenerate it after every change made to the data. Perhaps you could store a vector generation datetime and compare that to the updated_on field of your data with a system flow or task if you want more certainty. 
Freddy
Userlevel 5
Badge +16

A small update after some testing:

  • You have to strip HTML tags, as some elements are forbidden, and everything you send over is treated as text and thus used to match. 
  • When you have a larger dataset to check, the similarity check takes quite some time. I moved the check from the function to the view and that speeds things up quite a bit.. but nevertheless a search agains 255 documents (vector-embeddings) still takes up to 3 seconds.. so I would advise to do the similarity check in the background to reduce loading time for the user. 

for the rest it works quite nice. Good to have this connector out of the box. 

@Anne Buit  @Sander Kiesbrink any tips on treating larger documents? 

LEF - The future favors the brave. [lef.tec.br]

Reply


Terms & ConditionsCookie settings