Advanced scraping for website pages

24b8c97d384c9dfc4c006f7774d0c3f3.jpg-3265
A huge advancement for our Knowledge base engine.

Whenever you want to get your website data into our Knowledge base to create an OpenAI based autoresponder chatbot you'll have a totally new engine!

While the old engine only took data from the HTML page source code, the new one also executes the Javascript on the page using a real (headless) Chrome browser. In this way also HTML pages with content rendered on the browser side will be easily acquired and indexed by the RAG engine.

We also added a new optional "tunable" scraping (Advanced scrape) option, so you can choose what tags or CSS classes consider or ignore during the contents acquisition.

More info here: https://gethelp.tiledesk.com/articles/adding-a-knowledge-base/

What do you think about this update?