Ability to chunk and embed multiple HTML pages

[anjani-pothula]

As of today, oaim sandbox accepts a pdf file (from an OCI bucket or local) and an HTML page. Many times, all the content we would like to chunk and embed for a RAG chatbot will not be in a single HTML page. This limits us from using HTML input for ingestion. To overcome this limitation, the sandbox must:

1. Allow users to provide a list of HTML URLs from the embed and chunk page. (OR)
2. Crawl through all the URLs part of the HTML page given as the input. For example, if we give the TOC page as the input, the sandbox must be able to crawl through all the pages that are listed on the TOC, for chunking.