Hallucinations in LLMs Are Not a Bug in the Data
It's a feature of the architecture
Google has just released a preview version of its latest embedding model. This model is notable for one main reason. It can embed text, PDFs, images, audio, and video, making it a one-stop shop for embedding just about anything you’d care to throw at it.
If you’re new to embedding, you might wonder what all the fuss is about, but it turns out that embedding is one of the cornerstones of retrieval augmented generation or RAG, as it’s known. In turn, RAG is one of the most fundamental applications of modern artificial intelligence processing.
Quick Recap of RAG and Embedding
RAG is a method of chunking, encoding and storing information that can then be searched using similarity functions that match search terms to the embedded information. The encoding part turns whatever you’re searching into a series of numbers called vectors — this is what embedding does. The vectors (embeddings) are then typically stored in a vector database.