MNCH

TWIL 001

Posted on Dec 17, 2023

TWIL ~ This week I learned"

Highlight is that new Keras 3.0 is out.

nDCG Metric

In the paper about RankZephyr I found out Normalized Discounted Cumulative Gain (NDCG for short). Which seems to be and interesting and used metric for ranking systems.

NDCG (normalized discounted cumulative gain): NDCG is a measure of the effectiveness of a ranking system, taking into account the position of relevant items in the ranked list. It is based on the idea that items that are higher in the ranking should be given more credit than items that are lower in the ranking.

Wiki Page

nDCG vizualization

Vizualizations of Math

Thanks to hacker news (I lost the link to the original thread :() I ran into multiple interesting possibilities how to vizualize and present math stuff:

Gemini

Google released some materials about Gemini and early access to it through GCP (sadly, I was not able to get in). I have not seen the product videos and previews. But the report was easily readable. As we are getting used to - architecture is not revealed :(. On the other side, we got 10 pages of names of contributors. To be fair, Google revealed, they use transformers.

Jeff Bezos on Lex’s podcast

Just wow: https://www.youtube.com/watch?app=desktop&v=DcWqzZ3I2cY&ab_channel=LexFridman. Much more interesting dialogues than the ones with Elon. Just do not know why. It’s full of important stuff - teams for papercuts, truth telling, decision making, order of voicing opinions in meetings, memo vs powerpoint slides…. There is just too much on how to instigate good decisions.

RAG chunking

I started to look into chunking and this is a output I got from Edge GPT assistant:

In a typical retrieval-augmented generation (RAG) application, chunking is the process of breaking down the input text into smaller segments or chunks¹. The chunking strategy plays a crucial role in optimizing the relevance and accuracy of the content retrieved from a vector database⁶.

To create chunks based on semantic context using a Large Language Model (LLM), you can use advanced techniques such as content-based chunking⁷. This involves breaking up text based on meaning and sentence structure, for example, using part-of-speech tagging or syntactic parsing⁷. Another approach is sentence chunking, which consists of breaking the text into complete and grammatically correct sentences⁷.

As for generating metadata about the chunk, you can assign each chunk a unique identifier and a set of attributes, which describe its content, location, and relationship to other chunks⁷. This metadata can be used to optimize the retrieval process and improve the relevance of the results.

Remember, the choice of chunking strategy and the generation of metadata are not trivial tasks. They depend on several factors such as the type and amount of data, the nature and complexity of the queries, and the characteristics and performance of the model⁷. Therefore, it’s important to experiment with different strategies and evaluate their impact on your application’s performance⁵⁶⁷.

Source: Conversation with Bing, 12/12/2023