4/n) Next, I'm gonna try using the Contextual RAG technique by Anthropic as mentioned in one of their blogs.
01.05.2025 07:32
👍 0
🔁 0
💬 0
📌 0
4/n) Next, I'm gonna try using the Contextual RAG technique by Anthropic as mentioned in one of their blogs.
What I love about the OpenAI SDK is how easily it generates structured output exactly the way I need. Here’s the prompt I used to generate relevant and a sneak peek at what the final metadata for a verse looks like:
However, in my case, the libraries missed a few entities, so I switched entirely to GPT-4o for metadata extraction. This lets me control quality exactly the way I want.
Link to the blog: www.elastic.co/search-labs/...
1Just read the latest blog on Advanced RAG by Elastic Labs, loved the approach of enhancing retrieval accuracy using metadata.
They used NLP libraries to extract keywords, keyphrases, and entities, and GPT-4o to generate relevant questions from each chunk.
Used GPT-4o to generate interpretations for each verse. Each entry looks pretty solid so far. Could’ve experimented with models like Claude or LLaMA, but I’ll save that rabbit hole for after the basic version is done.
Here's how I usually write the prompt:
1) Start with a detailed, clear prompt - I try to provide as much information as I can and some context.
2) Then I ask ChatGPT for feedback and ask if there's anything missing.
3) Refine and test until it works well.
Next, I plan to write an interpretation for each verse and relate it to modern-day challenges people face. This would make it easier for the RAG system to retrieve and rank verses. This is the prompt I am planning to go with.
Got caught up with my viva last week, resuming work on GitaGPT now. Will try to get it done by the weekend.
Will keep ya'll updated.
Most likely going to use an LLM to make this happen let’s see what kind of prompts work best. Open to any suggestions you might have!
When people visit this platform, they’ll likely seek advice on modern-day issues like “job stress” or “overthinking.” Since the verses don’t directly mention these, I’ll need to interpret them in a way that relates to today’s challenges.
Update on GitaGPT!
Just scraped all the Bhagavad Gita verses, translations, and explanations from the website. The next challenge? Keyword search. The English is quite formal and traditional, which might make it harder for the search algorithm to discover relevant chunks.
holy-bhagavad-gita.org
I'm building a RAG-based system using verses from the Bhagavad Gita to help people easily seek timeless advice. Planning to use LLMs to simplify explanations so today’s youth can relate to modern-day challenges. Excited to see where this project leads!
3/n) We can calculate the mean and standard deviation of the trees planted by scaling the mean and standard deviation of the centuries scored accordingly.
This gives us a distribution having mean as 455.0 and standard deviation as 92.0
2/n) For every century Rohit scores, a non-profit plants 100 trees, plus 300 more on their own at the end of the campaign. How do we calculate the probability distribution of total trees planted from Rohit's score distribution? This is where density transformation shines!
1/n) Imagine a 3-match ODI series between India and South Africa. The probability distribution of Rohit Sharma's centuries in the series looks something like this:
Polishing my mathematics fundamentals to dive deeper into more complex generative and GAN-based models. It's a bit overwhelming to work on, but I'm sure I'll get the hang of it with practice. Here's a basic overview of Transforming Probability Densities:
- A Thread 🧵
6/n) If we can bring this system to remote corners where quality education is scarce, it could be a giant leap into the future. Next ambitious step? partnering with governments and NGOs to make education accessible, following the paths of @duolingoverde.bsky.social and @khanacademy.bsky.social.
5/n) This can be the foundations of a very sophisticated end-to-end autonomous educational system unlike anything we've seen before. This would further push the boundaries of what AI is capable of and its impact on making education more accessible and free for everyone
4/n) Now, the key here is asking the right questions so we can generate accurate student profiles. Using them we can mirror their thought processes, identify challenges, common mistakes and come up with creative solutions to help them overcome obstacles.
3/n) We can continuously adapt to students' preferences and learning styles by gathering real-time feedback. For those who learn best through visuals, we can create agents to build mind maps or diagrams. For story-driven learners, we can explain concepts through narratives.
2/n) Based on students' profiles, we can generate personalized examples and questions of varying difficulty levels on topics previously taught by the professor. This will increase cognitive engagement helping students absorb more knowledge and retain it longer.
1/n) Hyper-Personalized Student Profiles
Creating a detailed profile of each student, capturing their preferences, strengths, weaknesses, and interests through data on courses, extracurricular achievements, hobbies, and personal reflections from students about themselves.
Inspired by the incredible work @khanacademy.bsky.social and @duolingoverde.bsky.social are doing to make personalized education accessible, here are a few research directions I’d love to explore if given the chance.
You can check the paper out here:
arxiv.org/html/2410.04...
This is when I saw it as an opportunity to explore the pedagogical challenges and opportunities faced by students from diverse backgrounds and experiences, often with limited resources - challenges that AI has the potential to address.
Initially, I saw this as a product with great market fit and monetary potential. But then I paused and asked myself—is this the direction I want AI in ed-tech to take? Another expensive product for students who can afford it, turning education into just another business?
Recently got a chance to read this paper by some amazing folks at KAIST. It introduces a framework for building and evaluating personalized Pedagogical Conversational Agents (PCAs) to align with students' personality traits to enhance learning.
-🧵
Feel free to contribute to my project here:
github.com/AtharvaMaska...
My next step include the following things:
🤖Data Pre-processing.
🏋️Training our Deep Learning Model
📝Testing our model
2/n) Next, we loaded the metadata, which stores information about all audio files, and checked if the dataset is balanced or not. We found that the dataset was fairly balanced.
