How we made LlamaAI better for DeFi research than vanilla AI
Source: http://x.com/i/article/2041144423635894272
Lots of ppl ask me: Whats different between LlamaAI and just using some other AI with access to defillama API?
The short explanation is that LlamaAI has direct access to a database of our data that is fine-tuned for AI use
I'm spilling the beans on our secret sauce here, but just consider this example:
You want to calculate the chances of defi protocols getting hacked, to estimate your risk, so you want to know the % of defi protocols that got hacked historically, excluding small ones that have never reached 50m, to only include relevant ones
To answer this, claude with defillama api needs to query to find all our protocols and then send a query for each one of the 7k to find their full historical TVL and filter it.
Doing so it's gonna run into rate limits so it will need a system to internally rate limit. And this is GBs of data so when it starts reading it, its context window is gonna fill up, leading to degraded performance
By contrast LlamaAI is just gonna write:
`SELECT name FROM protocol_tvl_history GROUP BY name HAVING MAX(tvl) > 50e6;` This is way easier, much faster and less prone to mistakes.
But that's not all, from our research we found out that AI's ability to manipulate data changes a lot based on how the data is formatted, you can reduce data errors by an order of magnitude by using the right format.
So we wrote a test set of questions that LlamaAI users use and their correct answers, and then:
1. We run LlamaAI on all questions, then check correctness & where it got confused
2. We change parts of the DB where it got confused
3. We run tests again to see if correctness % has improved, if so we keep the changes
By doing this in a loop, we optimize our data for AI querying and that's how we can achieve much better results and lower errors vs just using claude/chatgpt with our data
Then on top of that we gave LlamaAI a crypto wallet so it can purchase other data sources, thus it has much wider access to data compared to vanilla AI
But the secret sauce is our fine-tuned database