Ola’s artificial intelligence model Krutrim was launched for the public in its beta version on February 26. The Krutrim family of large language models (LLMs) was announced on December 15, 2023. These LLMs have been trained in Indian languages specifically to cater to the Indian population.
Within a day of its beta launch, the AI model shows signs of errors according to X (formerly Twitter) users. One of the X posts on Krutrim that blew up was about the AI model claiming that it was created by OpenAI as opposed to Ola.
Something seem super fishy@KrutrimAI says it was created by OpenAI https://t.co/SNBoRDIzrl pic.twitter.com/m8NKyuIrcw
— Raghav Arora (@raghavar0) February 26, 2024
The company quickly addressed the situation, attributing the misinformation to data leakage from an open-source dataset used in its LLM’s fine-tuning. When asked who it is and who created it, the tool now correctly attributes its origin to the Ola team.
Errors in providing historical World Cup results:
To test whether Krutrim was truly making the errors that people were reporting, we tried the tool as well. Here’s how its responds to us compared to what it told X users. On the question of who won the World Cup in 1983, Krutrim replied to an X user that West Indies won it, when in fact, the Indian cricket team won the cricket world cup that year. It gave us the same response as well.
To give the AI a second chance, we rephrased the question and asked it “Who won the Cricket World Cup in 1983?” While it no longer responded with West Indies, its answer continued to remain incorrect.
Krutrim political neutrality:
As per a report by Indian Express, Krutrim was rating itself 7/10 on political neutrality. When we asked the AI the same question, it was even more confident about its neutrality than when Indian Express interacted with it.
Interestingly, Krutrim was giving polarizing responses even before it was launched for public use. On February 4, Ola CEO Bhavish Aggarwal posted screenshots on X comparing answers by Krutrim and ChatGPT when asked the question “Was India a country before the British Raj?”. While Krutrim answered affirmatively, ChatGPT said that it was not a single unified country prior to the establishment of the British Raj.
This response was also brought up in a closed-door discussion with the OpenAI on February 12. “What I’m seeing right now, which brings in from this point, is that [the] current government is extremely nationalistic and divisive in nature. So they are encouraging people to build models which push their narrative more,” a participant pointed out during the discussion, arguing that AI companies need to consider such biases and how they should be navigated.
The Indian government’s view on beta testing:
Given that Krutrim is still in the beta testing process, it makes sense that it is making mistakes and giving incorrect answers. However, the Indian government doesn’t look favorably on AI companies testing their products on the public. Speaking at the Digital News Publishers Association’s Conclave earlier this month, IT Minister Rajeev Chandrasekhar said that AI companies can “only offer AI that is safe and trusted in India. You can offer only models that are trained to deliver safe and trusted outputs in India. Even if you are a platform that use algo[rithm]s and not a full-blown AI model, those algo[rithm]s have to be safe and trusted.”
Chandrasekhar gave the example of Google Bard responding to a user question inaccurately because it was under trial and said that the government is “making it very clear that nobody can put a physically available platform on trial. You will have to sandbox that.”
Also read:
- Ola Launches Its Multilingual Artificial Intelligence Model Krutrim
- Highlights From OpenAI And Quantum Hub’s Discussion On AI Ahead Of 2024 Elections
- Govt Concerned About Asymmetry Between News Publishers And Social Media Platforms: IT Minister Rajeev Chandrasekhar Says
STAY ON TOP OF TECH NEWS: Our daily newsletter with the top story of the day from MediaNama, delivered to your inbox before 9 AM. Click here to sign up today!
