wordpress blog stats
Connect with us

Hi, what are you looking for?

Ola Krutrim now launched for the public in its beta version: Users report inaccurate answers

One of the X posts on Krutrim that blew up was about the AI model claiming that it was created by OpenAI as opposed to Ola.

Ola’s artificial intelligence model Krutrim was launched for the public in its beta version on February 26. The Krutrim family of large language models (LLMs) was announced on December 15, 2023. These LLMs have been trained in Indian languages specifically to cater to the Indian population.

Within a day of its beta launch, the AI model shows signs of errors according to X (formerly Twitter) users. One of the X posts on Krutrim that blew up was about the AI model claiming that it was created by OpenAI as opposed to Ola.

The company quickly addressed the situation, attributing the misinformation to data leakage from an open-source dataset used in its LLM’s fine-tuning. When asked who it is and who created it, the tool now correctly attributes its origin to the Ola team. 

Errors in providing historical World Cup results:

To test whether Krutrim was truly making the errors that people were reporting, we tried the tool as well. Here’s how its responds to us compared to what it told X users. On the question of who won the World Cup in 1983, Krutrim replied to an X user that West Indies won it, when in fact, the Indian cricket team won the cricket world cup that year. It gave us the same response as well.

To give the AI a second chance, we rephrased the question and asked it “Who won the Cricket World Cup in 1983?” While it no longer responded with West Indies, its answer continued to remain incorrect.

Krutrim political neutrality:

As per a report by Indian Express, Krutrim was rating itself 7/10 on political neutrality. When we asked the AI the same question, it was even more confident about its neutrality than when Indian Express interacted with it.

 

Interestingly, Krutrim was giving polarizing responses even before it was launched for public use. On February 4, Ola CEO Bhavish Aggarwal posted screenshots on X comparing answers by Krutrim and ChatGPT when asked the question “Was India a country before the British Raj?”. While Krutrim answered affirmatively, ChatGPT said that it was not a single unified country prior to the establishment of the British Raj.

This response was also brought up in a closed-door discussion with the OpenAI on February 12. “What I’m seeing right now, which brings in from this point, is that [the] current government is extremely nationalistic and divisive in nature. So they are encouraging people to build models which push their narrative more,” a participant pointed out during the discussion, arguing that AI companies need to consider such biases and how they should be navigated.

The Indian government’s view on beta testing:

Given that Krutrim is still in the beta testing process, it makes sense that it is making mistakes and giving incorrect answers. However, the Indian government doesn’t look favorably on AI companies testing their products on the public. Speaking at the Digital News Publishers Association’s Conclave earlier this month, IT Minister Rajeev Chandrasekhar said that AI companies can “only offer AI that is safe and trusted in India. You can offer only models that are trained to deliver safe and trusted outputs in India. Even if you are a platform that use algo[rithm]s and not a full-blown AI model, those algo[rithm]s have to be safe and trusted.”

Chandrasekhar gave the example of Google Bard responding to a user question inaccurately because it was under trial and said that the government is “making it very clear that nobody can put a physically available platform on trial. You will have to sandbox that.”

Also read:


STAY ON TOP OF TECH NEWS: Our daily newsletter with the top story of the day from MediaNama, delivered to your inbox before 9 AM. Click here to sign up today!


 

Written By

Free Reads

News

The ‘Reforming Intelligence and Securing America Act’ (RISAA) is a legislation to reauthorize Section 702 of the Foreign Intelligence Surveillance Act (FISA).

News

In its submission, the Interior Ministry said the decision to impose a ban was "made in the interest of upholding national security, maintaining public...

News

Among other things, the security requirements include data encryption and regular review and updated access permissions to reflect personnel changes.

MediaNama’s mission is to help build a digital ecosystem which is open, fair, global and competitive.

Views

News

NPCI CEO Dilip Asbe recently said that what is not written in regulations is a no-go for fintech entities. But following this advice could...

News

Notably, Indus Appstore will allow app developers to use third-party billing systems for in-app billing without having to pay any commission to Indus, a...

News

The existing commission-based model, which companies like Uber and Ola have used for a long time and still stick to, has received criticism from...

News

Factors like Indus not charging developers any commission for in-app payments and antitrust orders issued by India's competition regulator against Google could contribute to...

News

Is open-sourcing of AI, and the use cases that come with it, a good starting point to discuss the responsibility and liability of AI?...

You May Also Like

News

Google has released a Google Travel Trends Report which states that branded budget hotel search queries grew 179% year over year (YOY) in India, in...

Advert

135 job openings in over 60 companies are listed at our free Digital and Mobile Job Board: If you’re looking for a job, or...

News

By Aroon Deep and Aditya Chunduru You’re reading it here first: Twitter has complied with government requests to censor 52 tweets that mostly criticised...

News

Rajesh Kumar* doesn’t have many enemies in life. But, Uber, for which he drives a cab everyday, is starting to look like one, he...

MediaNama is the premier source of information and analysis on Technology Policy in India. More about MediaNama, and contact information, here.

© 2008-2021 Mixed Bag Media Pvt. Ltd. Developed By PixelVJ

Subscribe to our daily newsletter
Name:*
Your email address:*
*
Please enter all required fields Click to hide
Correct invalid entries Click to hide

© 2008-2021 Mixed Bag Media Pvt. Ltd. Developed By PixelVJ