Your face has a lot of value in Telangana these days. It can help you apply for a duplicate or renewed license without ever leaving the house. If you are an intermediate student, it can help you enrol in a degree college, again, without ever leaving the house. Interestingly, none of this would have been possible before the pandemic.

The youngest state of country has been evangelising the benefits of facial recognition technology to its people for many years now. However, the global Covid-19 pandemic has caused a shift of gears in the state government: in a matter of months, the tech has been deployed in two major areas — degree admissions and RTA services. Soon, it could also be used in the distribution of ration as well.

The use of facial recognition and artificial intelligence has also raised questions around citizens’ privacy and consent, especially considering how India still doesn’t have a law for personal data protection. However, the Telangana government is undeterred. In an interview with MediaNama, Jayesh Ranjan, the state’s IT and Industries Principal Secretary, said that when government work is concerned, “consent is implicit”. He also commented on concerns about algorithmic bias against minorities, emphasising that the tech was constantly being improved to tackle these things.

New facial recognition deployments are pandemic driven

Telangana has deployed facial recognition in two new areas since the pandemic began — degree admissions and Road Transport Authority (RTA) services.

  • Degree admissions: Telangana introduced facial recognition in degree admissions in June this year. The Telangana State Council of Higher Education (TSCHE) introduced the technology in its admission portal for prospective degree students — Degree Online Services Telangana (DOST).In theory, students have to enter their phone number, Aadhaar number and finally, upload a selfie. The DOST portal uses the selfie to query other personal details from the Board of Intermediate (BIE) database, which has the details of all students enrolled in all affiliated intermediate colleges in the state. Then the app will generate a “DOST ID” that students can then use to log into DOST’s website and apply for college admissions. This application is only available for students who have graduated from institutions affiliated with the BIE, since T-App Folio has access only to BIE’s database. It should be added that students who cannot use T-App Folio — or do not wish to — can also obtain DOST IDs directly from DOST’s website as long as they have an Aadhaar-linked mobile number.
  • RTA services: the state Road Transport Authority brought many of its services to the T-App Folio in July. As of September 28, users have access to 12 services, including driving license renewal, application for duplicate driving license and change of address. The RTA hopes this will reduce the need for citizens to come to its usually crowded offices during the pandemic. Applicants are authenticated through RTDAI using selfies that determine their “liveness”. These services are being offered through Syntizen, a Hyderabad-based company incubated in the state’s very own T-Hub programme. Syntizen also reportedly works with the Uttar Pradesh government to identify beneficiaries of the state’s Aadhaar-based welfare schemes.

Ranjan told MediaNama that the pandemic had indeed accelerated the deployment of facial recognition by the government, saying “the RTA application is completely pandemic-driven. RTA had no inclination to try [the technology], but because of the pandemic they realised that people are not coming. RTA is also a revenue generating department […] They realised they cannot just close shop and just sit idle.”

And it seems like these particular effects of the pandemic are here to stay. TSCHE vice-chairman and DOST convenor R Limbadri told The News Minute earlier this month that facial recognition was not a “temporary measure”, and that new technological features would be added every year.

Facial recognition in Telangana: An old and troubled story

Despite the absence of a personal data protection law, Telangana has until late used facial recognition — which involves the processing and storing of personal information — in three areas: pensions, elections and policing.

  • Pensions: Telangana has been using facial recognition to authenticate pensioners for many years now. Pensioners are supposed to use the “T App Folio” — an umbrella app for e-governance initiatives launched by the government in 2018  — to upload pictures of themselves to prove that they are alive. This authentication mechanism is powered by the state government’s Real Time Digital Authentication of Identity (RTDAI) architecture, the backbone of all facial recognition tech in Telangana (more on this later). In theory, all you need is a smartphone and the internet, and you can upload a freshly-taken selfie to the app, and you can bypass the otherwise cumbersome process of visiting the local Treasury department to get your pension.
  • Elections: Telangana made history in January this year by becoming the first state in the country to use facial recognition in elections. The State Election Commission (SEC) used the tech to authenticate voters at 10 polling booths in Kompally municipality, during statewide municipal elections. It was essentially a learning exercise for the SEC, which had hoped to use it in the Greater Hyderabad Municipal Elections (GHMC) scheduled early 2021. In a meeting held last week, State Election Commissioner C Parthasarathi was reportedly supportive of the use facial recognition technology for the GHMC elections, albeit after “thorough scrutiny”.
  • Police: Since 2018, the Telangana police have had access to the TSCOP app, which includes fingerprint and facial data of several criminals. The police are known to use the app to on random citizens without any warrant. In December 2019, for instance, Hyderabad police collected videos and photos of people attending a protest against the Citizenship Amendment Act (CAA).

The larger picture: Samagra Vedika and a ‘360 degree’ view

Any use of facial recognition or artificial intelligence in Telangana cannot be looked at as an individual exercise. It has to be put in context of the RTDAI and the Samagra Vedika (or Samagram, as it is sometimes known).

What is Samagra Vedika? In 2016, the Telangana government started working on combining its departmental databases for a “360 degree” view of citizens. In essence, databases from departments such as the treasury, education, health and transport have been combined to offer a consolidated view of a citizen. Some of these databases also have pictures of citizens, which is then fed into the RTDAI, to facilitate facial recognition checks. The system claims to not use Aadhaar.

In theory, anyone with access to Samagra Vedika can use a person’s name to query information such as address, phone number, education qualifications, passport and driving license information, vehicle ownership, income tax information, power bills and land ownership.

The architecture uses machine learning and artificial intelligence to even make relational maps of a person — it will connect their name to siblings, spouses, parents and so on. It also uses AI to distinguish between misspellings or variations in names, addresses.

Samagra Vedika was built over data collected during the Samagra Kutumba Survey, an integrated household survey conducted in 2014, just months after Telangana was carved out of Andhra Pradesh. It was praised by the Economic Survey 2018-19 for showing the “potential benefits of integrating data sets”.

In spite of this good coverage, however, the Samagra Vedika does not find mention in any state government website. The only known government document exists on the World Bank’s website, which features a presentation by the IT department detailing the uses of Samagra Vedika. The presentation was presumably prepared by the state’s IT department in December 2019.

Technologies used in Samagra Vedika, according to Telangana IT department’s presentation to the World Bank

It claims to be useful in weeding out “bogus” and “duplicate” beneficiaries in various welfare schemes, including ration cards, student scholarships, Aasara pension (for the elderly) and Rythu Bandhu (a state investment support scheme for land-owning farmers).

For instance, it highlights its success with Aasara pensions, noting that Samara Vedika was able to weed out 10% of over 65,000 new applications, saving the government more than Rs 16 crore. (See screenshot below)

Aasara pension case study

Aasara pension case study, where Samagra Vedika was used to filter applicants.

Similarly, it lists a pilot project from August 2016 in Hyderabad, when more than 100,000 ration cards were cancelled using analysis from Samagra Vedika. Ration card holders who were “well off” — decided on the basis of information about electricity bills, water bills, land ownership, vehicle ownership — were the ones that made this cut.

The department also notes public resistance to the move, after which around 19,000 reapplied for a card, and around 14,000 were able to get it back. At the end of the exercise, 86,000 cards stayed cancelled, apparently saving the state government Rs 4.6 crore every month since.

From this presentation, and from MediaNama’s interview with Ranjan, it seems clear that the state government wants to use Samagra Vedika in almost all welfare schemes. With no public disclosure available, it isn’t known where it has already been used, and where it will be in the future.

In March 2019, Huffington Post reported that Ranjan, on behalf of the state government, had offered to build a 360-degree profiling system which doesn’t use Aadhaar, just like Samagra Vedika. The offer had come just days after a Supreme Court court curtailed the use of Aadhaar to protect the privacy of Indian citizens.

Criticism of facial recognition sans law

Needless to say, the state government has faced criticism over the use of facial recognition. One of the most vocal critics has been Srinivas Kodali, an independent researcher. Kodali’s main argument has been that Telangana is using facial recognition in the complete absence of a law that governs these issues. “By bypassing the process of enacting a law,” he told MediaNama, “Telangana has been able to use RTDAI and Samagra Vedika with practically no oversight and consent of the people, effectively violating citizens’ privacy.”

The use of facial recognition tech has even been criticised by Hyderabad MP and AIMIM chief Asaduddin Owaisi, an ally of the ruling Telangana Rashtra Samithi (TRS) government. He had raised concerns about it first January, ahead of municipal elections. Last month, he raised them again, highlighting privacy concerns in relation to the Supreme Court’s Puttuswamy judgement upholding privacy as a fundamental right. He also noted that the databases used by facial recognition algorithms are “riddled with bias”. “For example, a database of arrested people will obviously have an over-representation of SCs, STs & Muslims,” he said in a tweet. He added that the tech was inaccurate when identifying people of dark complexion, women, persons with disabilities and the young.

Owaisi also promised to propose a Private Members’ Bill that would ban the use of facial recognition and remote biometric surveillance technology.

When asked about criticism from Owaisi, Ranjan said the Hyderabad MP was only unsure about the accuracy of the technology. “His feeling is that the software may not be able to pinpoint — suppose my picture is 30 years old, my appearance has changed — but obviously the technology has been perfected to handle all these kinds of things,” Ranjan said. He assured that said the software would improve using deep learning and machine learning.

The senior bureaucrat also noted that the system was nowhere near 100% accuracy, and hence not being used as the sole input for making decisions. For instance, if it is being used to verify a beneficiary of a welfare scheme, facial recognition data would only be used as supporting data. “Facial recognition should be an input, and then you can triangulate with may be three other inputs,” he said.

At the same time, Ranjan was far less accommodative of Kodali’s criticism. He said Kodali’s criticism came from the perspective of consent.

In government functions, consent is implicit — Jayesh Ranjan, IT Principal Secretary, Telangana

He took the example of a person applying for a passport. After submitting the form at the passport office, the officials initiate a process of background checks through the local police. The passport office sends the person’s details to a police officer, who is expected to check his antecedents. “The passport officer does not ask you ‘do I have the permission to send your details to Begumpet police station, or do I have the permission to send it to the bank central depository’. It is all implied— that if you have applied for a passport, this is the process,” he said.

Hence, Ranjan argued that by authenticating someone or automating a background check using facial recognition, the government is not doing something entirely new, but only automating a process done manually until now. “That is the only difference. No consent is required. We are doing something that otherwise is also being done manually. I have replaced that manual thing with a digital thing,” he said.

Ranjan also emphasised that Samagra Vedika and RTDAI does not use Aadhaar data. And data from voter IDs is only used in case of election-related work. “For non-election work, you can’t use voter ID. Aadhaar, we don’t use since it is not permitted,” he said.

It is important to remember here that Samagra Vedika doesn’t seem to be a static database. In case of facial recognition tech at least, the database takes in new facial photographs to improve the authentication accuracy using machine learning. Ranjan said that in the beginning, facial recognition using the Samagra Vedika database was only 72% accurate; now this has improved to 94%.

It seems clear that when a person is uploading selfies for “liveness checks” for DOST or RTA services or any other purpose, Samagra Vedika is indeed getting new data, which is then used to improve the accuracy of facial recognition.

Samagra Vedika not a combined database, just an algorithm

Ranjan also argued against claims that Samagra Vedika was an Orwellian database created after combining all departmental databases. The IT department has access to around 30 such databases, which remain in their own silos.

“No one has merged them together to create any kind of meta database. The only facility the only action we have taken is to create an algorithm which can pull out from all these databases. This is an algorithm,” Ranjan said.

However, it is still unclear as to how Samagra Vedika is used — that is, who is authorised to use it, and how this decision is taken. When asked about this, Ranjan said only he and his departmental colleagues had access to this algorithm. When requested for a little more detail, Ranjan politely refused.

He compared it to a bank: he said that if one were to ask a bank who amongst its employees has access to the details of all account details — which someone obviously does — the bank would refuse to answer. “Do we ask the bank who has [this information]? […] Do we ask the bank who has it, tell the name of the person, who is he, is he authorised? Many of these things are taken based on trust,” he said.

The situation was similar to the Aadhaar database. Someone has access to the entire Aadhaar database, and knows the numbers of every citizen in the country. “Who is that person? Even [the] Supreme Court went into Aadhaar security protocols. There is something called ‘security by design’. Even [the] Supreme Court did not ask that ‘tell me the name of the person who has access’ and all,” he said.

Ranjan reiterated that only people working in his department have access to the algorithm, and reminded that this was just that, an ‘algorithm’, not a database.

But how is the need to use Samagra Vedika decided?

Ranjan explained that Samagra Vedika could be used to query only one citizen at a time. And, he added, it is only used when a government department needs the IT department to check for this citizen’s eligibility to a welfare programme.
He gave the example of the two-bedroom-house programme, wherein the government has been allocating 2BHK homes to underprivileged members of the society (this is where Samagra Vedika has been of utmost use to the government, according to Ranjan). Suppose the GHMC were to filter through a batch of applicants, it would ask the IT department to use Samagra Vedika to query their antecedents. This information entails the applicant’s property tax (paid to GHMC), electricity bills, vehicle ownership and so on.

“GHMC will only have one piece of information, which is if you pay property tax or not. GHMC’s own database will not give them the comprehensive answer if [an applicant] is eligible for a 2BHK. But suppose I tell GHMC that I checked this person’s electricity bill record, and he pays Rs 20,000 electricity bill a month, GHMC will be immediately alerted, that he is not a poor person. Suppose I tell GHMC that there are three cars registered to this person, he will be disqualified,” he said.

Ranjan repeatedly assured that the rules governing the access to Samagra Vedika have indeed been formalised. “There are a lot of internal checks and balances. No unauthorised person has access to this,” he said. These rules, however, aren’t publicly available.

But isn’t this still too much power?

The state government has faced criticism from the political opposition about the use of Samagra Vedika. Dasoju Sravan, national spokesperson for the Congress party and a former MLA candidate during the Telangana Assembly elections in 2018, believes that TRS’ victory in the polls was a result of its “tailormade” electoral strategy, which it crafted using Samagra Vedika. He, and much of the Congress party in the state, believes that TRS has access to Samagra Vedika, which allows it to have far more in depth-information about the state’s populace than it should have had.

In a conversation with MediaNama, Sravan also argued that the Samagra Vedika was an invasion of privacy, and the state government should have taken the people’s consent before aggregating their data.

“The government claims that only the IT department has access to Samagra Vedika. But how can we simply believe this? How do we know some of these employees, or anyone else, are not supplying data to TRS?” — Dasoju Sravan, AICC spokesperson

When asked about the incorporation of facial recognition into Samagra Vedika, however, Congress hasn’t made up its mind. Dr Sravan said some members of the Congress party believe facial recognition tech could help reduce “bogus voting” during elections, while others want a ban on it. “We still have to discuss amongst ourselves on this subject,” he said.

How about passing a law, making the whole thing legal?

Arguably, a lot of criticism around Samagra Vedika could be solved by passing a law formalising its use, along with a laundry list of checks and balances. Srinivas Kodali, the independent researcher, argued that if Telangana does want to use Samagra Vedika and facial recognition along with it, it should pass a law.

Kodali explained: “Telangana is one of the first states to experiment with these technologies on its population. The bureaucrats are building a technology and e-governance system. But they never thought they needed a law because it concerns matters of governance, and that comes under the executive.”

However, “Supreme Court judgments have ruled that you can’t run these kinds of systems without laws, and that these systems need to be proportionate. None of this has been followed in Telangana,” he added.

When asked about the need for a law, Ranjan said the Telangana government is waiting for the national law — the Personal Data Protection Bill, 2019, which is under consideration of a Joint Parliamentary Committee.

“Based on [the PDP] law, if that law takes care of our interests, that is good enough. If that law still has some grey areas, where states can additionally legislate, we are happy to do it. We are just waiting for that,” he said. Ranjan said that the Telangana government has sent detailed comments on the PDP Bill, adding that it welcomes the law.

What’s the next frontier?

Ranjan, meanwhile, said that the state already sees value in facial recognition in view of the pandemic. “Right now, because of this pandemic, wherever biometric attendance is being used, this can be a very good alternative. This is one area that [facial recognition] can replace,” he said.

The state government also sees a potential opportunity in the distribution of ration. Ranjan said the verification of beneficiaries is still done using POS machines, which are based on fingerprints. Similarly, digital payments at MeeSeva centers in the state are also authenticated using fingerprints. In both these areas, he said, the government could try to use facial recognition in the near future.

Also read: