Facial recognition technology (FRT) could soon be used in the distribution of ration in Telangana. Ration shops make a great case for the use of FRT during the pandemic, said Jayesh Ranjan, the state’s IT and Industries secretary, in an interview to MediaNama. He told us that facial recognition tech could replace prevalent biometric authentication systems, which uses fingerprints, with a touch-less one. Ranjan also addressed criticism around the state’s use of facial recognition, artificial intelligence, and more. On a question about the need to collect citizens’ consent before collecting their data, Ranjan said it wasn’t necessary since “consent is implicit” in government work.

Ranjan also tried to defend Samagra Vedika, which Ranjan claimed was an algorithm to obtain information from several databases, rather than being one master database itself. Samagra Vedika has the power to query from more than 30 department databases (like the treasury, tax, GST, RTA and more) to create a “360 degree view” of virtually any citizen.  “No one has merged them together to create any kind of meta database,” Ranjan told us. He also assured us that Samagra Vedika was accessible only to employees of his department, and that there were “checks and balances” that govern this access.

We also talked about the paradox of the state government using a cutting-edge AI tool to predict Covid-19 hotspots, while simultaneously refusing to make more simple, people-facing apps such as a real-time dashboard to monitor vacancies of beds in hospitals (although a dashboard was announced in July, there has been no word on it since).

An edited excerpt of the interview: [Note: the excerpt has been slightly edited for brevity and clarity]

On use of Artificial Intelligence, and absence of public-facing tech

MediaNama: Telangana was one of the first states to resume manufacturing activity. How was this done?

Jayesh Ranjan: The decision was made based on an IT decision support system [based on AI] which told us that new cases were unlikely to happen in the industrial parks and areas and IT enclaves. With that confidence, we were able to take this bold decision that — okay, let’s reopen everything, without any restriction on number of staff or timings. So far, four months have gone by since the reopening started in May and all these predictions have turned right.

So there are tools. So when you said there isn’t a tool to predict beds, It is not that these tools can’t be created. The user department has felt that this is not a big requirement for us. If the health department tells us to create a tool, it will take an hour to create the tool […] So, many people, without having proper understanding, try to put the blame on the IT department that we are sleeping. The truth is that creating such a tool will take only 30 minutes.

MediaNama: So the capacity to create such applications does exists?

Jayesh Ranjan: There is no capacity, it’s a child’s play. Students in their first year of engineering can create this tool. It’s a simple thing. So there is no online facility to monitor how many beds are available, this is a very very elementary thing.
Recently, we created a tool for the Telangana State Medical Services & Infrastructure Development Corporation (TSMSIDC) which gives them real-time, minute-by-minute information about the consumption of medical supplies like PPE kits, masks at hospitals. It tells them in which hospital the number of masks us dipping below a minimum threshold, where you should reorder — all this is automated.

MediaNama: When was this deployed?

Jayesh Ranjan: More than three months ago. That is why you never come across [news] from any government hospital having run out of PPE kits, gloves and other medical supplies.

MediaNama: I would have to disagree with you on that. For instance, before [MediaNama] I was with [previous employer] where I had reported on the shortage of supplies at OGH [Osmania General Hospital in Hyderabad].

Jayesh Ranjan: It is not because the department of medical infrastructure doesn’t know they have run out [of supplies]. Sometimes there is a lag in ordering and receiving the supplies. The reason why PPEs are not supplied to OGH or to a Khammam hospital is not because the information is missing. It is not that they didn’t know the supplies are running low. We created a system wherein the moment you requisition 10 masks from the store, the software updates it in central database. These are very preliminary tools, it doesn’t require PhD level computer scientists. In engineering colleges, these are given as student projects. There is no question of capability. Suppose I tell our IT community for a tool, 100 of them will come forward.

The tools which are relevant for us, we have created. Those tools have worked well for us. Some of the decision support systems tools are very very advanced. The kind of algorithms they have created, they are the work of PhD level data scientists. They have access to very advanced IT tools, commuting tools and we are making use of it. Those tools which have not been created, have not been found relevant by the user department. If the user department feels that these are not in the larger context, they may have taken the judgement that making data available to the public will add to more panic or lead to some other imbalance, they have chosen against it. So to say that the tool is not there, it has

MediaNama: That it has not been created is not because it cannot be created but it is the decision of that department to not make it public?

Jayesh Ranjan: Yeah, but because of some reasons. It is not as if they have decided to not make it public based on whims and fancies. They must have looked at the larger context of the state and felt that this is not the right time to allow public access to these kinds of things. If there are some larger issues involved because of which some decision has been taken, we should respect that. See, people who take those decisions also have responsibilities. They are not irresponsible people or one should point fingers at them.

On use of AI and FRT because of pandemic

Jayesh Ranjan: We have ourselves created some solutions during the pandemic which will be use for governance. For example, as you know biometric attendance, you cannot give your fingerprint. So we have created a very good facial recognition, called RTDAI [Real-Time Digitial Authentication of Identity]. Some newspapers have written about. That is a good product we have created.

Similarly, e-office, like you handle all the files electronically, you don’t have to physically go to the office to clear a file. E-office is not our product but we have enabled all government departments including the 33 district collectorate offices to function on an e-office system. Similarly, we are in the process of setting up an online grievance redressal system.
Again, we are in the process of developing something which, tomorrow, once pandemic becomes normal also, will be of lasting use to the government.

MediaNama: In the last few months, we have seen a lot of activity in the area of facial recognition. In a span of two to three months, we have seen it being deployed in RTA and degree admissions. Has the pandemic actually accelerated the push towards the technology, or was it always on the cards?

Jayesh Ranjan: No no, the pandemic has definitely accelerated it. See, RTA application is completely pandemic driven. RTA had no inclination to try it out, but because of the pandemic they realised that people are not coming. RTA is also a revenue generating department. Every license you get some fees, some money. They realised that they cannot just close shop and just sit idle. And if your DL is expired, you do require a valid one. I am also handicapping people by forcing them to drive on the roads with an expired licence. I am creating public risk that way. So this has prompted them that the solution is available and they can’t run in the normal way.

On criticism of use of AI, Samagra Vedika

MediaNama: Now to come to criticism the govt has faced. Even the govt’s ally [Hyderabad MP and AIMIM chief] Asaduddin Owaisi has criticised the use of facial recognition. What is your assessment of this?

Jayesh Ranjan: See, the criticism which comes from Mr Owaisi is that he is not sure whether the technology is accurate or not. Because his feeling is that the software etc may not be able to pinpoint — suppose my picture is 30 years old, my appearance has changed — but obviously the technology has been perfected to handle all these kinds of things.

MediaNama: In January, when facial recognition was used in elections at Kompally — it was a learning exercise as we understand it — the accuracy at that time was 78%? Has there been any improvement?

Jayesh Ranjan: No, the accuracy was 87%. And, we found the causes in the 13% cases where it did not match. Largely, a dark background was one of the reasons — like if the picture is not standing out and is merging into the dark background. Obviously, as you say, we will now improve the algorithm. When we started the Samagra application [Samagra Vedika], the accuracy was only 72%, today we have brought it to 94%.

[Readers should note that, per an RTI filed by digital rights body the Internet Freedom Foundation, the Telangana government said that the accuracy of the facial recognition system used in the civil polls was 78%]

MediaNama: How?
Jayesh Ranjan: This is the whole principle of machine learning. There is a concept called ML. The more data you feed, all the inaccuracies are ironed out. This is how it will be. This is the meaning of big data.

MediaNama: What was the initial database based on? The photographs? The DL database or where exactly?

Jayesh Ranjan: All photographs that are available on public domain.

MediaNama: Could you be specific?

Jayesh Ranjan: There are 30 data sets in many of which pictures were available. Whichever database has pictures, we can access it.

[According to a presentation presumably prepared by the state IT department, Samagra Vedika has access to data of electricity bills, water bills, property records, pensions, vehicle database, ration cards etc.]

MediaNama: Does this also include voter ID and Aadhaar?

Jayesh Ranjan: No, voter ID is only used for election-related work. If you are using it for an election, then you will only use voter ID. For non-election work, you can’t use voter ID. Aadhaar, we don’t use since it is not permitted. Aadhaar is never used at all. So Mr Owaisi’s criticism is about his belief in the ability of the software. The software will improve using deep tech and machine learning. No software will start giving 100% accurate results on day one.

MediaNama: There is current research about inherent bias in FRT because of bias in the databases they use. That is a valid concern right, not just from activists but researchers as well?. Some law enforcement agencies in the US have decided to not use FRT because of this fact.

Jayesh Ranjan: There are all kinds of questions. You have to balance everything. Suppose you are going to use FRT for life and death decisions, obviously, I won’t recommend it. But if you are taking it for a general purpose — whether a person who is going to the college, is he that student or some other student. And I say he is not that student, and I debar him for that day’s attendance. It is not a life and death situation. A student’s one day’s attendance is stuck off because of FRT.

On whether government needs citizens’ consent for data collection

MediaNama: But isn’t that subjective? For instance, if that student has a very strict father who comes to learn he has bunked?

Jayesh Ranjan: No, no I am just giving an example. The point is that suppose some very life and death decisions are there and need 100% accuracy, we are still far from that. So to that extent we need multiple checks. FRT can be one of the inputs. It can definitely be an input which will help you make your judgement. FRT should be an input, and then you triangulate with maybe three other inputs, and then if you are sure the identity is proven then go ahead and do it.

To answer what people like [independent researcher] Srinivas Kodali say, their criticism comes more from a consent point-of-view. So my answer is like this, in any kind of government function, consent is implicit. Now let me give an example, let us say you apply for a passport. You go to the passport office in Secunderabad, you fill a form and you hand over that form in the passport office. Now does the passport officer pull out a passport instantly, put your name and hand it over to you? Is it done over the counter? [MediaNama: No It isn’t] So you don’t get the passport instantly because there is a process of investigation. The passport officer sends you details to a police officer, the police officer is expected to do some verification, he sends it to the bank to check whether you are involved in any bank fraud or any kind of bankruptcy or any kind of thing.

So when you apply for a passport, the passport officer does not ask you ‘Can I, or do I have the permission to send you details to Begumpet police station, or do I have the permission to send it to the bank central depository? It is all implied— that if you have applied for a passport, this is the process. If you apply for a ration card, if you apply for a house site, for some scholarship, pension it is presumed that these enquiries will be done, background verification will be done to check if you are eligible or not. What we have replaced is that instead of doing an arbitrary, subjective and manual enquiry, I have converted it into a digital enquiry with this kind of a sophisticated tool. So, this is the very simple logic that unfortunately people like Mr Kodali don’t understand. Whatever was done manually was automated. So earlier, the police officer would randomly come to my neighbour and ask ‘do you know him’, ‘is he involved in a criminal case’ in a very random kind of way, my neighbour will say ‘no no he is a good person. He is a fine gentleman and his signature will be taken. All this arbitrariness, subjectiveness is being taken away by doing enquiry using databases. That is the only difference. No consent is required. We are doing something that otherwise is also being done manually. I have replaced that manual thing to a digital thing.

MediaNama: A lot of people like Kodali have also said that in case the Samagra Vedika that the state has built, did not take the consent of the people to basically combine them and make a master database.

Jayesh Ranjan: There is no master database. Again, that is a very strong and serious misconception. I have clarified on many occasions that there is no meta database that ignorant people are presuming. The 30-31 databases we have access to remain in their own structures with their own commanding officers, in their own servers, with their own security protocols. No one has merged them together to create any kind of meta database. The only facility the only action we have taken is to create an algorithm which can pull out from all these databases. This is an algorithm.

On who can access Samagra Vedika

MediaNama: Who has access to this algorithm?

Jayesh Ranjan: Me, and my departmental colleagues. It is with the government officers.

MediaNama: Have the rules been formalised?

Jayesh Ranjan: Yeah yeah, of course. There are a lot of internal checks and balances. No unauthorized person has access to this.

MediaNama: Could you tell me a little more?

Jayesh Ranjan: I can’t. It is like this — someone can ask all the banking… hundreds of people who have bank accounts in a particular bank, how much money is there in the account, who has this information? So obviously, someone in the bank has. Do we ask the bank who has it, tell the name of the person, who is he, is he authorized? Many of these things are taken based on trust. You can say that someone in the country has access to the Aadhaar database, he knows the Aadhaar numbers of every citizen in the country. Who is that person, even Supreme Court went into Aadhaar security protocols. There is something called ‘security by design’. Even Supreme Court did not ask that ‘tell me the name of the person who has access’ and all.

I mean, whoever is working in our department has access. No one has access to this database, the access is to the algorithm. The databases remain with the department. For example, who are the car owners in the city. I mean someone has the database, RTA has the database. I have access to that database for only a particular individual. For example, suppose I want to check how many cars are there in your name, I won’t be able to check randomly.

MediaNama: If you had to use my name in that algorithm, you would perhaps know everything about me right?

Jayesh Ranjan: Yeah, of course. I will know everything about you, provided there is a need to know about you. I will not be doing it randomly.

MediaNama: How is this ‘need’ decided? What’s the process of you having to actually use that algorithm?

Jayesh Ranjan: Yeah yeah, so those needs are defined by eligibility under a particular government program. For example, in the transport department, there is a provision that if you are registering a second car, you have to pay a higher tax. So when you apply for a car, we run this algorithm to check whether you already have a car or not.

MediaNama: The RTA does this anyway right?

Jayesh Ranjan: Correct. That is what. So no one can question them. So the point is that I could have done this manually or this. When you come to the RTA office, I could have given you a form to fill in which there could have been one box ‘is this your second car’. You could have said no.

MediaNama: Yeah, but the RTA doesn’t need everything else right? It just needs to know how many vehicles I own.

Jayesh Ranjan: Exactly. So this is the principle.

MediaNama: The idea that somebody has access to my data, provided, as you say, after all checks and balances. They still have 360 degree data. How exactly is that decided? If you want to find out about me, my family or everyone that is connected to me. When is it necessary for you to do it? How do you decide that you need to do it?

Jayesh Ranjan: Only when you have applied for some government benefit and eligibility checks has to be run on you.

MediaNama: But that can be taken care of by individual departments right?

Jayesh Ranjan: No no, how? For example you apply for a house site. The revenue department decides whether you have to be given or not. Let us say you apply for a 2BHK house. For instance, GHMC has to decide whether you have to be given a 2BHK or not. So what information do they require?

MediaNama: Like, how much money do they make, how many cars they have?

Jayesh Ranjan: Precisely. GHMC won’t have this data. GHMC won’t know if you have four cars or two cars. GHMC will have only one information about you which is whether you pay property tax or not. GHMC’s own database will not give them the comprehensive answer if they are eligible for a 2BHK. But suppose I tell GHMC that I checked this person’s electricity bill record, and he pays Rs 20,000 electricity bill a month, GHMC will immediately be alerted, that he is not a poor person. Suppose I tell GHMC that there are three cars registered to this person, he will be disqualified.

Right now all this was done manually in an arbitrary way. GHMC will send some revenue inspector to your home. He will ask randomly, sometimes he will be bribed, then he will give kind of a false letter that ‘yeah he is eligible’. I mean everything was very ad-hoc, subjective. The errors of that process have been covered up using better use of data.

MediaNama: How does it happen? Is there a nodal officer in GHMC who coordinates this?

Jayesh Ranjan: They tell us the list, that these are the 20,000 people who have applied. Then once that kind of request comes, then tell this back: out of these 20,000, 18,000 are OK, and 2000 have cars to their names, or high electricity bills. What filters are to be used, we use and then get that information.

MediaNama: So GHMC itself does not have the authority to do it. You have the authority to do it. And, they can ask you?

Jayesh Ranjan: They have access only to their own database. If checking their database suffices and gives them the answer, they will stop there. In these kinds of cases, when their database has nothing to tell them, they have to get data from other databases, they bring it to our notice.

MediaNama: So the GHMC itself doesn’t have access to this database. Only people in your department, including yourself, have access. And you can provide this comprehensive data, and nobody else can?

Jayesh Ranjan: Absolutely. No one else can.

MediaNama: Could you also give me a little more examples like how you said 2BHK.

Jayesh Ranjan: All these things. 2BHK, pensions, ration cards. We have used it in ration card, we have weeded out lots of ineligible ration cards.

MediaNama: Any other welfare schemes?

Jayesh Ranjan: I mean all welfare schemes. Scholarships, pensions, ration cards, 2BHK, income certificate. This application is only used for verification of eligibility, whether you are eligible for a particular scheme. And this is data you only have provided. When you buy a car, obviously your name is registered. It is not that some data is being collected behind your back surreptitiously.

MediaNama: So it isn’t spying?

Jayesh Ranjan: There is no spying. All this database, you have come into this database by participating in a certain activity. Like suppose you have taken a commercial connection in your house, so your name will figure in the GST database. If you have some kind of large property where you exceed the property tax slab, then your name will come in that higher slab of the database. Nothing is collected afresh, there is no meta database of some crores of data lying anywhere. All data remains with the department, they are the custodians. They can change the database whichever way they want. The appreciation which we get from people who know about technology is that first of all there is no identifier. We don’t use Aadhaar. There is no common identifier. We only use names, father’s names and addresses. And you know in India there are lots of misspellings. Names, father’s names are spelled differently. Addresses are never precise. Despite all these imperfections, we have been able to create this algorithm.

Similarly, database formats are different. Someone is in Oracle, someone is in Office, someone is in some other format. Despite varying natures of database formats, despite this kind of inconsistencies, errors, we still have been able to create something. Of course, when we give this information, we tell that the accuracy is this much. Suppose I tell you 2000 are ineligible for 2BHK houses, you should do one more round of enquiry at your level before you blindly weed them out.
Every department uses this as an input not the final word.

On whether Telangana will pass laws to regulate use of FRT, AI

MediaNama: Is there any plan to pass legislation to make everything official, the use of RTDAI, Samagram.

Jayesh Ranjan: We are waiting for the national law, the PDP [Personal Data Protection Bill, 2019] law to come. Based on that law, if that law takes care of our interests, that is good enough. If that law still has some grey areas where states can additionally legislate, we are happy to do it. We are just waiting for that.

MediaNama: This has been one of the biggest criticisms, that there isn’t a law. 

Jayesh Ranjan: Yeah, the law is about to come.

MediaNama: I was just about to ask you: how you are preparing for PDP law

Jayesh Ranjan: No no, we welcome it. States were asked to give their comments, we have given detailed comments.

On where FRT could be introduced next

MediaNama: Where else will FRT be introduced?

Jayesh Ranjan: Right now because of this pandemic, wherever biometric attendance is being used, this can be a very good alternative. This is one area that we can replace all this biometric based attendance.

MediaNama: Anything specific?

Jayesh Ranjan: One good use case can be our ration shop. Lots of people take ration, their verification is done using POS machines, These POS machines are biometric based. That can be one area where we can try this out. Similarly digital payments — IF you go to our MeeSeva centres, lots of people go there to do digital payments using biometrics. This can be one area.

MediaNama: Any immediate plans? Or timeline?

Jayesh Ranjan: We have to discuss with the respective departments. But Civil supplies can be a very big opportunity for us.