You are reading it here first: India’s plans of deploying a controversial national level facial recognition surveillance tool are gathering pace. In a meeting held on Monday afternoon, between potential bidders for the Automated Facial Recognition System and officials from the National Crime Records Bureau, bidders enquired how many law enforcement agencies are expected to use the surveillance tool at the same time, and if there is a possibility to allow a consortium of companies to place a joint bid. Concerns around the quality standards of the 1.5 crore strong facial dataset which is to form the foundation of the surveillance tool, were also raised, as was the question if that dataset will include pictures of juveniles. NCRB’s deputy director Prashun Gupta, and joint director of the Crime and Criminal Tracking Network and Systems (CCTNS) Sanjay Mathur, were present at the virtual meeting.

The National Crime Records Bureau — the agency which has been tasked to deploy the AFRS — had released a revised tender last month, after about 3 months it started reworking it. The large scale surveillance tool is a centralised web application, and is expected to be the foundation for “a national level searchable platform of facial images”. In its current iteration, the tender document has said that the deployment of the surveillance tool will neither involve the installation of CCTV cameras nor will it connect to any existing CCTV camera anywhere. The deadline to submit bids for developing the tool is August 6.

The companies that seem interested in building the large-scale surveillance tech include not only state-owned Indian companies, but also companies from countries like Israel, France, and Japan. Moreover, even companies like EY, that don’t directly develop facial recognition services look keen on developing the system. Some of the companies that were present at the pre-bid conference, and also raised concerns, included: BECIL (Indian state-owned), Idemia (French), Tech5 (Swiss), Thales (French), Anyvision (Israeli), Vara Technology (Maharashtra-based), Innefu Labs (Delhi-based), EY, and NEC Corporation (Japanese). There were more participants at the meeting, but points were raised only by representatives of these companies.

Concerns raised by bidders at the pre-bid meeting

Number of users at any given time: Both the Broadcast Engineering Consultants India Limited (BECIL), an entity under the Information and Broadcasting Ministry, and Japan’s NEC Corporation raised questions about the number of concurrent users that will use the facial recognition system. A representative from BECIL asked about the number of video/image searches that will be carried out, and the amount of response time that the NCRB was expecting.

An NEC representative meanwhile asked how many match requests will the backend engine have to process per second. To this, NCRB’s Gupta replied:

“There will be approximately around 16,000 police stations, 7,000 “higher offices”, and we are also planning to give a mobile app which will put load on the system. If we consider giving this mobile app to 5 people in a police station, then that takes the number of users to 80,000. So if you design a system which can handle 2,500 requests at a time, I think that it will be able to take the load.” — Prashun Gupta, NCRB

Concerns against mandating NIST standardisation: The updated tender document for the facial recognition tool has mandated to have bidders or any member of a joint venture to have participated in NIST FRVT evaluation program by 2016 or later. Earlier, this was only a “desirable criteria”. However, several companies in the conference objected to this criteria. “NCRB should not be restrictive to NIST since it is only a benchmark. A fair chance has to be given to Indian companies. NCRB is open to ask for credentials, about projects in law enforcement etc. That is acceptable,” one company said.

  • One bidder argued in favour of NIST: However, not everyone was opposed to NIST certification being a mandatory requirement. “You’ve currently asked for NIST evaluation post 2016, but by the time we complete the project, it’ll be early 2021. So I request you to consider updating that condition to NIST evaluation post 2019,” Ameya Bhagwat director of business development at Swiss-based AI company Tech5 said.

Will the AFRS talk to similar international surveillance systems in the future? The discussion around NIST also led CCTNS’ joint director, Sanjay Mathur to hint that India’s surveillance tool could potentially communicate with similar tools deployed by other governments. “NIST participation isn’t mandatory, though bidders who have participated in it will receive some additional marks, but we have to follow that standard, so that the system can communicate with other systems as well in future, since most of these systems are based on the NIST standard, Mathur said.

Chances of a ‘backdoor’: Innefu Labs, a facial recognition and AI company, which counts Delhi Police among its clients argued that allowing foreign companies to build the surveillance tool could lead to potential security issues. “The kind of facial data which NCRB has in its repository isn’t very common facial recognition data which basically belongs to criminals and radical people. With foreign OEMs there is a chance of backdoor entrance. This is the only hindrance in the entire project. NCRB should be looking for Indian credentials,” a representative from this company said.

Concerns around matching faces modified after plastic surgery: Israel-based AI company AnyVision, which was also present at the meeting said that the requirement to match faces modified by plastic surgery needs a second look. “Plastic surgery is slightly open ended as it can end up changing features drastically. Either relax it or make it less open ended,” the company’s representative Sanjay Gandhi said.

To this, Gupta replied: “In the hundred test cases which we are going to give you there will be few cases for plastic surgery and let us see how the results are if everybody says it is same for everybody and if somebody gets an additional point. So, if somebody has a solution it is well and good if you lose marks on the couple of test cases.”

Concerns around image quality standards: Gandhi also enquired why the tender required the facial database to match ISO standards. Gupta said that CCTNS, which will be the among the biggest databases for the AFRS “is following e-gov standard of government of India, So that kind of image quality we are prescribing in CCTNS which is the primary source of data. There will be variation in data coming in from different sources, which is why this kind of standardisation is needed for basic benchmarking,” he said.

Gupta also said that in 1:1 and 1:N matches, there will be test cases which will be given to bidders so that they can test their facial recognition algorithm regarding compatibility with low quality cameras and high quality images. “In some of the cases there will be a need to change the attributes of the image itself to get it matched from the database, which is the live practical conditions,” he added.

Another representative from NEC Corporation asked if the test facial database worth 1.5 crore images will have all images taken in a controlled environment. He also asked if each of these images is unique, or if there are multiple images of the same person’s face. NCRB replied that images will not be taken in a controlled environment, and there could be multiple pictures of the same person if they have committed multiple crimes.

Will children’s faces be a part of the facial dataset? The same NEC representative also asked whether there will be images of children in this database, to which Gupta replied: “How does it matter?” He then added that there will be images of missing children in the database, but did not specify if the database will have images of children who might have been charged for a crime.

Video to video match: “My query is related to the technical evaluation criteria where there’s a clause to carry out video to video match. What is desired by the NCRB, and what is the non-ingested criteria,” asked Rohit Yadav, who is solutions manager for the public safety unit at France-based security company Idemia.

Gupta explained that a “video to video match is where you have two videos and there’s a person in both those videos. One there will be a repository of videos and  a video containing the same person will be given which has to be matched against the repository database. You should be able to locate that the person is available in which video in the repository and at what time, once you run the reference video against the repository.”

Allow a consortium of companies to bid: Maharashtra-based IT company Vara Technology, which was represented by its associate vice president Vikas Kumra enquired whether a consortium of companies could place a joint bid. The current tender allows only independent bidders or a joint venture company to bid for the developing the facial recognition tender. A representative from the French multinational Thales Group also raised a similar concern. “In a consortium, the legality exists, and the liability of both the parties will be there,” Srinivas Ayyagari, who was representing Thales said.

To this, Mathur said that this particular condition was discussed and the Home Ministry, and particularly its financial division decided that only joint ventures will be allowed to bid. He did not specify a reason as to why the Ministry came to this conclusion.

However, Kumra said that if they were to go for a joint venture, it’ll take them months to put something together, by when the deadline to submit bids would’ve already passed. “It could take months, a joint venture company, a new company, a startup and all the documentation process, everything takes few months, not few days,” he said. Mathur responded by saying that this is why only those companies who already are a part of a joint venture are perhaps in a better position to bid as a joint venture, and carry forward with the project “satisfactorily”.

Kumra persisted that the department should consider allowing a consortium of companies to place a bid, to which Mathur asked him to submit his query for consideration.

EY a potential bidder?

It appears that even Ernst and Young is interested in bidding for the facial recognition system, as a company representative essentially urged the NCRB to allow bidding from corporations who can procure separate products from different OEMs to put the system together. “Why should there be a requirement for a joint venture”, a person from Ernst and Young asked. “If I as EY purchase one product from Microsoft, in that case I don’t have to form a joint venture with the company. I’m bringing in their product, taking the responsibility for the performance of that product”. It’s worth noting that EY doesn’t manufacture most components that will go into the AFRS and as a result can’t bid as an independent bidder, but what they can do is buy things from several companies to put it together, essentially acting as an aggregator of sorts.

However, Gupta said that the decision to allow only independent companies and joint venture companies to bid was “well thought of” and “there were some reasons behind why we wanted it that way,” he said, without specifying what those reasons were.

Read: NCRB drops CCTV integration clause from updated facial recognition tender, eases bid qualification criteria for vendors

*Update on July 15, 1:15 pm: Innefu Labs, a Delhi-based AI company, told us that its representative had raised the point about backdoors. Story was updated with attribution to Innefu Labs.