Unlike drone operations in Delhi and Mumbai, some drones deployed in Amritsar as part of keeping an eye out on lockdown violators were far more sophisticated: They were equipped with an AI-based onboard software capable of detecting humans from around 400 feet away, along with the distance between two people, and could send the police the GPS coordinates of a person violating the lockdown norms. The drones were supplied to the police in Amritsar by a company called Skylark Labs, which has offices in Hyderabad, Amritsar and San Francisco, its CEO Amarjot Singh told MediaNama. The company started the surveillance for the police in about 4-5 areas in Amritsar in the middle of April, and did it for a couple of weeks, Singh said. 2 drones made by DJI (which has openly criticised India’s drone regulations) were used. The Times of India had first reported this.

“Police forces around the country used drones to assist in enforcing the lockdown and monitor areas, but all of that was essentially manual monitoring of live feeds. That way you can maybe fly many drones, but people will have to monitor all the feed manually,” Singh said. Skylark Labs’ drones on the other hand are equipped with an AI based system which can alert the police in case of any lockdown violations without someone having the monitor a live feed all the time, he claimed. He also told us that the operators flying drones for the police were Skylark Labs’ employees, and that there was no financial assistance offered by the police to them. We have reached out to Amritsar city police commissioner Sukhchain Singh Gill directly for more details. We had reached out his office previously on 16th May, and haven’t heard back from him.

How the company trained its AI to detect humans and the distance between them

The onboard software constantly analyses feed collected by the drone’s camera and is tasked with generating an alert in case it sees any lockdown violation. For instance, Singh explained, in the night when people aren’t allowed to roam, the drone will instantly generate an alert and send GPS coordinates of the violator to the police control room. Similarly, it can generate an alert if people violate social distancing norms, as the algorithm has been trained to gauge distance between two humans.

Fed with dataset of how humans look from a distance and height: But for that to be possible, the drone’s software would require to know how humans look from a particular distance and height, before it can analyse the distance between them. The software is essentially based on a machine learning model, Singh explained, when we asked him how it could ascertain when people are too close to each other. It has been trained on a dataset of aerial visuals to identify how a human being looks like from any given distance. Most of the aerial footage he said is collected by flying drones over the NIT Warangal campus, where the company has an office.

Read: Inside the deployment of drones in Delhi to contain COVID-19

Singh also shared a video with MediaNama (embedded below) to show how the drone’s onboard software perceives people from a distance. The drone was set to identify a distance of 6 feet between people, and any distance more than that showed a green box around the moving humans under a screen called “acceptable”. For people who were walking at distance of less than 6 feet, a red box surrounded them and they could be seen under a separate screen titles “not acceptable”. The video is also available on Skylark Labs’ YouTube channel.

Not just during the day, the software is also trained to detect people moving around in the night, which is not allowed, and highlighted them with a red box. This suggests that the drone’s camera is equipped with night-vision as well. Feeds from multiple drones can be seen on one screen at the police control centre. Apart from detecting humans and the distance between them in real-time, the software also records data on how many people it has detected and the number of alerts it has generated. This information, too, is generated in real-time as per the video.

Company claims no data retained: Singh claimed that as part of the project it did not collect any footage, and no data was retained. The only purpose the drones served was to send real-time feed into the police control centre along with the analysis that its software was doing. The video that Singh shared with MediaNama, which had footage from the drone’s view was collected “only to make that video” and was later destroyed, Singh claimed. He also claimed that the drone’s camera was “tilted” to an angle while capturing the video in order to avoid any kind of identification.

Read: Inside the deployment of drones in Mumbai to contain COVID-19

DGCA’s approval not taken

In what has emerged as a trend in drone surveillance of the lockdowns, the drones that took off as part of this surveillance exercise were not authorised by aviation regulator Directorate General of Civil Aviation. According to current drone regulations, a clearance is required from the DGCA before each drone takes flight. Called NPNT (no permission no takeoff), it is conceptually a green signal without which drones aren’t authorised to fly. However, it is worth noting that as a workaround this condition, the regulator has created a portal specifically for government bodies to deploy drones for COVID-19 related work such as aerial surveillance and photography and for public announcements. Government bodies can also use the portal to authorise third-party drone service providers to operate drones on their behalf. The surveillance in Amritsar which Skylark Labs did ended before the portal was launched.

Read: In Kerala and Telangana, police turn to drones to enforce COVID-19 lockdown