The algorithm will see you now

Last week, I set out to book a doctor’s appointment online. Before I even glimpsed the calendar, an algorithm had swiftly “triaged” me. With seemingly unnerving certainty, it informed me that a dermatologist wasn’t the answer—perhaps a lifestyle coach instead? The last time someone made such a leap, it was a distant aunt at a family gathering, peering at my dull skin and declaring, “You suffer from exhaustion.” 

But unlike my aunt, this algorithm wasn’t speculating. It really was “trying to help, beti”.

There was a time when artificial intelligence was distant and abstract. Now, it’s everywhere--our phones, at our banks, inside our grocery apps, and even overseeing our morning walks. The algorithm has arrived.

I first noticed this shift when the food delivery app sent me a notification: “Your usual dal-chawal combo is just a click away.” What made it conclude it was my “usual” fare? I had ordered it five times during a hectic month. The app had noticed and kept score. And was also judging me silently. But here’s the strange part. I don’t find all this creepy. Some days, I actually find it… comforting.

There’s a quiet appeal to a world where things predictably appear: music tailored to your mood, routes optimised for traffic, answers ranked by “relevance”. In a country where queues are long, forms are endless, and phone calls to customer care are like shouting into the void, the algorithm feels like a shortcut. A secret backdoor for the mildly overwhelmed.

The problem, of course, is when the algorithm misunderstands.

Like the time a friend applied for a loan and got rejected instantly. No human in the loop, just a silent “Based on your profile.” What was the AI factoring in, I wondered, her residential location or her previous month’s impetuous purchase? Or, when job applications disappear into applicant tracking systems, never to be seen again, because your CV didn’t have the right keywords.

The new gatekeepers don’t wear uniforms. 

The more I work in tech – especially legal tech – the more I see both the hope and the risk. AI has the power to bridge yawning access gaps, yes. But it also runs the risk of automating indifference. If a human bureaucrat is hard to reach, an algorithm is impossible to argue with. The algorithm doesn’t know how to listen, not really. It just calculates.

And yet, it’s here. It helps courts around the country manage case flows, supports students’ quest for scholarships, alerts farmers of impending pest attacks, and helps doctors prioritise patients. It is, in a very literal sense, seeing us now.

The problem is to the algorithm, you’re not a person – you’re a probability with a history.

I still haven’t booked that doctor’s appointment. I’m debating whether to trust the algorithm’s judgement or follow my own gut. Or, maybe I’ll ask ChatGPT.  Unsettlingly, it seems to know me pretty well!