Don't opt out: click here to learn more about our work.

How to trust AI with life-or-death decisions

A lecture on the ethics of consequential AI systems

How to trust AI with life-or-death decisions

As advances in AI reach new heights, there are certain traits that AI systems must have if humans are to trust them to make consequential decisions, according to U of T Computer Science Professor Emeritus Dr. Ronald Baecker at his lecture, “What Society Must Require of AI.” In an effort to learn more about how society will change as artificial intelligence (AI) advances, I attended the talk on June 5 and left more informed of the important role that people have to play in shaping this fast-changing technology.

How is today’s AI different from past technologies?

AI has been used in the field of computer science for decades, yet it has recently been accelerating at a strikingly fast pace, with machine learning catalyzing progress in the field.

Machine learning has improved pattern recognition to such an extent that it can now make consequential decisions normally made by humans. However, AI systems that apply machine learning can sometimes make mistakes.

For many AI systems, this result is acceptable. Such nonconsequential systems rarely make life-or-death decisions, explained Baecker, and their mistakes are “usually benign and can be corrected by trying again.”

But for consequential systems, which are AI-based software that addresses more complex problems, such mistakes are unacceptable.

Problems could arise when using AI to drive autonomous vehicles, diagnose medical conditions, inform decisions made in the justice system, and guide military drones. Mistakes in these areas could result in the loss of human life.

Baecker said that the research community must work to improve consequential AI, which he explains through his proposed “societal demands on AI.” He noted that these demands must give AI human-like attributes in order to improve the decisions that it makes.

Would you trust consequential AI?

When we agree to implement solutions for complex problems, said Baecker, we normally need to understand the “purpose and context” behind the solution suggested by a person or organization.

“If doctors, police officers, or governments make [critical] decisions or perform actions, they will be held responsible,” explained Baecker. “[They] may have to account or explain the logic behind these actions or decisions.”

However, the decision-making processes behind today’s AI systems are often difficult for people to understand. If we cannot understand the reasoning behind an AI’s decisions, it may be difficult for us to detect mistakes by the system and to justify its correct decisions.

Two questions we must answer to trust consequential AI

If a system makes a mistake, how can we easily detect it? The procedures that certain machine learning systems use cannot be easily explained, as their complexity ⁠— based on “hundreds of thousands of processing elements and associated numerical weights” ⁠— cannot be communicated to or understood by users. 

Even if the system works fine, how can we trust the results? For example, physicians reassure patients by explaining the reasoning for their treatment recommendations, so that patients understand what their decisions entail and why they are valid. It’s difficult to reassure users skeptical of an AI system’s decision when the decision-making process may be impossible to adequately explain.

It’s difficult to reassure users skeptical of an AI system’s decision when the decision-making process may be impossible to adequately explain.

Another real-life problem arises when courts use AI-embedded software to predict a defendant’s recidivism in order to aid in the setting of bonds. If that software system were inscrutable, then how could a defendant challenge the system’s reasoning on a decision that affects their freedom?

I found Baecker’s point fascinating: for society to be able to trust consequential AI systems, which may become integrated with everyday technologies, we must trust them like human decision-makers, and to do so, we must answer these questions.

Baecker’s point deserves more attention from us students, who, beyond using consumer technology every day, will likely experience the societal consequences of these AI systems once they are widely adopted.

Society must hold AI systems to stringent standards to trust them with life-or-death decisions

Baecker suggests that AI-embedded systems and algorithms must exhibit key characteristics of human decision-makers, with a list that he noted: “seems overwhelming.”

A trustworthy complex AI system, said Baecker, must display competence, dependability and reliability, openness, transparency and explainability, trustworthiness, responsibility and accountability, sensitivity, empathy, compassion, fairness, justice, and ethical behaviour. 

Baecker noted that the list is not exhaustive — it omits other attributes of true intelligence, such as common sense, intuition, context use, and discretion.

But at the same time, he also recognized that his list of requirements is an “extreme position,” which necessitates very high standards for a complex AI system to be considered trustworthy.

However, Baecker reinforced his belief that complex AI systems must be held to these stringent standards for society to be able to trust them to make life-or-death decisions.

“We as a research community must work towards endowing algorithmic agents with these attributes,” said Baecker. “And we must speak up to inform society that such conditions are now not satisfied, and to insist that people not be intimidated by hype and by high-tech mumbo-jumbo.”

“Society must insist on something like what I have proposed, or refinements of it, if we are to trust AI agents with matters of human welfare, health, life, and death.”

Where computers and clinics intersect

Raw Talk Podcast hosts expert panel discussions about AI’s role in healthcare

Where computers and clinics intersect

Experts in medicine, academia, and industry explored the promises and perils of the applications of artificial intelligence (AI) in health care during panel discussions with the Raw Talk Podcast on May 7. The event was organized by graduate students of U of T’s Institute of Medical Science.

The two panels, collectively named “Medicine Meets Machine: The Emerging Role of AI in Healthcare,” aimed to demystify sensationalism and clarify misconceptions about the growing field of study.

“On one hand, it seems like everyone has heard about [AI],” said Co-executive Producer Grace Jacobs. “But on the other hand, it seems like there’s a lot of misunderstanding and misconceptions that are quite common.”

How AI is used in health care

While discussing the reality of AI, several panelists emphasized that it should be viewed and treated as a tool. “It is statistics where you don’t have to predefine your model exactly,” said Dr. Jason Lerch of the University of Oxford.

Other speakers agreed that AI is an expansion of — or a replacement for — traditional statistics, image processing, and risk scores, as it can provide doctors with more robust and accurate information. However, final health care recommendations and decisions remain in the hands of doctors and patients.

“You always need a pilot,” said Dr. Marzyeh Ghassemi, a U of T assistant professor of computer science and medicine.

But what advantages can this tool provide? Ghassemi thinks it can assimilate clues from a wider range of patients’ conditions to predict treatment outcomes, replacing the experience-based intuition that doctors currently rely on.

Speaking on her time in the Intensive Care Unit as an MIT PhD student, Ghassemi said, “A patient would come in, and I swear they would look to me exactly the same as prior patients, and the… senior doctors would call it. They would say, ‘oh, this one’s not going to make it. They’re going to die.’ And I would say, ‘Okay… why?’ And they said, ‘I’m not sure. I have a sense.’”

“They used different words — gestalt, sense — but they all essentially said the same thing. ‘I just — I have a sense.'”

Doctors develop this sense by seeing many cases during their training, but they can intuit only the cases that they had personally experienced; AI algorithms can potentially understand many more cases using a wider dataset.

Accessing those cases requires access to patient data, and access to data requires conversations about consent and privacy. Ghassemi and Dr. Sunit Das, a neurosurgeon at St. Michael’s Hospital and Scientist at the Keenan Research Centre for Biomedical Science, said that “de-identification” — the removal of information that can be traced back to individual identities — protects privacy.

Large de-identified datasets from the United States and the United Kingdom are available for AI research, but generally, Canada lags behind these countries in making health data available for this purpose.

Dr. Alison Paprica, Vice-President of Health Strategy and Partnerships at the Vector Institute, agreed that data should be used for research, but argued that de-identification alone does not eliminate risk.

“You’re not just giving a dataset to anybody,” she said. “You’re giving a dataset to people who are extremely skilled at finding relationships and patterns and maybe piecing together information in ways that most people couldn’t. So I think there’s going to be heightened sensitivity around re-identification risk.”

Society must manage this risk and balance it against the benefits. “How do we balance that?” Paprica asked. She suggested that consulting all involved stakeholders could help strike that equilibrium.

Advice for scientists aiming to use AI in their research

So what advice did the panelists have for scientists hoping to harness the power of AI in their own research?

Ghassemi stressed the importance of knowing what you’re doing: researchers have created many tools that make AI research easy to implement, but conscientious scientists need to know the statistical and training principles behind the methods.

“If you’re not aware of how these things are trained,” she said, “it’s really easy to misuse them. Like, shockingly easy to misuse them.”

Other panelists advised users to take care when choosing data to train the algorithms. “A learning algorithm can’t overcome bad data that goes in, or can’t completely overcome it,” said Lerch.

Moderator Dr. Shreejoy Tripathy summed up a key takeaway on applying AI to health care: “Understand your data… And understand your algorithms.”

Exploring the gentrification of thrift stores

The ethics of splurging on the dollar bin

Exploring the gentrification of thrift stores

A nostalgic smell drifts from piles of stained mom jeans and multi-colored windbreakers at a local Queen Street thrift store. Loud alternative rock music blares through a broken-down speaker as herds of fashion-conscious youth crowd the sale rack — a familiar scene at this beloved consignment store. A group of rowdy, suburban teenagers gather at a nearby register as a cashier rings up several items in hand.

Vintage Levi’s shorts, $10. Old Raptors Jersey, $9. Ramones Band Tee, $10. Aviator Sunglasses, $5.

Once considered suitable only for the poor and working class, ‘thrifting’ has vastly expanded into mainstream culture, trickling its way to upper middle class hipsters and fashion bloggers. Major fashion magazines such as VOGUE have caught onto the ‘thrifting’ trend, with celebrity fashion gurus such as Alexa Chung utilizing thrift stores as a part of their style arsenal.

Can we blame Macklemore for donning your grandfather’s hand me downs?

With the average cost of living continuing to rise in Toronto, the rising prices of second hand clothing do not make it easier on those who truly need it most. Major thrift stores such as Value Village have been criticized by consumers for their drastic price changes.

Though a $20 jacket for most shoppers would be considered reasonable, some may consider it a stretch out of their budget. For consignment stores operating under a for-profit agenda, it seems contradictory to hike up the prices of donated clothing that is meant to benefit those in need. After all, shoppers flock to thrift stores for their low prices and unique finds. Is affordability a fashion fad?

Humble beginnings

Consignment stores made their way into larger cities in the early 1900s as a means of raising money for charitable causes. Charitable organizations such as The Salvation Army opened their own consignment stores under a religious agenda and, to this day, are still operating under their religious mission.  

The Salvation Army opened several Toronto buildings in the early 1900s, opening their first known ‘thrift-store’ in 1908.  The opening of consignment stores has also helped the increasing immigrant population, especially under a looming economic crisis.

The Yonge Street Mission (YSM), a local Toronto charity, opened its thrift store Double Take in 1999, to serve a growing immigrant community in the Regent Park neighbourhood. The store is a part of YSM’s Double Take program, which is a community training and employment initiative for people who may face obstacles with regular employment and those who are newcomers to Canada. Their goal is to help people living in poverty build stronger futures and work their way out of poverty.

The store, unlike many other thrift stores, prides itself in providing quality clothing for its customers at low prices that have not changed since its opening. Kathy Webster, manager of Double Take, explains that their pricing practises haven’t changed in the 10 years that she’s worked at the store, and they are simply to “provide low cost quality clothing and housewares and furniture.” However, Webster also says that, in order to cover costs, Double Take needs to “get some value out of the higher end things.”

“This has always been a destination for a lot of people… bargain hunters, students, people who actually live in Cabbagetown, which is just north of here [Regent Park]. It’s a very well-to-do area and a great many of the residents in Cabbagetown shop here because they know it’s good quality, it’s clean,” continues Webster. “They don’t feel that they’re walking into a thrift shop.”

Double Take also provides first-time employment for community members involved in the program. Clients of the programs offered at YSM are also issued gift certificates which they can use to shop at Double Take for interview or career clothing and household necessities.

When asked about the gentrification of thrift stores and the recent trends in thrifting culture, Webster says, “That doesn’t take away from the fact that people who don’t have a lot of money shop here. As far as I’m concerned, this [store] is open to everybody and I’m happy to see any kind of customers coming in.”

Thrifting culture takes a turn

In recent years, local business owners have caught onto the thrifting trend, opening their own consignment stores, though marketing it under a different agenda.

Fashionably Yours, a designer consignment store, curates a selection of second-hand luxury vintage clothing on their website and at their flagship Queen Street West location. Customers also have the option to sell their luxury second-hand items through them.

Plato’s Closet, a trendy second-hand clothing store, targets the hip, ‘fashion-forward’ teenager with locations across suburban communities in Ontario. Customers can sell their gently-used, unwanted clothing to the store, given that it meets certain conditions bonus points for brand names.

Startups such as Depop are also targeting fashion-conscious, trendy youth on a budget by creating apps for users to sell their second-hand clothing, accessories, and more.

So how exactly has the gentrification of thrift stores and the rise of ‘thrifting culture’ affected local charities?

In late 2016, Goodwill closed its Toronto locations as a result of a cash flow crisis. Faced with competition from fellow thrift stores such as Value Village, the company failed to remain financially stable despite receiving masses of clothing donations each day.

Thrifters weigh in

Meghan Garvida, a veteran thrifter and third-year student at U of T, describes her love story with the thrifting phenomenon and weighs in on its recent popularity. “I got into thrifting because my elementary school was right across the Value Village, so afterschool my friends and I would go to the thrift store, but technically, it originally started with my parents.” But when Garvida was growing up, “thrifting wasn’t well recognized. Honestly, people thought it was weird. Because it was such a cheaper option, my parents and I would go.”

Garvida prides herself on finding quality branded clothing at fraction prices. When asked about the gentrification of thrift stores and her experience with Value Village’s inflating prices, she shares a sentiment common among many longtime thrifters. “I’ve seen a big difference especially in their stores, like jackets used to be $7.99. And no-name brands are nowadays 17 dollars,” she laments. “This is a thrift store, this is meant for people who don’t have money and cannot afford to go to the mall.”

Nevertheless, Garvida explains that a few trend cycles ago, “it was all about the malls, what’s trending in stores.” Nowadays, however, “the trends are more focused, like vintage, retro vibes, and that’s something we don’t necessarily always have in store or… something that [every store] in the mall would have.”

Recent U of T graduate and avid thrifter Sila Naz Elgin came to thrifting in a similar way to Garvida. “Second-hand clothing and such isn’t looked at warmly in Turkish culture, so until I met someone who introduced me to Value Village in Waterloo I didn’t look at it positively,” explains Elgin in an email to The Varsity. “After that, it became a family affair, and was super convenient for an immigrant family that didn’t have a lot of money to find some really nice pieces.”

But Elgin’s fondness for thrifting is fading. “In recent years I’ve come to detest the industry because I think it’s turned into another one of those ‘fashion of the year’ fads,” she writes. “I used to be able to walk into a thrift store and find some genuine pieces from the 90s, which is kind of what I vibe for in my personal style. Lately, it became impossible; and I had workers respond with ‘90s aren’t in right now.’”

Josie Meza-Silva, a second-year student at U of T, shares Garvida’s concern over rising prices not only at major thrift stores but also at retail stores, alluding to the ‘low prices’ that attract thrifters in the first place. “You’d be lucky if you went to a mall and bought a plain t-shirt for less than $20, let alone big brand name prices. Even the thrift shops nowadays are getting expensive selling shirts at around $5 – $10 each.”

Echoing Elgin’s discontent, Meza-Silva continues, “Just the other day I went to a Value Village and I saw teenagers dressed in big brand name clothing with carts filled to the brim with clothes. The trendy culture of “thrifting” I’d say was influenced by the fashion industry. People who want to be seen as fashionable can’t afford the brand name “retro look” and turn to thrifting as a cheap alternative.”

While Mrinalini Fernandes, a third-year student at UTM, also notes that “Over the past couple of years I’ve noticed a large increase in cost at thrift stores and so the price has started to lose its appeal.,” she explains that “The main reason I continue to shop at thrift stores is because it’s a more sustainable and eco friendly way to recycle and reuse things no longer needed while reducing waste.”

Third-year U of T student Serena Viola also advocates the sustainability of thrifting culture. She writes that she doesn’t “think that something so versatile as thrifting should be a fad for fashion. At the end of the day, it is pre worn/used items that are being recycled- and they should be priced in a reasonable range.”

“Once, in Kensington Market I saw a pair of second hand levis jeans for over 80 dollars… The reason why this matters so much to me is because I thrift for environmental and humanitarian reasons. The fast fashion industry uses a lot of resources and is notorious for incredibly horrendous working conditions,” shares Viola.

The cost of self-expression

Overall, the search for self-expression seems to be a recurring theme in thrifting culture. Garvida explains, “When you go to a thrift store, you’ll have a one of a kind item and if someone asks where you got it, nobody else can find it. It’s something nowadays where people want to go to the thrift store to be that one-of-a-kind person.”  

She has a point.

Thrift stores evoke a certain aesthetic that seems awfully unique. Old clothes tell a story that mass-produced clothing cannot. Purchase a pair of culottes at a major retail store and you’re sure to have the same outfit as hundreds of other people. But purchase a pair of vintage jeans, and you stand out.

Shoppers want to be able to express their individuality while being able to save a few dollars, but does being ‘one of a kind’ come at an ethical cost? For-profit thrift stores that are meant to help struggling individuals have been evolving to cater to well-off clientele, instead of the economically disadvantaged. So who is really profiting?