Five Things That Scare Me About AI

As cinco coisas citadas são:

• Algorithms are often implemented without ways to address mistakes.
• AI makes it easier to not feel responsible.
• AI encodes & magnifies bias.
• Optimizing metrics above all else leads to negative outcomes.
• There is no accountability for big tech companies.

Com frequência, o problema que buscamos resolver com a aplicação de inteligências artificiais é o de eliminar burocracia (como exemplo, algoritmos avaliam quanto de crédito você recebe, faz sua triagem em planos de saúde e decide se você mostra tendência a reincidir criminalmente). No entanto, ver a aplicação dessa forma esconde algo: que a burocracia não foi eliminada, mas substituída por uma alternativa mais rápida e ilusoriamente vista como mais objetiva e, portanto, menos questionável. Um dos problemas disso é que a decisão algorítmica codifica nossos preconceitos e comete erros:

After the state of Arkansas implemented software to determine people’s healthcare benefits, many people saw a drastic reduction in the amount of care they received, but were given no explanation and no way to appeal. Tammy Dobbs, a woman with cerebral palsy who needs an aid to help her to get out of bed, to go to the bathroom, to get food, and more, had her hours of help suddenly reduced by 20 hours a week, transforming her life for the worse.

Outro problema é que a burocracia, automatizada ou não, distancia o humano da tomada de decisão e facilita que este não se veja como responsável:

This passing of the buck and failure to take responsibility is common in many bureaucracies. As Danah Boyd observed, “Bureaucracy has often been used to shift or evade responsibility. Who do you hold responsible in a complex system?” Boyd gives the examples of high-ranking bureaucrats in Nazi Germany, who did not see themselves as responsible for the Holocaust. Boyd continues, “Today’s algorithmic systems are extending bureaucracy.”

O artigo também ressalta que nenhum desses problemas é inerente aos algoritmos e aponta para possíveis soluções:

• Make sure there is a meaningful, human appeals process. Plan for how to catch and address mistakes in advance.
• Take responsibility, even when our work is just one part of the system.
• Be on the lookout for bias. Create datasheets for data sets.
• Choose not to just optimize metrics.
• Push for thoughtful regulations and standards for the tech industry.