The progress of machine decisions

• Classification problem in AI. It works and everything is sane as long as the statistical correlations between the features that we use and the classes hold in the future.
• Progress and its problems by Larry Laudan
  • Progress indicators tend to ignore non-empirical (or higher-order) problems.
  • Paradigm shifts are rather difficult to put in numbers that make them comparable with the object level discoveries.
• Crime prediction tools
  • Compas risk assessment tool.
  • HART -- another similar system
  • They sometimes might create feedback loops (over time).
• We rely on algorithms to match people with things (kind of ok) but also to match people with opportunities (hm...)
• FAT ML conference.
• Algorithmic fairness:
  • Statistical parity (outcome is independent from hidden vars).
  • Calibration (recall is independent from hidden vars).
  • Error rate (error rate is independent from hidden variables).

• Q/A
  • Being able to apply our principles with more foresight and consistency might allow us to see better that some of them might be not so good.
  • There's a study about Compas that says that you can get similar accuracy with just age and number of convictions.