Independence of mechanisms in machine learning and physics

• 0-th order: Approximation for understanding the world: study "independent" systems.
• 1-th order: the objects are not independent but interaction mechanisms are independent.

Causal Markov condition

• Every node is conditionally independent of its non-descendants given its parents.
• p(x1, ... xn) = ∏ p(xi|pi)

• Faithfulness is too weak / too strong.
• Instead use algorithmic independency (based on Kolmogorov complexity).
  • Conditional K(y|x) = number of bits to describe y(x)
  • K(y|x*) - conditional Kolmogorov complexity given shortest compression of x (the equations look a bit better).
  • I(x, y) -- mutual information
  • every algorithmic dependence is due to a causal relation.
• There are some methods of deriving causality from the distribution (See images)
  • if y = y(x) + ε then the noise will be independent in one direction but not in the other.

• Learning can be in causal and anti-causal direction.
  • Semi-supervised learns in the anti-causal direction but not so well in causal direction (bimodal examples).

• Elements of Causal Inference book

• Q/A
  • If we have time information, we can also assume "causes precede effects" to improve our cause-effect assumptions or to derive time information from data that has no time information but where we can infer causality.