How Machine Learning Works

Reward/Punishment mechanism on model prediction (forecasted) vs rating (actual).

• We tell the model if something is right or wrong each time a variable is tweaked.

"Dimension" = variable = one aspect of the subject in question.

• e.g. color of tree,

Gradient Descent

• Uses Objective Function equation to change the internal numbers towards the right direction.
• Becomes more accurate next time.

What is a model

A model is like a massive PNG file.

• It's a frozen data set.

Model Score

• ELO scoring (same as chess)

Scaling

• Scaling directly influences capability of a model.
• Why companies spend a lot of money training more data.

Fine Tuning

Scale.ai creates fine tuning documents to use.

Interpretability

Understanding what each parts of the LLM stack do.

• we are currently limited in what is actually going on in each step.

Hallucination

Compression leads to “dreaming” of answer

• document generation to feed in as new data is not helpful

Large quantity/Low quality

Current Developments (as of 2024)

System 2 = long running LLM thinking capability

• e.g.AlphaGo
  • started by imitating best players
  • then achieved self-improvements