Afterword

To continue deepening your knowledge on deep learning and deep learning systems, the following provide explicit offramps from SITP bridging you from nanochat to llama and from borscht to torch. Once you feel comfortable, you should graduate towards contributing to state of the art neural net architecture and deep learning systems at the frontier.

Good luck on your journey.
I’ll see you at work.

Afterword Table of Contents

1. Architecture: From nanochat to llama and deepseek

• moe

2. Systems: From borscht to torch

1. The Multidimensional Tensor

In which we transition from the borscht.Tensor and borscht.blas to the torch.Tensor and cuBLAS

From borscht.Tensor to torch.Tensor

From borscht.blas to cuBLAS

2. Standard Continuous Mathematics

In which we transition from borscht.linalg, borscht.distributions, borscht.lapack to torch.linalg, torch.distributions and cuSOLVER

From borscht.linalg to torch.linalg

Solvers

• torch.linalg.lstsq
• torch.linalg.solve_triangular
• torch.linalg.lu_solve

Decompositions

• torch.linalg.cholesky
• torch.linalg.qr
• torch.linalg.lu
• torch.linalg.svd
• torch.linalg.eig

From borscht.lapack to cuSOLVER

From borscht.distributions to torch.distributions

• torch.distributions.Normal
• torch.distributions.Categorial
• torch.distributions.Bernouilli
• torch.distributions.Uniform

3. Neural Network Primitives

In which we transition from borscht.nn, borscht.optim and borscht.dnn to torch.nn, torch.optim and cuDNN

From borscht.nn to torch.nn

From borscht.optim to torch.optim

From borscht.dnn to cuDNN