Technicals

View from the Palazzo Leoni Montanari di Vicenza Reinforcement learning In reinforcement learning we have an agent that wants to optimise its outcomes from operating in an environment e.g. a player playing a game, a person consuming and saving, a car driving through a street etc. The agent takes actions which affect the state of the environment and gives it corresponding payoffs. Actions at a given point in time can also affect the future environment, actions and payoffs....

View from the Peggy Guggenheim Collection, Venice. In the previous post we calculated the posterior distribution for a parameter we wanted to estimate. However, in practice, this is often not analytically possible and numerical simulations are needed. Here we evaluate the distribution using a Markov Chain Monte Carlo (MCMC) simulation technique. The idea is to use a Markov chain to generate samples from the distribution we want to evaluate where, as the chain evolves over time, the values it generates have the properties of the posterior’s probability distribution....

Ilke Sahin’s ‘Water Canals (Aquaducts)’ at Goldsmiths degree show 2024 This note reviews some basic aspects of Bayesian inference to refresh my knowledge of it and some of the probability distributions it often uses. 1. Finding the most probable parameters given the data using Bayes’ theorem We want to find the best hypothesis or model parameter(s) that describes the data. Bayes’ theorem allows us to write down a general formula for this:...

Printworks, London Transformers 3: Building and training a Transformer Having discussed how attention works and the structure of Transformers we’ll now implement a simple Transformer that translates German text into English. To do this we’ll take the attention mechanism and other network components that Andrej Karpathy developed in nanoGPT for language generation and reuse them to implement a Transformer for language translation. This uses the PyTorch framework. The language translation Transformer’s structure follows the example in François Chollet’s book ‘Deep Learning with Python’ which is written in Keras/TensorFlow....

Willis tower, Chicago, August 2023 Introduction to Infrastructure as Code - deploying a virtual machine In previous posts we spun up virtual machines from the consoles of cloud providers websites. However this has the disadvantage that it can be time consuming and the steps involved complicated and easy to forget. By contrast directly specifying the cloud computing that we want to access in code is more explicit and easier to reuse - it can also be version controlled....

Helix, White City Transformers 2: The structure of a Transformer Continuing the series of notes on Transformers post I’ll now look at the overall structure of Transformers and how attention is used within them in the context of language translation. I’ll cover: 1. The architecture of Transformers 2. How attention is included in the Transformer The previous post discussed the attention mechanism that underlies how Transformers work. 1. The architecture of Transformers 1....

“Let me Unsee” by Asbestos, Belfast, August 2023. The T in GPT In all the excitement around OpenAI’s ChatGPT two things are often not mentioned: the underlying architecture of Generative Pre-trained Transformer (GPT) models, the Transformer, was invented and made public by Google despite these models’ impressive capabilities, and their scale and complexity, the underlying mathematics involved (at least in terms of the components used) is relatively simple...

Patrick Staff’s ‘On Venus’ at the Serpentine gallery Deploying a web app with Continuous Integration and Continuous Deployment (CI/CD) Introduction The last post used a Serverless approach to regularly search Twitter for particular words and then stored the corresponding Tweet data in a cloud database. Here we develop a simple Streamlit app that accesses the database and creates a page showing average daily sentiment on the topic. We then deploy the app in the cloud....

La Cabane Éclatée aux 4 Salles, Daniel Buren at the Gori collection What is Serverless? In previous posts we spun up virtual machines to work in the cloud, but the cloud can also perform specific functions for us without us having to run any servers, so called serverless computing. The servers providing the services still exist, but they are managed by the cloud provider while we deal with the services directly....

Kings Cross underpass rotated, London In the last post we covered some basic operations with cloud machines, in this post we discuss using Docker containers to replicate environments in the cloud and how to containerise a Jupyter notebook. Docker has become a standard way to package applications enabling something that runs on one machine to run on another. It is a container which holds a virtualisation of an operating system within which applications can be packaged to allow them to move between machines....