Hello Large Language Models
An Initial Foray into Bloom using Google Colab
As part of my Master’s program Capstone project, I get to explore how to implement Large Language Models (LLMs). I’m sure this exploration will spawn a number of blog posts, but for starters, I wanted to just kick the tires on one and see how it works at a high level.
For the uninitiated, an LLM is a Natural Language Processing (NLP) machine learning model typically (and for now, exclusively) built using a deep learning approach known as a transformer. (“More than meets the i!” Sorry…) In a future post I may try to break down specifically what a transformer is and how it works, but for now, knowing it’s a large, complex neural networking approach that is capable of producing extremely sophisticated language-based machine learning models is good enough. The things you read about in the news where someone claims artificial intelligence has gained sentience? That’s an LLM. The other things you read about a machine learning model that can draw realistic images of ridiculous things (like “a dog wearing a ski mask driving a boat on liquid cotton candy” or whatever weirdness you can dream up) — that’s also powered by LLMs.
While there are a number of LLMs available for academic / research / killing time creatively purposes, most of them are only commercially available to the companies that created them. Since my end goal is to develop an application framework that could be deployed commercially (more on that later) without licensing constraints, this dramatically narrowed the number of possible LLMs I could choose from. As a result, I am starting with BLOOM, an open source LLM.
Getting started, I wanted to set up a quick playground just to kick the tires, and do so in such a way that my tire kicking process could be easily replicated. To do this, I turned to Google Colab. This is a free environment (for low resource users, but can include GPUs!) that looks and feels a lot like iPython / Jupyter Notebooks, but can easily integrate with both Google Workspace Drive and GitHub.
With the coding environment selected, I searched for instructions on how to install and use BLOOM, and came across this tutorial. I found (and reported) what I think were a couple of bugs in the instructions, but overall this was pretty helpful in terms of the core basics for getting started. I loosely followed the procedures and got BLOOM up and running in just a manner of a few minutes. (Full disclosure: actual time to get to this point was significantly longer, but if you just replicate my process, you should be up and running in less than 10–15 minutes, easy.) You can see what I did here, and actually access the notebook itself directly at this link.
A couple of notes:
- The free Colab environment is understandably resource constrained, and the 1.3 billion parameter model crashed the kernel when I attempted to load it. Thus, I switched to the 350 million parameter model, which for testing purposes was fine and easily fit in the resources available.
- The transformers package was not installed by default, so you’ll see a commented-out cell where I installed it. If you’re running this for the first time in your own Colab environment, you’ll want to run that line of code without the comment.
Otherwise, this was pretty straightforward. I was able to define a prompt, the number of words to return in response, and then use a few different approaches for the model to produce its predictions for what came next. Pretty neat stuff! And only the beginning…
