What Was AlexNet? The 2012 Model That Started the AI Boom

What was AlexNet? It was an image-recognition neural network built in 2012 by Alex Krizhevsky, Ilya Sutskever and Geoffrey Hinton at the University of Toronto. It won that year's ImageNet challenge by a wide margin, and that win is widely treated as the moment deep learning moved from a research niche to the centre of AI. Almost every AI product you use today traces back to the ideas it proved.

Age of AI Entrepreneur covers this moment in its Basic course The Road to the Age of AI, in the lesson Data, Chips and the Deep Learning Breakthrough; sign in with your email and an emailed code to open it. This post explains what AlexNet did, why it worked when earlier attempts had not, and what a founder can take from it.

What was AlexNet, in plain words?

AlexNet is a type of neural network called a convolutional neural network, built to look at a photo and say what is in it. It sorted images into 1,000 categories, from dog breeds to household objects. It had 60 million parameters, the adjustable numbers a network tunes during training.

The idea of neural networks was decades old. What was new was the scale: a much deeper network, trained on far more labelled images, using graphics chips to do the maths fast enough to be practical. If you want the vocabulary first, AI vs Machine Learning vs Deep Learning sets out how the terms fit together.

The ImageNet challenge and the 2012 result

ImageNet was a huge collection of labelled photos started in 2007 by Fei-Fei Li and her collaborators. It grew to over 14 million images, labelled by people through Amazon Mechanical Turk. Each year a contest, the ImageNet Large Scale Visual Recognition Challenge, asked teams to label a set of test images.

The three researchers entered as team SuperVision on 30 September 2012. The contest's main score was the top-5 error rate: how often the right label was not among the model's five best guesses. AlexNet scored 15.3%. The runner-up was more than 10.8 percentage points behind.

In a field where teams fought over small gains, a gap that size was hard to ignore. At a computer vision conference that year, the researcher Yann LeCun called it "an unequivocal turning point in the history of computer vision".

Three ingredients: data, chips and depth

AlexNet did not come from one new idea. It came from three things arriving at the same time.

• Data. ImageNet gave the network 1.2 million labelled training images, far more than earlier datasets. A deep network needs that much to learn without simply memorising.
• Chips. The team trained on two Nvidia GTX 580 graphics cards with 3GB of memory each, for five to six days. The network did not fit on one card, so they split it across two. Graphics chips do many small sums at once, which is exactly what training a neural network needs.
• Depth. The paper's main finding was that the depth of the model was essential to its performance. Depth was expensive to compute, which is why the chips mattered.

Take away any one of the three and the result likely does not happen in 2012. That pattern, where data, compute and method improve together, is the same one behind the large language models that came later, covered in What Is a Foundation Model?

What happened next

Soon after the win, Hinton, Sutskever and Krizhevsky formed a company, DNNResearch, and sold it to Google. Researchers rushed to apply deep networks and graphics chips to other problems. The approach spread from images to speech and then to language. The site's own blog post A Short History of AI: From Turing to ChatGPT follows the thread from there to today's chatbots.

It is worth keeping the earlier history in view too. AI had gone through long periods of disappointment before 2012, when funding and interest fell away. What two AI winters teach founders covers that side.

Why it still matters to a founder

AlexNet is a reminder that big shifts in AI often come from old ideas meeting new conditions. Neural networks were not new in 2012. Cheap, fast graphics chips and a large labelled dataset were. The people who noticed that the conditions had changed were early to the biggest opportunity in the field.

The same question is useful now. When a model gets cheaper, faster or able to read a new kind of input, some product that was impossible last year becomes possible. Your job as a builder is to notice which one, and to check it honestly rather than assume it.

A worked example: the AlexNet test for an AI idea

You can use the three ingredients as a quick check on any AI product idea. Here is how it works for one: a tool that reads photos of handwritten delivery notes and fills in a spreadsheet.

1. Data. Do good examples exist, and can you get them? Ask: "Where do I get 500 real delivery notes with the right answers written down?" If the honest answer is "nowhere", the idea is stuck before it starts.
2. Compute. Can today's models and chips do the job at a price that works? Try a handful of notes with an existing model that reads images before you build anything, and note what each one costs.
3. Method. Is there a known way to do this, or are you betting on research nobody has done? Reading printed and handwritten text from images is a well-studied problem. A product that needs a new scientific result is a research project, not a startup.

If all three answers are yes, you are in the same position AlexNet's team was in: the pieces exist and the job is to put them together well. If one is no, you know exactly where the risk sits.

Frequently asked questions

Who created AlexNet?

Alex Krizhevsky, with Ilya Sutskever and Geoffrey Hinton, at the University of Toronto in 2012. The network is named after Krizhevsky.

Why was AlexNet important?

It won the 2012 ImageNet challenge with a 15.3% top-5 error rate, more than 10.8 points ahead of the runner-up, and showed that deep networks trained on graphics chips could beat other methods by a wide margin.

Is AlexNet still used today?

Mostly as a teaching example and a historical reference. Its ideas, deep networks trained on lots of data with graphics chips, are what lasted.

What does AlexNet have to do with ChatGPT?

They are different kinds of model for different jobs, but both rest on the same recipe AlexNet made convincing: more data, more compute and deeper networks.

Get started

Open Age of AI Entrepreneur, sign in with your email and an emailed code, and take The Road to the Age of AI: thinking machines from Turing to the AI winters, the deep learning breakthrough, and transformers, scale and ChatGPT. It is one of 12 courses at three levels, and each lesson lists its sources.