Chapter 1: A Model on Your Machine
Thirty minutes from now a language model will be running on the computer in front of you, answering questions with no account, no network connection, and no bill. That part is easy, and it is deliberately the first thing this book does, because everything after it is about what that running model is.
What it is, is a server. The chat window you will see in this chapter is one client of it. VS Code will be another, your own scripts will be a third, and none of them are special. Once you have sent one HTTP request to the model by hand, the rest of the book is a set of increasingly useful things to send.
The chapter closes with the question that should come before any of this and usually comes after: when is a model on your own hardware the right tool, and when is it not.
In This Chapter
- 1.1 Thirty Minutes to a Running Model — Install LM Studio, download a small model, and get an answer from it.
- 1.2 It Is a Server, Not an App — Start the server, send it a request with
curl, and read the response the way a developer reads it. - 1.3 Where Things Are in LM Studio — The six surfaces of the application, framed by what this book uses each one for, and the command line that reaches the same server.
- 1.4 When Local Wins, and When It Does Not — Privacy, control and cost against the honest limits of what consumer hardware can run.
<a id="1a-thirty-minutes"></a>
1.1 Thirty Minutes to a Running Model
> Overview: Install LM Studio, download a small model, and get an answer from it.
The book uses LM Studio as its runtime because it shows you every setting that matters, which is the point of Part II. Chapter 5 shows the same server from two other runtimes, so nothing you learn is tied to the application.
Install LM Studio
Download the installer for your operating system from [lmstudio.ai/download](https://lmstudio.ai/download) and run it. The defaults are fine. When the installer or the first-run screen offers a choice of user level, choose Developer; if you miss it, turn it on afterward under Settings, in the developer section. Developer mode is what exposes the server tab this chapter needs.
IMPORTANT: The download page also offers Bionic, a separate product from the same company. Install LM Studio, the runtime that loads models and serves them, not Bionic. Everything in this book assumes LM Studio.
You do not need an account, and the application does not need to be signed in to anything.
Download a Model
LM Studio downloads models from Hugging Face, which Chapter 3 covers in depth. For now, open this link in a browser and let it hand the model to LM Studio:
[Qwen2.5 Coder 0.5B Instruct, Q4_K_M](lmstudio://open_from_hf?model=unsloth/Qwen2.5-Coder-0.5B-Instruct-GGUF&file=Qwen2.5-Coder-0.5B-Instruct-Q4_K_M.gguf)
If the link does not open the application, search inside LM Studio's Discover tab for unsloth/Qwen2.5-Coder-0.5B-Instruct-GGUF and choose the file Qwen2.5-Coder-0.5B-Instruct-Q4_K_M.gguf. The download is a few hundred megabytes.
Two parts of that file name will make full sense in Part II, and each deserves a sentence now:
0.5B is the model's size: roughly half a billion parameters, the learned numbers Chapter 2 explains. It is the smallest model in this book by a wide margin, chosen because it runs on almost anything, including a laptop with no usable GPU. Section 3.1 teaches you to read the rest of a name.Q4_K_M is the quantization: each parameter is stored in about four and a half bits instead of sixteen, which is why the file is small. Section 4.2 covers what that costs.
This is not a model you would use for real work. It is a model that proves the runtime works, and it has one genuinely useful job later, in Section 15.3.
Load It and Ask It Something
Open the Chat tab. In the model picker at the top of the window, choose the Qwen model you just downloaded and load it. Loading reads the weights from disk into memory; for this model it takes a few seconds.
Type a message and send it. The model is trained for code, so ask it for some:
Write a JavaScript function that returns the largest number in an array.
The answer will be plausible and, for a model this small, sometimes wrong. That is fine. You have a language model running on your own hardware, and the rest of the chapter is about what you can do with it that the chat window cannot.
Leave it loaded. Section 1.2 needs it.
<a id="1b-a-server-not-an-app"></a>
1.2 It Is a Server, Not an App
> Overview: Start the server, send it a request with `curl`, and read the response the way a developer reads it.
The chat window is not where LM Studio's value lies. Underneath it is an HTTP server that speaks the same wire format as OpenAI's API, and every editor, agent, and script in this book talks to that server. The chat window is one client of it. You are about to be another.
Start the Server
Open the Developer tab and turn the server on. It listens on http://127.0.0.1:1234 by default. The same thing from a terminal:
lms server start
lms is LM Studio's command-line interface, installed with the application. If your shell does not find it, the installer put it in a bin directory under your home folder's .lmstudio directory. The Developer tab has a button that adds it to your path.
Ask the server what it can serve:
curl http://127.0.0.1:1234/v1/models
{
"data": [
{ "id": "qwen2.5-coder-0.5b-instruct", "object": "model", "owned_by": "organization_owner" }
],
"object": "list"
}The id is the name the server knows the model by. It is not the file name, and it is the value every client will need. Copy it from here rather than guessing.
The Smallest Useful Request
Everything a chat application does reduces to one request. Here it is:
curl http://127.0.0.1:1234/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{
"model": "qwen2.5-coder-0.5b-instruct",
"messages": [
{ "role": "user", "content": "Reply with exactly the word: ready" }
],
"temperature": 0
}'A POST, a JSON body naming the model and a list of messages, and one setting. That is the whole protocol. An editor sends a longer message list and a few more settings; it is the same endpoint and the same shape.
This is the response LM Studio returned on the unified-memory machine, unedited:
{
"id": "chatcmpl-r8op12jbo2oly7iou38b",
"object": "chat.completion",
"created": 1788103972,
"model": "qwen2.5-coder-0.5b-instruct",
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "ready",
"reasoning_content": "",
"tool_calls": []
},
"logprobs": null,
"finish_reason": "stop"
}
],
"usage": {
"prompt_tokens": 36,
"completion_tokens": 2,
"total_tokens": 38,
"completion_tokens_details": { "reasoning_tokens": 0 }
},
"stats": {},
"system_fingerprint": "qwen2.5-coder-0.5b-instruct"
}Three fields matter now. Chapter 7 explains the rest, including the two empty ones.
`choices[0].message.content` is the answer. It is the only field the chat window shows you, and for a quick check it is the only one you need. The reasoning_content next to it is where a model trained to think before answering puts its thinking; this model does not, so it is empty.
`finish_reason` says why the model stopped. stop means it finished on its own. length means it ran out of room, and the answer you have is cut off. A program that reads content without reading finish_reason will one day hand someone half an answer and call it done.
`usage` is the bill, even though nobody is charging you. The model read 36 tokens and wrote 2. Chapter 2 explains what a token is. For now, notice that the prompt cost more than you typed: the model's own template wrapped your message in a default system prompt before the model saw it. Every request has a cost in tokens, and Chapter 4 shows that the cost is paid in memory as well as time.
The Sentence the Book Is Built On
A model that runs on your own hardware is a service you own. You can send it a request from anything that speaks HTTP, as often as you like, containing anything at all, and nothing about that request leaves the machine. Parts II and III are about making the service run well and connecting tools other people wrote to it. Part IV is about writing your own.
<a id="1c-where-things-are"></a>
1.3 Where Things Are in LM Studio
> Overview: The six surfaces of the application, framed by what this book uses each one for, and the command line that reaches the same server.
LM Studio has a sidebar with six entries. What follows is not a tour of the interface, which will move its buttons between versions. It is what this book uses each surface for, so that when a later chapter says "load the model with a 32k context" you know where that happens.
Chat is where you talk to a loaded model directly. The book uses it for one thing above all: Section 12.1's first diagnostic question, "does the model answer here?", which separates a broken model from a broken client in fifteen seconds. The panel beside the conversation shows the model's load settings and generation settings, and it shows tokens per second after every reply, which Chapter 4 uses as a first measurement.
Developer is the server. It starts and stops the server, shows which models are loaded and with what context length, lists the server's settings (Chapter 5 goes through them), and keeps the log. The log shows the request as it arrived, the template that was applied, and the error the runtime itself raised. Chapter 12 tells you to leave it open, and means it.
My Models is what is on disk. Each model has its own default settings here: the context length it loads with, how much of it goes to the GPU, its chat template, and its generation settings.
The chapter continues in the book.
