Google Colab CLI: Run A100 GPUs from Your Terminal
Use Google Colab CLI to launch an A100 from your terminal, run Ollama, monitor compute units, and export notebooks—with practical costs and limits to check.
Run an A100 on Google Colab from Your Terminal
Google Colab is usually treated like a browser notebook: open the site, pick a runtime, and hope the GPU you want is available. That's fine for a quick experiment. It gets annoying when you want to start sessions from your dev machine, run repeatable commands, monitor what's happening, and save results without babysitting the browser.
That's where Google's google-colab-cli comes in. It gives you terminal commands for creating and managing Colab sessions, running remote commands, checking compute usage, and exporting execution history.
In a small test on a Google AI Pro account, an A100 session was ready in about five seconds. Running a 26-billion-parameter quantized Gemma model through Ollama produced a reported peak of around 179 tokens per second.
Those numbers are promising - but they're not a guarantee. GPU availability, quotas, compute-unit rates, and model performance can all swing. This guide walks through the setup, what the CLI actually does, and how to think about whether it's worth it for your workload.
What Google Colab CLI does - and which package to install
First: make sure you install the right package. There's a colab-cli package on PyPI, and it's been around for years - but it's not the tool described here. In the test behind this guide, installing that older package pulled in outdated dependencies (including older versions of Click and Typer). It also didn't give terminal control of a remote GPU runtime - it mostly helped with uploading notebooks to Google Drive and opening them in a browser.
What you want is Google's google-colab-cli. The version used for the test was 0.7.4, which required Python 3.12 or later. Requirements can change, so check the current PyPI project page before installing. The main distinction is simple: this CLI should create and control a Colab runtime, not just help manage notebook files.
Install with an isolated tool environment
This example uses uv to install the CLI as a standalone tool with Python 3.12:
uv tool install --python 3.12 google-colab-cliInstalling it this way keeps the CLI's dependencies separate from your project's Python environment. That avoids the "why did my app's dependencies suddenly change?" problem when two tools share a virtual environment.
After installation, check the commands for the version you actually installed:
colab --helpIn the tested version, the help output included commands like new, exec, ssh, usage, status, log, url, run, and pay. Use that output as the source of truth. If something doesn't match an example you find elsewhere, don't assume the article is right - check the CLI help and current Google Colab docs.
What this tool changes
The big win is that Colab becomes scriptable from your normal terminal workflow. You can request a runtime, send commands to it, check its state, and pull logs back out - without treating the notebook UI as the center of your workflow.
It's useful for things like model evaluation, one-off GPU jobs, and experiments where you want a saved record. But it's still not "a guaranteed always-on cloud GPU." Colab availability is governed by account eligibility, resource availability, usage limits, and Google's rules. If you need predictable access or service-level guarantees, it's worth comparing it with a conventional cloud GPU instance before building something critical on Colab.
Set up authentication and launch an A100 session
On first use, run a command that requires account access, like:
colab usageThe CLI starts a Google OAuth flow and prints a URL. Open it in a browser, sign in with the Google account that has the relevant Colab subscription, approve access, and paste the returned authorization code back into the terminal. In the test, the account showed a balance of 200 compute units and no active assignments after authentication.
OAuth is typically one-time for your local environment, unless credentials expire or you reset them. Use the account that actually holds the subscription or compute balance. If you authenticate with the wrong Google account, you can lose an embarrassing amount of time debugging what's really an account mismatch.
Request the session
In the test, a named A100 session was created with:
colab new --session gemma-test --gpu A100The CLI reported the session as ready in roughly five seconds. That's what happened in that run - not a promise that every A100 request will start instantly. Queueing, capacity, account status, region, and current Colab policies can all affect startup time and what hardware you end up with.
Also, name your session on purpose. A stable session name makes it easier to read later commands and helps you avoid sending work to the wrong runtime when you're running multiple experiments. The example uses gemma-test; your own names should describe the workload or experiment, not just the GPU model.
Check the runtime before sending a large job
Before downloading a multi-gigabyte model, confirm the session is ready and has the hardware you asked for. For example:
colab status -s gemma-test
colab exec -s gemma-test "nvidia-smi"
In the tested run, the instance showed an NVIDIA A100-SXM4-40GB. It also reported 12 logical CPU cores, about 83.5 GB of host memory, and roughly 235.7 GB of NVMe storage. Treat these as "what this session had at the time," not as universal defaults.
This preflight check is worth it even when startup is fast. It catches the common "wrong session name / wrong GPU type / runtime not in the expected state" problems. Finding those issues before downloading a big model is a lot cheaper than discovering them afterward - especially when you're already paying in compute units.
Run Ollama and monitor the remote GPU
Once the session is ready, use colab exec to run commands on the remote machine. In the test, the workflow installed and started Ollama, downloaded the gemma4:26b model in GGUF Q4_K_M quantization, and sent a short prompt. The model download was about 18 GB, so setup took much longer than the initial session launch.
Ollama and Colab environments change over time, so the exact install commands may differ later. The reliable approach is to keep your setup steps in a script or documented sequence, run them against the named session, and verify each dependency before starting the real evaluation. For repeat runs, don't assume anything is clean just because "it worked last time" in an interactive session.
What the benchmark reported
The test asked the model for a two-sentence clinical summary of BRCA1's role in breast cancer. The reported output was 369 tokens in 2.16 seconds, with a peak generation rate of 179.42 tokens per second. Ollama showed a 32,768-token context window, and nvidia-smi reported about 20,072 MiB of GPU memory in use out of 40,960 MiB.
Still: this is just a snapshot. It's not something you should blindly compare to other machines. Token counts can vary depending on what gets included in the measurement, and throughput depends heavily on the prompt, output length, quantization, model build, runtime settings, warm-up state, and how the tool calculates tokens-per-second. Even the "total time" may come from different measurement windows. If you care about performance, run your own representative prompts and record the exact environment before making decisions based on this kind of result.
Also, the prompt was medical in topic. That doesn't make the output clinical advice. Language models can be fluent and plausible while still missing important context or making mistakes. For healthcare use, you need proper domain review, safeguards, and a validated workflow - not a single successful response.
Monitor the session in layers
The CLI isn't just for starting a GPU. It also helps you understand what the remote runtime is doing. A few useful commands:
colab usageshows remaining compute balance, current usage rate, and active assignments.colab status -s gemma-testreports the session's lifecycle state and recent execution info. In the test, the session showedRUNNINGwhen work was active andIDLEwhen no command was executing.colab exec -s gemma-test "nvidia-smi"exposes GPU details like memory use, temperature, and power draw.colab exec -s gemma-test "ollama ps"helps confirm whether Ollama has started and where it's running.
In this test, the model used the GPU, and Ollama released GPU memory after about five idle minutes. That can be helpful if you don't want models lingering between tasks - but double-check the behavior for the Ollama version and configuration you're actually using.
Export a record of the experiment
After you run your commands, export the session's execution history with:
colab log -s gemma-test -o colab_a100_gemma4_replay.ipynbIn the test, this produced a 728 KB notebook that captured the recorded inputs, outputs, and test logs. This is one of the most practical parts of the workflow: you get an inspectable artifact, not just terminal scrollback (and not just "I wrote some notes somewhere").
Saving the notebook helps with reproducibility, but it doesn't magically make everything reproducible. If a result matters, include model identifiers, quantization, package versions, hardware details, prompts, and relevant settings alongside the exported file. If someone else has to re-run it, the notebook needs to contain enough context for them to understand what you actually ran and what might affect the outcome.
The CLI also provides a colab url command (in the tested version), which returns a notebook link you can open in a browser. That's handy when you want a graphical view of the runtime and its resource charts. In practice, the terminal and browser are complementary - you don't have to choose one forever.
Understand compute units, costs, and session management
A Colab subscription that includes compute isn't the same thing as unlimited free GPU access. In the tested account, Google AI Pro showed a balance of 200 compute units. The A100 session rate was reported as 5.30 units per hour, and the balance dropped from 200.00 to 197.28 during a run that included environment setup, an 18 GB model download, and about 30 minutes of inference.
The simple estimate is:
200 compute units ÷ 5.30 units per hour ≈ 37.7 hoursThat calculation can help with planning, but only if you treat both numbers as "from this test," not as a universal rule. Rates can differ by hardware, account, product terms, and Google's current allocation policy. Your 200-unit starting balance can also depend on your subscription terms. Check the usage screen and official Google Colab information before you assume a fixed monthly allowance or price.
Separate subscription value from marginal cost
If you're already paying for a subscription, the GPU time can feel free, because you don't see a separate charge every time you launch a session. But it's still a limited resource tied to a paid plan. The real question isn't just "Is it free right now?" It's whether the overall subscription value makes sense - and whether your GPU work actually fits within what you're allocated.
The test also described a pay-as-you-go option for buying additional compute units, including an example price of about $9.99 for 100 units. Pricing and terms can change, so verify the current price and expiry rules in Google's pricing UI before budgeting. Don't reuse an old article's number as your final estimate.
At the measured test rate, spending 200 units would correspond to about 37.7 A100 hours. But idle sessions, setup work, model downloads, failed jobs, and reruns all consume time too. If a typical experiment takes 20 minutes of billed runtime, you might end up with just over 100 runs. In practice, leave overhead and a buffer - don't plan to burn every last unit.
Use short-lived jobs for work that doesn't need an interactive session
If a script can run unattended, the tested CLI supported an ephemeral job pattern:
colab run -s gemma-eval --gpu A100 my_benchmark.pyBased on the reported behavior, this starts a suitable runtime, runs your script, collects output, and shuts the runtime down when the job finishes. Check the command's help for supported flags and cleanup behavior before relying on it for important work.
This is usually a better fit than keeping an interactive session open for batch evaluation. The trade-off is on you: the script needs to be self-contained. It should install or verify dependencies, download or locate its model, write outputs somewhere you can retrieve, and fail in a way that makes problems obvious.
For interactive development, explicitly stop the session when you're done - and confirm it's no longer active. For scheduled or production workloads, compare this with a managed cloud GPU service. Colab tends to work best when jobs are occasional and flexible; it's a worse choice when you need guaranteed GPU capacity at a fixed time, predictable hourly costs, or uptime requirements.
Conclusion
Google Colab CLI makes Colab much easier to use if you prefer terminal-driven workflows. In the test, it authenticated via OAuth, launched an A100 session, ran Ollama with a quantized Gemma model, exposed usage and hardware info, and exported the session history as a notebook. That's a solid way to use a subscription's compute allocation for occasional model experiments without moving every step into the browser.
The key caveat: Colab is still quota-based and variable. Treat the measured 5.30-unit hourly rate and the 179-token-per-second result as observations, not guarantees. Check current package requirements, account terms, hardware availability, and pricing - and run a small representative job before committing to a workflow.
If your workload is occasional, interruptible, and easy to package as a script, try the official CLI with one measured experiment. Track units used, export the run, and stop the session when it's no longer doing useful work. If you need predictable capacity or uptime, you'll probably be happier paying for a dedicated GPU service instead.