How to SSH into a Google Cloud VM Without gcloud

gcloud compute ssh is convenient but it is a dependency. The moment you need to ssh into a GCP VM from a CI runner without gcloud installed, a colleague's laptop, or a fresh machine where you have not yet run gcloud auth login, the convenience evaporates. The underlying truth: a GCP Compute Engine VM is just a Linux box listening on TCP/22. As long as (a) a firewall rule permits SSH from your source IP and (b) the VM has a public key it trusts, plain OpenSSH connects in like any other server. Below: the three authentication paths (metadata keys, OS Login, IAP TCP forwarding), the exact no-gcloud procedure, a ~/.ssh/config block that drops the -i and username repetition, a troubleshooting matrix for "Permission denied" and "Connection refused", and a comparison table for gcloud vs OpenSSH vs PuTTY vs IAP.

How do I SSH into a Google Cloud VM without gcloud?

Generate a key pair with ssh-keygen -t ed25519 -C YOUR_USERNAME -f ~/.ssh/gcp_ed25519. Add the public key to the VM via the Console (VM instances, click the VM, Edit, SSH Keys, Add item, paste gcp_ed25519.pub). The username in the key comment becomes the Linux username on the VM, so make it match. Get the external IP from the Console. Connect with ssh -i ~/.ssh/gcp_ed25519 YOUR_USERNAME@EXTERNAL_IP. The GCP guest agent reads the metadata, sees your key, and writes it to ~/.ssh/authorized_keys for that Linux user. No gcloud installed, no gcloud auth login, no Google SDK. Works identically from macOS, Linux, WSL, and any CI runner with stock OpenSSH.

Jump to:

• Why bother going without gcloud
• The three authentication paths
• Approach A: Plain OpenSSH with metadata keys
• Approach B: OS Login (IAM-integrated SSH)
• Approach C: IAP TCP forwarding
• The ~/.ssh/config block I use for GCP VMs
• Connecting from Windows: PuTTY and OpenSSH
• Comparison: gcloud vs OpenSSH vs PuTTY vs IAP
• Troubleshooting: Connection refused, Permission denied
• What to do next
• FAQ

Why bother going without gcloud

Three situations come up often enough that learning the no-gcloud path pays for itself:

1. CI/CD pipelines. A GitHub Actions runner, self-hosted Jenkins, Drone. Installing gcloud and authenticating with a service-account JSON works, but if all you need is ssh-and-run-a-command, dropping a public key in metadata and using stock OpenSSH is simpler. One secret, no SDK install, no token refresh logic.
2. Scripts on a server that is not Google-managed. A monitoring box pulls log files from a GCP VM. A backup orchestrator rsyncs to it. These do not need gcloud, they need ssh.
3. A fresh machine. New laptop, no Google SDK, want to debug a VM right now. With a key already in metadata, plain ssh user@ip is the fastest path.

The gcloud convenience also masks what is actually happening, which makes failures harder to debug. Knowing that GCP VMs are plain SSH targets with a metadata-driven authorized_keys mechanism is the single most useful piece of GCP knowledge a sysadmin can carry around.

The three authentication paths

GCP gives you three ways to authorize a public key against a VM. Pick by operational model.

For the "ssh without gcloud" goal, A is the simplest and B is the most team-friendly. C effectively requires gcloud or a custom IAP-tunnel client.

Approach A: Plain OpenSSH with metadata keys

Step 1: Generate a key pair locally.

ssh-keygen -t ed25519 -C ishan -f ~/.ssh/gcp_ed25519

The -C ishan comment matters. GCP uses the comment field as the Linux username on the VM. The default ssh-keygen comment is USER@HOSTNAME, which makes the username USER@HOSTNAME literally. Set it explicitly.

Ed25519 is the right choice over RSA in 2026: faster, shorter, equivalent or better security.

Step 2: Add the public key to the VM.

Easiest no-gcloud path is the Console: Compute Engine, VM instances, click the VM, Edit, scroll to SSH Keys, Add item, paste the contents of ~/.ssh/gcp_ed25519.pub, Save. The guest agent inside the VM picks up the metadata change within a few seconds and writes the key to /home/ishan/.ssh/authorized_keys.

For project-wide keys (every VM in the project trusts them): Compute Engine, Metadata, SSH Keys tab, Add item. Per-VM keys are the right default for security.

Step 3: Find the VM's external IP.

Console: Compute Engine, VM instances, the External IP column. Reserve a static external IP (free while attached to a running VM) to make this a one-time problem and put it in ~/.ssh/config.

Step 4: Connect.

ssh -i ~/.ssh/gcp_ed25519 ishan@34.122.15.42

That is all. The VM sees the public key in metadata, the guest agent has placed it in authorized_keys, OpenSSH negotiates the key exchange, and you are in. For a useful primer on the SSH flags, the SSH cheat sheet has the option reference.

Firewall reminder: the default default-allow-ssh rule on a fresh GCP project permits TCP/22 from 0.0.0.0/0. If your project has been hardened or uses a custom VPC, add a rule that allows SSH from your source IP (VPC network, Firewall, Create firewall rule, Targets, Source IPv4 ranges, Protocols and ports: tcp 22).

Approach B: OS Login (IAM-integrated SSH)

OS Login is Google's answer to "how do I manage SSH access for a team of fifty engineers." Public keys live in Google IAM tied to user identities, not in metadata. The guest agent fetches them per-VM and syncs authorized_keys automatically.

Enable OS Login project-wide via Compute Engine, Metadata, Add enable-oslogin = TRUE. Or per-VM with the same metadata key.

Add your public key to your OS Login profile:

gcloud compute os-login ssh-keys add --key-file=~/.ssh/gcp_ed25519.pub

Yes, this uses gcloud. Adding the key to your OS Login profile is one of the few operations the Console UI does not cover end-to-end. You only do this once per laptop; after that, plain ssh works.

Find your OS Login username. OS Login derives the Linux username from your Google account email: cloudysanfrancisco@gmail.com becomes cloudysanfrancisco_gmail_com. Find yours:

gcloud compute os-login describe-profile --format="value(posixAccounts[0].username)"

Once you have it, plain ssh works from anywhere:

ssh -i ~/.ssh/gcp_ed25519 cloudysanfrancisco_gmail_com@34.122.15.42

Advantage over metadata keys: revoking access is a one-line IAM change instead of a per-VM metadata edit. Disadvantage: requires roles/compute.osLogin (or roles/compute.osAdminLogin for sudo) granted explicitly to each user.

For teams above five engineers, OS Login is the right answer. For solo work and CI service accounts, metadata keys are simpler.

Approach C: IAP TCP forwarding

Identity-Aware Proxy TCP forwarding tunnels SSH through Google's edge without requiring a public IP on the VM. The VM stays on a private RFC1918 address, the firewall allows SSH only from IAP's edge range, and authentication happens through Google IAM at the proxy layer.

The official command is gcloud:

gcloud compute ssh VM_NAME --tunnel-through-iap --zone ZONE

To use IAP without gcloud, you have to talk to the IAP REST API directly. There is an iap-tunnel client library in Go that some teams package as a small binary, and the IAP team publishes a tunnel daemon you can run alongside ssh -o ProxyCommand. Both paths involve a Google-issued OAuth token, which is the part gcloud usually handles for you.

In practice, the no-gcloud IAP path is rare. Either accept gcloud as a dependency on machines that need IAP, or expose a public IP and use Approaches A or B with a tight single-source-IP firewall allowlist.

The firewall rule allowing IAP traffic:

Source IPv4 ranges: 35.235.240.0/20
Protocols and ports: tcp:22

That CIDR is Google's IAP edge range. The rule must exist on the VPC before IAP tunneling works.

The ~/.ssh/config block I use for GCP VMs

Typing ssh -i ~/.ssh/gcp_ed25519 ishan@34.122.15.42 repeatedly wears out a keyboard. ~/.ssh/config fixes it.

Host gcp-web-01
    HostName 34.122.15.42
    User ishan
    IdentityFile ~/.ssh/gcp_ed25519
    IdentitiesOnly yes
    ServerAliveInterval 60
    ServerAliveCountMax 3

Host gcp-*
    User ishan
    IdentityFile ~/.ssh/gcp_ed25519
    IdentitiesOnly yes
    StrictHostKeyChecking accept-new

Notes:

• IdentitiesOnly yes stops OpenSSH from trying every key in ssh-agent before falling back to the right one. Without this, GCP can temporarily lock the account after too many wrong-key attempts.
• ServerAliveInterval 60 keeps the connection alive across NATs and corporate firewalls that drop idle TCP sessions.
• StrictHostKeyChecking accept-new auto-accepts the host key on first connection but refuses if it changes later. The safe middle ground.

Once this is in place, the rest of this article is one command: ssh gcp-web-01.

Connecting from Windows: PuTTY and OpenSSH

Windows 10 1809+ and all of Windows 11 ship the OpenSSH client. The Linux-style flow works identically from cmd, PowerShell, or Windows Terminal. The config file lives at %USERPROFILE%\.ssh\config.

For teams with years of PuTTY sessions:

1. Generate the key pair with PuTTYgen (Ed25519, 256-bit).
2. Export the public key in OpenSSH format (right pane of PuTTYgen, "Public key for pasting into OpenSSH authorized_keys file"). Paste this into GCP metadata as in Approach A.
3. Save the private key as a .ppk file for PuTTY's use.
4. PuTTY session: Host Name = EXTERNAL_IP, Port = 22, Connection, SSH, Auth, Credentials, Private key file = the .ppk.
5. Connection, Data, Auto-login username = ishan (or your OS Login derived username).

Save the session by name. For migrating saved PuTTY sessions across Windows machines, see How to Export and Import PuTTY Sessions and Settings. PuTTY stores everything in the Windows registry, and the export-via-regedit workflow saves a portable .reg file that imports cleanly on a fresh machine.

Comparison: gcloud vs OpenSSH vs PuTTY vs IAP

Plain OpenSSH covers 90% of the workload at zero gcloud dependency. gcloud is convenient for daily work but not necessary for production access. IAP is the right answer when you genuinely cannot expose a public IP, but that constraint is rarer than it sounds.

Troubleshooting: Connection refused, Permission denied

ssh: connect to host X.X.X.X port 22: Connection refused

The VM is reachable but nothing is listening on port 22, or the firewall is blocking the SYN.

• The VM might be down. Check Console, VM instances, status.
• A firewall rule blocks TCP/22. Check VPC, Firewall for a rule with protocols: tcp:22 that targets the VM and has your IP in Source IPv4 ranges.
• sshd crashed on the VM. Rare on stock images. Recovery: use the Console's serial console or boot a recovery VM that attaches the disk.

Permission denied (publickey)

Network works, key is not accepted.

• Wrong username. Metadata path: username is the key's comment field. OS Login path: the IAM-derived form like cloudysanfrancisco_gmail_com.
• Wrong key. Run ssh -v to see which key OpenSSH offered. Fix with -i ~/.ssh/gcp_ed25519 or ~/.ssh/config.
• Key not yet propagated. Wait 10 seconds.
• Metadata keys and OS Login conflict. OS Login wins when enable-oslogin = TRUE. Either disable OS Login or use the OS Login flow.
• /home/USER/.ssh/authorized_keys has wrong permissions. Use the Console's SSH-in-browser button and run chmod 600 ~/.ssh/authorized_keys && chmod 700 ~/.ssh.

Add -vvv to ssh for full debug output. The Offering public key: and Server accepts key: lines usually pinpoint the issue within 30 seconds.

For scripting the no-gcloud path against many VMs, a Bash for loop over ssh calls is the simplest pattern, and Bash while loops handle the retry-until-ready case. To fetch instance metadata over the REST API in scripts, the curl cheat sheet covers the bearer-token header pattern.

What to do next

• The companion in the GCP cluster: How to Add a Persistent Disk to a Google Cloud VM for the storage side, and How to Increase Google Cloud VM Disk Size Without Rebooting for the live-resize flow.
• SSH Cheat Sheet: the flag and option reference covering everything above.
• How to Export and Import PuTTY Sessions and Settings: the Windows-side workflow for migrating saved sessions, and the registry layout that makes PuTTY config portable.
• Curl Cheat Sheet: when scripting the no-gcloud path and needing to hit the GCP REST API directly.
• The canonical reference: Google Cloud's Connect to a Linux VM guide covers all four methods with official syntax.
• The OpenSSH manual pages are the upstream source of truth for ssh, ssh_config, and authorized_keys behavior.

FAQ