Pulumi & Google Cloud minimum viable build
30 Jun 2022
Here are notes about how I created a single resource inside a Google Cloud playground from A Cloud Guru.
CLI tools
Pulumi CLI
I was working from a Linux-like desktop environment (Windows Subsystem for Linux), so first things first, I installed Pulumi’s CLI tool into that virtual machine with this command:
curl -fsSL https://get.pulumi.com | sh
As prompted by the installer script, I closed my command-line terminal and re-opened it. To validate that I’d correctly installed Pulumi’s command-line tool, I made sure that this command gave me a version number on my screen:
pulumi version
Oh yeah, also this:
pulumi login
Google Cloud SDK
According to Google’s official installation instructions linked to by Pulumi, it was worth checking if my WSL Ubuntu installation had something called “snap” installed. This returned a version number, so I guess it does:
snap --version
I ran this command:
snap install google-cloud-cli --classic
Oops, nevermind …
Interacting with snapd is not yet supported on Windows Subsystem for Linux.
This command has been left available for documentation purposes only.
Okay, let’s try their generic Linux instructions. First I download the latest version:
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-cli-392.0.0-linux-x86_64.tar.gz
Then I uncompress that download into a folder in my Linux VM’s home directory called google-cloud-sdk
:
tar -xf "google-cloud-cli-392.0.0-linux-x86_64.tar.gz"
Then I run a script within it so that my Linux environment comes to recognize various commands as legitimate (in this one, the ~
is because I have google-cloud-sdk
right below my Linux home directory):
~/google-cloud-sdk/install.sh
I let the installer put its settings into ~/.bashrc
when it asked.
As prompted by the installer script, I closed my command-line terminal and re-opened it. Then I ran the following command:
~/google-cloud-sdk/bin/gcloud init
Oh, ouch:
Welcome! This command will take you through the configuration of gcloud.
Your current configuration has been set to: [default]
You can skip diagnostics next time by using the following flag:
gcloud init --skip-diagnostics
Network diagnostic detects and fixes local network connection issues.
Checking network connection...done.
Reachability Check passed.
Network diagnostic passed (1/1 checks passed).
You must log in to continue. Would you like to log in (Y/n)? Y
You are authorizing gcloud CLI without access to a web browser. Please run the following command on a machine with a web browser and copy its output back here. Make sure the installed gcloud version is 372.0.0 or newer.
gcloud auth login --remote-bootstrap="https://accounts.google.com/o/oauth2/auth?response_type=code&client_id=12345678987654321.apps.googleusercontent.com&scope=openid+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fuserinfo.email+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcloud-platform+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fappengine.admin+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fsqlservice.login+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Fcompute+https%3A%2F%2Fwww.googleapis.com%2Fauth%2Faccounts.reauth&state=DX9eINBqvyTaVtIRGF1jEIY0dJZu83&access_type=offline&code_challenge=1abcdef23456789GHIJKLMN0987654321opqrs&code_challenge_method=S256&token_usage=remote"
Interesting … AWS’s CLI is smart enough to open my Windows web browser when in WSL, but GCP’s CLI isn’t.
OK, anyway, docs say to try gcloud init --console-only
, which implies I didn’t need to fully qualify gcloud
anyway so let’s see:
gcloud --version
Yup, that works. So let’s try this:
gcloud init --console-only
Yay, that worked better for letting me log Google Cloud’s CLI into my A Cloud Guru playground.
Although yikes – it says “Pick cloud project to use” and says I’ve already got 3 of them.
Enter verification code: 4/0AX4XfWg88xh4zUFFY1hW-3pu17vWQO6gH4HffQviU3H10-3dPnayw4NJiZdWBPozBvM1vA
You are logged in as: [[email protected]].
Pick cloud project to use:
[1] custom-images-654321
[2] custom-images-staging
[3] playground-s-12-3ab45c67
[4] Enter a project ID
[5] Create a new project
Please enter numeric choice or text value (must exactly match list item):
How would I know which one to pick? I just got here! I think I’ll try Create a new project and call it “my-first-project
.”
Oh, it didn’t like that:
WARNING: Project creation failed: HttpError accessing <https://cloudresourcemanager.googleapis.com/v1/projects?alt=json>: response: <{'vary': 'Origin, X-Origin, Referer', 'content-type': 'application/json; charset=UTF-8', 'content-encoding': 'gzip', 'date': 'Thu, 30 Jun 2022 18:19:18 GMT', 'server': 'ESF', 'cache-control': 'private', 'x-xss-protection': '0', 'x-frame-options': 'SAMEORIGIN', 'x-content-type-options': 'nosniff', 'server-timing': 'gfet4t7; dur=79', 'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"', 'transfer-encoding': 'chunked', 'status': 409}>, content <{
"error": {
"code": 409,
"message": "Requested entity already exists",
"status": "ALREADY_EXISTS"
}
}
>
Please make sure to create the project [my-first-project] using
$ gcloud projects create my-first-project
or change to another project using
$ gcloud config set project <PROJECT ID>
Oh, darnit, I see now back in my web browser that up in the top nav, toward the left, there’s a drop-down that has just one project ID in it … and it was the 3rd one of the 3: “playground-s-12-3ab45c67
”. Oops.
OK, let’s try again:
gcloud init --console-only
Option 1 looks good:
Pick configuration to use:
[1] Re-initialize this configuration [default] with new settings
[2] Create a new configuration
Please enter your numeric choice: 1
Still option 1:
Choose the account you would like to use to perform operations for this configuration:
[1] [email protected]
[2] Log in with a new account
Please enter your numeric choice: 1
Yay, here’s that screen again. I’ll pick #3 this time:
Pick cloud project to use:
[1] custom-images-654321
[2] custom-images-staging
[3] playground-s-12-3ab45c67
[4] Enter a project ID
[5] Create a new project
Please enter numeric choice or text value (must exactly match list item): 3
Gabe Weiss says to go ahead and pick a default region and zone.
OK, Google seems happy:
Your project default Compute Engine zone has been set to [us-central1-a].
You can change it by running [gcloud config set compute/zone NAME].
Your project default Compute Engine region has been set to [us-central1].
You can change it by running [gcloud config set compute/region NAME].
Created a default .boto configuration file at [/home/smit1234linux/.boto]. See this file and
[https://cloud.google.com/storage/docs/gsutil/commands/config] for more
information about configuring Google Cloud Storage.
Your Google Cloud SDK is configured and ready to use!
* Commands that require authentication will use [email protected] by default
* Commands will reference project `playground-s-12-3ab45c67` by default
* Compute Engine commands will use region `us-central1` by default
* Compute Engine commands will use zone `us-central1-a` by default
Run `gcloud help config` to learn how to change individual settings
This gcloud configuration is called [default]. You can create additional configurations if you work with multiple accounts and/or projects.
Run `gcloud topic configurations` to learn more.
Some things to try next:
* Run `gcloud --help` to see the Cloud Platform services you can interact with. And run `gcloud help COMMAND` to get help on any gcloud command.
* Run `gcloud topic --help` to learn about advanced features of the SDK like arg files and output formatting
* Run `gcloud cheat-sheet` to see a roster of go-to `gcloud` commands.
lol, getting GCP’s CLI is taking so long my playground’s going to expire before I test anything w/ Pulumi.
At this point, I might as well figure out the right way to validate that I’m really logged in – could come in handy later. I think I want some sort of command like aws sts get-caller-identity
…
Bingo! Thanks, bilcy on StackOverflow.
gcloud auth list
Okay, that was a lot, but apparently I now have everything installed and logged in.
Pulumi-GCP authorization
Only it looks like maybe now I still set up Pulumi to know how to talk to Google on my behalf? I’ll come back to that later if it doesn’t work.
OK, yup, it was a problem when I tried pulumi up
:
Previewing update (my-first-stack)
Type Name Plan Info
+ pulumi:pulumi:Stack my-first-pulumi-project-my-first-stack create
└─ gcp:storage:Bucket my-bucket 1 error
Diagnostics:
gcp:storage:Bucket (my-bucket):
error: 1 error occurred:
* Attempted to load application default credentials since neither `credentials` nor `access_token` was set in the provider block. No credentials loaded. To use your gcloud credentials, run 'gcloud auth application-default login'. Original error: google: could not find default credentials. See https://developers.google.com/accounts/docs/application-default-credentials for more information.
Oh neat, Google has a whole document dedicted to “gcloud auth application-default login
”. OK, I trust running it. I’m going to do it from my home directory, and it seems I need to add the --no-launch-browser
flag to the end of it due to my weird Google-not-handling-WSL-very-well issue.
Credentials saved to file: [/home/my-linux-username/.config/gcloud/application_default_credentials.json]
These credentials will be used by any library that requests Application Default Credentials (ADC).
Quota project "playground-s-12-3ab45c67" was added to ADC which can be used by Google client libraries for billing and quota. Note that some services may still bill the project owning the resource.
Files
Then I created the following file structure at C:\example\hellopulumi
:
.
├── Pulumi.my-first-stack.yaml
├── Pulumi.yaml
└── Main.yaml
Pulumi.my-first-stack.yaml
config:
gcp:project: playground-s-12-3ab45c67
There used to be a “pulumi:template: aws-yaml
” setting in this file when Pulumi’s CLI tool helped me generate a sample, but taking it out didn’t seem to hurt anything, so it’s not in my “minimum viable build” code.
Pulumi.yaml
name: my-first-pulumi-project
runtime: yaml
description: The description of my first Pulumi project
(Note: If you run pulumi up
with just the above 2 files, you can create an empty Pulumi project+stack, which is great if all you want to do next is run pulumi import ...
to grab some configurations out of things you built in a cloud provider through their web console, so as to see what they look like.)
Main.yaml
resources:
# Create a GCP resource (Storage Bucket)
my-bucket:
type: gcp:storage:Bucket
properties:
location: US
outputs:
# Export the DNS name of the bucket
bucketName: ${my-bucket.url}
(Not that I actually knew what to write here. I stole it from Pulumi’s built-in samples, which created it in Pulumi.yaml
, and then I refactored it into Main.yaml
.)
Commands
Add a bucket
Then, from within C:\example\hellopulumi
, I ran:
pulumi up
It hung, but Ctrl+C got it going for some reason.
When asked, “Do you want to peform this update?” I arrowed down to yes and hit Enter.
In about 5 seconds, Pulumi told me it was done.
Verify it’s there
Heading over to my GCP playground’s Cloud Storage dashboard at https://console.cloud.google.com/storage/browser?project=playground-s-12-3ab45c67&prefix=
, I could see that indeed, suddenly a Cloud Storage bucket existed. Neat.
Delete the bucket
I’m sure there are ways to clean up that aren’t quite so “nuclear,” but here’s how I did so: I deleted every last resource in “my-first-stack
:”
pulumi destroy -s my-pulumi-username/my-first-pulumi-project/my-first-stack
Verify it’s gone
Back to the GCP playground’s Cloud Storage web dashboard at https://console.cloud.google.com/storage/browser?project=playground-s-12-3ab45c67&prefix=
.
Yup, I’m back to no buckets.
Try a database
(Still also need a Pulumi.some-stack-name.yaml
file and a Pulumi.yaml
file, but they’re more or less the same.)
Main.yaml
resources:
# Create a GCP resource (Storage Bucket)
my-pulumi-database-instance:
type: gcp:sql:DatabaseInstance
properties:
databaseVersion: SQLSERVER_2019_STANDARD
name: hello-pulumi-db
region: us-central1
rootPassword: a-really-great-password # Root username becomes "sqlserver"
settings:
tier: db-custom-2-8192
ipConfiguration:
authorizedNetworks:
- name: "My own computer"
value: "123.231.123.231/32"
collation: SQL_Latin1_General_CP1_CI_AS
diskSize: 100
locationPreference:
zone: us-central1-b
my-new-db-user:
type: gcp:sql:User
properties:
instance: ${my-pulumi-database-instance.name}
name: extra-database-user
password: another-really-great-password
my-extra-database:
type: gcp:sql:Database
properties:
instance: ${my-pulumi-database-instance.name}
name: cool-extra-db