Salesforce, Python, SQL, & other ways to put your data where you need it

Need event music? 🎸

Live and recorded jazz, pop, and meditative music for your virtual conference / Zoom wedding / yoga class / private party with quality sound and a smooth technical experience

Random handy CumulusCI notes

13 Jul 2021 πŸ”– salesforce
πŸ’¬ EN

Table of Contents

Below are some miscellaneous Salesforce scratch org / packaging / SFDX / CumulusCI / VSCode / Git notes I don’t want to forget.

My colleague set up a Git repository with the following structure:

.
β”œβ”€β”€ .github
β”‚   └── PULL_REQUEST_TEMPLATE.md
β”œβ”€β”€ datasets
β”‚   └── mapping.yml
β”œβ”€β”€ force-app
β”‚   └── main
β”‚       └── default
β”‚           └── objects
β”‚               └── Legacy_Custom_Object__c
β”‚                   β”œβ”€β”€ fields
β”‚                   β”‚   β”œβ”€β”€ Field_One__c.field-meta.xml
β”‚                   β”‚   └── Field_Two__c.field-meta.xml
β”‚                   β”œβ”€β”€ listViews
β”‚                   β”‚   └── All.listView-meta.xml
β”‚                   β”œβ”€β”€ recordTypes
β”‚                   β”‚   β”œβ”€β”€ Type_One.recordType-meta.xml
β”‚                   β”‚   └── Type_Two.recordType-meta.xml
β”‚                   └── Legacy_Custom_Object__c.object-meta.xml
β”œβ”€β”€ orgs
β”‚   β”œβ”€β”€ beta.json
β”‚   β”œβ”€β”€ dev.json
β”‚   β”œβ”€β”€ feature.json
β”‚   └── release.json
β”œβ”€β”€ robot
β”‚   └── OurPackageName
β”‚       └── tests
β”‚           └── create_contact.robot
β”œβ”€β”€ .gitignore
β”œβ”€β”€ cumulusci.yml
β”œβ”€β”€ README.md
└── sfdx-project.json

And the following .gitignore:

# Salesforce / SFDX / CCI
.cci
.sfdx
/src.orig
/src

# Python
*.pyc
__pycache__

# Robot Framework results
robot/OurPackageName/results/

# Editors
*.sublime-project
*.sublime-workspace
.vscode
.idea
.project
.settings

# Misc
.DS_Store

I did a git clone to my local machine so I could work on it.

All right, what do I do?

VSCode Peacock color theme

Updated /.vscode/settings.json with a nice Peacock color that’ll help me remember what’s what.

Didn’t need to .gitignore the new .vscode folder because it’s already in there. Yay.

Git branch

Used GitHub Desktop to create a new branch newfeature and Publish it to GitHub.

Verified that it appears on GitHub as β€œeven with main.”

Verified that VSCode is showing up with the correct branch selected in my git control panel.

Configure a dev hub username in my local codebase

I created a file /.sfdx/sfdx-config.json whose contents are:

{ "defaultdevhubusername": "my-company-hub-org-nickname" }

Manipulating a Salesforce scratch org

Create a scratch org

In my VSCode terminal, at the root of my repo, run:

`sfdx auth:list`

I see the β€œhub” in the list for the production org I’m going to want to spin up a scratch from, so that seems promising. Pretty sure I can skip any sort of sfdx force:auth:web:login --setdefaultdevhubusername --setalias my-hub-org command (my-hub-org being an alias for handy use).

Also, since there’s clearly a lot of boilerplate already in the repo, I’m guessing I don’t have to do anything along the lines of sfdx force:project:create --projectname mywork --template standard or, alternatively, cci project init (which is normally the first thing I’d apparently do after git init if I were setting up my own project from scratch, using a source format of sfdx, not mdapi, and ).

The Summit Events docs pointed out that I can confirm this by running the following command:

cci project info

Oh, cute, time to upgrade CCI.

pipx upgrade cumulusci

I ran cci org list and, unsurprisingly, I don’t have any scratch orgs spinning yet.

Time to create one:

cci flow run dev_org --org dev

Took about a minute or two.

This fails with an error of ERROR running force:org:create: This command requires a dev hub org username set either with a flag or by default in the config. Run this command for more information about debugging errors: cci error --help if I didn’t set up sfdx-config.json

Now there’s a scratch org under the dev row of the output of cci org list.

Open the scratch org

Now I’ll open the dev org in my web browser:

cci org browser --org dev

Make and track changes

  1. If I ever needed to retrieve point-and-click changes from the dev scratch org onto my local machine, I could use cci task run list_changes --org dev and cci task run retrieve_changes --org dev to do so.
  2. If I make any changes on my local machine that I want to push to the dev scratch org, I can run cci task run dx_push --org dev.
    • I haven’t tested thoroughly, but it looks like this command might only push changes that are committed to my local git repository, not just ones that are saved to disk on my local machine.
    • Maybe I’m just doing things wrong, but I’m finding the deployment & test-running workflow annoying compared to the way I have coding set up against more traditional sandboxes w/ a package.xml file, so I’ve jumped to coding in Developer Console instead of VSCode for this project (using retrieve_changes when I’m happy), since it’s a small project and not worth my time to fight VSCode over.

When I have things to push to GitHub, I can use ordinary git commands.

Delete org and recreate with dependent packages

Oops, this one needs an internal package, the Education Data Architecture (β€œEDA”), and the Marketing Cloud Connector in it.

https://cumulusci.readthedocs.io/en/main/cheat_sheet.html

cci org scratch_delete dev

Great, cci org list shows dev as still a row in the table, but nothing there.

In /cumulusci.yml, I added this under the project outermost property, right below its package subproperty and above its git subproperty:

    dependencies:
        - version_id: '1234567890' # Dependency 1:  Corporate internal package
        - github: 'https://github.com/SalesforceFoundation/EDA' # Dependency 2:  EDA
        - namespace: 'et4ae5' # Dependency 3:  MarketingCloud Connector version 230.1
          version: 230.1 

Let’s create it again:

cci flow run dev_org --org dev

Phew – that’s a lot slower – EDA’s a big boy to install. I think last time it might’ve taken 20 minutes?

The org-creation logs ended like this:

?[32m2021-07-08 16:07:15:?[0m [InProgress]: next check in 1 seconds
?[32m2021-07-08 16:07:16:?[0m [Done]
?[32m2021-07-08 16:07:17:?[0m [Success]: Succeeded
?[32m2021-07-08 16:11:51:?[0m ?[31mException in task dependencies.update_dependencies?[0m

<class 'requests.exceptions.RetryError'>: An error occurred while making a request to GitHub: HTTPSConnectionPool(host='api.github.com', port=443): Max retries exceeded with url: /repos/SalesforceFoundation/EDA (Caused by ResponseError('too many 401 error responses'))
Run this command for more information about debugging errors: cci error --help

But luckily, opening the org went just fine despite the error at the end of the logs.

I can see hed__Trigger_Handler__c, an EDA object, in the Object Manager in the org’s Setup.

cci org browser --org dev

Open the scratch org manually in a different web browser

Editing custom metadata and switching to classic aren’t working in Firefox from a VSCode-opened browser session, so I need to fire up Chrome and log in manually.

cci org browser --org dev --url-only

I copy the URL displayed & paste it into Chrome.

Tip: Ctrl+Click makes VSCode give me an option to copy rather than open. Keeps me from having to carefully select the URL from beginning to end.

Much better – now I can see the body of https://....lightning.force.com/lightning/setup/CustomMetadata/home.

Deploying the code I write in a scratch org into actual orgs

Update 9/22/21: it’s been a while, and people are ready to test & retest my code up the QA chain of orgs.

Log into a Salesforce sandbox

Okay, so opening up VSCode and navigating to the folder where I was developing my codebase, I tried:

cci org list

The sandbox I want to push the code to is not there.

How about this?

sfdx auth:list

The sandbox I want to push the code to is not there.

β”ŒScratch Orgs───────┬──────┬─────────┬─────────┬────────┐
β”‚ Name    β”‚ Default β”‚ Days β”‚ Expired β”‚ Config  β”‚ Domain β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ beta    β”‚         β”‚ 1    β”‚ -       β”‚ beta    β”‚        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ dev     β”‚ +       β”‚ 7    β”‚ -       β”‚ dev     β”‚        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ feature β”‚         β”‚ 1    β”‚ -       β”‚ feature β”‚        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ qa      β”‚         β”‚ 7    β”‚ -       β”‚ qa      β”‚        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ release β”‚         β”‚ 1    β”‚ -       β”‚ release β”‚        β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”˜

β”ŒConnected Orgs──┬──────────┬─────────┐
β”‚ Name β”‚ Default β”‚ Username β”‚ Expires β”‚
β””β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Trying to add it via the SFDX CLI (this requires me to log in w/ my browser):

sfdx auth:web:login -r https://customdomain--sandboxname.my.salesforce.com -a CleverAliasName

Listing again:

sfdx auth:list

Okay, now it’s there, but I really would prefer to play w/ CumulusCI since that’s what I wrote down directions for.

Per Connect persistent orgs, I should try:

cci org connect CleverAliasName --sandbox --login-url https://customdomain--sandboxname.my.salesforce.com

(Sad, this doesn’t have a –urlonly option. Luckily, the β€œapprove?” button slows things down and lets me capture the URL and copy/paste it to a different browser where I’m actually logged in.)

(Also, when I had to run this again the same day from another project’s command line prompt because it wasn’t showing up in cci org list from there, there was no browser login … it just sailed through.)

cci org list

Okay, great:

cci org list
β”ŒScratch Orgs───────┬──────┬─────────┬─────────┬────────┐
β”‚ Name    β”‚ Default β”‚ Days β”‚ Expired β”‚ Config  β”‚ Domain β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ beta    β”‚         β”‚ 1    β”‚ -       β”‚ beta    β”‚        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ dev     β”‚ +       β”‚ 7    β”‚ -       β”‚ dev     β”‚        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ feature β”‚         β”‚ 1    β”‚ -       β”‚ feature β”‚        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ qa      β”‚         β”‚ 7    β”‚ -       β”‚ qa      β”‚        β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ release β”‚         β”‚ 1    β”‚ -       β”‚ release β”‚        β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”˜

β”ŒConnected Orgs─────────────┬─────────┬──────────────────────────────────┬────────────┐
β”‚ Name                      β”‚ Default β”‚ Username                         β”‚ Expires    β”‚
β”œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”Όβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€
β”‚ CleverAliasName           β”‚         β”‚ me@example.com.sandboxname       β”‚ Persistent β”‚
β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

Push the repo to the sandbox

Okay, here we go.

I’m in a Git-tracked folder, I’ve done a cci task run dx_push call before and it worked … let’s go for broke!

cci task run dx_push --org CleverAliasName

Interesting – that got rejected because this particular sandbox is a Full sandbox, and it looks like it only works against β€œDeveloper” & β€œDeveloper Pro” sandboxes.

All right … what else can I try … maybe this:

cci task run deploy --org CleverAliasName

Hmmm – nope.

2021-09-22 15:59:15: Org info updated, writing to keychain
2021-09-22 15:59:15: Beginning task: Deploy
2021-09-22 15:59:15: As user: me@example.com.sandboxname
2021-09-22 15:59:15: In org: 0012345432123454321
2021-09-22 15:59:15:
2021-09-22 15:59:15: src not found.
2021-09-22 15:59:15: Deployment package is empty; skipping deployment.

Ah, here we go – apparently the cool kids now put stuff in src folders but this project had code folders more like force-app\main\default\class and force-app\main\default\trigger

cci task run deploy --org CleverAliasName --path force-app\main\default    

All right … it’s not happy with me now because there are prereqs for that codebase missing … fair enough … this isn’t going the normal packaging-dependent way w/ scratch orgs and whatnot, so I guess I have to go open up a prereq package & repeat these steps.

2021-09-22 16:19:10: Org info updated, writing to keychain
2021-09-22 16:19:10: Beginning task: Deploy
2021-09-22 16:19:10: As user: me@example.com.sandboxname
2021-09-22 16:19:10: In org: 0012345432123454321
2021-09-22 16:19:10:
2021-09-22 16:19:10: Converting from SFDX to MDAPI format.
2021-09-22 16:19:18: Cleaning meta.xml files of packageVersion elements for deploy
2021-09-22 16:19:18: Payload size: 16252 bytes
2021-09-22 16:19:18: Pending
2021-09-22 16:19:19: [Pending]: next check in 1 seconds
2021-09-22 16:19:20: [Pending]: next check in 1 seconds
2021-09-22 16:19:21: [Done]
2021-09-22 16:19:23: [Success]: Succeeded

Okay … did that, it went through, trying again.

Okay, it’s still not happy with me on account of … let’s see … weird things that look like they belong to before I wrote my code?

Oh. I was pushing from the main branch of my codebase, but it turns out that my most recent updates are on a different branch. Used GitHub Desktop to switch branches & re-fetch. Now my codebase looks like what I wrote.

I’ll have to remember to get the boss to incorporate a pull request once this all goes through. Probably should do it now, but arguably maybe it’s fine for us to test, first.

Here we go again:

cci task run deploy --org CleverAliasName --path force-app\main\default    

Success?

2021-09-22 16:24:13: Org info updated, writing to keychain
2021-09-22 16:24:13: Beginning task: Deploy
2021-09-22 16:24:13: As user: me@example.com.sandboxname
2021-09-22 16:24:13: In org: 0012345432123454321
2021-09-22 16:24:13:
2021-09-22 16:24:13: Converting from SFDX to MDAPI format.
2021-09-22 16:24:22: Cleaning meta.xml files of packageVersion elements for deploy
2021-09-22 16:24:22: Payload size: 43672 bytes
2021-09-22 16:24:22: Pending
2021-09-22 16:24:23: [Pending]: next check in 1 seconds
2021-09-22 16:24:24: [Pending]: next check in 1 seconds
2021-09-22 16:24:25: [Pending]: next check in 2 seconds
2021-09-22 16:24:27: [Pending]: next check in 2 seconds
2021-09-22 16:24:30: [Pending]: next check in 2 seconds
2021-09-22 16:24:32: [Pending]: next check in 3 seconds
2021-09-22 16:24:35: [InProgress]: Processing Type: CustomObject
2021-09-22 16:24:36: [InProgress]: Processing Type: ApexClass
2021-09-22 16:24:38: [InProgress]: Processing Type: ApexClass
2021-09-22 16:24:39: [InProgress]: Processing Type: PermissionSet
2021-09-22 16:24:40: [InProgress]: Processing Type: PermissionSet
2021-09-22 16:24:42: [Done]
2021-09-22 16:24:43: [Success]: Succeeded

Cool! I see one of the triggers I meant to deploy when I visit β€œsetup” via this URL:

https://customdomain--sandboxname.lightning.force.com/lightning/setup/ApexTriggers/page?address=%2Fsetup%2Fbuild%2FallTriggers.apexp

Off to here to run the unit test classes I just deployed:

https://customdomain--sandboxname.lightning.force.com/lightning/setup/ApexTestQueue/home

Hooray, they pass! Time to let users know they can test my code.

Run a specific Apex test class

Darnit, we made some changes and things here failing.

Here’s the CumulusCI command to run a specific test class:

cci task run run_tests --org CleverAliasName --test_name_match TheApexClassName

Output:

2021-09-29 11:52:29: Org info updated, writing to keychain
2021-09-29 11:52:29: Beginning task: RunApexTests
2021-09-29 11:52:29: As user: me@example.com.sandboxname
2021-09-29 11:52:29: In org: 0012345432123454321
2021-09-29 11:52:29:
2021-09-29 11:52:29: Running query: SELECT Id, Name FROM ApexClass WHERE NamespacePrefix = null AND (Name LIKE 'prefctr_Contact_TEST')
2021-09-29 11:52:29: Found 1 test classes
2021-09-29 11:52:29: Queuing tests for execution...
2021-09-29 11:52:30: Completed: 0  Processing: 0  Queued: 1
2021-09-29 11:52:31: Completed: 0  Processing: 0  Queued: 1
2021-09-29 11:52:32: Completed: 0  Processing: 0  Queued: 1
2021-09-29 11:52:33: Increased polling interval to 2 seconds
2021-09-29 11:52:33: Completed: 0  Processing: 0  Queued: 1
2021-09-29 11:52:35: Completed: 0  Processing: 0  Queued: 1
2021-09-29 11:52:37: Completed: 0  Processing: 1 (prefctr_Contact_TEST)  Queued: 0
...
2021-09-29 11:53:56: --------------------------------------------------------------------------------
2021-09-29 11:53:56: Pass: 1  Retried: 0  Fail: 1  CompileFail: 0  Skip: 0
2021-09-29 11:53:56: --------------------------------------------------------------------------------
2021-09-29 11:53:56: --------------------------------------------------------------------------------
2021-09-29 11:53:56: Failing Tests
2021-09-29 11:53:56: --------------------------------------------------------------------------------
2021-09-29 11:53:56: 1: TheApexClassName.theApexMethodName - Fail
2021-09-29 11:53:56:    Message: System.AssertException: Assertion Failed: My custom error message here: Expected: 3, Actual: 1
2021-09-29 11:53:56:    StackTrace: Class.TheApexClassName.theApexMethodName: line 205, column 1
Class.TheApexClassName.theApexMethodName: line 53, column 1

Cool – there’s what I want, that My custom error message here: Expected: 3, Actual: 1 right toward the end. Wasn’t getting that when a deploy was failing out on account of test failures, even with cci error info.

Incorporating some production data into future scratch orgs

Turns out someone wanted to work a Contact.Custom_Field__c field into the code I’d written – which now means there’s a dependency upon that field in my codebase.

I should probably spin off the existence of a Contact.Custom_Field__c field into its own project, since it’s not strictly essential to my code, but just to play, I’m going to see what it’d take to add that field’s definition to my project.

Git prep

I used GitHub Desktop to spin up a new branch (wow, they make it easy to make sure you don’t click the wrong buttons) and verified that VSCode is now showing that I’m working in that branch.

Downloading code from production

I created a file at manifest\get_my_field.xml:

<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Package xmlns="http://soap.sforce.com/2006/04/metadata">
	<types>
    <members>Contact.Custom_Field__c</members>
		<name>CustomField</name>
	</types>
	<version>51.0</version>
</Package>

Then I fetched the current definition of Contact.Custom_Field__c out of production with the following command:

cci task run retrieve_unpackaged --path force-app\main\default --org CleverAliasName --package-xml manifest\get_my_field.xml

But, sadly, it downloaded the definition of Contact.Custom_Field__c into a file called force-app\main\default\objects\Contact.object with content like this:

<?xml version="1.0" encoding="UTF-8"?>
<CustomObject xmlns="http://soap.sforce.com/2006/04/metadata">
    <fields>
        <fullName>Custom_Field__c</fullName>
        <externalId>false</externalId>
        ...
        <unique>false</unique>
    </fields>
</CustomObject>

It also created a weird clone of manifest\get_my_field.xml at force-app\main\default\package.xml.

That’s all right – close enough – I just needed the <fields> list.

I had some other custom fields in this and other projects, so I had a pretty good idea how they should be structured.

I copied the entire Contact folder from a project that happened to contain nothing but a Different_Field__c file in it into my project and carefully did surgery until I had a force-app\main\default\objects\Contact\fields\Custom_Field__c.field-meta.xml file that looked like this:

<?xml version="1.0" encoding="UTF-8"?>
<CustomField xmlns="http://soap.sforce.com/2006/04/metadata">
    <fullName>Custom_Field__c</fullName>
    <externalId>false</externalId>
    ...
    <unique>false</unique>
</CustomField>

File cleanup

Then I deleted 3 files I no longer needed:

  1. force-app\main\default\objects\Contact.object
  2. force-app\main\default\package.xml
  3. manifest\get_my_field.xml

Git work

I used Git features in VSCode to β€œcommit” the new force-app\main\default\objects\Contact\fields\Custom_Field__c.field-meta.xml to the β€œbranch” of my Git project I’d just spun up, and GitHub Desktop to push my changes up to my new branch on the remote repository. I’m not sure the remote bit was necessary for cci, but I did it just in case.

Does it scratch org?

I ran cci org list to pick a blank scratch org spot (I liked that feature only lives a day) and ran cci flow run dev_org --org feature. It’s going to take a good 30 minutes to spin up since it’s some big Salesforce packages as dependencies.

DARN – about 5 minutes in, it failed.

2021-10-01 10:28:25: ============================================================
2021-10-01 10:28:25: Initializing flow: FlowCoordinator (dev_org)
2021-10-01 10:28:25: Set up an org as a development environment for unmanaged metadata
2021-10-01 10:28:25: ============================================================
2021-10-01 10:28:25:
2021-10-01 10:28:25: Verifying and refreshing credentials for the specified org: feature.
2021-10-01 10:28:25: Creating scratch org with command: sfdx force:org:create  -f orgs/feature.json -w 120 -n --durationdays 1 -a GitHubRepoName__GitBranchName adminEmail=my_github_commit_email@example.com
2021-10-01 10:31:33: Successfully created scratch org: 0012345432123454321, username: me@example.com.scratchorgname
2021-10-01 10:31:33: Generating scratch org user password with command: sfdx force:user:password:generate -u me@example.com.scratchorgname
2021-10-01 10:31:42: Getting org info from Salesforce CLI for me@example.com.scratchorgname
2021-10-01 10:31:50: Org info updated, writing to keychain
2021-10-01 10:31:50: ------------------------------------------------------------
2021-10-01 10:31:50: Organization:
2021-10-01 10:31:50:   Username: me@example.com.scratchorgname
2021-10-01 10:31:50:     Org Id: 0012345432123454321
2021-10-01 10:31:50:   Instance: CS00
2021-10-01 10:31:50: ------------------------------------------------------------
2021-10-01 10:31:50:
2021-10-01 10:31:50: ------------------------------------------------------------
2021-10-01 10:31:50: Steps:
2021-10-01 10:31:50: Description: Set up an org as a development environment for unmanaged metadata
2021-10-01 10:31:50:
2021-10-01 10:31:50: Flow Steps
2021-10-01 10:31:50: 1) flow: dependencies [from current folder]
2021-10-01 10:31:50:     1) task: update_dependencies
2021-10-01 10:31:50:     2) task: deploy_pre
2021-10-01 10:31:50: 2) flow: deploy_unmanaged
2021-10-01 10:31:50:     0) task: dx_convert_from
2021-10-01 10:31:50:        when: project_config.project__source_format == "sfdx" and not org_config.scratch
2021-10-01 10:31:50:     1) task: unschedule_apex
2021-10-01 10:31:50:     2) task: update_package_xml
2021-10-01 10:31:50:        when: project_config.project__source_format != "sfdx" or not org_config.scratch
2021-10-01 10:31:50:     3) task: deploy
2021-10-01 10:31:50:        when: project_config.project__source_format != "sfdx" or not org_config.scratch
2021-10-01 10:31:50:     3.1) task: deploy
2021-10-01 10:31:50:          when: project_config.project__source_format == "sfdx" and org_config.scratch
2021-10-01 10:31:50:     4) task: uninstall_packaged_incremental
2021-10-01 10:31:50:        when: project_config.project__source_format != "sfdx" or not org_config.scratch
2021-10-01 10:31:50:     5) task: snapshot_changes
2021-10-01 10:31:50: 3) flow: config_dev
2021-10-01 10:31:50:     1) task: deploy_post
2021-10-01 10:31:50:     2) task: update_admin_profile
2021-10-01 10:31:50: 4) task: snapshot_changes
2021-10-01 10:31:50: ------------------------------------------------------------
2021-10-01 10:31:50:
2021-10-01 10:31:50: Starting execution
2021-10-01 10:31:50: ============================================================
2021-10-01 10:31:50:
2021-10-01 10:31:50: ------------------------------------------------------------
2021-10-01 10:31:50: Running task: update_dependencies
2021-10-01 10:31:50: ------------------------------------------------------------
2021-10-01 10:31:50:
2021-10-01 10:31:50: Options:
2021-10-01 10:31:50:   resolution_strategy: preproduction
2021-10-01 10:31:50:   packages_only: False
2021-10-01 10:31:50: Beginning task: UpdateDependencies
2021-10-01 10:31:50:
2021-10-01 10:31:50: Resolving dependencies...
2021-10-01 10:36:25: Exception in task dependencies.update_dependencies


Bad credentials. Verify that your personal access token is correct and that you are authorized to access this resource.

Run this command for more information about debugging errors: cci error --help

Okay, but cci org list shows it as existing, and I can totally log into it with cci org browser --org feature --url-only. That’s weird.

Let’s poke around the org and see what it’s got in it.

Okay, yeah, that’s messed up that the Apex Classes list at https://....lightning.force.com/lightning/setup/ApexClasses/home is empty. I definitely wrote Apex in there.

Okay, interesting – it spun up a scratch org but didn’t put my code into it. Annoying.

I wonder if this has to do with Windows chronically forgetting the GitHub Personal Access Token I keep trying to get it to remember so that I can authenticate to my company’s private repositories, one of which is a dependency for scratch orgs…

Trailhead seems to have some advice under Connect CumlusCI To Your GitHub Account.

cci service connect github github.com

I punched in my GitHub username, token, and the e-mail address I’d like to see attached to commits, and it asked me:

A default service already exists for service type github. Would you like to set this service as the new default? [y/N]:

Have I done this before? Anyway, I chose y. If things work, I’ll be happy. Also, now that I think about it, maybe I did change my PAT in GitHub since the last time I did this.

Service github:github.com is now connected
Service github:github.com is now the default for service type: github.

Now time to cci org scratch_delete feature and cci flow run dev_org --org feature. Cross your fingers for me.

Okay, this is going a LOT better already…

2021-10-01 11:03:39: ============================================================
2021-10-01 11:03:39: Initializing flow: FlowCoordinator (dev_org)
2021-10-01 11:03:39: Set up an org as a development environment for unmanaged metadata
2021-10-01 11:03:39: ============================================================
2021-10-01 11:03:39:
2021-10-01 11:03:39: Verifying and refreshing credentials for the specified org: feature.
2021-10-01 11:03:39: Creating scratch org with command: sfdx force:org:create  -f orgs/feature.json -w 120 -n --durationdays 1 -a GitHubRepoName__GitBranchName adminEmail=my_github_commit_email@example.com
2021-10-01 11:04:11: Successfully created scratch org: 0012345432123454321, username: me@example.com.scratchorgname
2021-10-01 11:04:11: Generating scratch org user password with command: sfdx force:user:password:generate -u me@example.com.scratchorgname
2021-10-01 11:04:18: Getting org info from Salesforce CLI for me@example.com.scratchorgname
2021-10-01 11:04:26: Org info updated, writing to keychain
2021-10-01 11:04:26: ------------------------------------------------------------
2021-10-01 11:04:26: Organization:
2021-10-01 11:04:26:   Username: me@example.com.scratchorgname
2021-10-01 11:04:26:     Org Id: 0012345432123454321
2021-10-01 11:04:26:   Instance: CS00
2021-10-01 11:04:26: ------------------------------------------------------------
2021-10-01 11:04:26:
2021-10-01 11:04:26: ------------------------------------------------------------
2021-10-01 11:04:26: Steps:
2021-10-01 11:04:26: Description: Set up an org as a development environment for unmanaged metadata
2021-10-01 11:04:26:
2021-10-01 11:04:26: Flow Steps
2021-10-01 11:04:26: 1) flow: dependencies [from current folder]
2021-10-01 11:04:26:     1) task: update_dependencies
2021-10-01 11:04:26:     2) task: deploy_pre
2021-10-01 11:04:26: 2) flow: deploy_unmanaged
2021-10-01 11:04:26:     0) task: dx_convert_from
2021-10-01 11:04:26:        when: project_config.project__source_format == "sfdx" and not org_config.scratch
2021-10-01 11:04:26:     1) task: unschedule_apex
2021-10-01 11:04:26:     2) task: update_package_xml
2021-10-01 11:04:26:        when: project_config.project__source_format != "sfdx" or not org_config.scratch
2021-10-01 11:04:26:     3) task: deploy
2021-10-01 11:04:26:        when: project_config.project__source_format != "sfdx" or not org_config.scratch
2021-10-01 11:04:26:     3.1) task: deploy
2021-10-01 11:04:26:          when: project_config.project__source_format == "sfdx" and org_config.scratch
2021-10-01 11:04:26:     4) task: uninstall_packaged_incremental
2021-10-01 11:04:26:        when: project_config.project__source_format != "sfdx" or not org_config.scratch
2021-10-01 11:04:26:     5) task: snapshot_changes
2021-10-01 11:04:26: 3) flow: config_dev
2021-10-01 11:04:26:     1) task: deploy_post
2021-10-01 11:04:26:     2) task: update_admin_profile
2021-10-01 11:04:26: 4) task: snapshot_changes
2021-10-01 11:04:26: ------------------------------------------------------------
2021-10-01 11:04:26:
2021-10-01 11:04:26: Starting execution
2021-10-01 11:04:26: ============================================================
2021-10-01 11:04:26:
2021-10-01 11:04:26: ------------------------------------------------------------
2021-10-01 11:04:26: Running task: update_dependencies
2021-10-01 11:04:26: ------------------------------------------------------------
2021-10-01 11:04:26:
2021-10-01 11:04:26: Options:
2021-10-01 11:04:26:   resolution_strategy: preproduction
2021-10-01 11:04:26:   packages_only: False
2021-10-01 11:04:27: Beginning task: UpdateDependencies
2021-10-01 11:04:27: 
2021-10-01 11:04:27: Resolving dependencies...
2021-10-01 11:04:28: Collecting dependencies from Github repo https://github.com/Salesforce/...
...
2021-10-01 11:04:30: In Progress
2021-10-01 11:04:31: In Progress
...

Well over 5 minutes in and still a lot of In Progress indicators. I think I just had changed my GitHub PAT (personal access token) and had forgotten to put the change into the CumulusCI command line tool.

Now I’m just waiting to see if force-app\main\default\objects\Contact\fields\Custom_Field__c.field-meta.xml was the right code to set up in my repository to make sure that all scratch orgs spun up from this repository have a Custom_Field__c field on Contact.

(Again, I’m not convinced I shouldn’t break this field definition out into its own dependent repository, but this is a good baby step just to make sure I even got the definition right.)

Okay, it’s loaded up. cci org browser --org feature --url-only and open it up…

  • Awesome, my custom Apex classes show up at https://....lightning.force.com/lightning/setup/ApexClasses/home.
  • Awesome, my Custom_Field__c field shows up at https://....lightning.force.com/one/one.app#/setup/ObjectManager/Contact/FieldsAndRelationships/view
  • BAD – several unit tests at https://....lightning.force.com/lightning/setup/ApexTestQueue/home fail.

Okay, they’re all failing due to … a FIELD_CUSTOM_VALIDATION_EXCEPTION … looks like one of those managed package dependencies needs to have some data reconfigured in it…

Not finding anything in Validation Rules for Contact (https://....lightning.force.com/lightning/setup/ObjectManager/Contact/ValidationRules/view) for production vs. this scratch org…

Okay, I think maybe I found the setting – had to add β€œCustom Field” as a picklist value for a different custom field that came with one of the packages. Running tests again.

Darnit, that wasn’t it either.

Ooooooh, I found it. That actually was close to the issue – my codebase said to use a picklist value that was like in production, not like as the package gets installed by default, and they’re so close I hadn’t noticed a subtle spelling change we’ve made in our orgs.

  • I wonder if I can add fixing that picklist to my (or the to-be-created new dependent) package.

Okay, that error’s out of the way, but now a test is failing because the scratch org is coming with Standard Contact duplicate rule turned on and it doesn’t like how similar my test records are to each other. I see that we have that deactivated in prod, so buh-bye. This must be something that was released since the last time I was developing this git repository against a scratch org.

  • I wonder if I can add fixing that to a package as well.

(Yes, all tests passed once I deactivated Standard Contact duplicate rule and Standard Contact Matching Rule.)

Phew!

Do it all again

All right, I guess I have to work through this again. Download some more data out of prod w/ packages I’ll later delete, incorporate, try a new scratch org, etc.

Let’s see if doing >cci task run dx_push --org feature after adding a little force-app\main\default\objects\Contact\fields\thepackage__thefield__c.field-meta.xml file with only a <valueSet> inside it works:

<?xml version="1.0" encoding="UTF-8"?>
<CustomField xmlns="http://soap.sforce.com/2006/04/metadata">
    <valueSet>
        <valueSetDefinition>
            <sorted>false</sorted>
            <value>
                <fullName>...</fullName>
                <default>false</default>
                <label>...</label>
            </value>
            ...
            <value>
                <fullName>...</fullName>
                <default>false</default>
                <label>...</label>
            </value>
        </valueSetDefinition>
    </valueSet>
</CustomField>

Nope.

D:\Users\gump1149\Documents\GitRepositories\sf\eda\edaprefctr202103github>cci task run dx_push --org feature
An update to CumulusCI is available. To install the update, run this command: pipx upgrade cumulusci
2021-10-01 14:35:25: Getting org info from Salesforce CLI for me@example.com.scratchorgname
2021-10-01 14:35:36: Org info updated, writing to keychain
2021-10-01 14:35:36: Beginning task: SFDXOrgTask
2021-10-01 14:35:36: As user: me@example.com.scratchorgname
2021-10-01 14:35:36: In org: 0012345432123454321
2021-10-01 14:35:36:
2021-10-01 14:35:36: Running command: sfdx force:source:push -u me@example.com.scratchorgname
2021-10-01 14:35:44: STATE     FULL NAME                        TYPE         PROJECT PATH
2021-10-01 14:35:44: ────────  ───────────────────────────────  ───────────  ────────────────────────────────────────────────────────────────────────────────────
2021-10-01 14:35:44: Conflict  Contact.thepackage__thefield__c  CustomField  force-app\main\default\objects\Contact\fields\thepackage__thefield__c.field-meta.xml
2021-10-01 14:35:45: Return code: 1
ERROR:  We couldn't complete the push operation due to conflicts. Verify that you want to keep the local versions, then run "sfdx force:source:push -f" with the --forceoverwrite (-f) option.
ERROR running force:source:push:  Source conflict(s) detected.

Hmmm – okay, what if I just pull the changes I made manually and see what those look like … cci task run list_changes --org feature shows CustomField: Contact.thepackage__thefield__c and DuplicateRule: Contact.Standard_Contact_Duplicate_Rule as β€œchanged components in the scratch org.” Looks good, guess I’ll get them. (To be fair, this is the way I should’ve done this, not weird hacky stuff in VSCode.) cci task run retrieve_changes --org feature

Okay, that pulled down force-app\main\default\duplicateRules\Contact.Standard_Contact_Duplicate_Rule.duplicateRule-meta.xml with its <isActive>false</isActive>:

<?xml version="1.0" encoding="UTF-8"?>
<DuplicateRule xmlns="http://soap.sforce.com/2006/04/metadata" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
    <actionOnInsert>Allow</actionOnInsert>
    <actionOnUpdate>Allow</actionOnUpdate>
    <alertText>Duplicate Alert</alertText>
    <description>Duplicate rule for contact records.</description>
    <duplicateRuleFilter xsi:nil="true"/>
    <duplicateRuleMatchRules>
        <matchRuleSObjectType>Contact</matchRuleSObjectType>
        <matchingRule>Standard_Contact_Match_Rule_v1_1</matchingRule>
        <objectMapping xsi:nil="true"/>
    </duplicateRuleMatchRules>
    <isActive>false</isActive>
    <masterLabel>Standard Contact Duplicate Rule</masterLabel>
    <operationsOnInsert>Alert</operationsOnInsert>
    <operationsOnInsert>Report</operationsOnInsert>
    <operationsOnUpdate>Report</operationsOnUpdate>
    <securityOption>EnforceSharingRules</securityOption>
    <sortOrder>1</sortOrder>
</DuplicateRule>

And fully filled in force-app\main\default\objects\Contact\fields\thepackage__thefield__c.field-meta.xml:

<?xml version="1.0" encoding="UTF-8"?>
<CustomField xmlns="http://soap.sforce.com/2006/04/metadata">
    <fullName>thepackage__thefield__c</fullName>
    <externalId>false</externalId>
    ...
    <type>Picklist</type>
    <valueSet>
        <valueSetDefinition>
            <sorted>false</sorted>
            <value>
                <fullName>...</fullName>
                <default>false</default>
                <label>...</label>
            </value>
            ...
            <value>
                <fullName>...</fullName>
                <default>false</default>
                <label>...</label>
            </value>
        </valueSetDefinition>
    </valueSet>
</CustomField>

One last scratch org rebuild and test run

Now cci task run dx_push --org feature doesn’t error out, so I guess I’ll try cci org scratch_delete feature, cci flow run dev_org --org feature (20+ minutes), & cci task run run_tests --org feature.

Hooray – it passed!

Reorganize (one day)

Now all I have to figure out is exactly what GitHub repository these 3 files really belong in:

  1. objects\Contact\fields\Custom_Field__c.field-meta.xml (the one my codebase refers to)
  2. objects\Contact\fields\thepackage__thefield__c.field-meta.xml (to override the valid picklist values set up by its managed package, since my codebase refers to this field and sets a value in it)
  3. default\duplicateRules\Contact.Standard_Contact_Duplicate_Rule.duplicateRule-meta.xml (which trips up my codebase if I don’t deactivate it)

But that’s another project for another day.

--- ---