Salesforce, Python, SQL, & other ways to put your data where you need it

Need event music? 🎸

Live and recorded jazz, pop, and meditative music for your virtual conference / Zoom wedding / yoga class / private party with quality sound and a smooth technical experience

CCI on a new computer

10 Aug 2022 🔖 salesforce
💬 EN

Table of Contents

I got a new computer. I moved some allegedly-CCI-ready projects stored in folders tracked as “repositories” with Git onto it. However, the SFDX & CCI command-line interfaces weren’t yet installed onto the computer.

I had already installed VSCode (code-editing & command-running software) and Python (which CCI uses) onto the computer from the Microsoft Store, which I verified by opening a Terminal tab in VSCode and executing the command python3 --version. (As expected, the response displayed the word “Python” followed by a version number.)

But sfdx --version and cci version commands returned errors like “The term ‘sfdx’ is not recognized as the name of a cmdlet, function, script file, or operable program. Check the spelling of the name, or if a path was included, verify that the path is
correct and try again.”

Pipx

Per https://cumulusci.readthedocs.io/en/stable/get-started.html#id3, I tried pipx --version and realized that although I had PIP installed (pip --version), I didn’t have PIPX installed, so I guess I have to do that too.

python -m pip install --user pipx

Plus manually adding some folder-paths to my local user’s “PATH” Windows environment variable:

  1. %USERPROFILE%\.local\bin
  2. %USERPROFILE%\AppData\Local\Packages\PythonSoftwareFoundation.Python.RIDICULOUSLY_VERSION_SENSITIVE_FOLDER_NAME_HERE\LocalCache\local-packages\PythonRIDICULOUSLY_VERSION_SENSITIVE_FOLDER_NAME_HERE\Scripts (this is a little different than the docs at CCI because I use Windows-store Python on this computer. I figured out what to type by looking at the warning from installing PIPX saying that it wasn’t yet in the PATH. Note that “RIDICULOUSLY_VERSION_SENSITIVE_FOLDER_NAME_HERE” isn’t really in the foldernames – I just wanted this article to not date itself too badly.)

Anyway, after saving that, I exited & re-opened VSCode and then … okay, weird, pipx --version still claims there is no such command as pipx. Weird.

CCI

Anyway, it works from a standalone PowerShell window, so I’m going to install CCI with that and then hope that CCI commands work from VSCode terminals later on, because it’ll be a huge nuisance if I can’t use CCI commands from an area right below where I’m editing Salesforce configuration files.

pipx install cumulusci

It installed, cci version worked but said I needed to edit the Windows Registry to set LongPathsEnabled under HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\FileSystem to 1 instead of 0, so I escalated to Windows admin rights and did so. When I ran cci version again, the message about long paths disappeared.

That said, I still don’t have cci working in PowerShell-inside-VSCode. Annoying!

SFDX

Installing the SFDX CLI tool was downloading & double-clicking a .exe file from https://developer.salesforce.com/tools/sfdxcli. It wanted me to have admin rights to my computer, so it took care of setting PATH variables and such.

OK, actually, it didn’t – it put C:\Program Files\sfdx\bin into the PATH variable of my Windows admin user’s account, not into the system-wide PATH variable. I escalated to admin privileges and fixed that.

Now I can see sfdx --version output in a fresh PowerShell (though frustratingly, still not within VSCode – maybe I’ll just uninstall Microsoft Store VSCode and reinstall it by downloading the .exe from the software’s website; I haven’t added a lot of extensions to it yet; I’m sick of not having “open in VSCode” in the context menu anyway, and maybe reinstalling will help with that, too).

Next

From a PowerShell session that’s in any old unrelated-to-Salesforce folder on my computer, I did sfdx auth:list and saw that I haven’t logged into anything with the SFDX CLI yet. Of course not, I just installed it! So I ran this (substituting for my-clever-alias and the instanceurl appropriately):

sfdx force:auth:web:login --setalias my-clever-alias --instanceurl https://customdomain.my.salesforce.com/

And got this:

Successfully authorized [email protected] with org ID 00D123456789876543

Plus, it shows up with sfdx auth:list.

Sidetrack downloading production metadata

Then I got sidetracked on a totally different endeavor … I put the my-clever-alias for the prod org I’d just logged into in the defaultdevhubusername of a \.sfdx\sfdx-config.json file in a different project.

In a PowerShell prompt of this other C:\sidetrack project, I did git init and then cci flow run dev_org --org beta but, oops, that’s not what I meant to do. Shoot. I meant to log into the production org. cci org list only shows scratch orgs, so I need to, from within C:\sidetrack, do (substituting for another-clever-alias and the login-url appropriately) (note the lack of a --sandbox option because I was connecting to a production org):

cci org connect another-clever-alias --login-url https://customdomain.my.salesforce.com

Yay, now another-clever-alias also shows up with cci org list.

Let’s try this from the C:\sidetrack PowerShell prompt:

cci task run generate_dataset_mapping --org another-clever-alias

https://cumulusci.readthedocs.io/en/stable/data.html#generate-dataset-mapping

[08/10/22 18:21:43] Org info updated, writing to keychain
                    Beginning task: GenerateMapping
                    As user: [email protected]
                    In org: 00D123456789876543

                    Collecting sObject information
[08/10/22 18:22:03] Creating mapping schema datasets/mapping.yml

Error: [Errno 2] No such file or directory: 'datasets/mapping.yml'

Oh, right, I did forget to make a \datasets\mapping.yml file in my project.

I guess I’ll start by filling it in like this:

Contacts:
    sf_object: Contact
    table: Contact

Okay, wow, cci task run generate_dataset_mapping --org another-clever-alias just replaced the contents of \datasets\mapping.yml with a toooooon of detail. (Sadly, not exactly the detail I wanted.)

Here’s another starter:

Application_Controls:
  sf_object: Application_Control__c
  table: Application_Control__c
Programs:
  sf_object: Program__c
  table: Program__c
Requirements:
  sf_object: Requirement__c
  table: Requirement__c
Requirement_Items:
  sf_object: Requirement_Item__c
  table: Requirement_Item__c
Questions:
  sf_object: Question__c
  table: Question__c
Question_Dependency:
  sf_object: Question_Dependency__c
  table: Question_Dependency__c

Once I transformed it into a bigger file with generate_dataset_mapping, I tried:

cci task run extract_dataset -o mapping datasets/mapping.yml -o sql_path datasets/data.sql --org another-clever-alias

OK, that was going to take forever because there was too much data in it.

I need to parse the datasets/mapping.yml generated and cull out the ones besides the 6 I explicitly specified.

Also, do I need Plan__c or Term?

OK so Ctrl + k + 0 in VSCode folded things nicely, and then I was able to delete all but 6, and then I just expanded lookups until I found references not in my 6 and deleted those too.

Much better!

[08/17/22 18:27:02] Extracting data for sObject Application_Control__c
                    Downloading and importing records
                    Done! (Total: 17)
                    Extracting data for sObject Program__c
[08/17/22 18:27:03] Downloading and importing records
                    Done! (Total: 688)
                    Extracting Record Types for Program__c
                    Extracting data for sObject Requirement__c
                    Downloading and importing records
                    Done! (Total: 169)
                    Extracting data for sObject Requirement_Item__c
[08/17/22 18:27:04] Downloading and importing records
                    Done! (Total: 341)
                    Extracting Record Types for Requirement_Item__c
                    Extracting data for sObject Question__c
[08/17/22 18:27:05] Downloading and importing records
                    Done! (Total: 1076)
                    Extracting Record Types for Question__c
                    Extracting data for sObject Question_Dependency__c
[08/17/22 18:27:06] Downloading and importing records
                    Done! (Total: 372)
                    Extracting Record Types for Question_Dependency__c

Oh – OK I also want to change the word that comes after the generated Insert to match my pattern.

Application_Controls:
  sf_object: Application_Control__c
  table: 'UST_SALESFORCE.UST_SF_KT_APPLICATION_CONTROL_EDA'
Programs:
  sf_object: Program__c
  table: 'UST_SALESFORCE.UST_SF_KT_PROGRAM_EDA'
Requirements:
  sf_object: Requirement__c
  table: 'UST_SALESFORCE.UST_SF_KT_REQUIREMENT_EDA'
Requirement_Items:
  sf_object: Requirement_Item__c
  table: 'UST_SALESFORCE.UST_SF_KT_REQUIREMENT_ITEM_EDA'
Questions:
  sf_object: Question__c
  table: 'UST_SALESFORCE.UST_SF_KT_QUESTION_EDA'
Question_Dependency:
  sf_object: Question_Dependency__c
  table: 'UST_SALESFORCE.UST_SF_KT_QUESTION_DEPENDENCY_EDA'

Forget that.

c:\example>sfdx plugins:install sfdmu
sfdx force:org:list

Right, it’s my-clever-alias which is [email protected].

{
    "objects": [
        {
            "query": "SELECT all FROM Application_Control__c",
            "operation": "Readonly"
        }
    ]
}
cd sfdmu
sfdx sfdmu:run --sourceusername [email protected] --targetusername csvfile
--- ---