SFDX Upgrade Error

Recently, I upgraded the Salesforce CLI, sf, and ran into the following error:

Error (10): Metadata API request failed: Cannot read properties of null (reading ‘split’)

If you look closely at your Salesforce projects in VS Code or via your favorite OS Explorer, you will see that there are two hidden folders.

  • .sf
  • .sfdx

These folders contains metadata, tools, apex classes, and other tools. Regardless of the OS or VS Code that you are running – the error or errors that result from an cli upgrade manifes themselfes when trying to work with existing Salesforce projects.

I removed the .sf and .sfdx hidden folders from my project. Then, ran 

sf config:set target-org=myorg

I was able to work with my project without encountering any additionals errors.

Salesforce Development with CI/CD Pipelines Part 1

Overview

Salesforce supports the deployment of source code and metadata via CI/CD pipelines.  In this document, we will refer to the term “metadata deployment”, which will implicitly include code.  DevSecOps teams rely on CI/CD pipelines for integration and deployment of metadata during the development and operational lifecycle.  The key concept of “continuous integration/delivery” is to deliver value to both DevSecOps and the business continuously.  You might be wondering why I am bringing the business into this discussion?  Everything that we do in DevSecOps is based on delivering the most value upfront (Agile/Scrum principle) to the business.  If our DevSecOps teams are constantly dealing with integration and software delivery issues, then the value delivery is not streamlined nor realized in a timely manner.   This is not a good predicament to be in.   CTO’s and other leaders will have lengthy meetings and a long laundry list of actions items that will need the precious time of our DevSecOps team; thus, shifting away the focus from DevSecOps to mundane task completion and status reporting.  We don’t want to be the program that triggers such laundry list nor the oversight that comes with such predicament.

There are many benefits of having a CI/CD pipelines.  A couple that come to mind are the speed in which teams can integrate their work with other teams, identify collisions internally and externally, speed to fix defects within the development life cycle (the earlier the better) and the speed to deliver the features to business stake holders.  Yes, I am simplifying this discussion since we can write an entire article series on benefits of a CI/CD pipeline.

In this blog series, we are going to build a Salesforce CI/CD pipeline on GitLab.   Our approach will be different from the Salesforce trailheads in that we will focus on using sfdx to push our project to the Salesforce Org of choice upon a commit event in the repository.  The preferred approach is to use unlocked packages first.  This document assumes that you are not able to create unlocked packages due to the complexity of your organization (i.e. – Multiple vendors working on different applications that share objects).   

The Getting Started section will get us going with some housekeeping items that we have to get done before jumping into GitLab. NOTE: This article series requires proficient knowledge of Salesforce DX. If you are not there yet, go thru all the trailheads 🙂

What are we going to do?

This is what we are going to do in Part 1 of the series.

  • Create a Connected Application
    • Create a self-signed certificate for the connected application.
  • Create a Permission Set

In Part 2 of this series, will have the steps to configure GitLab CI/CD pipelines.

Getting Started with OpenSSL

We start by creating a self signed certificate.  This certificate is for the CI/CD Pipeline to present and retrieve an access token from Salesforce. You don’t have to get a Cert signing authority to sign your certificate request.  If you chose to go that route, the steps will be the same; however, you will have to follow your process to get the certificate signed.  Before we get started with the creation of a self-signed certificate, make sure that you have openssl installed on your favorite OS.  Openssl is available for all operating systems.  If you don’t have OpenSSL installed, take a look in the reference section on how to install Openssl on your operating system and the command to create a self-signed certificate.

Once you have the self signed certificate, you need to encrypt it because we don’t want anyone who has access to the GitLab project to use it.  In the sample project, the directory assets stores the encrypted certificate.   We will add the password to decrypt the file as a GitLab CI/CD Variable. 

Pre-Work Checklist:

  • Create a self signed certificate. Or get a signed certificate from your certificate authority.
  • Encrypt the self signed certificate and place it in your project folder (i.e. – assets/)

Encrypt the certificate

Navigate to the directory where the certificate is stored and run the following command:

:>openssl enc -aes-256-cbc -salt -pbkdf2 -pass pass:<password> -d -in assets/server.key.enc -out assets/server.key

Openssl Command Reference:

enc

we are telling openssl to encrypt

aes-256-cbc

the encryption cypher to use.

salt

is used to prevent dictionary attacks.

pbkdf2

cryptography key derivation function used to reduce password brute force attacks.

pass

password used to encrypt the file

-P

print out the salt, key and IV used

-in

input file (assets is the directory that I used in my project)

-out

output file

Create a Salesforce Connected Application

We need to create a connected application that we will use to get an access token from Salesforce.   The following 10 steps:

  1. Create your connected app, and complete its basic information.
  2. In the API (Enable OAuth Settings) area of the page, select Enable OAuth Settings.  (Refer to the image below)
  3. If you’re setting up a connected app for an external application on a device with limited input or display capabilities, such as TVs, appliances, or command-line applications, select Enable for Device Flow.
    A callback URL isn’t used in the device flow. However, when this flow is enabled, the value for the callback URL defaults to a placeholder. You can specify a callback URL if needed, such as when this same client is being used for a different flow.
  4. Enter the callback URL (endpoint) that Salesforce calls back to your application during OAuth. It’s the same as the OAuth redirect URI.  Depending on which OAuth flow you use, the URL is typically the one that a user’s browser is redirected to after successful authorization. Because this URL is used for some OAuth flows to pass an access token, the URL must use secure HTTPS or a custom URI scheme.
    If you enter multiple callback URLs, at run time Salesforce matches the callback URL value specified by the app with one of the values in Callback URL. It must match one of the values to pass validation. Separate multiple callback URLs with line breaks. The callback URL field has a limit of 2000 characters, cumulatively. If you enter several URLs and they exceed this limit, create another connected app to manage more callback URLs.
  5. Check the Use digital Signatures box.
  6. Click on the Browse button and upload the self-signed certificate. (i.e. – server.key)
  7. Select the OAuth scopes to apply to the connected app. OAuth scopes define permissions for the connected app, which are granted as tokens after the app is authorized. The OAuth token name is in parentheses.
    1. Access and manage your data (api)
    2. Perform requests on your behalf at any time (refresh_token, offline_access)
    3. Provide access to your data via the Web (web)
  8. Check the Require Secret  for Web Server Flow Checkbox.
  9. Check the Require Secret for Refresh Token Flow Checkbox.

The image below depicts the setting that were outlined in the previous 9 steps.

SFDC Connected App Settings

Next, retrieve the Consumer Key for your Connected App. From Setup -> Apps -> App Manager: Select the connected application that you created and click on the drop-down menu on the left.  Select View and Copy the key and save it in a secure place.  

You will need this key for the GitLab CI/CD Pipeline Variables (CONSUMER_KEY  & PROD_CONSUMER_KEY).  Note:  If you are deploying to multiple Salesforce orgs, then you will need to create a connected app for each org.

Connected App Consumer Key

Update the Oauth Policy for the connected app.  Failure to do so will result in a Grant JWT error.

Permission Set Creation

Next, we need to create a permission set and assigned it to the Salesforce administrators/DevSecOps team that will operate the CI/CD pipeline.   Basically, the permission set does not need administrative privileges, but you must be able to do all things that relate to deploying metadata.  The key to the permission set, is to assigned the Connected App under the “Assigned Connected Apps” section of the profile.  We will refer to the permission set as CICD Permission Set.

Then, you must assign the permission set to the DevSecOps team.  You will need an administrator or delegated administrator privileges that can assign permission sets to users.

Permission Set Menu

Testing the Permission Set & Connected App

Congratulations on reaching this section.  Its time to test the permission set and connected app configuration with the sfdx auth:jwt:grant command.  You must use the connected apps consumer key and pass it as a parameter to clientid. This command will grant the client an access token.

:>sfdx auth:jwt:grant --clientid  3MVG9GYWKbMgBvbBlahBlahBlahBlahBlahBlah --jwtkeyfile <path to>/server.key --username devops.user1@salesforce.com --instanceurl https://test.salesforce.com --json

clientid

This is the Consumer Key of the connected application.

jwtkeyfile

This is the self-signed certificate that we created a long time ago.  Make sure that you are using the decrypted version.

username

The username that was granted the CI/CD permission set.

instanceurl

The login URL of your Salesforce Org

Summary

We created a Salesforce Connected Application that we are going to use from our CI/CD pipeline to push metadata to Salesforce Sandboxes. Part 1 is foundational and you need to have this done before moving on to Part 2. There are several advance security concepts when creating a connected application. Do review the documentation on connected applications. We are creating a JWT Bearer token flow and granting the connected application access to the Salesforce API, Data and permission to perform requests on your behalf. DO NOT share your consumer key and do keep it save. Review the JWT bearer token flow, it’s important to understand how OAUTH 2.0 works and it will be on your CTA exam.

JWT Error Message

If you see the following error message, make sure that you revisit the Oauth Policies for the connect application and make  sure that it’s set to:  “Admin Approved users are pre-authorized”

Error Message:

We encountered a JSON web token error, which is likely not an issue with Salesforce CLI. Error authenticating with JWT config due to: user hasn't approved this consumer.

Reference

How to create a self signed certificate:

https://devcenter.heroku.com/articles/ssl-certificate-self

SFDX Usernames & Orgs

Salesforce DX Sandbox Config

Before you start developing with Salesforce DX, I recommend that you set your org and default username. The setdefaultusername parameter sets all Salesforce projects to use this org/scratch-org when using sfdx commands. This command is very helpful if you support multiple Salesforce projects.

sfdx cli command
sfdx force:auth:web:login --instance-url https://test.salesforce.com --setdefaultusername --set-alias my-alias
  • instance-url: provide the url for your Salesforce domain that you want to authenticate with.
  • setdefaultusername: tell sfdx to use this username/org for all projects.

How to check which username is set?

sfdx force:org:list

The output will list all Orgs that your sfdx client is connected too. The org with the (U) next to the Alias is your default username.

SFDX Unable to Authorize Org

Featured

Overview

Recently, I ran into this error while refreshing my Oauth Token on VS Code. I tried to re-authorize the org and it failed. I had never encounter this type of failure. Usually, clearing out the .sfdx/<username>.json file and reauthorizing the org worked. This time it was different.. The error lingered and I had no additional information to debug. My next step was to research and troubleshoot.

Did I mention that I run Linux?

I recently had applied a bunch of package updates to my linux desktop and thought that my firewall was blocking tcp traffic into port 1717. I did a quick check and that was not the case. BTW – the local nodejs server runs only while the OAUTH workflow is in progress. Once the flow completes, the local server is shutdown.

http://localhost:1717/OauthRedirect?code=jedeiojropu2iou23u2io3uj

If you want to modify the port that the request should redirect, then on your sfdx-project.json add the oauthLocalPort key.

"namespace": "",
    "sfdcLoginUrl": "https://login.salesforce.com",
    "sourceApiVersion": "50.0",
    "oauthLocalPort": 7171

How to Fix this error

I logged out of all my Orgs on VS Code. Go to the command palette and look for:

SFDX: Log out of all Authorized Orgs

A quick search on the web and I stumbled on this stack exchange post, which was extremely helpful.

Stake Exchange Article

Authorize a Device

Salesforce provides many secure flows for authorizing applications and devices. In this case, authorizing my desktop (device) was the best option. First, run the following command on the terminal in VS Code. Or open a terminal and go to sfdx project directory.

sfdx force:auth:device:login -r https://test.salesforce.com

Make sure that you specify -r or –instanceurl flag. Sandboxes use the test.salesforce.com.

The outcome of a successful authenticated device follows.

â–¶ sfdx force:auth:device:login -r https://test.salesforce.com
=== Action Required!
Enter <CODE> user code in the verification URL https://test.salesforce.com/setup/connect

Login successful for testuser@test.com. You can now close the browser.

Final Step

In VS Code, command palette, select SFDX:Authorize an Org. You should see your org listed in the command box. Select the org and you are all set.

Just in case, here’s the command that VS Code runs behind the scenes. You can run the command in liue of the VS Code approach.

sfdx force:config:set defaultusername=testuser@test.com

Note: the username must match the username that you use on the step Authorize a Device.