SFDCQuery — Building and publishing my first Elixir Library

Lincoln Rodrigues — Mon, 15 Apr 2024 23:59:09 +0000

Working with Salesforce, after each feature delivered, I usually need to investigate how are the Accounts and Leads, but, when having access to the account, you can see the data in the Developer console except for my use-case which remains to me fetching the data through Query API and Bulk Search API since in I only have the OAuth Credentials of the instances stored (encrypted of course). To improve productivity while developing or investigating any possible issue with the customer's instances, I built the library SFDCQuery.

Before sharing the steps for building it, here's where you can find it:
https://github.com/linqueta/sfdc-query
https://hex.pm/packages/sfdc_query
https://hexdocs.pm/sfdc_query/readme.html

All right, first of all, having mix installed, it's time to create the project:

▶ mix new sfdc_query           
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/sfdc_query.ex
* creating test
* creating test/test_helper.ex
* creating test/sfdc_query_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd sfdc_query
    mix test

Run "mix help" for more commands.

Having the project created, we need to figure out its shape allowing the clients to create their implementations, it means, Elixir behaviors!

Building the Client

For connecting to a Salesforce application, the first step is to execute the oAuth which can be done using the Salesforce CLI, and once it's done, you have the following information:

▶ sfdx force:auth:web:login -r https://test.salesforce.com

Successfully authorized me@linqueta.com with org ID ABC102030

▶ sfdx force:org:display --o me@linqueta.com

=== Org Description

 KEY              VALUE                                                                                                            
 ──────────────── ──────────────────────────────────────────────────────────────────────────────────────────────────────────────── 
 Access Token     YOUR_CURRENT_ACCESS_TOKEN
 Api Version      60.0                                                                                                             
 Client Id        CLIENT_ID                                                                                                      
 Connected Status Connected                                                                                                        
 Id               ABC102030                                                                                    
 Instance Url     https://linqueta.my.salesforce.com                                                                 
 Username         me@linqueta.com

The most important data for connecting to the Query API are the fields Access Token, Api Version, and Instance Url. To hold this info, the module SFDCQuery.Config was created as:

defmodule SFDCQuery.Config do
  @moduledoc """
  The configuration of the SFDCQuery library.
  """

  alias SFDCQuery.Config

  defstruct [:instance_url, :access_token, :version, :logs]

  @type t :: %__MODULE__{
          instance_url: String.t(),
          access_token: String.t(),
          version: String.t(),
          logs: boolean()
        }

  @spec new(%{
          required(:instance_url) => String.t(),
          required(:access_token) => String.t(),
          required(:version) => String.t() | integer(),
          optional(:logs) => boolean()
        }) :: t()
  def new(%{instance_url: nil}), do: raise(ArgumentError, "instance_url is required")
  def new(%{access_token: nil}), do: raise(ArgumentError, "access_token is required")
  def new(%{version: nil}), do: raise(ArgumentError, "version is required")

  def new(%{instance_url: instance_url, access_token: access_token, version: version} = args) do
    %Config{
      instance_url: instance_url,
      access_token: access_token,
      version: parse_version(version),
      logs: parse_boolean(args[:logs])
    }
  end

  defp parse_boolean(nil), do: false
  defp parse_boolean(val) when is_binary(val), do: String.downcase(val) == "true"
  defp parse_boolean(bool), do: bool

  defp parse_version(version) when is_integer(version), do: "#{version}.0"
  defp parse_version(version) when is_binary(version), do: version
end

But, each client probably needs to have their way of building their Config, so, the Client layer allows to have it easily:

defmodule SFDCQuery.Client.Behaviour do
  @callback create(any()) :: SFDCQuery.Config.t()
end

defmodule SFDCQuery.Client.Default do
  @behaviour SFDCQuery.Client.Behaviour

  alias SFDCQuery.Config

  @doc """
  Builds a new client configuration with the given arguments.
  It fetches the values from the environment variables if they are not provided.
    SFDC_QUERY_INSTANCE_URL
    SFDC_QUERY_ACCESS_TOKEN
    SFDC_QUERY_REFRESH_TOKEN
    SFDC_QUERY_VERSION
    SFDC_QUERY_LOGS_ENABLE
  """
  @impl true
  def create(args \\ %{}) do
    Config.new(%{
      instance_url: args[:instance_url] || fetch_env("SFDC_QUERY_INSTANCE_URL"),
      access_token: args[:access_token] || fetch_env("SFDC_QUERY_ACCESS_TOKEN"),
      version: args[:version] || fetch_env("SFDC_QUERY_VERSION"),
      logs: args[:logs] || fetch_env("SFDC_QUERY_LOGS_ENABLED")
    })
  end

  defp fetch_env(key) do
    case System.get_env(key) do
      "" -> nil
      value -> value
    end
  end
end

It means that the library client can create their own SFDC Client by implementing the Client's behavior. Here's an example:

defmodule MyApp.CustomerSFDCClient do
  @behaviour SFDCQuery.Client.Behaviour

  alias SFDCQuery.Config

  alias MyApp.Ecto

  import Ecto.Query

  @impl true
  def create(customer_id) do
    credentials = from(c in Credentials, where: c.customer_id == ^customer_id) |> Repo.one()

    Config.new(%{
      instance_url: credentials.salesforce_instance_url,
      access_token: credentials.salesforce_access_token,
      version: credentials.salesforce_version,
      logs: false
    })
  end
end

Querying Salesforce data

Once we have the instance's credentials in memory, we need to call SFDC Query API. To allow making HTTP requests, I choose Req since it's simple enough for just one API endpoint:

Req.request(
  method: :get,
  url: "#{config.instance_url}/services/data/v#{config.version}/query",
  headers: [{"Authorization", "Bearer #{config.access_token}"}],
  params: [q: "Select Id, Name from Account LIMIT 10"]
)

The module SFDCQuery.RestAPI makes the Req request exampled above and parses the response handling possible errors:

defp handle({:ok, %Req.Response{status: 200, body: %{"records" => records}}}), do: {:ok, records}
defp handle({:ok, %Req.Response{status: _, body: body}}), do: {:error, body}
defp handle({:error, _} = error), do: error

Here's the shape of a successful query response:

{:ok,
 %Req.Response{
   status: 200,
   body: %{
     "done" => true,
     "records" => [
        %{"Id" => "001U8000005CeutIAC"},
        %{"Id" => "001U8000005cJN0IAM"},
        %{"Id" => "001U8000005cRAnIAM"},
        %{"Id" => "001U8000005oz2rIAA"}
      ],
     "totalSize" => 4
   }
 }}

Parsing the data

SFDCQuery allows to query SFDC through the Query API but it's not the most important thing in this library. Parsing the response and printing it if makes sense improves a lot the productivity of the developer who needs to check or handle the records returned. To parse the response, SFDCQuery implements a parser for a Map, a JSON, and a CSV.

# Map
SFDCQuery.Client.Default.create(args)
|> SFDCQuery.query("SELECT Id, Name, Website From Account LIMIT 3")
|> SFDCQuery.Parser.Map.parse()

{:ok,
 [
    %{
      attributes: %{
        "type" => "Account",
        "url" => "/services/data/v60.0/sobjects/Account/001U8000005CeutIAC"
      },
      Name: "Page",
      Id: "001U8000005CeutIAC",
      Website: nil
    },
    %{
      attributes: %{
        "type" => "Account",
        "url" => "/services/data/v60.0/sobjects/Account/001U8000005cJN0IAM"
      },
      Name: "Nike",
      Id: "001U8000005cJN0IAM",
      Website: "https://www.nike.com/"
    },
    %{
      attributes: %{
        "type" => "Account",
        "url" => "/services/data/v60.0/sobjects/Account/001U8000005cRAnIAM"
      },
      Name: "Google",
      Id: "001U8000005cRAnIAM",
      Website: "google.com"
    }
  ]
}

# CSV
SFDCQuery.Client.Default.create(args)
|> SFDCQuery.query("SELECT Id, Name, Website From Account LIMIT 3")
|> SFDCQuery.Parser.CSV.parse()

{:ok, "Id,Name,Website\n001U8000005CeutIAC,Page,\n001U8000005cJN0IAM,Nike,https://www.nike.com/\n001U8000005cRAnIAM,Google,google.com"}

# JSON
SFDCQuery.Client.Default.create(args)
|> SFDCQuery.query("SELECT Id, Name, Website From Account LIMIT 3")
|> SFDCQuery.Parser.JSON.parse()

{:ok, "[{\"attributes\":{\"type\":\"Account\",\"url\":\"/services/data/v60.0/sobjects/Account/001U8000005CeutIAC\"},\"Name\":\"Page\",\"Id\":\"001U8000005CeutIAC\",\"Website\":null},{\"attributes\":{\"type\":\"Account\",\"url\":\"/services/data/v60.0/sobjects/Account/001U8000005cJN0IAM\"},\"Name\":\"Nike\",\"Id\":\"001U8000005cJN0IAM\",\"Website\":\"https://www.nike.com/\"},{\"attributes\":{\"type\":\"Account\",\"url\":\"/services/data/v60.0/sobjects/Account/001U8000005cRAnIAM\"},\"Name\":\"Google\",\"Id\":\"001U8000005cRAnIAM\",\"Website\":\"google.com\"}]"}

The library client can implement their parser using SFDCQuery.Parser.Behaviour :

defmodule MyApp.SFDCFileParser do
  @behaviour SFDCQuery.Parser.Behaviour

  alias SFDCQuery.Query

  @impl true
  def parse({:error, _} = error), do: error
  def parse({:ok, %Query{records: records}}), do: {:ok, save_file(records)}

  defp save_file(records) do
    ...
  end
end

Showing the data

Here the magic happens! After querying the API, you can check the data as a view, first using the table implementation:

SFDCQuery.Client.Default.create(args)
|> SFDCQuery.query("SELECT Id, Name, Website From Account LIMIT 3")
|> SFDCQuery.View.Table.show()

SELECT Id, Name, Website From Account LIMIT 3
-------------------------------------------------------
| Id                 | Name   | Website               |
-------------------------------------------------------
| 001U8000005CeutIAC | Page   |                       |
| 001U8000005cJN0IAM | Nike   | https://www.nike.com/ |
| 001U8000005cRAnIAM | Google | google.com            |

You can check JSON and CSV too:

SFDCQuery.Client.Default.create(args)
|> SFDCQuery.query("SELECT Id, Name, Website From Account LIMIT 3")
|> SFDCQuery.View.JSON.show()

SELECT Id, Name, Website From Account LIMIT 3
-------------------------------------------------------
[
  {
    "attributes": {
      "type": "Account",
      "url": "/services/data/v60.0/sobjects/Account/001U8000005CeutIAC"
    },
    "Name": "Page",
    "Id": "001U8000005CeutIAC",
    "Website": null
  },
  {
    "attributes": {
      "type": "Account",
      "url": "/services/data/v60.0/sobjects/Account/001U8000005cJN0IAM"
    },
    "Name": "Nike",
    "Id": "001U8000005cJN0IAM",
    "Website": "https://www.nike.com/"
  },
  {
    "attributes": {
      "type": "Account",
      "url": "/services/data/v60.0/sobjects/Account/001U8000005cRAnIAM"
    },
    "Name": "Google",
    "Id": "001U8000005cRAnIAM",
    "Website": "google.com"
  }
]

SFDCQuery.Client.Default.create(args)
|> SFDCQuery.query("SELECT Id, Name, Website From Account LIMIT 3")
|> SFDCQuery.View.CSV.show()

SELECT Id, Name, Website From Account LIMIT 3
-------------------------------------------------------
Id,Name,Website
001U8000005CeutIAC,Page,
001U8000005cJN0IAM,Nike,https://www.nike.com/
001U8000005cRAnIAM,Google,google.com

The behaviour SFDCQuery.View.Behaviour allows the client to define new views as they want.

Publishing the library

Once the project is tested, it's time to publish it to hex.pm. The steps to do it are simple:

Before publishing, you need to append some package details to the mix.env file:

def package do
  [
    name: "sfdc_query",
    files: ~w(lib .credo.exs .formatter.exs mix.exs README*),
    licenses: ["Apache-2.0"],
    links: %{"GitHub" => "https://github.com/linqueta/sfdc-query"}
  ]
end

Create an account at hex.pm
In your terminal, step a login into your account defining a local password: mix hex.user whoami
Publish the package: mix hex.publish
And it's ready for being used in other projects being delivered by hex.pm: https://hex.pm/packages/sfdc_query

Documentation

The library ExDoc allows to your project have the docs at hex.pm based on the documentation inside the modules and function-related. To install it, in you mix.env you need to add a docs section and later execute it with mix docs . Once call mix hex.publish , ExDoc automatically triggers to build new docs for the version selected and publishes it to hex.pm docs

defp docs do
  [
    source_ref: "v#{@version}",
    main: "readme",
    formatters: ["html"],
    extras: ["README.md"]
  ]
end

And here it is: https://hexdocs.pm/sfdc_query/readme.html

How to test your gem in a Rails application

Lincoln Rodrigues — Sun, 02 Feb 2020 22:33:46 +0000

Many gems in RubyGems require Rails to work as well and many of them are developed to use together with a Rails application like a plugin. How is known, the best way to ensure that some code is working as well is writing unit tests for it. But, for gems that depend or work together Rails, there is a nice way to test it into a Rails application, the name of it is Dummy tests.

The meaning of the word dummy is a kind of mannequin or replica of humans, but in ruby context, it's a kind of scope for you test your gem, in this case, a Rails application. I've added recently dummy tests in my gem rails-healthcheck in this pull request to check if this gem is rightly imported and have responded in success and failure cases as is waited.

These are the steps to add dummy tests into your gem using RSpec:

Create a Rails application into folder /spec:

rails new spec/dummy --skip-gemfile --skip-git --api

Require your gem in the dummy application:

A Rails application will require all gems into Gemfile's environments blocks using bundler, but, here we wanna test your local gem:

If your gem requires to run some setup, it's the moment to do it.

Add the gem rspec-rails as a development dependency:

Require dummy files and rspec-rails in tests:

Code your test:

For this example, my gem provides the method rgb_to_hex and it is used in the dummy's model Color. This model has the attributes red, green and blue.

Run rspec to test them:

rspec spec/color_spec.rb # or just rspec for all tests

# Color
#   #configure
#     with Dim Gray color
#       is expected to eq "3B3638"
#     with White Smoke color
#       is expected to eq "F9F7F7"
# Finished in 0.05234 seconds (files took 3.83 seconds to load)
# 2 examples, 0 failures

This example is so basic, but with dummy test you can test routes, controllers, helpers any what you want in a Rails application, for example, in the gem rails-healthcheck was tested all responses of Healthcheck::HealthcheckController#check mounted in the route /healthcheck or any route set in the initializer.

Watch my blog to see more posts.

DEV Community: Lincoln Rodrigues