#81: Building Structs from External Data

Published February 11, 2019

Elixir 1.7

Phoenix 1.4

In this episode let’s look at how we can leverage structs to format data from external sources. To start off let’s open an iex session and let’s define a simple struct - we’ll call Person - giving it two fields: age and name.

Now we can create a person like this with the %Person{} struct or if we have a map of attributes we want to use we’ll want to create our struct using the struct function. This also works with a keyword list of data.

Now here we’ve been creating structs where the keys are atoms, but instead let’s try to create a struct where the keys are strings. And a person struct is returned, but because we used strings none of the data was included.

> defmodule Person do
..>   defstruct [:name, :age]
..> end
{:module, Person,
 <<70, 79, 82, 49, 0, 0, 5, 192, 66, 69, 65, 77, 65, 116, 85, 56, 0, 0, 0, 183,
   0, 0, 0, 18, 13, 69, 108, 105, 120, 105, 114, 46, 80, 101, 114, 115, 111,
   110, 8, 95, 95, 105, 110, 102, 111, 95, 95, ...>>,
 %Person{age: nil, name: nil}}
> %Person{age: 75, name: "Mick"}
%Person{age: 75, name: "Mick"}
> attrs = %{age: 75, name: "Mick"}
%{age: 75, name: "Mick"}
> struct(Person, attrs)
%Person{age: 75, name: "Mick"}
> struct(Person, [age: 75, name: "Mick"])
%Person{age: 75, name: "Mick"}
> struct(Person,  %{"age" => 75, "name" => "Mick"})
%Person{age: nil, name: nil}

Alright now that we’ve seen the basics of creating a struct, let’s take a quick look at the application we’re working with. Here we have a simple Elixir Phoenix application that lists some albums and instead of using a database to load the albums from, we’re actually fetching the album data from an API.

Ok, now that we’ve seen our application in action let’s look at the code that’s doing the work. We’ll open our project in our editor and go to the album_controller.ex and there are two two actions defined, “index” and “show”. Both are fetching albums using the recording.ex module, so let’s open that.

Now both of these functions are doing essentially the same thing. They’re fetching the album data from a JSON API, which happens to be another app I have running locally on port 4001.

Then it’s taking the body of the response, decoding it with Jason.decode, then taking the data returned and building an album struct with Album.build.

Let’s open the album.ex module to see how it’s building the struct. And first we can see that there’s a struct defined with some fields about our albums. Then in the build function, it’s taking the attributes passed in, and, since they’re strings, mapping them to atoms, before finally creating an album struct.

While this works, mapping data like this can be tedious and takes time away from building the more interesting parts of an application. It would be great to not have to define these types of functions and luckily for us, we can use the ExConstructor package to do just that.

ExConstructor makes it easy to create structs from external data. Specifically it allows us to create structs where it doesn’t matter if the keys are strings, atoms, camel case, or snake case. Let’s update our application to use exconstructor.

The first thing we’ll want to do is grab it from Hex. Then we’ll open our Mixfile and add exconstructor to our list of dependencies.

mix.exs

...
defp deps do
...
{:exconstructor, "~> 1.1"},
...
end
...

And now we’ll go to the command line and install exconstructor:

$ mix deps.get
...
New:
  exconstructor 1.1.0

Alright let’s open the module we want to use exconstructor with, in this case the album.ex module and to use exconstructor we’ll simply include use ExConstructor after defining our struct with defstruct.

ExConstructor will give us a constructor function we can use called new so we can go ahead and remove the existing build function.

lib/teacher/recordings/album.ex

defmodule Teacher.Recordings.Album do
  defstruct [:id, :artist_name, :album_category, :album_title, :album_year, :album_summary]
  use ExConstructor

end

Then we can open our recordings.ex module and we’ll replace the calls to Album.build with Album.new. With that we should be good to go.

lib/teacher/recordings/recordings.ex

defmodule Teacher.Recordings do
  alias Teacher.Recordings.Album

  def fetch_albums do
    response = HTTPoison.get!("http://localhost:4001/api/albums")

    response.body
    |> Jason.decode!()
    |> Enum.map(&Album.new/1)
  end

  def fetch_album(id) do
    response = HTTPoison.get!("http://localhost:4001/api/albums/#{id}")

    response.body
    |> Jason.decode!()
    |> Album.new()
  end

end

Let’s go to the command line and start our server:

$ mix phx.server
...

And if we go back to our browser - and reload the page - perfect our page is still working telling us that ExConstructor is working and building our structs for us.

Now let’s say that the API we are getting our album data from is now returning the data we need in a different format. Instead of the keys being in snake case, which is how we’ve defined them in our struct, let’s say they are now returned in camel case. If we weren’t using ExConstructor we’d have to map our data like we were doing before, but ExConstructor handles the conversion for us.

Let’s try loading the page again, but with the new format from the API. And perfect - our app still works because exconstructor is converting the data to snake case for us. In this episode we saw how ExConstructor handles the tedious conversions when building a struct from external data, allowing us focus on building our application.