Do you have a Rails background?

Are you tired of looking at outdated or incomplete tutorials on how to build a JSON API using Elixir and Phoenix?

Then, read on my friend!

Introduction

I have found there are mainly two types of tutorials one should write:

  • Scoped, focused tutorials.
  • Full step-by-step tutorials.

Scoped, focused tutorials, should be used to explain techniques, like this one: Fluid SVGs with Vue.js.

But, full step-by-step tutorials should be used to teach about new tech stacks.

Going from zero to fully working prototype without skipping steps.
With best practices baked-in, presenting the best libreries available for a given task.
I really like tutorials that take this holistic approach.

So, this won’t just be about how to generate a new Phoenix API only app. That’s easy enough, you just need to pass the --no-brunch --no-html to mix phx.new.

This tutorial is about creating a small, but fully operational JSON API for web applications.


What we’ll do:

  • Create a new API-only Phoenix application —skip HTML and JS stuff.
  • Create a User schema module (model) and hash its password —because storing plain text passwords in the database is just wrong.
  • Create a Users endpoint —so you can get a list of, create or delete users!
  • CORS configuration —so you can use that frontend of yours that runs on another port / domain.
  • Create a Sign in endpoint —using session based authentication through cookies.

If you are interested on doing auth with JWTs, check this other tutorial out.


Let me be clear about something…

I’m just starting with Elixir / Phoenix, if there are any omissions or bad practices, bear with me, notify me and I’ll fix them ASAP.

This is the tutorial I wish I had available when I was trying to learn about how to implement a JSON API with Elixir / Phoenix… But I digress.


Prerequisites

Install Elixir

We will start by installing Erlang and Elixir using the asdf version manager —using version managers is a best practice in development environments.

Install Hex

Let’s take on Hex the package manager now:

mix local.hex

You can now print some info about your Elixir stack with:

mix hex.info

You’ll see something along the lines of:

Hex:    0.17.3
Elixir: 1.6.4
OTP:    20.3.2

Built with: Elixir 1.5.3 and OTP 18.3

Install Phoenix

Now let’s install the framework:

mix archive.install https://github.com/phoenixframework/archives/raw/master/phx_new.ez

Install PostgreSQL

PostgreSQL is the default database for new Phoenix apps, and with good reason: It’s a solid, realiable, and well engineered relational DB.

About REST clients

You might need to get a REST client so you can try out your API endpoints.

The two most popular ones seem to be Postman and Advanced Rest Client I tested both of them and I can say liked neither —at least on their Chrome app incarnations— as one didn’t display cookie info, and the other didn’t send declared variables on POST requests. ¬¬

In any case if you want to try them out:

  • You can get Postman here.
  • You can get ARC here.

If you are using a web frontend library like Axios, then your browser’s developer tools should be enough:

If you go with Axios dont’ forget to pass the configuration option withCredentials: true, this will allow the client to send cookies along when doing CORS requests.

Or you can just use good ol’ curl it works really well!
I’ll show you some examples on how to test out your endpoints from the CLI.


Create a new API-only Phoenix application

Generate the app files

In your terminal:

mix phx.new my-app --app my_app --module MyApp --no-brunch --no-html

From the command above:

  • You’ll see my-app as the name for the directory created for this application.
  • You’ll see my_app used in files and directories inside my-app/lib e.g. my_app.ex.
  • You’ll find MyApp used everywhere since it’s the main module for your app.

    For example in my-app/lib/my_app.ex:

    defmodule MyApp do
      @moduledoc """
      MyApp keeps the contexts that define your domain
      and business logic.
    
      Contexts are also responsible for managing your data, regardless
      if it comes from the database, an external API or others.
      """
    end

Create the development database

If you created a new DB user when installing PostgreSQL, add its credentials to config/dev.exs and config/test.exs. Then execute:

cd my-app
mix ecto.create

NOTE:

You can drop the database for the dev environment with:

mix ecto.drop

If you’d like to drop the database for the test environment, you’d need to:

MIX_ENV=test mix ecto.drop

Start the development server

From your terminal:

mix phx.server

Visit http://localhost:4000 and bask in the glory of a beautifully formatted error page. :)
Don’t worry though, we’ll be adding a JSON endpoint soon enough.

Router Error

Errors in JSON for 404s and 500s

If you don’t like to see HTML pages when there is an error and instead want to receive JSONs, set debug_errors to false in your config/dev.ex, and restart your server:

config :my_app, MyAppWeb.Endpoint,
  # ...
  debug_errors: false,
  # ...

Now, visiting http://localhost:4000 yields:

{ "errors": { "detail": "Not Found" } }

To stop the development server hit CTRL+C twice.


User schema (model)

We’ll be generating a new User schema (model) inside an Auth context.

Contexts in Phoenix are cool, they serve as API boundaries that let you organize your application code in a better way.

Generate the User schema and Auth context modules

mix phx.gen.context Auth User users email:string:unique password:string is_active:boolean

From above:

  • Auth is the context’s module name.
  • User is the schema’s module name.
  • users is the DB table’s name.
  • After that comes some field definitions.

The migration generated from the command above looks like this:

defmodule MyApp.Repo.Migrations.CreateUsers do
  use Ecto.Migration

  def change do
    create table(:users) do
      add :email, :string#, null: false
      add :password, :string
      add :is_active, :boolean, default: false, null: false

      timestamps()
    end

    create unique_index(:users, [:email])
  end
end

Adjust it to your liking —i.e. not allowing null for emails— then run the new migration:

mix ecto.migrate

If you want to read some info about this generator, execute:

mix help phx.gen.context

Hash a user’s password on saving

Add a new dependency to mix.exs:

  defp deps do
    [
      # ...
      {:bcrypt_elixir, "~> 1.0"}
    ]
  end

This is Bcrypt, we will use it to hash the user’s password before saving it; so we don’t store it as plain text inside the database.

Fetch the new app dependencies with:

mix deps.get

Change lib/my_app/auth/user.ex to look like this:

defmodule MyApp.Auth.User do
  # ...
  def changeset(user, attrs) do
    user
    |> cast(attrs, [:email, :password, :is_active])
    |> validate_required([:email, :password, :is_active])
    |> unique_constraint(:email)
    |> hash_user_password()
  end

  defp hash_user_password(%Ecto.Changeset{valid?: true, changes: %{password: password}} = changeset) do
    change(changeset, password: Bcrypt.hash_pwd_salt(password))
  end

  defp hash_user_password(changeset) do
    changeset
  end
end

Notice the call and definitions of hash_user_password/1.

What this does is run the changeset through that function, and if the changeset happens to have a password key, it’ll use Bcrypt to hash it.


Running Bcrypt.hash_pwd_salt("hola") would result in something like "$2b$12$sI3PE3UsOE0BPrUv7TwUt.i4BQ32kxgK.REDv.IHC8HlEVAkqmHky".

That strange looking string is what ends up being saved in the database instead of the plain text version.


Fix the tests

Run the tests for your project with:

mix test

Right now they will fail with:

  1) test users create_user/1 with valid data creates a user (MyApp.AuthTest)
     test/my_app/auth/auth_test.exs:32
     Assertion with == failed
     code:  assert user.password() == "some password"
     left:  "$2b$12$PUK73EqrvBTuOi2RiVrkOexqiVS.wIwbOtyR0EtzQLpbX6gaka8T2"
     right: "some password"
     stacktrace:
       test/my_app/auth/auth_test.exs:36: (test)

  2) test users update_user/2 with valid data updates the user (MyApp.AuthTest)
     test/my_app/auth/auth_test.exs:43
     Assertion with == failed
     code:  assert user.password() == "some updated password"
     left:  "$2b$12$cccPJfQD3seaBc8pHX8cJO/549lojlAjNNi/qo9QY0K7a7Zm5CaNG"
     right: "some updated password"
     stacktrace:
       test/my_app/auth/auth_test.exs:49: (test)

That’s because of the change we just made, the one that hashes the password.

But this is easily fixed by changing those assertions to use Bcrypt.verify_pass/2.

Open the file test/my_app/auth/auth_test.exs and change these lines:

# ...
test "create_user/1 with valid data creates a user" do
  # ...
  assert user.password == "some password"
end
# ...
test "update_user/2 with valid data updates the user" do
  #...
  assert user.password == "some updated password"
end
# ...

To:

# ...
test "create_user/1 with valid data creates a user" do
  # ...
  assert Bcrypt.verify_pass("some password", user.password)
end
# ...
test "update_user/2 with valid data updates the user" do
  # ...
  assert Bcrypt.verify_pass("some updated password", user.password)
end
# ...

Now mix test should yield no errors.


Users endpoint

Generate a new JSON endpoint

Let’s generate the users JSON endpoint, since we already have the Auth context and User schema available, we will pass the --no-schema and --no-context options.

mix phx.gen.json Auth User users email:string password:string is_active:boolean --no-schema --no-context

Fix the tests

Now, if you try and run your tests you’ll see this error:

== Compilation error in file lib/my_app_web/controllers/user_controller.ex ==
** (CompileError) lib/my_app_web/controllers/user_controller.ex:18: undefined function user_path/3

It’s complaining about a missing user_path/3 function.

You need to add a resources line to lib/my_app_web/router.ex.
Declaring a resource in the router will make some helpers available for its controller —i.e. user_path/3.

defmodule MyAppWeb.Router do
  # ...
  scope "/api", MyAppWeb do
    pipe_through :api
    resources "/users", UserController, except: [:new, :edit]
  end
end

Nevertheless, tests will still complain.
To fix them we need to make two changes:

  1. Comment out the password line in lib/my_app_web/views/user_view.ex:

    defmodule MyAppWeb.UserView do
      # ...
      def render("user.json", %{user: user}) do
        %{id: user.id,
          email: user.email,
          # password: user.password,
          is_active: user.is_active}
      end
    end

    We don’t need —nor should we— be sending the hashed password in our responses.

  2. Since we won’t be receiving the hashed password in the response, comment out these password lines in test/my_app_web/controllers/user_controller_test.exs:

    # ...
    assert json_response(conn, 200)["data"] == %{
      "id" => id,
      "email" => "some email",
      "is_active" => true
      # "password" => "some password"
    }
    # ...
    assert json_response(conn, 200)["data"] == %{
      "id" => id,
      "email" => "some updated email",
      "is_active" => false
      # "password" => "some updated password"
    }
    # ...

Tests should be fine now.

Create a couple of users

Using IEx

You can run your app inside IEx (Interactive Elixir) —this is akin to rails console— with:

iex -S mix phx.server

Then create a new user with:

MyApp.Auth.create_user(%{email: "asd@asd.com", password: "qwerty"})

Using curl

If you have curl available in your terminal, you can create a new user through your endpoint using something like:

curl -H "Content-Type: application/json" -X POST \
-d '{"user":{"email":"some@email.com","password":"some password"}}' \
http://localhost:4000/api/users

CORS configuration

You’ll need to configure this if you plan on having your API and frontend on different domains.
If you don’t know what CORS is, have a look at this: Cross-Origin Resource Sharing (CORS).

That said, here we have two options:

I’ll be using Corsica in this tutorial, as it have more features for configuring CORS requests —if you want a less strict libray, try CorsPlug out.

Add this dependency to mix.exs:

  defp deps do
    [
      # ...
      {:corsica, "~> 1.0"}
    ]
  end

Fetch new dependencies with:

mix deps.get

Add plug Corsica to lib/my_app_web/endpoint.ex just above the router plug:

defmodule MyAppWeb.Endpoint do
  # ...
  plug Corsica, origins: "http://localhost:8080",
    log: [rejected: :error, invalid: :warn, accepted: :debug],
    allow_headers: ["content-type"],
    allow_credentials: true

  plug MyAppWeb.Router
  # ...
end

You can pass a list to origins that can be composed of strings and regular expressions.

In my case, the rule above will accept CORS requests from a Vue.js frontend that uses Axios for HTTP requests —Vue.js development servers go up on port 8080 by default.


Simple authentication

Verify a user’s password

Let’s add some functions to the lib/my_app/auth/auth.ex file to verify a user’s password:

defmodule MyApp.Auth do
  # ...
  def authenticate_user(email, password) do
    query = from u in User, where: u.email == ^email
    query |> Repo.one() |> verify_password(password)
  end

  defp verify_password(nil, _) do
    # Perform a dummy check to make user enumeration more difficult
    Bcrypt.no_user_verify()
    {:error, "Wrong email or password"}
  end

  defp verify_password(user, password) do
    if Bcrypt.verify_pass(password, user.password) do
      {:ok, user}
    else
      {:error, "Wrong email or password"}
    end
  end
end

sign_in endpoint

Then add a new sign_in endpoint to lib/my_app_web/router.ex:

defmodule MyAppWeb.Router do
  # ...
  scope "/api", MyAppWeb do
    # ...
    resources "/users", UserController, except: [:new, :edit]
    post "/users/sign_in", UserController, :sign_in
  end
end

sign_in controller function

Finally add the sign_in function to lib/my_app_web/controllers/user_controller.ex:

defmodule MyAppWeb.UserController do
  # ...
  def sign_in(conn, %{"email" => email, "password" => password}) do
    case MyApp.Auth.authenticate_user(email, password) do
      {:ok, user} ->
        conn
        |> put_status(:ok)
        |> render(MyAppWeb.UserView, "sign_in.json", user: user)

      {:error, message} ->
        conn
        |> put_status(:unauthorized)
        |> render(MyAppWeb.ErrorView, "401.json", message: message)
    end
  end
end

Notice that we are rendering views inside MyAppWeb not inside MyApp.

Define sing_in.json and 401.json views

In lib/my_app_web/user_view.ex add this:

defmodule MyAppWeb.UserView do
  # ...
  def render("sign_in.json", %{user: user}) do
    %{data:
      %{user:
        %{id: user.id, email: user.email}}}
  end
end

In lib/my_app_web/error_view.ex add this:

defmodule MyAppWeb.ErrorView do
  # ...
  def render("401.json", %{message: message}) do
    %{errors: %{detail: message}}
  end
  # ...
end

You can try the sign_in endpoint now.

Try out your new endpoint with curl

Let’s restart our development server and send some POST requests to http://localhost:4000/api/users/sign_in.

Good credentials

curl -H "Content-Type: application/json" -X POST \
-d '{"email":"asd@asd.com","password":"qwerty"}' \
http://localhost:4000/api/users/sign_in -i

You’ll receive a 200 with:

{
  "data": {
    "user": { "id": 1,  "email": "asd@asd.com" }
  }
}

Bad credentials

curl -H "Content-Type: application/json" -X POST \
-d '{"email":"asd@asd.com","password":"not the right password"}' \
http://localhost:4000/api/users/sign_in -i

You’ll get a 401 with:

{ "errors": { "detail": "Wrong email or password" } }

Sessions

Add plug :fetch_session to your :api pipeline in lib/my_app_web/router.ex:

defmodule MyAppWeb.Router do
  # ...
  pipeline :api do
    plug :accepts, ["json"]
    plug :fetch_session
  end
  # ...
end

Save authentication status

Now let’s modify our sign_in function in lib/my_app_web/controllers/user_controller.ex:

defmodule MyAppWeb.UserController do
  # ...
  def sign_in(conn, %{"email" => email, "password" => password}) do
    case MyApp.Auth.authenticate_user(email, password) do
      {:ok, user} ->
        conn
        |> put_session(:current_user_id, user.id)
        |> put_status(:ok)
        |> render(MyAppWeb.UserView, "sign_in.json", user: user)

      {:error, message} ->
        conn
        |> delete_session(:current_user_id)
        |> put_status(:unauthorized)
        |> render(MyAppWeb.ErrorView, "401.json", message: message)
    end
  end
end

Protect a resource with authentication

Modify your lib/my_app_web/router.ex to look like this:

defmodule MyAppWeb.Router do
  use MyAppWeb, :router

  pipeline :api do
    plug :accepts, ["json"]
    plug :fetch_session
  end

  pipeline :api_auth do
    plug :ensure_authenticated
  end

  scope "/api", MyAppWeb do
    pipe_through :api
    post "/users/sign_in", UserController, :sign_in
  end

  scope "/api", MyAppWeb do
    pipe_through [:api, :api_auth]
    resources "/users", UserController, except: [:new, :edit]
  end

  # Plug function
  defp ensure_authenticated(conn, _opts) do
    current_user_id = get_session(conn, :current_user_id)

    if current_user_id do
      conn
    else
      conn
      |> put_status(:unauthorized)
      |> render(MyAppWeb.ErrorView, "401.json", message: "Unauthenticated user")
      |> halt()
    end
  end
end

As you can see we added a new pipeline called :api_auth that’ll run requests through a new :ensure_authenticated plug function.

We also created a new scope "/api" block that pipes its requests through :api then through :api_auth and moved resources "/users" inside.


Isn’t it amazing the way you can define this stuff in Phoenix?!
Composability FTW!


Fix the tests

Obviously all our MyAppWeb.UserController tests are broken now because of the requirement to be logged in. I’ll leave here how the fixed test/my_app_web/controllers/user_controller_test.exs file looks like:

defmodule MyAppWeb.UserControllerTest do
  use MyAppWeb.ConnCase

  alias MyApp.Auth
  alias MyApp.Auth.User
  alias Plug.Test

  @create_attrs %{email: "some email", is_active: true, password: "some password"}
  @update_attrs %{email: "some updated email", is_active: false, password: "some updated password"}
  @invalid_attrs %{email: nil, is_active: nil, password: nil}
  @current_user_attrs %{email: "current user email", is_active: true, password: "some current user password"}

  def fixture(:current_user) do
    {:ok, current_user} = Auth.create_user(@current_user_attrs)
    current_user
  end

  def fixture(:user) do
    {:ok, user} = Auth.create_user(@create_attrs)
    user
  end

  setup %{conn: conn} do
    {:ok, conn: conn, current_user: current_user} = setup_current_user(conn)
    {:ok, conn: put_req_header(conn, "accept", "application/json"), current_user: current_user}
  end

  describe "index" do
    test "lists all users", %{conn: conn, current_user: current_user} do
      conn = get conn, user_path(conn, :index)
      assert json_response(conn, 200)["data"] == [
        %{"id" => current_user.id,
          "email" => current_user.email,
          "is_active" => current_user.is_active}
      ]
    end
  end

  describe "create user" do
    test "renders user when data is valid", %{conn: conn} do
      conn = post conn, user_path(conn, :create), user: @create_attrs
      assert %{"id" => id} = json_response(conn, 201)["data"]

      conn = get conn, user_path(conn, :show, id)
      assert json_response(conn, 200)["data"] == %{
        "id" => id,
        "email" => "some email",
        "is_active" => true}
    end

    test "renders errors when data is invalid", %{conn: conn} do
      conn = post conn, user_path(conn, :create), user: @invalid_attrs
      assert json_response(conn, 422)["errors"] != %{}
    end
  end

  describe "update user" do
    setup [:create_user]

    test "renders user when data is valid", %{conn: conn, user: %User{id: id} = user} do
      conn = put conn, user_path(conn, :update, user), user: @update_attrs
      assert %{"id" => ^id} = json_response(conn, 200)["data"]

      conn = get conn, user_path(conn, :show, id)
      assert json_response(conn, 200)["data"] == %{
        "id" => id,
        "email" => "some updated email",
        "is_active" => false}
    end

    test "renders errors when data is invalid", %{conn: conn, user: user} do
      conn = put conn, user_path(conn, :update, user), user: @invalid_attrs
      assert json_response(conn, 422)["errors"] != %{}
    end
  end

  describe "delete user" do
    setup [:create_user]

    test "deletes chosen user", %{conn: conn, user: user} do
      conn = delete conn, user_path(conn, :delete, user)
      assert response(conn, 204)
      assert_error_sent 404, fn ->
        get conn, user_path(conn, :show, user)
      end
    end
  end

  defp create_user(_) do
    user = fixture(:user)
    {:ok, user: user}
  end

  defp setup_current_user(conn) do
    current_user = fixture(:current_user)
    {:ok, conn: Test.init_test_session(conn, current_user_id: current_user.id), current_user: current_user}
  end
end

Add missing tests

In test/my_app/auth/auth_test.exs, test the authenticate_user/2 function:

defmodule MyApp.AuthTest do
  # ...
  describe "users" do
    # ...
    test "authenticate_user/2 authenticates the user" do
      user = user_fixture()
      assert {:error, "Wrong email or password"} = Auth.authenticate_user("wrong email", "")
      assert {:ok, authenticated_user} = Auth.authenticate_user(user.email, @valid_attrs.password)
      assert user == authenticated_user
    end
  end
end

In test/my_app_web/controllers/user_controller_test.exs, test the sign_in endpoint:

defmodule MyAppWeb.UserControllerTest do
  # ...
  describe "sign_in user" do
    test "renders user when user credentials are good", %{conn: conn, current_user: current_user} do
      conn = post conn, user_path(conn, :sign_in, %{email: current_user.email, password: @current_user_attrs.password})
      assert json_response(conn, 200)["data"] == %{
        "user" => %{
          "id" => current_user.id,
          "email" => current_user.email}}
    end

    test "renders errors when user credentials are bad", %{conn: conn} do
      conn = post conn, user_path(conn, :sign_in, %{email: "nonexistent email", password: ""})
      assert json_response(conn, 401)["errors"] == %{"detail" => "Wrong email or password"}
    end
  end
  # ...
end

Endpoint testing with curl

Try to request a protected resource, like /api/users with:

curl -H "Content-Type: application/json" -X GET \
http://localhost:4000/api/users \
-c cookies.txt -b cookies.txt -i

You’ll get:

{ "errors": { "detail": "Unauthenticated user" } }

Let’s login with:

curl -H "Content-Type: application/json" -X POST \
-d '{"email":"asd@asd.com","password":"qwerty"}' \
http://localhost:4000/api/users/sign_in \
-c cookies.txt -b cookies.txt -i

You’ll get:

{
  "data": {
    "user": { "id": 1, "email": "asd@asd.com" }
  }
}

Now, try requesting that protected resource again:

curl -H "Content-Type: application/json" -X GET \
http://localhost:4000/api/users \
-c cookies.txt -b cookies.txt -i

You’ll see:

{
  "data": [
    { "is_active": false, "id": 1, "email": "asd@asd.com" },
    { "is_active": false, "id": 2, "email": "some@email.com" }
  ]
}

Success!


Bonus section

Customize your 404s and 500s JSON responses

In lib/my_app_web/views/error_view.ex:

defmodule MyAppWeb.ErrorView do
  # ...
  def render("404.json", _assigns) do
    %{errors: %{detail: "Endpoint not found!"}}
  end

  def render("500.json", _assigns) do
    %{errors: %{detail: "Internal server error :("}}
  end
  # ...
end

Links


This was a long one, that’s it for now folks!


— lt

Feedback & comments

Get in touch on Twitter

Or by good ol' email at adriandcs@gmail.com