martinthenth

martinthenth

Goal - A parameter validation library based on Ecto

Hi fellow Elixirists! :wave:

About 8 months ago, I wrote a blog post on validating Phoenix controller parameters using Ecto Changesets in combination with Ecto Embedded Schemas (Using Ecto changesets for API request parameter validation).

Since then, I found that writing embedded schemas for validating parameters to be quite tedious. It required setting up new modules, a lot of boilerplate code, and complex file organisation.

So I decided to write a library that replaces the boilerplate code with a simple syntax, that can be written inside controllers, and is still based on Ecto Changesets to leverage its validation capabilities.

It’s called Goal, and it offers those capabilities with a nice syntax:

defmodule MyApp.SomeController do
  import Goal
  import Goal.Syntax

  def create(conn, params) do
    with {:ok, attrs} <- validate_params(params, schema()) do
      ...
    end
  end

  defp schema do
    defschema do
      required :uuid, :string, format: :uuid
      required :name, :string, min: 3, max: 3
      optional :age, :integer, min: 0, max: 120
      optional :gender, :enum, values: ["female", "male", "non-binary"]

      optional :data, :map do
        required :color, :string
        optional :money, :decimal
        optional :height, :float
      end
    end
  end
end

It features:

  1. Compatibility with all primitive types from Ecto.Schema
  2. Customisable regexes for email, password, and url formats, so you can maintain compatibility with your production system
  3. Unlimited nested maps and lists of maps
  4. trim and squish to trim and collapse whitespaces
  5. optional and required arguments for defining optional and required fields
  6. An extended version of Ecto.Changeset.traverse_errors/2 (aptly called Goal.Changeset.traverse_errors/2) that works with Goal and Ecto’s existing functionality

You can use Goal for validating Phoenix controller action parameters, but it will work with any data source as long as it’s represented in a map.

I aim for Goal to cover all parameter validations that Ecto.Changeset offers for database fields. I think I covered most with the 0.1.1 version; but if you’d like another validation, then please contribute :pray: or open an issue on GitHub.

It was inspired by Ruby’s dry-schema, and I had to borrow code from Ecto. Thank you for making such awesome libraries. :bow:

Feedback and suggestions are very welcome :pray:. For example, I was thinking about generating the chain of Ecto.Changeset functions at compile-time, to crunch out some additional usecs :racing_car:

Most Liked

stefanchrobot

stefanchrobot

This looks great! How about supporting this syntax:

defschema schema do
  required ...
end

I’d love to see the functionality of Goal incorporated into Ecto. Did you consider that? If so, there’s one thing to think about: the Goal schema seems to be controller-specific so it makes sense to have required, min, max, etc. as options. On the other hand Ecto schemas are reused across the changeset functions (each function has its own set of required/optional/validations). I still think it would make sense to support what Goal is using, maybe with a different macro (there’s schema and embedded_schema, so maybe validation_schema?).

fuelen

fuelen

required :uuid, :string, format: :uuid
optional :gender, :enum, values: ["female", "male", "non-binary"]

if the library is based on Ecto, then why not to use built-in types? I mean this

required :uuid, Ecto.UUID
optional :gender, Ecto.Enum, values: ["female", "male", "non-binary"]
martinthenth

martinthenth

Goal 0.1.3 released :tada:

I recently switched to LiveViews and found that I have the same need to validate incoming parameters as with JSON APIs. Since LiveViews depend on changesets for validation, I decided to expose the Goal.build_changeset(params, schema) function in the library.

I find using database Ecto.Schemas in LiveViews, like the examples in the LiveView docs do, a bit problematic because I often need fields that aren’t defined in the Ecto.Schema. Embedded schemas are great, but it’s a lot of boilerplate…

Plus, often my changesets have required fields like foreign keys that I cannot validate in the LiveView so the changeset is never valid; a state that I use in my forms to enable/disable form submit buttons. :thinking:

Goal solves all that with a sweet syntax.

Using Goal for LiveViews is super easy:

defmodule MyApp.SomeLiveView do
  import Goal.Syntax

  def handle_event("validate", %{"some" => some_params}, socket) do
    changeset =
      some_params
      |> Goal.build_changeset(schema())
      |> Map.put(:action, :validate)

    {:noreply, assign(socket, :changeset, changeset)}
  end

  defp schema do
    defschema do
      required :id, :uuid
      required :name, :string, max: 20
      optional :age, :integer, min: 0, max: 120
      optional :gender, :enum, values: ["female", "male", "non-binary"]
      optional :address, :string

      optional :data, :map do
        required :color, :string
        optional :money, :decimal
        optional :height, :float
      end
    end
  end
end
martinthenth

martinthenth

Goal 0.2.0 released today! :tada:

This version contains some breaking, but very convenient changes to the library.

  1. It adds the defparams macro which makes defining parameter validation schemas easier than before, because it encapsulates the schemas in a function that you can call elsewhere.
  2. It adds conveniences like validate/1 and /2, and changeset/1 and /2 that can be like MySchema.validate(:new, params) and MySchema.changeset(:new, params)

Let’s take a look at the new defparams macro:

Example with a schema in a separate module

Define a schema in a separate module:

defmodule MyAppWeb.MySchema do
  use Goal

  defparams :create do
    required :id, :uuid
    optional :name, :string
    optional :age, :integer, min: 0
  end
end

And then use it in a controller or LiveView:

defmodule MyApp.SomeController do
  use MyApp, :controller
  
  alias MyAppWeb.MySchema

  def create(conn, params) do
    with {:ok, attrs} <- MySchema.validate(:create, params)) do
      ...
    else
      {:error, changeset} -> {:error, changeset}
    end
  end
end

You can have the schema inside the controller, it will work the same.

Example with LiveView

Version 0.2.0 adds conveniences for working with LiveViews, called changeset/2 and validate/2. They are used to build and validate a changeset that can be rendered in a LiveView (also works in HTML-controllers):

defmodule MyApp.SomeLiveView do
  use MyApp, :live_view

  alias MyAppWeb.MySchema

  def mount(params, _session, socket) do
    changeset = MySchema.changeset(:new, %{})
    socket = assign(socket, :changeset, changeset)

    {:ok, socket}
  end

  def handle_event("validate", %{"some" => params}, socket) do
    changeset = MySchema.changeset(:new, params)
    socket = assign(socket, :changeset, changeset)

    {:noreply, socket}
  end

  def handle_event("save", %{"some" => params}, socket) do
    with {:ok, attrs} <- MySchema.validate(:new, params)) do
      ...
    else
      {:error, changeset} -> {:noreply, assign(socket, :changeset, changeset)}
    end
  end
end

With these additions, it looks like Goal is running out of major functional improvements. The library’s behavior is very similar to Ruby’s DRY params and other mature parameter validation libraries. So I think Goal may reach v1.0.0 soon :open_mouth:

martinthenth

martinthenth

I decided to add a function for outbound key recasing, because I need it in my apps and it seems intuitive that if you recase incoming parameters you would like to recase outbound parameters as well. :smile:

An example (with Phoenix 1.7 views):

config :goal,
  recase_keys: [to: :camel_case]

defmodule MyAppWeb.UserJSON do
  import Goal

  def show(%{user: user}) do
    %{data: %{first_name: user.first_name}}
    |> recase_keys()
  end

  def error(%{changeset: changeset}) do
    errors =
      changeset
      |> Goal.Changeset.traverse_errors(&translate_error/1)
      |> recase_keys()

    %{errors: errors}
  end
end

iex(1)> UserJSON.show(%{user: %{first_name: "Jane"}})
%{data: %{firstName: "Jane"}}
iex(2)> UserJSON.error(%Ecto.Changeset{errors: [first_name: {"can't be blank", [validation: :required]}]})
%{errors: %{firstName: ["can't be blank"]}}

The recase_keys/1 function should only be used when you are in control of the data that is being recased.

The updated docs are available on Hex: Goal 0.2.3

Where Next?

Popular in Libraries Top

zoltanszogyenyi
Hey everyone :wave: Excited to join this forum - I am one of the founders and current project maintainers of a popular and open-source U...
New
sabiwara
Dune is a sandbox for Elixir and aims to safely evaluate user-provided code. You can try it out using this basic Elixir playground made ...
New
cjen07
parameterized pipe in elixir: |n&gt; edit: negative index in |n&gt; and mixed usage with |&gt; are supported example: use ParamP...
New
josevalim
EDIT: since Ecto 3.0 final version is out, this post was amended to use the final versions in the instructions below. Hi everyone, We a...
New
tfwright
After working on it for a couple of months and using it in production for most of that time, today I’ve released LiveAdmin, a LiveView ba...
New
ahamez
Hi everyone, I’ve been working on this protobuf library for 3 years. We use it in the company I work for, EasyMile, to communicate with ...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 42633 214
New
ericlathrop
I built a silly site for Halloween that uses Phoenix Channels on the backend, and React on the frontend. I had many problems integrating ...
New
bryanjos
Hi, I just published version 0.23.0 of Elixirscript. Most of the changes are around JavaScript interop now that Elixirscript uses the ...
New
Azolo
Hey everyone, I just released WebSockex which is a Elixir WebSocket client. WebSockex strives to work as a OTP special process, be RFC6...
New

Other popular topics Top

chrismccord
Phoenix 1.4.0 released Phoenix 1.4 is out! This release ships with exciting new features, most notably with HTTP2 support, improved deve...
688 30048 115
New
srinivasu
How to handle excepions in elixir? Suppose i have A, B, C ,D, E modules. and each module has get() function. A.get() method will call th...
New
openscript
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
New
joaquinalcerro
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
sergio
I couldn’t find any guides that worked well with Phoenix 1.6.0 and esbuild. I hope this helps people test the waters and eases you into t...
New
nsuchy
Hi. I’ve noticed that Windows Powershell has it’s own IEX command and you cannot access Elixir’s IEX due to the conflict. This isn’t a cr...
New
stefanchrobot
What’s the safe way to decode a JSON string into a struct? I want to avoid calling String.to_atom. Jason.decode can give me a map with st...
New
vonH
When I run the Plug and I recompile I wind up having to use Ctrl C to quit iex and start again. Witht the help of rlwrap I can use the cu...
New
msaraiva
Surface is an experimental library built on top of Phoenix LiveView and its new LiveComponent API that aims to provide a more declarative...
564 42633 214
New

Sub Categories:

We're in Beta

About us Mission Statement