ntd23

ntd23

Private, a lib to test private functions in 2022

Hi again ppl,

I really like the idea of test my private functions, mainly after read this article of @pragdave, but the library that he create is a little old and the article too, the last modification on code was 2 years ago . So, I have two questions:

  • I get much risk using this lib, even it being “stable”?
  • Is really worth test many private functions as I can or the article already get a strong oposit response?

Marked As Solved

pragdave

pragdave

Author of Programming Elixir

FWIW, my personal belief is that no one knows enough about software development to be able to make rules: it’s just too young a discipline.

But, I also believe that:

  • defining helper functions as private sets a demarcation layer in my code: if the code is to be used by a third party, then making a function private means that I’m free to change it without worrying about unintended consequences.

  • tests can be helpful for some functions. Which ones? The ones where there’s doubt; the ones which are pivotal to the rest of the code; the ones where the tests help you explore the problem.

These two things do not necessarily correlate. I often find myself writing internal (private) functions which need tests.

“But, Dave” say those in the never-test-a-private-function camp. “Why not just test them indirectly via the public functions that use them?”

Because, I say (outside of quotes), that is often a lot of work. The higher level the function, the more state it assumes, and the harder the test is to write. And, typically, when you invoke these higher level functions, you’re calling a number of lower-level ones. The test loses focus.

Let’s look at this a different way. Let’s think about the demarcation of code in GenServers.

When I write GenServers, I typically split them into two files. One file contains the server stuff: start_link, init, and all the handle_xxx calls, the other contains the implementation.

Here’s part of a typical server:

defmodule ExMdns.Server.ReceiveDNSPackets do

  alias ExMdns.Impl.PubSub
  alias ExMdns.Impl.ReceiveDNSPackets, as: Impl 

  use GenServer
  require Logger
  
  @me __MODULE__

  def start_link(_) do
    GenServer.start_link(__MODULE__, [], name: @me)
  end

  def init(_) do
    PubSub.join()
    { :ok, Impl.initial_state }
  end

  def handle_info({ :pubsub, msg=%PubSub.PacketReceived{}}, state) do
    state = Impl.handle_incoming_packet(state, msg.data)
    { :noreply, state }
  end

  def handle_info({ :pubsub, _ }, state), do: { :noreply, state }

end

None of these functions contain any logic (apart from pattern matching on their parameters. Instead, they delegate to a second module that contains the actual logic.

The functions in this second module contain the implementation of the logic behind the server. The server module calls an implementation function with the server state as a parameter, along with any additional parameters from the handle_xxx. The implementation function returns an updated state.

Here’s the implementation corresponding to that server:

defmodule ExMdns.Impl.ReceiveDNSPackets do

  alias ExMdns.Impl.PubSub
  require Logger

  def initial_state() do
    []
  end

  def handle_incoming_packet(state, raw_packet) do 
    try do
      DnsPackets.decode(raw_packet.data)
      |> publish_replies()
    rescue
      e ->
        Logger.error("Ignoring invalid DNS packet: #{inspect raw_packet}")
        Logger.error(inspect e)
    end
    state
  end

  defp publish_replies(packet) when packet.header.qr == :reply do
    for answer <- packet.answers do
      PubSub.publish(%PubSub.AnswerReceived{ answer: answer })
    end
  end

  defp publish_replies(packet) do
    Logger.info("ignoring query\n#{inspect packet, pretty: true}")
  end
end

Why is this relevant to testing? Because I can test the implementation functions standalone, without firing up a GenServer. And it’s easy to do, because my tests just create the state to pass in, and then verify the state that comes back. In fact, I rarely run any genservers in test: My application.ex looks like this:

def start(_type, _args) do
    children = case Mix.env do 
      :test -> 
        [ ]
      _ -> 
        [
          Server.Cache,
          Server.ReceiveDNSPackets,
          Server.UdpInterface,
          # Registration,
        ]
    end

    opts = [strategy: :one_for_one, name: ExMdns.Supervisor]
    Supervisor.start_link(children, opts)
  end

What has this to do with private functions?

It’s an illustration of how I believe that testing convenience trumps following rules.

Typically, Elixir developers would put the implementation functions in the same module as the server stuff. They’d then struggle with whether they should be private. Purity insists that they should: they are accessed via the GenServer.call interface, and so shouldn’t be exposed externally. Make 'em private, and then test via GenServer calls. Or, make 'em public, test them directly, and feel guilty.

Instead, this two-module approach sidesteps the dogma. In my apps, nothing in a subdirectory of lib/ should be accessed externally. Instead, I have a top-level api.ex that delegates each API function, calling .Server files if the implementation of the function uses a GenServer, and calling the implementation files directly for stateless functions.

It then becomes immediately obvious to any external user of the code what they’re allowed to use: anything with more than one module name in the path is internal to my application. It may be public, but that’s for my convenience and not for your use.

Do I do this consistently? No: to be honest I’m experimenting with each app I write. This is where I am now.

TLDR; forget what people tell you is right. Do what you feel makes your code easier to work with.

Also Liked

dorgan

dorgan

I have been using Patch — patch v0.13.0 quite a lot recently. It’s very well documented and behaves as advertised. It’s a powerful alternative to Mox though it can also be used alongside it.

Private functions frequently go untested because they are difficult to test. Developers are faced with a few options when they have a private function.

  1. Don’t test the private function.
  2. Test the private function circuitously by calling some public functions.
  3. Make a public wrapper for the private function and test that.
  4. Change the visibility to public and put a comment with some form of, “This is public just for testing, this function should be treated as though it’s private.”

Patch provides a new mechanism for testing private functions, expose/2.

dorgan

dorgan

Also, code doesn’t need to be complex to be hard to test.

The behaviour of a genserver with no public api that all it does is periodically call itself to perform some side effect is built on entirely private behavior that should not be used by any other module. I’ve had to deal with a bunch of them and they are always nasty to test.

Whatever approach you take, you’re ripping the module open and testing its internals, not any “observed behavior through public api”. Theres no public api.

The process calling itself every X time is not something you can test via public api, so giving up on idealistic notions of how code should be tested in favor of pragmatism is best. This point is also raised in the Testing Elixir book by Andrea Leopardi, in the Testing OTP chapter.

You can decide to move some functions to a different module and test that because now it’s no longer “private” and you can test it “normally”, but imho that’s just lying to yourself and if theres no other reason for that module to exist(to split complex logic into smaller files, to name an example), it’s just an indirection that brings very little value.

At that point, just test the behaviour callbacks directly by providing some state. This may sound controversial, but if theres no client api then no solution will appeal to everyone’s taste. FWIW this would be just like testing a custom Ecto type by calling the callbacks directly instead of creating a schema and persisting it to indirectly test dump and cast.

1000 times this

tcoopman

tcoopman

Tests are also code that you need to maintain. Delete tests that don’t have any value anymore. Nothing wasteful about that.

al2o3cr

al2o3cr

IMO if a function is complicated enough to merit its own test it’s probably complicated enough to merit a @doc - and putting one on a private function emits a warning (see many discussions, like this one for more).

dimitarvp

dimitarvp

I haven’t reviewed the library but I don’t see why it would be unstable or dangerous to use? It’s a pretty small-scoped library so it might just be considered complete and does not need further changes.

This is a broad question and nobody can answer that for your project except yourself. I personally do my best to extract out all the bits and pieces that my higher-level business functions are using, into smaller utility functions, and usually make them public throughout the project. Doesn’t hurt anything IMO.

So:

  • Have smaller amount of private functions. Abstract away things as long as they make sense as separate functions (utilize your experience and knowledge to make that call; there are no deadly consequences if you get it wrong).
  • Those private functions that end up remaining private are not important enough to try and test in isolation. You will test the behaviour of your system e.g. sending particular combination of parameters to your controller and will then check if certain DB objects have been created / updated / deleted, certain emails have been sent, or message queues filled, notifications sent etc. The private functions are not important there. If the test fails you’ll start to track what went wrong and you’ll fix everything along the way, the private functions included.

Where Next?

Popular in Questions Top

Harrisonl
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
New
dokuzbir
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
New
myronmarston
The Elixir Typespec docs show the following syntax for keyword lists in typespecs: # ... | [key: type] # keyword lis...
New
makeitrein
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
New
aadeshere1
I have a another noob question about loop. Since elixir is immutable, while loop is not directly possible. total = 10 while total != 0 ...
New
alice
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
chewm
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
New
jc00ke
Expanding on this topic: https://forum.elixirforum.net/t/map-typespec-question/19217 Let’s say I have a map with required and optional k...
New
jay1
Why is it that the mnesia database isn’t the most preferred database for use in Elixir/Phoenix?
New

Other popular topics Top

Tee
can someone please explain to me how Enum.reduce works with maps
New
aalberti333
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
fireproofsocks
I’m working on defining a simple Ecto schema for a table (in PostGres), but I don’t see where I can define a column as NOT NULL. Conside...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
fireproofsocks
Forgive me if this is obvious, but how does one delete a database record WITHOUT selecting it first? https://hexdocs.pm/ecto/Ecto.Repo.h...
New
baxterw3b
Hi guys, i’m new in the Elixir world, and i have to say, that i love it! i’m having some problem to understand anonymous functions with ...
New
WestKeys
Currently suffering from paralysis by [HTTP client] analysis. This is rather unusual in Elixirland as there tends to be consensus on the ...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New

We're in Beta

About us Mission Statement