I’ve debugged my share of C NIFs in the past, so I hope I can provide some help. I pulled your branch and reproduced the segfault. Locally I added some fprintf-based logging to exqlite_close and noticed something that made my raise an eyebrow, so I thought I’d share. If I’m off base here, please let me know.

An fprintf statement in exqlite_close prevents the segfault on my system, so there is definitely a timing element here (race condition). I printed the memory addresses for conn and conn->db and saw many duplicate addresses, which made me wonder if exqlite_close is being called multiple times on the same resource, resulting in a double-free-like segfault when running with high concurrency.

BTW, thanks for this library. I am a fan.

I’m not familiar with DBConnection, but I’ve written similar libraries in the past. When you do DBConnection.start_link, is it calling your connect one time and then sharing the result among the pool of 50? If that’s the case, I wonder if creating a single enif resource and allowing it to be “disconnected” from multiple concurrent processes in the pool is the source of the race condition. The DBConnection README says it calls disconnect automatically:

DBConnection also takes all of the care necessary to handle
failures, and it shuts down the connection and the socket
whenever the client does not check in the connection to avoid
recycling sockets/connections in a corrupted state (such as a socket
that is stuck inside a transaction).

(It looks like DBConnection was written with network sockets in mind. Due to the nature of SQLite, obviously there is no network socket, so the disconnect behavior may not be optimal for your use case.)

In my experience, it’s safest to ensure that an enif resource gets a dedicated BEAM process so that you can carefully control the access to it. At the very least, I assume you only want a maximum of 1 exqlite_close on a given resource; You can build that guarantee by serializing calls to Connection.close on a dedicated BEAM process. Otherwise, you can’t guarantee there are not 2 scheduler threads freeing the memory simultaneously.

We get spoiled by the BEAM scheduler and garbage collection, but when it comes to NIFs sadly we have to worry about all these issues again. I don’t believe the BEAM garbage collector is causing any problems in this case.

Edit: in my haste I probably jumped to some wrong conclusions about DBConnection. I’m going to spend some more time and try to get more concrete results.

Not necessarily, but I tied this pretty closely to being used by ecto_sqlite3.

I could use poolboy or nimble pool for pooled resources, but when it comes to Ecto, you have to implement the DBConnection interface in order to have it play nice with ecto_sql.