zwippie

zwippie

Nimble_csv dump with each column quoted

I want to dump some data to a CSV file where each column value is separated with a semicolon and wrapped inside double quotes. The semicolon is no problem, but the double quotes are.

This does not work, no double quotes:

NimbleCSV.define(MyParser, separator: ";", escape: "\"")
data = [["firstname", "lastname"], ["Foo", "Bar"]]
IO.iodata_to_binary MyParser.dump_to_iodata(data)

That will return:

"firstname;lastname\nFoo;Bar\n"

But i want this:

"\"firstname\";\"lastname\"\n\"Foo\";\"Bar\""

How to wrap each column with double quotes?

Marked As Solved

peerreynders

peerreynders

iex(1)> data = [["\"firstname\"", "\"lastname\""], ["\"Foo\"", "\"Bar\""]]
[["\"firstname\"", "\"lastname\""], ["\"Foo\"", "\"Bar\""]]
iex(2)> IO.iodata_to_binary MyParser.dump_to_iodata(data)
"\"\"\"firstname\"\"\";\"\"\"lastname\"\"\"\n\"\"\"Foo\"\"\";\"\"\"Bar\"\"\"\n"
iex(3)> 

… so not exactly.

rfc4180

  1. Each field may or may not be enclosed in double quotes (however
    some programs, such as Microsoft Excel, do not use double quotes
    at all). If fields are not enclosed with double quotes, then
    double quotes may not appear inside the fields.

NimbleCSV has no problem consuming

"\"firstname\";\"lastname\"\n\"Foo\";\"Bar\""

it just doesn’t seem interested in producing it (likely to avoid unnecessarily bloating the output).

iex(1)> MyParser.parse_string("\"firstname\";\"lastname\"\n\"Foo\";\"Bar\"",[headers: false])
[["firstname", "lastname"], ["Foo", "Bar"]]
iex(2)> 
  1. Fields containing line breaks (CRLF), double quotes, and commas
    should be enclosed in double-quotes.

NimbleCSV will only use the double quotes when absolutely necessary, e.g.:

iex(1)> data = [["firstname", "lastname"], ["Fo\no", "Bar"]]
[["firstname", "lastname"], ["Fo\no", "Bar"]]
iex(2)> IO.iodata_to_binary MyParser.dump_to_iodata(data)   
"firstname;lastname\n\"Fo\no\";Bar\n"
iex(3)> data = [["firstname", "lastname"], ["Fo\"o", "Bar"]]
[["firstname", "lastname"], ["Fo\"o", "Bar"]]
iex(4)> IO.iodata_to_binary MyParser.dump_to_iodata(data)   
"firstname;lastname\n\"Fo\"\"o\";Bar\n"
iex(5)> 

that last example also demonstrating

  1. If double-quotes are used to enclose fields, then a double-quote
    appearing inside a field must be escaped by preceding it with
    another double quote.

Also Liked

zwippie

zwippie

Thank you for your great explanation! Considering rfc4180, it all makes sense of course. I would welcome a force_escape option for NimbleCSV though.

Now I’m going to find out how many clients rely on our current (quoted) CSV output format.

nallwhy

nallwhy

Simple implementation

Where Next?

Popular in Questions Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
Jim
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
New
chensan
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
New
Fl4m3Ph03n1x
Background Let’s assume I have a typical GenServer that receives messages as requests, does some operation in a DB and returns responses....
New
LegitStack
I’m trying to make a websocket server in Phoenix or raw Elixir. I heard about gun, I think I could use cowboy, but since I’m not that sma...
New
belgoros
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
New
sabri
Can someone explain the settings of pool_size of Ecto in config file? and what is the recommend size? Thanks
New
Codball
Mix format works fine if run from the cmd. I’ve followed this to facilitate the implementation into VSC which involves downloading an ext...
New
joeerl
Hello again - after a longish gap I’ve decided I really must dig into Elixir and see what’s been happening here - so I have a few questio...
New

Other popular topics Top

yurko
Here are few pieces of (common) Linux knowledge that we use for reasonably small one server apps. We use Ubuntu but this should work for ...
New
William
I would like to know that is there any online source for learning Phoenix Framework for building E-Commerce Store? Any advantage on build...
New
bsollish-terakeet
Credo is smart enough to check for (something like) this: assert length(the_list) == 0 with this response: Checking if an enum is empt...
New
SoCreat
i’m a new one to elixir which editor can i use vs code? or atom? Thanks! :smiley:
New
hariharasudhan94
I would like to know what is the best IDE for elixir development?
New
ovidiubadita
Hey all, I discovered Elixir and I love it. I always wanted to learn a functional programming and I intended to go for Haskell, but afte...
New
beno
I will often find my self writing things similar to: case some_value do nil -> something() "" -> something() _ -> someth...
New
AstonJ
by Lance Halvorsen Elixir and Phoenix are generating tremendous excitement as an unbeatable platform for building modern web application...
460 27162 124
New
aesmail
Hello guys, I have finally made it. I created an admin interface for a framework. It’s been on my todo list for years and with the curre...
New
magnetic
Hey :wave:t3: Elixir community, I’ve been learning Elixir, and working on some side projects. My editor of choice is VSCode, and althoug...
New

We're in Beta

About us Mission Statement