Questions & Help>Questions

Explorer.DataFrame - load_ndjson json

#explorer
imed-code

imed-code

Explorer.DataFrame - load_ndjson json

Hello,

I’m trying to load a json API message with Explorer.DataFrame.

My api is returning a list like bellow:

resp = [%{"idade" => "8", "nome" => "Mirela"}, %{"idade" => "40", "nome" => "Sergio"}]

When I try to run the following code:

require Explorer.DataFrame, as: DF
DF.load_ndjson!(Jason.encode!(resp), lazy: true)

I got an error:

** (ErlangError) Erlang error: :nif_panicked
    (explorer 0.5.7) Explorer.PolarsBackend.Native.df_load_ndjson("...

But if I change to read the first element it works as well.

require Explorer.DataFrame, as: DF
DF.load_ndjson!(Jason.encode!(List.first(resp)))

How to read the entire data ?

Thank you.

1 comment
#explorer
0 552 1

First Post!

al2o3cr

al2o3cr

The ndjson parser expects JSON objects separated by newlines - for instance, here’s an example from Explorer’s tests:

That is not what Jason.encode!(resp) will produce - that result will have [ at the beginning and , between the objects. You’d need to encode each item in resp separately.

HOWEVER

In general, re-encoding like that should make you suspect that you’re not using the best API for your situation.

The better choice is Explorer.DataFrame.new, in particular the last example labeled “From row data:”.

Post #2

Where Next?

View thread on forum (has 1 response!)
explorer
Home Questions & Help>Questions
#explorer
0 554 1
Last post

Popular in Questions Top

shahryarjb
Questions & Help>Questions

How to convert Date to DateTime which Ecto needs
Hello, I get Persian date from my client and convert it to normal calendar like this: def jalali_string_to_miladi_english_number(persi...
#ecto#datetime
15 14989 5
New
aalberti333
Questions & Help>Questions

Enum.map over list of key/value pairs with a map as the value
As the title describes, I’m trying to run Enum.map() over a list of key/value pairs, where the value is a map. My data looks like this: ...
#enummap
7 19677 6
New
Harrisonl
Questions & Help>Questions

Deploying Elixir into ECS causing many "'global' at node :"xxxxx@10.0.X.X" requested disconnect from node :"xxxx@10.0.X.X" in order to prevent overlapping partitions"
We have an ECS cluster with 4 services, where each task joins a single cluster, via discovery ECS discovery service. Currently when I de...
#deployment
2 85219 3
New
dokuzbir
Questions & Help>Questions

How to convert list to string without losing brackets
Hello, I am trying to convert my lists to string without losing brackets.For start i have 3 map. They look like these buyer = %{ id: ...
26 14426 10
New
ashish173
Questions & Help>Questions

Difference in between :utc_datetime and :naive_datetime in Ecto
I am using Ecto timestamps with postgres, I can see the timestamps() use the :naive_dateime but for my use case I wanted to store the ti...
#ecto#timestamps
79 27907 15
New
chewm
Questions & Help>Questions

how configure visual studio code for elixir intellisense
Hi guys, nice to meet you to the whole forum, I’m new here, I’m trying to configure visual studio code for elixir, right now the intellis...
#code-editors#vscode#vscode-elixir
7 14611 11
New
Patoshizzle
Questions & Help>Questions

(Postgrex.Error) FATAL 28P01 (invalid_password) password authentication failed for user “postgres”
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID<0.412.0> terminating ** (Postgrex.Error) FATAL...
#ecto#postgres#troubleshooting
10 28696 20
New
hpopp
Questions & Help>Questions

Access Behaviour on Structs
To simplify some tasks at work, I wrote and published this package yesterday. It’s a simple macro that enables Access behaviour on struct...
#structs
31 13547 7
New
belgoros
Questions & Help>Questions

Regex question for hyphen match
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
#learning-elixir
9 12939 10
New
romenigld
Questions & Help>Questions

Docker run Error loading shared library libstdc++.so.6 and libgcc_s.so.1
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
/phoenix#deployment#docker-compose
30 19932 15
New

Other popular topics Top

Brian
Questions & Help>Questions

Import a module from a file into IEX
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
#iex
35 32139 16
New
sorentwo
News>Announcing

Oban - Reliable and Observable Job Processing
Hello! tl;dr Announcing Oban, an Ecto based job processing library with a focus on reliability and historical observability. After spen...
/oban#ecto#postgres#job-processing
977 41022 311
New
joaquinalcerro
Questions & Help>Questions

What is the best approach for fetching large amount of records from postgresql with ecto
Hi there, I am working with Ecto-Postgresql and I need to call all of the records from a specific table but the table has 40,000 record...
#ecto
126 24587 27
New
jerry
Questions & Help>Questions

Ecto query using like/ilike in query
Good day to you all. I have been struggling to get a query involving like and ilike to work. Can anyone assist me on this, please? pro...
#ecto
17 16504 10
New
_russellb
Questions & Help>Questions

Web scraping tools
I want to try my hand at web scraping. What tools/libraries do I need to use. I’m hoping to turn this into something professional so don’...
#web-scraping
123 19127 45
New
Jim
Questions & Help>Questions

Help with HTTPoison POST
As a follow up to my earlier question: I have the code compiling and running but not getting a successful login from the rest server. ...
#troubleshooting
15 14640 7
New
chrismccord
News>Phoenix News

Phoenix v1.3.0-rc.0 released
As promised, the first release candidate of Phoenix 1.3.0 is out! This release focuses on code generators with improved project structure...
/phoenix#phoenix-release
434 29347 97
New
Fl4m3Ph03n1x
Questions & Help>Questions

Learning Elixir, frst impressions ( plz don't kill me ! )
About me? ( if you have nothing better to do than reading about some random guy in the internet :stuck_out_tongue: ) Hello all, this is ...
#learning-elixir
371 25801 61
New
9mm
Questions & Help>Questions

Elixir way to conditionally update a map
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
#maps
55 19905 29
New
lucidguppy
Questions & Help>Questions

How do I read a file and enumerate its lines
I have a super simple question about elixir - how would I take a file like this foo bar baz and output a new file that enumerates th...
#how-to-question
20 15454 3
New
Back to top

Questions & Help>Questions

Questions Questions ❯

Latest on Elixir Forum

Elixir Forum ❯

Sponsor Spotlight

Manning

Develop your skills with books, videos, and courses.

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

ElixirCasts

Producing high quality Elixir screencasts since 2017.

Potions

The deployment platform built for Elixir: PaaS ease, VPS control.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Forums

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Pragmatic Bookshelf

Practical resources that improve the lives of professional developers.

ElixirCasts

Producing high quality Elixir screencasts since 2017.

Potions

The deployment platform built for Elixir: PaaS ease, VPS control.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement