Questions & Help>Questions

How to create an Explorer DataFrame from and Adbc Result? Is going via Adbc.Result.to_map the correct way?

#explorer#adbc
vegabook

vegabook

How to create an Explorer DataFrame from and Adbc Result? Is going via Adbc.Result.to_map the correct way?

I’m loving Adbc which is super fast! I’m creating Explorer DataFrames like this:

Adbc.Connection.query!(Suprabonds.AdbcConn, "Select * from trace") 
|> Adbc.Result.to_map 
|> Explorer.DataFrame.new

Just wondering if this is the best way? For large numbers of rows the to_map intermediate stage is taking quite a long time.

Does Explorer use Apache Arrow under the hood as well? If so can we go directly from an Adbc.Result to an Explorer.DataFrame without the to_map stage?

What’s the fastest way of getting from Adbc.Result to Explorer.DataFrame?

3 comments
#explorer
1 117 3

Marked As Solved

josevalim

josevalim

Creator of Elixir

Use Explorer.DataFrame.from_query!(Suprabonds.AdbcConn, "Select * from trace") instead. It will allocate the underling arrow data only once and pass it directly to Explorer.

1
Post #2

Where Next?

View thread on forum (has 3 responses!)
explorer
adbc
Home Questions & Help>Questions
#explorer#adbc
1 119 3
Last post

Popular in Questions Top

sergio_101
Questions & Help>Questions

Pattern matching against a string
I am VERY much an elixir newbie. I have taken one elixir course and one phoenix course on Udemy. During that course, I saw the instructor...
#pattern-matching
27 26685 14
New
jononomo
Questions & Help>Questions

Where / How does the Mix environment variable get set?
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
#mix
31 22791 9
New
lastday4you
Questions & Help>Questions

How to check Elixir version
I wanted to check elixir version in phoenix because i found that my elixir is 1.5 but when i use Enum.chunk_by it said the function is un...
#how-to-question
11 18647 3
New
makeitrein
Questions & Help>Questions

Extracting numbers from a string
Hey all, just started picking up Elixir last week and am writing a scraper as a learning project. Baby step #1 is extracting the number ...
#code-review#regex#troubleshooting
38 14381 20
New
chensan
Questions & Help>Questions

(Postgrex.Error) ERROR 42804 (datatype_mismatch): column "" cannot be cast automatically to type integer
I have a User schema with a :from_id field set to type :string: defmodule TweetBot.Repo.Migrations.CreateUsers do use Ecto.Migration ...
#ecto
29 13294 4
New
mathew4509
Questions & Help>Questions

Iterating through a list
I have a list say x = ["23gh", "56kh", "97mh"] I would like to pass each element to Val in each iteration. Say, in iteration 1 -------...
#book-programming-elixir#enumerables#enum#list
3 12090 5
New
gonzofish
Questions & Help>Questions

Ecto: how to join three tables?
I’m currently trying to understand how to join three tables using Ecto. All the examples I’ve seen use 2, so maybe I’m just missing somet...
/phoenix#ecto
21 13889 7
New
chrisalley
Questions & Help>Questions

How to describe many contexts in ExUnit without a hierarchy
ExUnit now has describe blocks which is a welcome addition coming from RSpec. In the docs, it states that nested hierarchies of describe ...
#testing#exunit
66 16773 4
New
belgoros
Questions & Help>Questions

Regex question for hyphen match
I’m not a pro in using Regex and can’t figure out why the following behaviour happens, especially if we take into account the difference ...
#learning-elixir
9 12939 10
New
9mm
Questions & Help>Questions

Elixir way to conditionally update a map
I am constructing a JSON object (map) and I need to conditionally set a field. I’m trying to write proper elixir-way code… and I’m at a l...
#maps
55 19905 29
New

Other popular topics Top

JakeBecker
Chat & Discussions>Discussions

ElixirLS - the Elixir Language Server
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
#elixir-ls
1140 51847 244
New
William
Learning Resources>Chat/questions

Resources for building an e-commerce store in Phoenix?
I would like to know that is there any online source for learning Phoenix Framework for building E-Commerce Store? Any advantage on build...
/phoenix#ecommerce
137 23142 64
New
albydarned
Questions & Help>Questions

Erlang and Elixir on Apple Silicon/M1 Chip
Hello all! I am typing this post from my new MacBook Pro with the M1 chip. I’m loving it so far, and will probably use it as my daily dr...
#erlang#troubleshooting
121 24599 65
New
openscript
Questions & Help>Questions

How to set up the Elixir SDK in Intellij IDEA with the intellij-elixir plugin?
Hello! Sorry for this astonishing simple question, but I’m really stuck. I try to set up the intellij-elixir plugin, but I don’t know ho...
17 22671 9
New
AstonJ
Learning Resources>Courses

Elixir for Programmers (PragDave)
You’re a programmer, so you don’t need spoon feeding with the conventional drivel about “this is an integer.” No. You need to know what’s...
#elixir-for-programmers-course
201 29468 78
New
malloryerik
Learning Resources>Guides/Tuts/Tips/Info

How to use 'phoenix-heex' templates in VSCode
Hi, this is for people who, like me, have had some friction using .html.heex templates in VSCode. The solution seems to be, in a hyphena...
/phoenix#vscode#heex
118 16932 34
New
jononomo
Questions & Help>Questions

Where / How does the Mix environment variable get set?
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
#mix
31 22791 9
New
chrismccord
News>Phoenix News

Phoenix v1.5.0 released!
This release brings a number of exciting features, including integration with the new Phoenix LiveDashboard and Phoenix LiveView. There h...
/phoenix#phoenix-release
379 28447 94
New
alice
Questions & Help>Questions

What are the main benefits of Elixir compared to Clojure?
Hey, Just curious what are the main benefits of Elixir compared to Clojure? When is Elixir more useful than Clojure and vice versa? Th...
#clojure
223 32979 47
New
electic
Questions & Help>Questions

DateTime Format to string?
Hi, I am new to Elixir. I am trying to use the DateTime component to insert a date into MySQL however the there seems to be no way to fo...
#datetime
19 24687 16
New
Back to top

Questions & Help>Questions

Questions Questions ❯

Latest on Elixir Forum

Elixir Forum ❯

Sponsor Spotlight

ElixirCasts

Producing high quality Elixir screencasts since 2017.

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Manning

Develop your skills with books, videos, and courses.

WyewWorks

Enabling companies to succeed by building software people love.

Potions

The deployment platform built for Elixir: PaaS ease, VPS control.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

Categories:

Sub Categories:

Forums

Our Sponsors

Dashbit

Build Elixir applications with speed and confidence.

EEF

Supporting innovation across the BEAM ecosystem.

Coders51

We build reliable cloud platforms for business-critical systems.

AppSignal

Catch errors, track performance, monitor hosts and more.

Manning

Develop your skills with books, videos, and courses.

WyewWorks

Enabling companies to succeed by building software people love.

Potions

The deployment platform built for Elixir: PaaS ease, VPS control.

Pragmatic Studio

Courses that'll move you from confusion to "Aha, now I get it!"

View all our sponsors ❯

We're in Beta

About us Mission Statement