shotleybuilder

shotleybuilder

Unicode for emoji - string concatenation

Hi,

I’m not understanding the behaviour when including unicode inside a string which is then string concatenated. The resulting string is not known when streamed into a file.

Here’s something that works:
"❔ foobar" -> "❔ foobar"
This works
"\u2754 foobar" -> "❔ foobar"

This doesn’t work:
"❔" <> "foobar" -> �
Neither does this
"\u2754" <> "foobar" -> �

What am I missing?

Most Liked

NobbZ

NobbZ

+U274c is represented by these 3 bytes in utf8: 0xE2 0x9D 0x8C.

If your terminal doesn’t support utf8, then you should probably inspect on a byte level rather than printed representation.

As you have not specified your terminals encoding, basically everything can be triggered by these bytes.

Assuming latin-X, which is often used in the windows world, then the 8x and 9x bytes are not used and can cause weirdnesses.

kip

kip

ex_cldr Core Team
iex> << 0x1F517 :: utf8 >>
"🔗"

iex> << 0xF0, 0x9F, 0x94, 0x97 >>
"🔗"

iex> << 0xF0, 0x9F, 0x94, 0x97>> == << 0x1f517 :: utf8 >>
true

d8 3d dd 17 is the UTF16 representation. Elixir is a UTF8 language where the encoding is f0 9f 94 97.

kip

kip

ex_cldr Core Team

With a little bit of erlang magic:

iex> :erlang.binary_to_list << 0x1f517 :: utf8 >>       
[240, 159, 148, 151]

I have a unicode library that examines code points. For example:

iex> Unicode.category << 0x1F49A :: utf8 >>             
[:So]
iex> Unicode.properties << 0x1F49A :: utf8 >>                                                                          
[[:emoji, :grapheme_base]]

Unfortunately the Regex module doesn’t support the Unicode character class [:So:]. Or any other Unicode character classes. I have another lib unicode_set that has some support for Unicode character classes. But regexes not yet - soon though.

LostKobrakai

LostKobrakai

iex(2)> IEx.configure inspect: [base: :hex]
:ok
iex(3)> << 0x1f517 :: utf8 >>
<<0xF0, 0x9F, 0x94, 0x97>>

Where Next?

Popular in Questions Top

senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
Brian
What is the proper way to load a module from a file in to IEX? In the python world, doing something like this pretty standard: from ....
New
sergio
In Ruby, I can go: User.find_by(email: "foobar@email.com").update(email: "hello@email.com") How can I do something similar in Elixir? ...
New
dotdotdotPaul
Okay, I'm having a heck of a time trying to figure out how to best handle the validation of belongs_to associations in Ecto. I'm sure I'...
New
itssasanka
Hi all, Trying to get some more clarity over utc_datetime and naive_datetime for Ecto: https://hexdocs.pm/ecto/Ecto.Schema.html#module-...
New
ycv005
I have followed this StackOverflow post to install the specific version of Erlang. And When I am running mix ecto.setup then getting fol...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New
Kagamiiiii
Student &amp; New to elixir. Nice language. I want to convert a english character, e.g. “a”, which is stored in a variable, to it’s asci...
New
lk-geimfari
What is most correct way to open, read and parse JSON file with poison? For example if we have example.json file in root of some projec...
New
Mooodi
Given a string, how can I get access to its character by index? Enum.at("my_string", 2) doesn't work. Or rather, not char, but a substr...
New

Other popular topics Top

minhajuddin
I have seen a lot of code which picks the first element from a list using Enum.at(0) instead of List.first. Is there a reason why people ...
New
josevalim
Hi everyone, One of the features added to Elixir early on to help integration with Erlang code was the idea of overridable function defi...
New
Patoshizzle
After calling mix ecto.create I get this error: 17:00:32.162 [error] GenServer #PID&lt;0.412.0&gt; terminating ** (Postgrex.Error) FATAL...
New
JakeBecker
TL;DR: I’ve just released an implementation of Microsoft’s IDE-independent Language Server Protocol for Elixir. It adds language support ...
1140 51847 244
New
peerreynders
Manning 2016 Halloween weekend sale via Deal of the Day Friday, October 28 - Half off all MEAPs - code WM102816LT Saturday, October 29 ...
326 29600 154
New
senggen
Erlang/OTP 25 [erts-13.2.2] [source] [64-bit] [smp:8:8] [ds:8:8:10] [async-threads:1] 15:22:35.803 [error] gen_event {lager_file_backend...
New
jononomo
I am trying to figure out how Mix knows whether the environment is test, dev, or prod -- where is this set? Thanks.
New
pmjoe
I have a relationship of love and hate with Elixir. Lots of things are just absolutely right, but there are some things that are kind of ...
New
lessless
I believe there are people here who are dealing with CSV files import on the daily basis, and since Excel is a really popular tool there ...
New
romenigld
I am trying to run a deploy with docker and I successfully runned with this command: docker build -t romenigld/blog-prod . but when I t...
New

We're in Beta

About us Mission Statement