My goal is to speed up performance of ASCII-only binary converting to lower case. I do not need any languages other than English. I've wrote and compare some variants:
Binary comprehension:
binary_comprehension(Binary) ->
<< <<if
C >= $A andalso C =< $Z -> C - $A + $a;
true -> C
end >>
|| <<C>> <= Binary >>.
List comprehension:
list_comprehension(Binary) ->
L = binary_to_list(Binary),
Lower =
[if
C >= $A andalso C =< $Z -> C - $A + $a;
true -> C
end || C <- L],
list_to_binary(Lower).
And regular string:lowercase.
And surprisingly list comprehension beat all others:
1> timer:tc(fun() -> lists:foreach(fun(_) -> tolower:list_comprehension(<<"QWEQWEIQEKQHWKEHKQWHEKQHWKEQWEKHQWLKL">>) end, L100000) end).
{267603,ok}
2> timer:tc(fun() -> lists:foreach(fun(_) -> tolower:binary_comprehension(<<"QWEQWEIQEKQHWKEHKQWHEKQHWKEQWEKHQWLKL">>) end, L100000) end).
{324383,ok}
3> timer:tc(fun() -> lists:foreach(fun(_) -> string:lowercase(<<"QWEQWEIQEKQHWKEHKQWHEKQHWKEQWEKHQWLKL">>) end, L100000) end).
{319819,ok}
Any ideas why double list conversion + comprehension is much faster than just binary transformation?
Maybe you know more powerful optimisation?
Update:
I also found that list-of-char version of string is also fast:
string_lowercase(Binary) ->
L = binary_to_list(Binary),
Lower = string:lowercase(L),
list_to_binary(Lower).
Run:
39> timer:tc(fun() -> lists:foreach(fun(_) -> tolower:string_to_lower(<<"QWEQWEIQEKQHWKEHKQWHEKQHWKEQWEKHQWLKL">>) end, L100000) end).
{277766,ok}
I made some modification of the code and changed test case. Test changes is not mandatory but I personally like more this way:
-module(tolower).
-compile(export_all).
u2l(C) when C >= $A andalso C =< $Z -> C + 32;
u2l(C) -> C.
binary_comprehension(Binary) ->
<< << (u2l(C)) >> || <<C>> <= Binary >>.
list_comprehension(Binary) ->
list_to_binary([u2l(C) || C <- binary_to_list(Binary)]).
list_recur(Binary) -> list_recur(binary_to_list(Binary), []).
list_recur([], Result) -> lists:reverse(Result);
list_recur([C | Tail], Result) when C >= $A andalso C =< $Z ->
list_recur(Tail, [(C + 32) | Result]);
list_recur([C | Tail], Result) ->
list_recur(Tail, [C | Result]).
string_to_lower(Binary) ->
list_to_binary(string:lowercase(binary_to_list(Binary))).
test() ->
L100000 = lists:seq(1, 100000),
TL0 = <<"QWEQWEIQEKQHWKEHKQWHEKQHWKEQWEKHQWLKL">>,
TL = binary:copy(TL0, 100000),
{R0, _} = timer:tc(fun() -> lists:foreach(fun(_) -> tolower:binary_comprehension(TL0) end, L100000) end),
{R1, _} = timer:tc(tolower, binary_comprehension, [TL]),
{R2, _} = timer:tc(tolower, list_comprehension, [TL]),
{R3, _} = timer:tc(tolower, list_recur, [TL]),
{R4, _} = timer:tc(string, lowercase, [TL]),
{R5, _} = timer:tc(tolower, string_to_lower, [TL]),
io:format("~n1.binary_comprehension = ~10w~n2.binary_comprehension = ~10w~n3. list_comprehension = ~10w~n4. list_recur = ~10w~n5. lowercase = ~10w~n6. string_to_lower = ~10w~n",
[R0,R1,R2,R3,R4,R5]).
Erlang shell shows that elapsed time not consistent due to concurrent nature of system. But the best time is for binary_comprehension as expected.
62> c(tolower).
tolower.erl:2: Warning: export_all flag enabled - all functions will be exported
{ok,tolower}
63> l(tolower).
{module,tolower}
64> tolower:test().
1.binary_comprehension = 109000
2.binary_comprehension = 94000
3. list_comprehension = 312001
4. list_recur = 344001
5. lowercase = 469002
6. string_to_lower = 218000
ok
65> tolower:test().
1.binary_comprehension = 140998
2.binary_comprehension = 93999
3. list_comprehension = 327994
4. list_recur = 296996
5. lowercase = 155997
6. string_to_lower = 280996
ok
66> tolower:test().
1.binary_comprehension = 124998
2.binary_comprehension = 93998
3. list_comprehension = 327995
4. list_recur = 296995
5. lowercase = 452993
6. string_to_lower = 202997
ok
67> tolower:test().
1.binary_comprehension = 125000
2.binary_comprehension = 94000
3. list_comprehension = 312000
4. list_recur = 282000
5. lowercase = 171000
6. string_to_lower = 266000
ok
Time on line 5 is different of time on line 6 because when you invoke string:lowercase/1 with binary argument it processed as utf8 sequence. When you invoke string:lowercase/1 with string argument utf8 processing is avoided. See code of string.erl from OTP for details.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With