Creative Commons License

Unless otherwise expressly stated, all original material of whatever nature created by and included in this weblog is licensed under a Creative Commons License.

Make grep 50x faster, 15.12.2013 14:33

Found this neat trick in Brendan Gregg's Blazing Performance with Flame Graphs talk.

Switching to LANG=C improved performance by 2000x

In a quick test I directly got a performance gain of factor 50.22.
This is quite an achievement for only changing one environment variable.

real:~# du -sh /var/log/querylog 
148M	/var/log/querylog
real:~# time grep -i e /var/log/querylog > /dev/null 

real	0m12.807s
user	0m12.437s
sys	0m0.068s
real:~# time LANG=C grep -i e /var/log/querylog > /dev/null

real	0m0.255s
user	0m0.196s
sys	0m0.052s

I suspect that the performance gain may vary quite a lot depending on the search pattern. Also, please note that this trick only works when you know that the involved files and search patterns are ASCII only.

(via Standalone Sysadmin)

blog comments powered by Disqus