Ratelimit misbehavior

Eduardo_Schoedler · May 4, 2016, 11:39pm

Hi,

Some unbounds are misbehaving here

We are under attack, as you can see:

# ./top-domain.sh
  41285 amazonaws.com
  41919 googleapis.com
  42303 fbcdn.net
  59289 root-servers.net
  61485 whatsapp.net
  75474 akamai.net
167995 facebook.com
181346 google.com
208779 akamaihd.net
384725 315ye.zj.cn

No doubt, when looking into the log, there is a lot of queries going on:

# tail -n 1000 unbound.log | grep '315ye.zj.cn'
[1462404661] unbound[32679:7] info: x.x.x.x heghnlxhdjd.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x rgdtbdcpire.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x gtkvcringtidqt.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x ashcjctstvm.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x knypkraxsfcfob.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x irmfuzchub.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x evajqb.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x mlshiz.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x kbedwjgrgb.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x ihkdutetkx.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x cbwhmlolylyfozgt.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x yjinkfudktsj.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x ynmlktcdit.315ye.zj.cn. A IN
[1462404661] unbound[32679:4] info: x.x.x.x ir.315ye.zj.cn. A IN
[1462404661] unbound[32679:5] info: x.x.x.x qnefwbmhktat.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x qanthkajjdo.315ye.zj.cn. A IN
[1462404661] unbound[32679:4] info: x.x.x.x sh.315ye.zj.cn. A IN
[1462404661] unbound[32679:4] info: x.x.x.x mhgxitap.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x mtwbmvetidkbov.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x mfezktetldm.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x fldrduebgre.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x ifgb.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x dhbsbrqlatz.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x gzebilwhgl.315ye.zj.cn. A IN
[1462404661] unbound[32679:5] info: x.x.x.x mlktchobab.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x qnehgfwt.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x ercpmnmlovwr.315ye.zj.cn. A IN
[1462404661] unbound[32679:5] info: x.x.x.x ufevershyvmv.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x yvorcxanet.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x qxafih.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x arqfmjcrev.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x wncrmvohmnclqbux.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x ypejmtmhklkhex.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x qlyzyh.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x ijovajovwzqzcv.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x ejwxojsdulmd.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x mxohwbid.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x ed.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x ipuzehgfuf.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x ct.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x alkbcp.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x ilunwruryl.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x sdedkj.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x exkbsf.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x kxexyl.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x wtqvafcfcpynotkl.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x wvmnijebmnwb.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x czkvahipspyzypsp.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x atqvqxwruderuj.315ye.zj.cn. A IN
[1462404661] unbound[32679:4] info: x.x.x.x ozstgdcped.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x alozon.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x afaxcdyfipavynix.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x cfereredizmfktcd.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x qj.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x jgdgywedmhy.315ye.zj.cn. A IN
[1462404661] unbound[32679:5] info: x.x.x.x gpmputgd.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x mlyhot.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x cn.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x cjaxojix.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x qvopovizkxkn.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x gb.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x qpilofyjgzqt.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x mfczcfwletylip.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x crmbyjejcvojan.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x uxijabmhenmp.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x cfqn.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x qnehgfwt.315ye.zj.cn. A IN
[1462404661] unbound[32679:5] info: x.x.x.x ypkpstinkjcpwn.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x wpqpalip.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x ojojexgxuxkp.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x afwx.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x ydet.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x ubgnehytstyn.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x cxszorejad.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x qdwnunyd.315ye.zj.cn. A IN
[1462404661] unbound[32679:5] info: x.x.x.x yxonmpulmlgv.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x pfexcmznbub.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x wvcxwxovahkpkj.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x iz.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x ijcvgrifcfif.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x sfat.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x ozwr.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x ad.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x sd.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x sxipsnuvwlybcxmp.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x mfif.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x exkpinodon.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x uv.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x wvsfyxklelcfmh.315ye.zj.cn. A IN
[1462404661] unbound[32679:7] info: x.x.x.x izklyhmrqfotoh.315ye.zj.cn. A IN
[1462404661] unbound[32679:5] info: x.x.x.x gdofqrit.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x yhgpexsn.315ye.zj.cn. A IN
[1462404661] unbound[32679:1] info: x.x.x.x gzwd.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x yxofqbuxchapwdqx.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x ghwvulohmjkzgx.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x wzuxsxivyhqd.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x ofir.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x mp.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x glsbwl.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x irsxmpkrqlsp.315ye.zj.cn. A IN
[1462404661] unbound[32679:4] info: x.x.x.x wlgjslobkd.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x ijmhqfixqxaxut.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x edwjqd.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x op.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x qbqj.315ye.zj.cn. A IN
[1462404661] unbound[32679:4] info: x.x.x.x yvytefatgj.315ye.zj.cn. A IN
[1462404661] unbound[32679:5] info: x.x.x.x ulinkd.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x ytmrmh.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x yjinkfudktsj.315ye.zj.cn. A IN
[1462404661] unbound[32679:6] info: x.x.x.x gt.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x ihof.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x mnytsripyrqv.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x aruzihur.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x urslyxcdmdmnmd.315ye.zj.cn. A IN
[1462404661] unbound[32679:3] info: x.x.x.x sdirknkz.315ye.zj.cn. A IN
[1462404661] unbound[32679:0] info: x.x.x.x mzodkloruz.315ye.zj.cn. A IN
[1462404661] unbound[32679:2] info: x.x.x.x mrahej.315ye.zj.cn. A IN

But unbound-control told me there is no attack:

# unbound-control ratelimit_list +a | grep '315ye.zj.cn'
315ye.zj.cn. 0 limit 25

My unbound is:
Version 1.5.8
linked libs: libevent 2.0.16-stable (it uses epoll), OpenSSL 1.0.1 14 Mar 2012
linked modules: dns64 validator iterator
BSD licensed, see LICENSE in source package for details.
Report bugs to unbound-bugs@nlnetlabs.nl

My system:

# cat /etc/issue
Ubuntu 12.04.5 LTS \n \l

# uname -a
Linux dns 3.13.0-68-generic #111~precise1-Ubuntu SMP Fri Nov 6
18:17:31 UTC 2015 x86_64 x86_64 x86_64 GNU/Linux

What can be the problem?

Thank you.

Best regards,

hdais · May 5, 2016, 11:44pm

Hi, Eduardo:

It seems that all nameservers of "315ye.zj.cn" (ns1.22.cn, ns2.22.cn)
are completely down and no response; In Unbound "infra" database all
NS of "315ye.zj.cn"
should be marked as "rto 120000", which means "not responsible".

$ unbound-control dump_infra | grep 315ye.zj.cn
121.12.104.72 315ye.zj.cn. ttl 4 ping 0 var 94 rtt 376 rto 120000 tA 3
tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0
other 0
121.12.104.73 315ye.zj.cn. ttl 0 ping 0 var 94 rtt 376 rto 120000 tA 3
tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0
other 0
218.66.171.136 315ye.zj.cn. ttl 6 ping 0 var 94 rtt 376 rto 120000 tA
3 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0
other 0
218.66.171.137 315ye.zj.cn. ttl 2 ping 0 var 94 rtt 376 rto 120000 tA
3 tAAAA 0 tother 0 ednsknown 0 edns 0 delay 0 lame dnssec 0 rec 0 A 0
other 0

In this case Unbound stops resolving names under the zone (returns
SERVFAIL for user queries) for a while.

Unbound's "ratelimit" feature ratelimits number of queries from
Unbound to nameservers,
not from user to Unbound. So my guess is: Unbound should already had
stopped resolving
"315ye.zj.cn" because all the NSs are down, so its "ratelimit" feature
no longer detect
excessive queries to "315ye.zj.cn" nameservers.

Regards,

Eduardo_Schoedler · May 6, 2016, 12:47am

Hi Daisuke,

Thank you for the response.

This same behaviour is occurring to all domains that has being attacked. Do you think is the same reason (nameservers tango down)?

Regards,