← Back to team overview

rohc team mailing list archive

Re: Oops in arm kernel 4.4.32 on d_tcp_parse_packet

 

Yakir,

This is a known problem. I discovered it while fuzzing the decompressor
last month. I cherry-picked it for you on the 2.1.x bugfix branch along
with some other fixes.

Please give a try to the 2.1.x branch:
https://github.com/didier-barvaux/rohc/commits/2.1.x

All those fixes should be soon released in a future 2.1.1 release.

Regards,
Didier


Le Thu, 22 Mar 2018 03:51:38 +0000,
Yakir Matusovsky <yakir.matusovsky@xxxxxxxxxxx> a écrit :

> Following my concern here, I’ve continued to debug the ko and got the
> following to protect from oops,
> 
> [ decomp/d_tcp.c:1009 d_tcp_parse_ir_cr()]
> context->decompressor->contexts[base_cid] == NULL
> 
> Means, I’ve added a red code below in d_tcp.c,
> 
>             /* check whether the decoded base CID is allowed by the
> decompressor */ if(base_cid > context->decompressor->medium.max_cid)
>             {
>                         rohc_decomp_warn(context, "unexpected Base
> CID %zu received: MAX_CID " "was set to %zu", base_cid,
>                                          context->decompressor->medium.max_cid);
>                         goto error;
>             }
> 
>            if(context->decompressor->contexts[base_cid] == NULL)
>             {
>                          rohc_decomp_warn(context,
> "context->decompressor->contexts[base_cid] == NULL\n"); goto error;
>             }
> 
>             base_context = context->decompressor->contexts[base_cid];
> 
> Please advise the best way forward…
> 
> Regards,
> Yakir
> From: Rohc
> <rohc-bounces+yakir.matusovsky=mimomax.com@xxxxxxxxxxxxxxxxxxx> on
> behalf of Yakir Matusovsky <yakir.matusovsky@xxxxxxxxxxx> Date:
> Thursday, 22 March 2018 at 4:19 PM To: "rohc@xxxxxxxxxxxxxxxxxxx"
> <rohc@xxxxxxxxxxxxxxxxxxx> Subject: [Rohc] Oops in arm kernel 4.4.32
> on d_tcp_parse_packet
> 
> Hi
> 
> I’ve experienced the problem below a few times already and want to
> get to the bottom of it. I have a data link and I initiate a remote
> SSH session (to the node over a link). I get this crash (my kernel
> panics on oops),
> 
> > [  458.076694] Internal error: Oops: 17 [#1] ARM  
> [  458.081321] Modules linked in: mdl_driver(O) rohc(O) fpga_driver(O)
> [  458.087958] CPU: 0 PID: 505 Comm: kworker/0:2 Tainted: G
> O    4.4.32 #3 [  458.095859] Hardware name: Generic AM33XX
> (Flattened Device Tree) [  458.102485] Workqueue: ReceiveFramesQueue
> mac__frame_rx_task [mdl_driver] [  458.109651] task: cc729100 ti:
> cc552000 task.ti: cc552000 [  458.115691] PC is at
> d_tcp_parse_packet+0x64c/0x1194 [rohc] [  458.121729] LR is at
> d_tcp_parse_packet+0x5e0/0x1194 [rohc] [  458.127608] pc :
> [<bf077c34>]    lr : [<bf077bc8>]    psr: 80000013 [  458.127608]
> sp : cc553940  ip : cc553940  fp : cc5539b4 [  458.139709] r10:
> 00000000  r9 : 00000029  r8 : cc6a25e7 [  458.145215] r7 : cc6a25e2
> r6 : 00000002  r5 : bf0ce8b4  r4 : cb175800 [  458.152100] r3 :
> cc558080  r2 : cc580000  r1 : 00000000  r0 : 00000000 [  458.158992]
> Flags: Nzcv  IRQs on  FIQs on  Mode SVC_32  ISA ARM  Segment none
> [  458.166513] Control: 10c5387d  Table: 8b07c019  DAC: 00000051
> [  458.172567] Process kworker/0:2 (pid: 505, stack limit =
> 0xcc552208) [  458.179252] Stack: (0xcc553940 to 0xcc554000)
> [  458.183851] 3940: bf0a1b32 bf09fa89 000003e1 bf07e554 00000002
> 00000008 c0097ac4 c00970c0 [  458.192487] 3960: cb0f8000 cb175810
> cb175800 cb17580c cc55398c cc55398c cc5539bc cc553988 [  458.201109]
> 3980: bf072ce4 bf0ce8c0 bf09fcac cc553b6c bf0775e8 cb175800 bf0ce8b4
> cc553b98 [  458.209748] 39a0: cc580000 bf07e6c0 cc553ae4 cc5539c0
> bf054be8 bf0775f8 3880cb87 00000000 [  458.218374] 39c0: 08c27650
> 00000000 cc6a25dd 00000033 00000005 0000002e 00000000 cc553b98
> [  458.227008] 39e0: cb175810 cb178000 cc553a40 c00470dc c0056458
> c002f5b0 c06248c0 20070013 [  458.235638] 3a00: cb175810 cc552000
> cc3714a4 cc552000 cc553b88 00000020 cb178000 cb168000 [  458.244267]
> 3a20: 00000006 00000005 bf07e6c0 00000001 00000001 cc6a25dd 00000000
> 0000002e [  458.252905] 3a40: c03f2ba4 c03f0b20 3880cb87 00000000
> 08c27650 00000000 cc139154 00000001 [  458.261541] 3a60: cb0e46c0
> cb0e46c0 cc139154 7fffffff cc3714a4 cc552000 3880cb87 00000000
> [  458.270174] 3a80: 08c27650 00000000 cc6a25dd 00000033 00000005
> 0000002e 3880cb87 00000000 [  458.278805] 3aa0: 08c27650 00000000
> cc6a25dd 00000033 00000005 0000002e 000047f1 cc580000 [  458.287441]
> 3ac0: cc553c98 cc590000 cc553c58 00000033 cc553c78 cc553bd0 cc553bcc
> cc553af0 [  458.296072] 3ae0: bf055fc0 bf0532e0 3880cb87 00000000
> 08c27650 00000000 cc6a25dd 00000033 [  458.304703] 3b00: 00000000
> 00000033 cc553c58 cc553c78 cc553b6c c05c5738 00000000 a0000093
> [  458.313333] 3b20: 00000000 cc339400 cc339400 cc553c8c cc553b54
> cc553b40 c0047b50 c004c3ec [  458.321967] 3b40: 00000000 a0000013
> 00000000 cc553cc0 60000013 cc339400 cc339400 cc553c8c [  458.330606]
> 3b60: cc553b84 cc553b70 c004ee00 00000001 00000001 00000008 cc553b01
> cb175800 [  458.339236] 3b80: 00000006 00000001 cc339400 00000001
> 00000000 00000000 00000020 c004f300 [  458.347875] 3ba0: cc553bd4
> cc6a0000 cc6a25dd 0000000c cc580000 cc5c0000 cc6a0498 00000000
> [  458.356507] 3bc0: cc553ce4 cc553bd8 bf0d02c4 bf055c5c 3880cb87
> 00000000 08c27650 00000000 [  458.365148] 3be0: cc6a25dd 00000033
> 00000000 00000033 cc553c58 cc553c78 cc553c98 cc339400 [  458.373784]
> 3c00: 00000000 00023e12 3880cb87 00000640 cc6a0ad8 00000000 cc6a25dd
> 0000000c [  458.382413] 3c20: 00000000 0000000e 00000001 0000000c
> 00000001 00000000 3880cb87 00000000 [  458.391049] 3c40: 08c27650
> 00000000 cc6a25dd 00000033 00000000 00000033 00000000 00000000
> [  458.399682] 3c60: 00000000 00000000 cc6a04a6 00000632 00000000
> 00000000 00000000 00000000 [  458.408307] 3c80: 00000000 00000000
> cc6a0ad8 000001f4 00000000 00000000 00000000 00000000 [  458.416946]
> 3ca0: 00000000 00000000 cc6a0cdb 000001e5 00000000 00000000 c00649f0
> cc6a0000 [  458.425581] 3cc0: bf0db600 cc6a2480 cc6a0480 cc6a25dd
> 00000001 cc6a0f90 cc553d3c cc553ce8 [  458.434213] 3ce0: bf0c9d88
> bf0cfbe8 cc6a2c1e 00000640 cc553d08 00000000 00000008 00000000
> [  458.442848] 3d00: ab7a1aa8 ab7a1aa8 3880cb87 00023e12 cc553d3c
> cc6a0000 bf0db600 cc6a15da [  458.451479] 3d20: cc6a2c90 bf0d7f9b
> cc6a2000 bf0d8387 cc553d9c cc553d40 bf0ca5dc bf0c9b80 [  458.460104]
> 3d40: 00000000 00000000 00000000 00000000 00000000 ab000000 00000000
> 00010037 [  458.468744] 3d60: 00000037 00000037 00000000 00000000
> 00000000 bf0df358 00000001 00000038 [  458.477368] 3d80: 00000000
> 00000000 bf0df77c 00000644 cc553dcc cc553da0 bf0d64f4 bf0c9fac
> [  458.486009] 3da0: 00000000 00ab0000 00000000 bf0df77c 00000646
> 00000000 00000000 00000646 [  458.494639] 3dc0: cc553e0c cc553dd0
> bf0d666c bf0d6348 00002010 000f4240 bf0db640 bf0df77c [  458.503271]
> 3de0: 00000000 00000001 00000000 bf0deba6 00000037 00000000 00000000
> 00000037 [  458.511902] 3e00: cc553e54 cc553e10 bf0d6984 bf0d6528
> 00000000 bf0deba6 00000037 00000000 [  458.520536] 3e20: 8f489f0c
> 00000000 cc553e98 bf0deb9e 00000037 00000002 bf0df140 bf0db640
> [  458.529167] 3e40: 00000000 00000008 cc553e84 cc553e58 bf0d7a50
> bf0d6904 00000037 00000000 [  458.537791] 3e60: bf0deba6 00000037
> 08c1d53a bf0deb9e 0000003f bf0debdd cc553ec4 cc553e88 [  458.546423]
> 3e80: bf0d3d80 bf0d79c0 00000044 ccd7f700 cc553eac cc6f3200 3880cb87
> 00023de8 [  458.555058] 3ea0: 00000000 00000000 bf0deb9e 00000044
> bf0df140 bf0db640 cc553eec cc553ec8 [  458.563693] 3ec0: bf0d3eec
> bf0d3bd8 cc6f3200 bf0de338 c05c4f8c 00000000 ccd7f700 00000000
> [  458.572329] 3ee0: cc553f2c cc553ef0 c003eb64 bf0d3dc0 c05c4f8c
> c05c4f8c c05c70c0 c05c4f9c [  458.580963] 3f00: cc6f3218 cc6f3200
> c05c4f8c c05c4f8c c05c70c0 c05c4f9c cc6f3218 00000008 [  458.589596]
> 3f20: cc553f64 cc553f30 c003f694 c003e9b4 c003f3e0 00000000 00000000
> cc6f0600 [  458.598224] 3f40: 00000000 cc6f3200 c003f3e0 00000000
> 00000000 00000000 cc553fac cc553f68 [  458.606860] 3f60: c0043a8c
> c003f3ec cc6f0600 00000000 00000000 cc6f3200 00000000 cc553f7c
> [  458.615498] 3f80: cc553f7c 00000000 cc553f88 cc553f88 cc6f0600
> c00439b8 00000000 00000000 [  458.624133] 3fa0: 00000000 cc553fb0
> c000efb8 c00439c4 00000000 00000000 00000000 00000000 [  458.632765]
> 3fc0: 00000000 00000000 00000000 00000000 00000000 00000000 00000000
> 00000000 [  458.641397] 3fe0: 00000000 00000000 00000000 00000000
> 00000013 00000000 625f4449 735f7572 [  458.650014] Backtrace:
> [  458.652881] [<bf0775ec>] (d_tcp_parse_packet [rohc]) from
> [<bf054be8>] (d_decode_header+0x1918/0x2854 [rohc]) [  458.663336]
> r10:bf07e6c0 r9:cc580000 r8:cc553b98 r7:bf0ce8b4 r6:cb175800
> r5:bf0775e8 [  458.671617]  r4:cc553b6c [  458.674556] [<bf0532d4>]
> (d_decode_header [rohc]) from [<bf055fc0>]
> (rohc_decompress3+0x374/0x18dc [rohc]) [  458.684747]  r10:cc553bd0
> r9:cc553c78 r8:00000033 r7:cc553c58 r6:cc590000 r5:cc553c98
> [  458.693027]  r4:cc580000 [  458.695946] [<bf055c50>]
> (rohc_decompress3 [rohc]) from [<bf0d02c4>]
> (rohc_cms_decompress+0x6e8/0xd10 [mdl_driver]) [  458.706943]
> r10:00000000 r9:cc6a0498 r8:cc5c0000 r7:cc580000 r6:0000000c
> r5:cc6a25dd [  458.715221]  r4:cc6a0000 [  458.718051] [<bf0cfbdc>]
> (rohc_cms_decompress [mdl_driver]) from [<bf0c9d88>]
> (rx_decompress+0x214/0x3dc [mdl_driver]) [  458.729327]  r10:cc6a0f90
> r9:00000001 r8:cc6a25dd r7:cc6a0480 r6:cc6a2480 r5:bf0db600
> [  458.737603]  r4:cc6a0000 [  458.740426] [<bf0c9b74>]
> (rx_decompress [mdl_driver]) from [<bf0ca5dc>]
> (mdl_driver_rx_tasklet+0x63c/0x6ac [mdl_driver]) [  458.751895]
> r10:bf0d8387 r9:cc6a2000 r8:bf0d7f9b r7:cc6a2c90 r6:cc6a15da
> r5:bf0db600 [  458.760177]  r4:cc6a0000 [  458.763005] [<bf0c9fa0>]
> (mdl_driver_rx_tasklet [mdl_driver]) from [<bf0d64f4>]
> (mac__enqueue_data+0x1b8/0x1e0 [mdl_driver]) [  458.774834]
> r10:00000644 r9:bf0df77c r8:00000000 r7:00000000 r6:00000038
> r5:00000001 [  458.783116]  r4:bf0df358 [  458.785944] [<bf0d633c>]
> (mac__enqueue_data [mdl_driver]) from [<bf0d666c>]
> (mac_frag__newcue+0x150/0x1e4 [mdl_driver]) [  458.797321]
> r9:00000646 r8:00000000 r7:00000000 r6:00000646 r5:bf0df77c
> r4:00000000 [  458.805672] [<bf0d651c>] (mac_frag__newcue
> [mdl_driver]) from [<bf0d6984>] (mac_frag__process_frame+0x8c/0x11c
> [mdl_driver]) [  458.817499]  r10:00000037 r9:00000000 r8:00000000
> r7:00000037 r6:bf0deba6 r5:00000000 [  458.825781]  r4:00000001
> [  458.828618] [<bf0d68f8>] (mac_frag__process_frame [mdl_driver])
> from [<bf0d7a50>] (extract_data_frame+0x9c/0xb4 [mdl_driver])
> [  458.840529]  r10:00000008 r9:00000000 r8:bf0db640 r7:bf0df140
> r6:00000002 r5:00000037 [  458.848812]  r4:bf0deb9e [  458.851650]
> [<bf0d79b4>] (extract_data_frame [mdl_driver]) from [<bf0d3d80>]
> (rx_one_frame+0x1b4/0x1e8 [mdl_driver]) [  458.862740]  r6:bf0debdd
> r5:0000003f r4:bf0deb9e [  458.867776] [<bf0d3bcc>] (rx_one_frame
> [mdl_driver]) from [<bf0d3eec>] (mac__frame_rx_task+0x138/0x17c
> [mdl_driver]) [  458.878875]  r8:bf0db640 r7:bf0df140 r6:00000044
> r5:bf0deb9e r4:00000000 [  458.886052] [<bf0d3db4>]
> (mac__frame_rx_task [mdl_driver]) from [<c003eb64>]
> (process_one_work+0x1bc/0x2ec) [  458.896326]  r9:00000000
> r8:ccd7f700 r7:00000000 r6:c05c4f8c r5:bf0de338 r4:cc6f3200
> [  458.904529] [<c003e9a8>] (process_one_work) from [<c003f694>]
> (worker_thread+0x2b4/0x3f0) [  458.913157]  r10:00000008 r9:cc6f3218
> r8:c05c4f9c r7:c05c70c0 r6:c05c4f8c r5:c05c4f8c [  458.921443]
> r4:cc6f3200 [  458.924132] [<c003f3e0>] (worker_thread) from
> [<c0043a8c>] (kthread+0xd4/0xec) [  458.931746]  r10:00000000
> r9:00000000 r8:00000000 r7:c003f3e0 r6:cc6f3200 r5:00000000
> [  458.940029]  r4:cc6f0600 [  458.942719] [<c00439b8>] (kthread)
> from [<c000efb8>] (ret_from_fork+0x14/0x3c) [  458.950346]
> r7:00000000 r6:00000000 r5:c00439b8 r4:cc6f0600 [  458.956337] Code:
> e12fff35 ea000097 e5923018 e793a106 (e59a3008) [  458.962893]
> ---[ end trace f6e6f365816638a5 ]--- [  458.967792] Kernel panic -
> not syncing: Fatal exception in interrupt [  458.974494] Rebooting in
> 30 seconds..
> 
> Traces prior to crash show the following
> 
> [  485.966625] [ decomp/rohc_decomp.c:770 rohc_decompress3()]
> decompress the 51-byte packet #1
> 
> [  485.982491] [ decomp/rohc_decomp.c:3792
> rohc_decomp_parse_padding()] skip 4 byte(s) of padding
> 
> [  485.998560] [ decomp/rohc_decomp.c:1017 d_decode_header()]
> decompressor received 0 bytes of feedback for the same-side
> associated compressor
> 
> [  486.018811] [ decomp/rohc_decomp.c:3728 rohc_decomp_decode_cid()]
> add-CID present (0xe9) contains CID = 9
> 
> [  486.035899] [ decomp/rohc_decomp.c:3862
> rohc_decomp_find_context()] ROHC packet is an IR, IR-CR or IR-DYN
> packet
> 
> [  486.053606] [ decomp/rohc_decomp.c:3885
> rohc_decomp_find_context()] profile ID 0x0006 found in IR(-CR|-DYN)
> packet
> 
> [  486.071494] [ decomp/rohc_decomp.c:3901
> rohc_decomp_find_context()] context with CID 9 not found
> 
> [  486.087742] [ decomp/rohc_decomp.c:3964
> rohc_decomp_find_context()] create new context with CID 9 and profile
> 'IP/TCP' (0x0006)
> 
> [  486.106872] [ decomp/rohc_decomp.c:1162 d_decode_header()] decode
> packet with profile 'IP/TCP' (0x0006)
> 
> [  486.123763] [ decomp/d_tcp.c:596 tcp_detect_packet_type()] try to
> determine the header from first byte 0xfc
> 
> [  486.141014] [ decomp/rohc_decomp.c:1194 d_decode_header()] decode
> packet as 'IR-CR'
> 
> [  486.156079] [ decomp/rohc_decomp.c:1367 rohc_decomp_decode_pkt()]
> parse packet type 'IR-CR' (32)
> 
> [  486.172331] [ decomp/d_tcp.c:748 d_tcp_parse_packet()] rohc_length
> = 46, large_cid_len = 0
> 
> [  486.188024] [ decomp/d_tcp.c:937 d_tcp_parse_ir_cr()] B = 1 =>
> Base CID is present in packet
> 
> [  486.203905] [ decomp/d_tcp.c:939 d_tcp_parse_ir_cr()] CRC7 = 0x00
> 
> [  486.217307] [ decomp/d_tcp.c:969 d_tcp_parse_ir_cr()] 1-byte small
> base CID = 2
> 
> [  486.231980] [ decomp/d_tcp.c:993 d_tcp_parse_ir_cr()] IR-CR asks
> to replicate the Base CID 2 in the CID 9
> 
> [  486.249052] Unable to handle kernel NULL pointer dereference at
> virtual address 00000008
> 
> It is well reproducible if I reboot the device (local or remote) and
> do from local ssh root@remoteIP right after boot.
> 
> I use 2.1.0 rohc.ko loadable module, I wonder if anything like that
> happened before. Pretty sure it is the kernel fix branch.
> 
> Thanks!
> 
> Regards,
> Yakir Matusovsky
> MiMOMax Wireless Ltd.
> 


Follow ups

References