← Back to team overview

opensand team mailing list archive

Re: ST stops working

 

Hi Julien,

Thanks for your letter!

Here are my answers:

On 25/07/2013 17:08, Julien BERNARD wrote:
Hi Andrei,
While  carrying  out a series of experiments I faced a problem when at
some point Satellite Terminal stops working.
The  testbed  topology  is  presented in the attached file. I generate
multiple traffic flows (1 to 9) from the Sender to the Receiver in the
direction  from GW to ST using iperf with MSS = 1400 bytes. The buffer
of the GW (max_fifo) reduced to the value of 2500.
At  some  point  of the experiments (after 40-100 seconds form flow(s)
start)  the  ST  stops  processing  traffic (sometimes ST indicator in
OpenSand  Manager  turns red, sometimes it's still green but TCP flows
don't flow).

I tried to reproduce your problem in your conditions but among many tests I only encounter a crash a couple of times and this is not enough to investigate.

My tests:
 - I configure GW fifo size to 2500 and  start OpenSAND
- I launch 9 or more iperf flows with the command "iperf -c 192.168.19.1 -p 3004 -M 1400 -t 500 -m" in a workstation behind the GW Am I missing something ? Do you have a problem for each test or is it random ?
Yes, that's right. I use the same configuration and send traffic from WS behind the GW to WS behind the ST. And I don't run opensnad-network and opensand-daemon services on both workstations. I found that WS-daemon adds route to the network behind ST or GW. I use static routing instead. Please let me know if it's essential to run WS-daemon services and if they have another functions besides routing. As I said in previous letters, all machines are virtual machines hosted on one single machine. And may be I should add that I use 1.0 version of OpenSand. The problem itself is random but in my case it can be reproduced on the second-forth experimental try after starting OpenSand. In fact, this problem appeared only when I started testing GW-to-ST direction. In ST-to-GW direction TCP worked fine with BE buffers in the range from 3000 to 30 000. With buffer less than 3000 TCP performance decreases and with very low buffers (don't remember exactly - may be 50-100) iperf refused to establish connection and send traffic.

In  the  core.log  I  see  the  following errors after which ST stops
working:

Jul 24 15:28:40 router st: [encap] [Mpeg.cpp:deencapsulate():193] ERR: [Mpeg::Context::deencapsulate] cannot create a burst of packets, drop packet Jul 24 15:28:42 router st: [encap] [Mpeg.cpp:deencapMpeg():515] ERR: [Mpeg::Context::deencapMpeg] MPEG frame(s) lost (MPEG CC = 12, CTXT CC = 8), reset context, sync on PUSI Jul 24 15:28:42 router st: [encap] [Mpeg.cpp:deencapMpeg():515] ERR: [Mpeg::Context::deencapMpeg] MPEG frame(s) lost (MPEG CC = 15, CTXT CC = 2), reset context, sync on PUSI Jul 24 15:28:43 router st: [dvb_rcs_tal] [bloc_dvb_rcs_tal.cpp:connectToQoSServer():1138] [Tal][BlocDVBRcsTal::connectToQoSServer] service on TCP/12000 is not available Jul 24 15:28:44 router st: [encap] [Mpeg.cpp:deencapMpeg():536] ERR: [Mpeg::Context::deencapMpeg] PUSI not set in current MPEG frame, drop it
.............................
Jul 24 15:28:56 router st: [encap] [../../src/common/IpPacketHandler.h:build():89] ERR: [IpPacketHandler::build] cannot get IP version Jul 24 15:28:56 router st: [encap] [Ule.cpp:deencapUle():397] ERR: [Ule::Context::deencapUle] cannot create a IP packet, drop the ULE packet Jul 24 15:28:56 router st: [encap] [Ule.cpp:deencapsulate():191] ERR: [Ule::Context::deencapsulate] cannot create a burst of packets, drop packet Jul 24 15:28:56 router st: [encap] [../../src/common/IpPacketHandler.h:build():89]
The errors seems to be in the encapsulation plugins, could you try with GSE on forward link instead of MPEG/ULE ? You may have to install libopensand-gse-encap-plugin on hosts and libopensand-gse-encap-plugin-manager on manager.
Ok, thanks for suggestion! I'll try with GSE and tell you the result.

Best regards,
Andrei





Could you please tell me what can cause this problem?

I think there is a bug in OpenSAND appearing at high throughput but I need a way to reproduce it easily in order to fix it.

Best regards,




Follow ups

References