← Back to team overview

kernel-packages team mailing list archive

[Bug 1575407] Re: Trusty + 3.19 (lts-vivid) PERF wrong cpu-migration counter

 

It was brought to my attention traces from the buggy execution.

>From the perf statistics:

    201054.649876      task-clock (msec)         #    0.999 CPUs utilized          
             9,547      context-switches          #    0.047 K/sec                  
                 6      cpu-migrations            #    0.000 K/sec                  
             6,427      page-faults               #    0.032 K/sec                  
   777,038,506,852      cycles                    #    3.865 GHz                     [66.66%]
     1,799,766,809      stalled-cycles-frontend   #    0.23% frontend cycles idle    [50.00%]
   518,104,985,732      stalled-cycles-backend    #   66.68% backend  cycles idle    [50.01%]
   819,194,804,419      instructions              #    1.05  insns per cycle        
                                                  #    0.63  stalled cycles per insn [66.67%]
    90,811,473,192      branches                  #  451.676 M/sec                   [50.00%]
     2,203,470,400      branch-misses             #    2.43% of all branches         [49.99%]

     201.248973668 seconds time elapsed

**** We had 6 cpu-migrations.

>From tracing the execution we had exactly 6 set_task_cpus coming from
bottom halves (from the IRQ handler):

# coming from interrupt handler (1)

    blackscholes-15443 [000]  5599.360541: function:             set_task_cpu
    blackscholes-15443 [000]  5599.360543: kernel_stack:         <stack trace>
=> set_task_cpu (c0000000000eaa48)
=> try_to_wake_up (c0000000000ee998)
=> process_timeout (c00000000013f2e8)
=> call_timer_fn (c00000000013f154)
=> run_timer_softirq (c00000000013fd9c)
=> __do_softirq (c0000000000b5854)
=> irq_exit (c0000000000b5e28)
=> timer_interrupt (c00000000001fb94)
=> decrementer_common (c000000000002758)
    
 # coming from interrupt handler (2)

    blackscholes-15443 [000]  5600.800540: function:             set_task_cpu
    blackscholes-15443 [000]  5600.800542: kernel_stack:         <stack trace>
=> set_task_cpu (c0000000000eaa48)
=> try_to_wake_up (c0000000000ee998)
=> commit_timeout (c0000000003d52f8)
=> call_timer_fn (c00000000013f154)
=> run_timer_softirq (c00000000013fd9c)
=> __do_softirq (c0000000000b5854)
=> irq_exit (c0000000000b5e28)
=> timer_interrupt (c00000000001fb94)
=> decrementer_common (c000000000002758)

# coming from interrupt handler (3)

    blackscholes-15443 [000]  5603.840601: function:             set_task_cpu
    blackscholes-15443 [000]  5603.840603: kernel_stack:         <stack trace>
=> set_task_cpu (c0000000000eaa48)
=> try_to_wake_up (c0000000000ee998)
=> hrtimer_wakeup (c000000000141a2c)
=> __run_hrtimer (c000000000141798)
=> hrtimer_interrupt (c0000000001427f8)
=> __timer_interrupt (c00000000001f480)
=> timer_interrupt (c00000000001fb90)
=> decrementer_common (c000000000002758)

# coming from interrupt handler (4)

    blackscholes-15444 [000]  5629.340575: function:             set_task_cpu
    blackscholes-15444 [000]  5629.340577: kernel_stack:         <stack trace>
=> set_task_cpu (c0000000000eaa48)
=> try_to_wake_up (c0000000000ee998)
=> wake_up_worker (c0000000000cf004)
=> __queue_work (c0000000000d1a0c)
=> queue_delayed_work_on (c0000000000d2244)
=> bnx2x_schedule_sp_task (d00000001bbe28cc)
=> bnx2x_msix_sp_int (d00000001bbfa200)
=> handle_irq_event_percpu (c0000000001287d0)
=> handle_irq_event (c000000000128ab8)
=> handle_fasteoi_irq (c00000000012cf74)
=> generic_handle_irq (c0000000001279c8)
=> __do_irq (c000000000010f60)
=> call_do_irq (c0000000000248ac)
=> do_IRQ (c000000000011150)
=> hardware_interrupt_common (c0000000000025d8)

# coming from interrupt handler (5)

    blackscholes-15444 [000]  5636.625580: function:             set_task_cpu
    blackscholes-15444 [000]  5636.625582: kernel_stack:         <stack trace>
=> set_task_cpu (c0000000000eaa48)
=> try_to_wake_up (c0000000000ee998)
=> hrtimer_wakeup (c000000000141a2c)
=> __run_hrtimer (c000000000141798)
=> hrtimer_interrupt (c0000000001427f8)
=> __timer_interrupt (c00000000001f480)
=> timer_interrupt (c00000000001fb90)
=> decrementer_common (c000000000002758)

# coming from interrupt handler (6)

    blackscholes-15444 [000]  5640.856531: function:             set_task_cpu
    blackscholes-15444 [000]  5640.856533: kernel_stack:         <stack trace>
=> set_task_cpu (c0000000000eaa48)
=> try_to_wake_up (c0000000000ee998)
=> process_timeout (c00000000013f2e8)
=> call_timer_fn (c00000000013f154)
=> run_timer_softirq (c00000000013fd9c)
=> __do_softirq (c0000000000b5854)
=> irq_exit (c0000000000b5e28)
=> timer_interrupt (c00000000001fb94)
=> decrementer_common (c000000000002758)

-- 
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1575407

Title:
  Trusty + 3.19 (lts-vivid) PERF wrong cpu-migration counter

Status in linux package in Ubuntu:
  Confirmed

Bug description:
  It was brought to my attention that:

  In a PowerPC based server, PERF seems to acuse cpu-migrations when
  only a single cpu is activated.

  ## perf

  Performance counter stats for 'CPU(s) 0':

  15027.888988      task-clock (msec)                 #    1.000 CPUs utilized [100.00%]
  25,206                     context-switches                 #    0.002 M/sec [100.00%]
  3,518                       cpu-migrations                     #    0.234 K/sec [100.00%]
  639                           page-faults                           #    0.043 K/sec                  
  41,545,780,384    cycles                                      #    2.765 GHz [66.68%]
  2,868,753,319       stalled-cycles-frontend    #    6.91% frontend cycles idle [50.01%]
  30,162,193,535    stalled-cycles-backend     #   72.60% backend  cycles idle [50.01%]
  11,161,722,533    instructions                          #    0.27  insns per cycle        
                                                                                     #    2.70  stalled cycles per insn [66.68%]
  1,544,072,679      branches                                #  102.747 M/sec [49.99%]
  52,536,867            branch-misses                     #    3.40% of all branches [49.99%]

  15.027768835 seconds time elapsed

  ## lscpu

  Architecture:          ppc64le
  Byte Order:            Little Endian
  CPU(s):                128
  On-line CPU(s) list:   0
  Off-line CPU(s) list:  1-127
  Thread(s) per core:    1
  Core(s) per socket:    1
  Socket(s):             1
  NUMA node(s):          2
  Model:                 8335-GCA
  L1d cache:             64K
  L1i cache:             32K
  L2 cache:              512K
  L3 cache:              8192K
  NUMA node0 CPU(s):     0
  NUMA node8 CPU(s):     

  So either task migrations are being done to offline cpus or perf is
  accounting it wrong.

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1575407/+subscriptions


References