1. 22 Jul, 2009 4 commits
  2. 18 Jul, 2009 5 commits
  3. 13 Jul, 2009 3 commits
    • Roel Kluin's avatar
      perf_counter tools: Fix index boundary check · 23cdb5d5
      Roel Kluin authored
      Keep index within event_type_descriptors[]
      Signed-off-by: default avatarRoel Kluin <roel.kluin@gmail.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      LKML-Reference: <4A5A7F0B.4070106@gmail.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      23cdb5d5
    • Chris Wilson's avatar
      perf_counter: Fix the tracepoint channel to perfcounters · d4d7d0b9
      Chris Wilson authored
      Fix a missed rename in EVENT_PROFILE support so that it gets
      built and allows tracepoint tracing from the 'perf' tool.
      
      Fix a typo in the (never before built & enabled) portion in
      perf_counter.c as well, and update that code to the
      attr.config changes as well.
      Signed-off-by: default avatarChris Wilson <chris@chris-wilson.co.uk>
      Cc: Ben Gamari <bgamari.foss@gmail.com>
      Cc: Jason Baron <jbaron@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Steven Rostedt <rostedt@goodmis.org>
      LKML-Reference: <1246869094-21237-1-git-send-email-chris@chris-wilson.co.uk>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      d4d7d0b9
    • Daniel Qarras's avatar
      perf_counter, x86: Extend perf_counter Pentium M support · f1c6a581
      Daniel Qarras authored
      I've attached a patch to remove the Pentium M special casing of
      EMON and as noticed at least with my Pentium M the hardware PMU
      now works:
      
       Performance counter stats for '/bin/ls /var/tmp':
      
             1.809988  task-clock-msecs         #      0.125 CPUs
                    1  context-switches         #      0.001 M/sec
                    0  CPU-migrations           #	 0.000 M/sec
                  224  page-faults              #	 0.124 M/sec
              1425648  cycles                   #    787.656 M/sec
               912755  instructions             #	 0.640 IPC
      
      Vince suggested that this code was trying to address erratum
      Y17 in Pentium-M's:
      
        http://download.intel.com/support/processors/mobile/pm/sb/25266532.pdf
      
      But that erratum (related to IA32_MISC_ENABLES.7) does not
      affect perfcounters as we dont use this toggle to disable RDPMC
      and WRMSR/RDMSR access to performance counters. We keep cr4's
      bit 8 (X86_CR4_PCE) clear so unprivileged RDPMC access is not
      allowed anyway.
      
      Cc: Vince Weaver <vince@deater.net>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Stephane Eranian <eranian@googlemail.com>
      LKML-Reference: <new-submission>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      f1c6a581
  4. 11 Jul, 2009 6 commits
    • Arnaldo Carvalho de Melo's avatar
      perf report: Introduce -n/--show-nr-samples · e3d7e183
      Arnaldo Carvalho de Melo authored
      [acme@doppio pahole]$ perf report -ns comm,dso,symbol -d /lib64/libc-2.10.1.so -C pahole | head -17
          21.94%      32101  [.] _int_malloc
          20.10%      29402  [.] __GI_strcmp
          16.77%      24533  [.] __tsearch
          12.61%      18450  [.] malloc_consolidate
           6.42%       9394  [.] _int_free
           6.28%       9191  [.] __tfind
           4.56%       6678  [.] __GI___libc_free
           4.46%       6520  [.] _IO_vfprintf_internal
           2.59%       3786  [.] __malloc
           1.17%       1716  [.] __GI_memcpy
      [acme@doppio pahole]$
      Signed-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1247325517-12272-5-git-send-email-acme@redhat.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      e3d7e183
    • Arnaldo Carvalho de Melo's avatar
      perf_counter tools: PLT info is stripped in -debuginfo packages · a25e46c4
      Arnaldo Carvalho de Melo authored
      So we need to get the richer .symtab from the debuginfo
      packages but the PLT info from the original DSO where we have
      just the leaner .dynsym symtab.
      
      Example:
      
      | [acme@doppio pahole]$ perf report --sort comm,dso,symbol > before
      | [acme@doppio pahole]$ perf report --sort comm,dso,symbol > after
      | [acme@doppio pahole]$ diff -U1 before after
      | --- before	2009-07-11 11:04:22.688595741 -0300
      | +++ after	2009-07-11 11:04:33.380595676 -0300
      | @@ -80,3 +80,2 @@
      |       0.07%  pahole ./build/pahole              [.] pahole_stealer
      | -     0.06%  pahole /usr/lib64/libdw-0.141.so   [.] 0x00000000007140
      |       0.06%  pahole /usr/lib64/libdw-0.141.so   [.] __libdw_getabbrev
      | @@ -91,2 +90,3 @@
      |       0.06%  pahole [kernel]                    [k] free_hot_cold_page
      | +     0.06%  pahole /usr/lib64/libdw-0.141.so   [.] tfind@plt
      |       0.05%  pahole ./build/libdwarves.so.1.0.0 [.] ftype__add_parameter
      | @@ -242,2 +242,3 @@
      |       0.01%  pahole [kernel]                    [k] account_group_user_time
      | +     0.01%  pahole /usr/lib64/libdw-0.141.so   [.] strlen@plt
      |       0.01%  pahole ./build/pahole              [.] strcmp@plt
      | [acme@doppio pahole]$
      Signed-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1247325517-12272-4-git-send-email-acme@redhat.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      a25e46c4
    • Arnaldo Carvalho de Melo's avatar
      perf report: Make the output more compact · 021191b3
      Arnaldo Carvalho de Melo authored
      When we filter by column content we may end up with a column
      that has the same value for all the lines. So remove that
      column and tell its unique value on the top, as a comment.
      
      Example:
      
        [acme@doppio pahole]$  perf report --sort comm,dso,symbol -d ./build/libdwarves.so.1.0.0 -C pahole | head -15
        # dso: ./build/libdwarves.so.1.0.0
        # comm: pahole
        # Samples: 58409
        #
        # Overhead  Symbol
        # ........  ......
        #
            20.93%  [.] tag__recode_dwarf_type
            14.94%  [.] namespace__recode_dwarf_types
            10.38%  [.] cu__table_add_tag
             6.69%  [.] __die__process_tag
             5.05%  [.] die__process_function
             4.70%  [.] list__for_all_tags
             3.68%  [.] tag__init
             3.48%  [.] die__create_new_parameter
        [acme@doppio pahole]$
      Signed-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1247325517-12272-3-git-send-email-acme@redhat.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      021191b3
    • Arnaldo Carvalho de Melo's avatar
      strlist: Introduce strlist__entry and strlist__nr_entries methods · 27d0fd41
      Arnaldo Carvalho de Melo authored
      The strlist__entry method allows accessing strlists like an
      array, will be used in the 'perf report' to access the first
      entry.
      
      We now keep the nr_entries so that we can check if we have just
      one entry, will be used in 'perf report' to improve the output
      by showing just at the top when we have just, say, one DSO.
      
      While at it use nr_entries to optimize strlist__is_empty by not
      using the far more costly rb_first based implementation.
      Signed-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1247325517-12272-2-git-send-email-acme@redhat.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      27d0fd41
    • Arnaldo Carvalho de Melo's avatar
      perf report: Tidy up reporting of symbols not found · 60c1baf1
      Arnaldo Carvalho de Melo authored
      Always printing the level info about if it is in the kernel,
      hypervisor or userspace as that is in the hist_entry.
      Signed-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1247325517-12272-1-git-send-email-acme@redhat.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      60c1baf1
    • Arnaldo Carvalho de Melo's avatar
      perf report: Adjust column width to the values sampled · 52d422de
      Arnaldo Carvalho de Melo authored
      Auto-adjust column width of perf report output to the
      longest occuring string length.
      
      Example:
      
      [acme@doppio pahole]$  perf report --sort comm,dso,symbol | head -13
      
          12.79%   pahole  /usr/lib64/libdw-0.141.so    [.] __libdw_find_attr
           8.90%   pahole  /lib64/libc-2.10.1.so        [.] _int_malloc
           8.68%   pahole  /usr/lib64/libdw-0.141.so    [.] __libdw_form_val_len
           8.15%   pahole  /lib64/libc-2.10.1.so        [.] __GI_strcmp
           6.80%   pahole  /lib64/libc-2.10.1.so        [.] __tsearch
           5.54%   pahole  ./build/libdwarves.so.1.0.0  [.] tag__recode_dwarf_type
      [acme@doppio pahole]$
      
      [acme@doppio pahole]$  perf report --sort comm,dso,symbol -d /lib64/libc-2.10.1.so | head -10
      
          21.92%   pahole  /lib64/libc-2.10.1.so  [.] _int_malloc
          20.08%   pahole  /lib64/libc-2.10.1.so  [.] __GI_strcmp
          16.75%   pahole  /lib64/libc-2.10.1.so  [.] __tsearch
      [acme@doppio pahole]$
      
      Also add these extra options to control the new behaviour:
      
        -w, --field-width
      
      Force each column width to the provided list, for large terminal
      readability.
      
        -t, --field-separator:
      
      Use a special separator character and don't pad with spaces, replacing
      all occurances of this separator in symbol names (and other output) with
      a '.' character, that thus it's the only non valid separator.
      Signed-off-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Paul Mackerras <paulus@samba.org>
      LKML-Reference: <20090711014728.GH3452@ghostprotocols.net>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      52d422de
  5. 10 Jul, 2009 5 commits
  6. 05 Jul, 2009 5 commits
    • Frederic Weisbecker's avatar
      perf report: Add "Fractal" mode output - support callchains with relative overhead rate · 805d127d
      Frederic Weisbecker authored
      The current callchain displays the overhead rates as absolute:
      relative to the total overhead.
      
      This patch provides relative overhead percentage, in which each
      branch of the callchain tree is a independant instrumentated object.
      
      This provides a 'fractal' view of the call-chain profile: each
      sub-graph looks like a profile in itself - relative to its parent.
      
      You can produce such output by using the "fractal" mode
      that you can abbreviate via f, fr, fra, frac, etc...
      
      ./perf report -s sym -c fractal
      
      Example:
      
           8.46%  [k] copy_user_generic_string
                      |
                      |--52.01%-- generic_file_aio_read
                      |          do_sync_read
                      |          vfs_read
                      |          |
                      |          |--97.20%-- sys_pread64
                      |          |          system_call_fastpath
                      |          |          pread64
                      |          |
                      |           --2.81%-- sys_read
                      |                     system_call_fastpath
                      |                     __read
                      |
                      |--39.85%-- generic_file_buffered_write
                      |          __generic_file_aio_write_nolock
                      |          generic_file_aio_write
                      |          do_sync_write
                      |          reiserfs_file_write
                      |          vfs_write
                      |          |
                      |          |--97.05%-- sys_pwrite64
                      |          |          system_call_fastpath
                      |          |          __pwrite64
                      |          |
                      |           --2.95%-- sys_write
                      |                     system_call_fastpath
                      |                     __write_nocancel
      [...]
      Signed-off-by: default avatarFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Anton Blanchard <anton@samba.org>
      Cc: Jens Axboe <jens.axboe@oracle.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1246772361-9960-5-git-send-email-fweisbec@gmail.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      805d127d
    • Frederic Weisbecker's avatar
      perf_counter tools: callchains: Manage the cumul hits on the fly · e05b876c
      Frederic Weisbecker authored
      The cumul hits are the number of hits of every childs of a node
      plus the hits of the current nodes, required for percentage
      computing of a branch.
      
      Theses numbers are calculated during the sorting of the branches of
      the callchain tree using a depth first postfix traversal, so that
      cumulative hits are propagated in the right order.
      
      But if we plan to implement percentages relative to the parent and not
      absolute percentages (relative to the whole overhead), we need to know
      the cumulative hits of the parent before computing the children
      because the relative minimum acceptable number of entries (ie: minimum
      rate against the cumulative hits from the parent) is the basis to
      filter the children against a given rate.
      
      Then we need to handle the cumul hits on the fly to prepare the
      implementation of relative overhead rates.
      Signed-off-by: default avatarFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Anton Blanchard <anton@samba.org>
      Cc: Jens Axboe <jens.axboe@oracle.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1246772361-9960-4-git-send-email-fweisbec@gmail.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      e05b876c
    • Frederic Weisbecker's avatar
      perf report: Change default callchain parameters · 94a8eb02
      Frederic Weisbecker authored
      The default callchain parameters are set to use the flat mode and never
      filter any overhead threshold of backtrace.
      
      But flat mode is boring compared to graph mode.
      Also the number of callchains may be very high if none is
      filtered.
      
      Let's change this to set the graph view and a minimum overhead of 0.5%
      as default parameters.
      Reported-by: default avatarIngo Molnar <mingo@elte.hu>
      Signed-off-by: default avatarFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Anton Blanchard <anton@samba.org>
      Cc: Jens Axboe <jens.axboe@oracle.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1246772361-9960-3-git-send-email-fweisbec@gmail.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      94a8eb02
    • Frederic Weisbecker's avatar
      perf report: Use a modifiable string for default callchain options · be903885
      Frederic Weisbecker authored
      If the user doesn't provide options to tune his callchain output
      (ie: if he uses -c without arguments) then the default value passed
      in the OPT_CALLBACK_DEFAULT() macro is used.
      
      But it's parsed later by strtok() which will replace comma separators
      to a zero. This may segfault as we are using a read-only string.
      
      Use a modifiable one instead, and also fix the "100%" default
      minimum threshold value by turning it into a 0 (output every callchains)
      as it was intended in the origin.
      Signed-off-by: default avatarFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Anton Blanchard <anton@samba.org>
      Cc: Jens Axboe <jens.axboe@oracle.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1246772361-9960-2-git-send-email-fweisbec@gmail.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      be903885
    • Frederic Weisbecker's avatar
      perf report: Warn on callchain output request from non-callchain file · 91b4eaea
      Frederic Weisbecker authored
      perf report segfaults while trying to handle callchains from a non
      callchain data file.
      
      Instead of a segfault, print a useful message to the user.
      Reported-by: default avatarJens Axboe <jens.axboe@oracle.com>
      Reported-by: default avatarArnaldo Carvalho de Melo <acme@redhat.com>
      Signed-off-by: default avatarFrederic Weisbecker <fweisbec@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Anton Blanchard <anton@samba.org>
      Cc: Jens Axboe <jens.axboe@oracle.com>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      LKML-Reference: <1246772361-9960-1-git-send-email-fweisbec@gmail.com>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      91b4eaea
  7. 04 Jul, 2009 1 commit
    • Eric Dumazet's avatar
      x86: atomic64: Inline atomic64_read() again · a79f0da8
      Eric Dumazet authored
      Now atomic64_read() is light weight (no register pressure and
      small icache), we can inline it again.
      
      Also use "=&A" constraint instead of "+A" to avoid warning
      about unitialized 'res' variable. (gcc had to force 0 in eax/edx)
      
        $ size vmlinux.prev vmlinux.after
           text    data     bss     dec     hex filename
        4908667  451676 1684868 7045211  6b805b vmlinux.prev
        4908651  451676 1684868 7045195  6b804b vmlinux.after
      Signed-off-by: default avatarEric Dumazet <eric.dumazet@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <4A4E1AA2.30002@gmail.com>
      [ Also fix typo in atomic64_set() export ]
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      a79f0da8
  8. 03 Jul, 2009 11 commits
    • Ingo Molnar's avatar
      x86: atomic64: Clean up atomic64_sub_and_test() and atomic64_add_negative() · ddf9a003
      Ingo Molnar authored
      Linus noticed that the variable name 'old_val' is
      confusingly named in these functions - the correct
      naming is 'new_val'.
      Reported-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <alpine.LFD.2.01.0907030942260.3210@localhost.localdomain>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      ddf9a003
    • Ingo Molnar's avatar
      x86: atomic64: Improve atomic64_xchg() · 3a8d1788
      Ingo Molnar authored
      Remove the read-first logic from atomic64_xchg() and simplify
      the loop.
      
      This function was the last user of __atomic64_read() - remove it.
      
      Also, change the 'real_val' assumption from the somewhat quirky
      1ULL << 32 value to the (just as arbitrary, but simpler) value
      of 0.
      Reported-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <tip-05118ab8859492ac9ddda0154cf90e37b0a4a0b0@git.kernel.org>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      3a8d1788
    • Ingo Molnar's avatar
      x86: atomic64: Export APIs to modules · 1fde902d
      Ingo Molnar authored
      atomic64_t primitives are used by a handful of drivers,
      so export the APIs consistently. These were inlined
      before.
      
      Also mark atomic64_32.o a core object, so that the symbols
      are available even if not linked to core kernel pieces.
      
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <tip-05118ab8859492ac9ddda0154cf90e37b0a4a0b0@git.kernel.org>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      1fde902d
    • Eric Dumazet's avatar
      x86: atomic64: Improve atomic64_read() · 67d7178f
      Eric Dumazet authored
      Optimize atomic64_read() as a special open-coded
      cmpxchg8b variant. This generates nicer code:
      
      arch/x86/lib/atomic64_32.o:
      
         text	   data	    bss	    dec	    hex	filename
          435	      0	      0	    435	    1b3	atomic64_32.o.before
          431	      0	      0	    431	    1af	atomic64_32.o.after
      
      md5:
         bd8ab95e69c93518578bfaf0ea3be4d9  atomic64_32.o.before.asm
         2bdfd4bd1f6b7b61b7fc127aef90ce3b  atomic64_32.o.after.asm
      
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <alpine.LFD.2.01.0907021653030.3210@localhost.localdomain>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      67d7178f
    • Paul Mackerras's avatar
      x86: atomic64: Code atomic(64)_read and atomic(64)_set in C not CPP · 8e049ef0
      Paul Mackerras authored
      Occasionally we get bugs where atomic_read or atomic_set are
      used on atomic64_t variables or vice versa.  These bugs don't
      generate warnings on x86 because atomic_read and atomic_set are
      coded as macros rather than C functions, so we don't get any
      type-checking on their arguments; similarly for atomic64_read
      and atomic64_set in 64-bit kernels.
      
      This converts them to C functions so that the arguments are
      type-checked and bugs like this will get caught more easily. It
      also converts atomic_cmpxchg and atomic_xchg, and
      atomic64_cmpxchg and atomic64_xchg on 64-bit, so we get
      type-checking on their arguments too.
      
      Compiling a typical 64-bit x86 config, this generates no new
      warnings, and the vmlinux text is 86 bytes smaller.
      
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <alpine.LFD.2.01.0907021653030.3210@localhost.localdomain>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      8e049ef0
    • Ingo Molnar's avatar
      x86: atomic64: Fix unclean type use in atomic64_xchg() · 199e2378
      Ingo Molnar authored
      Linus noticed that atomic64_xchg() uses atomic_read(), which
      happens to work because atomic_read() is a macro so the
      .counter value gets u64-read on 32-bit too - but this is really
      bogus and serious bugs are waiting to happen.
      
      Fix atomic64_xchg() to use __atomic64_read() instead.
      
      No code changed:
      
      arch/x86/lib/atomic64_32.o:
      
         text	   data	    bss	    dec	    hex	filename
          435	      0	      0	    435	    1b3	atomic64_32.o.before
          435	      0	      0	    435	    1b3	atomic64_32.o.after
      
      md5:
         bd8ab95e69c93518578bfaf0ea3be4d9  atomic64_32.o.before.asm
         bd8ab95e69c93518578bfaf0ea3be4d9  atomic64_32.o.after.asm
      Reported-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <alpine.LFD.2.01.0907021653030.3210@localhost.localdomain>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      199e2378
    • Ingo Molnar's avatar
      x86: atomic64: Make atomic_read() type-safe · 32171208
      Ingo Molnar authored
      Linus noticed that atomic64_xchg() uses atomic_read(), which
      happens to work because atomic_read() is a macro so the
      .counter value gets u64-read on 32-bit too - but this is really
      bogus and serious bugs are waiting to happen.
      
      Change atomic_read() to be a type-safe inline, and this exposes
      the atomic64 bogosity as well:
      
        arch/x86/lib/atomic64_32.c: In function ‘atomic64_xchg’:
        arch/x86/lib/atomic64_32.c:39: warning: passing argument 1 of ‘atomic_read’ from incompatible pointer type
      Reported-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <alpine.LFD.2.01.0907021653030.3210@localhost.localdomain>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      32171208
    • Ingo Molnar's avatar
      x86: atomic64: Reduce size of functions · 3ac805d2
      Ingo Molnar authored
      cmpxchg8b is a huge instruction in terms of register footprint,
      we almost never want to inline it, not even within the same
      code module.
      
      GCC 4.3 still messes up for two functions, under-judging the
      true cost of this instruction - so annotate two key functions
      to reduce the bloat:
      
      arch/x86/lib/atomic64_32.o:
      
         text	   data	    bss	    dec	    hex	filename
         1763	      0	      0	   1763	    6e3	atomic64_32.o.before
          435	      0	      0	    435	    1b3	atomic64_32.o.after
      
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <alpine.LFD.2.01.0907021653030.3210@localhost.localdomain>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      3ac805d2
    • Ingo Molnar's avatar
      x86: atomic64: Improve atomic64_add_return() · 824975ef
      Ingo Molnar authored
      Linus noted (based on Eric Dumazet's numbers) that we would
      probably be better off not trying an atomic_read() in
      atomic64_add_return() but intead intentionally let the first
      cmpxchg8b fail - to get a cache-friendly 'give me ownership
      of this cacheline' transaction. That can then be followed
      by the real cmpxchg8b which sets the value local to the CPU.
      Reported-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Cc: Eric Dumazet <eric.dumazet@gmail.com>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <alpine.LFD.2.01.0907021653030.3210@localhost.localdomain>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      824975ef
    • Eric Dumazet's avatar
      x86: atomic64: Improve cmpxchg8b() · 69237f94
      Eric Dumazet authored
      Rewrite cmpxchg8b() to not use %edi register but a generic "+m"
      constraint, to increase compiler freedom in code generation and
      possibly better code.
      
      Cc: Linus Torvalds <torvalds@linux-foundation.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <alpine.LFD.2.01.0907021653030.3210@localhost.localdomain>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      69237f94
    • Eric Dumazet's avatar
      x86: atomic64: Improve atomic64_read() · aacf682f
      Eric Dumazet authored
      Linus noticed that the 32-bit version of atomic64_read() was
      being overly complex with re-reading the value and doing a
      retry loop over that.
      
      Instead we can just rely on cmpxchg8b returning either the new
      value or returning the current value.
      
      We can use any 'old' value, which will be faster as it can be
      loaded via immediates. Using some value that is not equal to
      the real value in memory the instruction gets faster.
      
      This also has the advantage that the CPU could avoid dirtying
      the cacheline.
      Reported-by: default avatarLinus Torvalds <torvalds@linux-foundation.org>
      Cc: Peter Zijlstra <a.p.zijlstra@chello.nl>
      Cc: Mike Galbraith <efault@gmx.de>
      Cc: Paul Mackerras <paulus@samba.org>
      Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
      Cc: Frederic Weisbecker <fweisbec@gmail.com>
      Cc: David Howells <dhowells@redhat.com>
      Cc: Andrew Morton <akpm@linux-foundation.org>
      Cc: Arnd Bergmann <arnd@arndb.de>
      LKML-Reference: <alpine.LFD.2.01.0907021653030.3210@localhost.localdomain>
      Signed-off-by: default avatarIngo Molnar <mingo@elte.hu>
      aacf682f